When I try to send a test message to a private (but previously public) channel, I get a 422 error. When I check the logs, I see this:
Test provider failed {:error_key=>"chat_integration.provider.slack.errors.channel_not_found", :request=>"", :response_code=>"200", :response_body=>"{\"ok\":false,\"error\":\"channel_not_found\"}"}
This setup worked when the channel was private. I tried re-installing the Slack app and deleting the rules and recreating them. When I use @username rule or a public channel that seems to work well.