That is configurable, albeit not easily.
For personal chats, the default is being auto-notified for every message. This can be turned down for each chat thread by the user.
For channels, the default is as you describe. Users can change that to every message if they wish.
The last time I looked, the only way to change the default behaviour was to delve into the Rails console. Getting this into the UI could be a worthy feature request
Does this work as is, or does it require the whitelisted app as well?