Suggestion: Add some language to the top of the notifications tab of the preferences page to briefly explain the nuances about what Discourse means by the word “notification”. A first attempt:
Below, control which kinds of events you’ll receive notifications and how you’ll receive them. You can also control the notification level for categories and tags here.
By default, notifications will first appear as an indicator above your avatar. You can also turn on desktop notifications if you want to be alerted when the app is open but not in focus. But if you’re not around, we’ll email you for the important ones.
In the various notification drop downs (like the one in the screenshot below), it may also be nice to link to the notifications preferences page to help people discover this.
Visit your notification preferences to refine how you receive notifications.
I did a little “hallway usability testing” yesterday. After showing someone the how they could subscribe as “Watching First Post” to the tag for the new weekly newsletter we’re piloting, I watched him click around various places to try and grok what these words in the dropdown really mean: “You will only be notified of the first post in each new topic in this tag”.
At first I thought he was confused about the differences between watching and tracking, etc. But after a minute or two I asked him to speak aloud about what he was doing as he was exploring the preferences pages.
In the end, what stood out to me was his statement that, “what I’m really to figure out is ‘what is a notification?’ Is that a desktop notification? an email? what does notification really mean?”
I then pointed out this part of one of the pages he had clicked through (the Emails tab of the preferences page):
That helped to clarify, but he spent most of his time on the Notifications tab while exploring, which is what led me to suggest that we add this language to that tab.
Another observation is that he did not ever click on the nested Categories or Tags submenus under Notifications until I pointed them out, which is why I suggested also including links to them in the verbiage above.