Are email notifications sent out for unlisted topics?

Unlisting does not prevent any notifications. So if you @mention a user, they will be notified. And if they choose to receive emails for being @mentioned, then they will be emailed even if the topic is hidden.

If you link to someone’s post or quote them, they will also be notified. Even if you don’t do any of that, users who are watching the category and/or tag will still be notified, so if you don’t want anyone to be notified, you need to use a private category.