Thanks for this @JammyDodger !
So I checked a topic ID that I never visited and could not find myself in the list. But when I visit that topic and rerun the query I appear in the list with notification level = 1. Is this expected? (note that auto-tracking is off!)
Then, for the topics that were causing spurious mails to many many users I found a huuuge list of users with notif. level = 1. Some of these are staged used which actually never logged in on the web and also did not participate in the thread.
Finally, I found that the topics with mass-subscriptions were mostly from before the time the site went online and users could log in. Also the number of subscribers is pretty stable and seems to correspond to the number of users the site had at that time.
I should note that at some point I enabled the setting allow changing staged user tracking but then I disabled that. I also enabled/disabled the mute all categories by default at some point.
Maybe this all somehow caused this confusion?
In the end I should probably just suggest the users to unsubscribed from the unwanted topics.