OK, I figured most of this out. (Isn’t realtime reporting fun? 
)
Apparently, there’s a setting called max users notified per group mention, which (you guessed it) limits the max number of users that can be notified by a mention.
Setting it to a high value like 9999 (we only have 400 users) looks like it works as expected.
The admin screen clearly describe what this setting does:
Maximum number of users that may receive a notification if a group is mentioned (if threshold is met no notifications will be raised)
However, this isn’t mentioned when you get the just-in-time warning:
@team, I propose that this is a UX bug. If the number you’re mentioning is higher than the max users notified per group mention setting, then Discourse should extend the above warning with some copy like this:
By mentioning @group_name, you would normally notify X people. But because this exceeds the maximum number of users that may receive a notification, no notifications will be raised.
I’m not sure how exactly multiple group mentions are handled, so I can’t offer suggestions in that case.
In related news, messaging @trust_level_0 also doesn’t seem to work – it only sends 10 messages and then ceases. Is there a similar setting that prevents messaging users? I wasn’t able to find it.