Not much. Your easy options are to either turn off user profiles so that no one can see any profiles or allow people to see all of them.
A plugin could probably override the user list or hide users who are not in the same primary group as the user looking. If it would be OK for the other users to be in the JSON load, it might be possible for a theme component to hide users that don’t match the user’s group.
This is something that comes up from time to time.
If you really want people not to see anyone else your best bet is a separate forum for each group. You could do it with multisite, so there’s still be only a single container.