How to make AI summaries available to specific user groups only

I set it up, but it still seems to be showing to the wrong user.