Hi — could you please check the Discourse site setting invite only and disable it if it’s enabled?
Disabling invite only will allow new users to sign in or register using OAuth2.
Once they’re signed in, you can use your group-membership rules (automation or manual assignment) to grant them access to the private category.