Restricting Access to categories

Hm ok. Not sure how I can restrict non-members (in Wordrpess membership entering the forum via SSO) then from accessing the forum other than restricting the categories for certain user groups without losing the badge feature. Am I missing something and the way I implemented it is wrong in the first place?