Have you removed any permissions from the everyone group? If you haven’t done that, you’ll see this message:
This category is public, everyone can see, reply and create posts. To restrict permissions, remove one or more of the permissions granted to the “everyone” group.
In this state, when you click Save, nothing will be saved.
Does it persist through a page reload? If not, it’s probably not being saved correctly. Are you behind a subfolder or proxy? Do any errors show up in the JS console (F12)?
If it does, is there anything in your admin logs? Check /admin/logs/staff_action_logs and filter by “change category settings”.