On the category edit modal of the category you are trying to change the permissions for, have a look at its “General” tab. You should see a list of the category’s subcategories. Something like this:
What the error message you are seeing means is that one or more of the subcategories allows the group “everyone” to access it. The group “everyone” includes all users on your site. You will either need to change the subcategory’s permissions so that it can only be accessed by admins, or remove the parent category from the subcategory so that it is no longer a subcategory of the category you are trying to edit. You can remove the parent category here:
