What happens to members accounts

That’s correct. Though if all your categories are limited by group and a user is not in any group anymore, then they will have a sad experience when they log in! You may want to also suspend their user accounts, unless you want to create one or more categories they are still allowed to access.