There is no “this group cannot”.
So it sounds like if there are any categories that should be accessible to “everyone,” then you’ll need to remove “everyone” from the other categories.
You don’t need to add “l1” and “l2” since “L2” by definition includes L1.
Most communities have most categories open to everyone. If your community is one where most people are not full members and are excluded, you’ll need to remove everyone from all of the categories. If there are more than a dozen or so, you might want to search here to find out how to use the Rails console to act on a bunch of categories at once.