Although that is perfectly possible from a technical point of view, I think the approach would be over complicated, especially since “at the start” is hard to define/detect, and you want to avoid that the plugin keeps adding B back after the site admin removed it. Also, when a new private category is added, the plugin would need to add it to the setting, but it would need to be able to see the difference between a new category (add) and a category that was previously removed by the admin (don’t re-add).
I would opt for a include_private_categories setting which starts out empty, and the plugin would simply process all public categories, AND the categories in include_private_categories. That will give you a lot less headaches.
Also, from a functional point of view, that makes it more easy to spot which private categories are being shared (in your approach one would have to have a list of categories in their mind and then determine which ones are missing by comparing them to the ones in the setting).