Topics in a muted category, or individually muted topics, will still show in latest. They won’t ever generate notifications or unread counts, though.
What you propose would not work. Muting a category simply means, set all new topics in that category to a notification state of “muted” for you at the time of creation. There is no mechanism for somehow automagically deciding a topic in a category you explicitly muted, is no longer muted.
Sounds like it would be easier to just browse your site by category.