Actually, what you see as Breadcrumbs on category and tag lists is the nav-bar. It’s just styled so that it only shows selected items, and no drop-downs.
I don’t want to add drop-down filters here because that would pretty much go against the direction of the theme. I do think of having breadcrumbs on all routes though. What I’d like is a look that always repeats the current selection on the sidebar as a breadcrumb indicator. E.g. for the Latest list:
So I hope to add this as an optional component a bit later 
I can totally see that more navigation options make sense on your forum though! The Panels component as such could probably go well with your setup?