The category icons component overrides a core category link renderer to add its icons. The sidebar, for good reason, doesn’t use the same renderer, so we need to add a way to extend/override the sidebar category links in core before a component can do something here.
Similar story for tags, but it’s probably a little easier given that we are currenly showing a default icon for each tag in the sidebar.