The plugin that adds this feature to the sidebar is
You create subcategories like categories. The only difference is that you select a parent category
Create a category in Discourse
There is also a guide about how to create the boxes Display boxes for subcategories above topic list