See How to override a template.hbs from a plugin - #4 by tshenry. I think that’s the magic your looking for.
It’s not where I said, but you stick it in the assets tree with the right name. It’s impossible to understand until you get it and then it’s totally obvious.
Edit : Hey @JQ331. I just realized that I think I did what you want in s plugin yesterday, but I’m so bad at it I didn’t realize that’s what you’re looking for. See https://github.com/literatecomputing/discourse-max-topics-per-month. It’s very simple so there’s not much to get in the way of finding the example.