This link may help. From my understanding you will maybe need to spin up a new VPS for your mail server. As Discourse team do not support trying to run discourse with a mail server on the same vps as it can break things and make troubleshooting very difficult.
This topic from mailcow seems to be for setting up a mail server.
After I believe most smpt setup is pretty standard. Smpt user name and password.
Be sure to research things carefully as the mail server can break your project if email ends up being blocked.
Depending on community size some services like Brevo com have free tiers. Brevo gives 300 emails/day on free tier.
Looks like they are running discourse so likely a good place to ask them.