Used the smtp server that was provided by the web host we got our VPS from. Used Ssl ports etc.
Here is even where it gets more interesting. Emails sent through that same smtp using ssl ports still works ok.
So, Emails through - https://mydomain.com still works fine even till now.
Emails through - https://ourDiscourseInstance.mydomain.com is blocked regardless of the port used/ssl used.
My understanding is that the block is for Discourse/other Forum tools.