This is what I was missing. It is working now with TLS.
I also got DKIM working so mails should not be spammed anymore.