If you want to debug your smtp then see Troubleshoot email on a new Discourse install - #547 by Nikolai_Patrushev or perhaps Troubleshooting Outgoing Email. Discourse-setup isn’t really the best way.
Just run the rake task yourself, then edit the yml and destroy and start the container then exit and try again.