OK, here’s the report:
I created a DS1_V2 (~$55/month) virtual machine running Ubuntu 16.04 LTS. I did
wget -qO- https://get.docker.com/ | sh
apt -y upgrade
./discourse-setup
It worked just fine.
Oh. You see? I knew there would be head banging. Turns out, not being able to access your machine is a feature, not a bug. You need to open the ports, like this.
Once I managed to convince Azure to open port 80 so that one could access it via a web browser, I enabled Let’s Encrypt and rebuilt. That worked too. (And to make sure that there was no outbound filtering, I sent an activation message with Mailgun. No shenanigans were required.).