Discourse setup on aws ec2 doesn't work with default IPv4 DNS

Sure! See that one one else who uses an .amazonaws.com domain name uses Let’s Encrypt and then wait a week. If you don’t want to register a domain, that would be the next thing that I’d try.