I think the external reverse proxy is the key - I don’t think this is solvable at all, or at least not easily, from just within the Docker container.
I’ve solved it now by having the Discourse Docker listen on a Unix socket and front it with a Caddy on the same machine (outside Docker). The Caddy setup is trivial, includes Let’s Encrypt, and now it works as expected for IPv6 as well.
forum.example.com {
proxy / unix:/var/discourse/shared/web-only/nginx.http.sock {
transparent
websocket
}
}