Depending upon what you’d want to change in the nginx configuration, the easiest way would be to use an external nginx as reverse proxy to the discourse container and make your changes there.
Alternatively, if you want to make changes directly to your internal nginx server, you may have to create a hook in your yml file. It will vary based on what you want to achieve but this topic will give you a hint Multisite Configuration with Let's Encrypt and no reverse proxy