I would create a template that you include in the docker container which is created by the launcher.
For example like this rate limiting template which makes changes to the nginx setup: discourse_docker/templates/web.ratelimited.template.yml at main · discourse/discourse_docker · GitHub
You would include your own template in the templates: list of your container yml configuration.
The next time you rebuild it, your changes to the nginx configuration will be included.