In a discussion on minimizing downtime during Discourse maintenance, emonunix asks for best practices for tasks like changing resource settings or applying major updates, which typically require a lengthy launcher rebuild app. pfaffman suggests using a two-container setup (web + data) where the new container builds while the old one runs, reducing downtime to the time needed to launch the new container — provided sufficient RAM is available. For true zero-downtime, a load balancer can route traffic to the new container only after it’s fully started, followed by shutting down the old one and running post-update migrations. Ethsim2 asks about failover with two data containers, to which pfaffman replies that while possible with PostgreSQL mirroring, it’s complex and he prefers migrating to a new VM for major reconfigurations. itsbhanusharma suggests using a read replica, which pfaffman confirms as a viable option for hot-swapping if the primary fails. merefield notes that Discourse is highly stable, making failover setups unnecessary for most, though useful for high-availability or hosted environments. They caution that rebuilds are the riskiest moments, and while two-container setups allow testing before commit, rollbacks are rarely attempted once migrations run — fixing errors in-place is preferred.
I’d like to open a discussion on the best practices for performing core maintenance tasks on a Discourse instance while minimizing or eliminating downtime.
Tasks like changing critical resource settings (e.g., UNICORN_WORKERS, DISCOURSE_SIDEKIQ_WORKERS, DISCOURSE_DB_POOL) or applying major updates typically require a launcher rebuild app which can take a significant amount of time, sometimes 30 minutes or more.
My question is: What are the recommended strategies for system administrators to perform these essential updates and configuration changes with the least amount of user-facing downtime?
Are there any advanced techniques, like blue/green deployments or other zero-downtime deployment strategies, that are supported or recommended for Discourse? Or is the standard rebuild process the only supported method, and the focus should be on optimizing the rebuild time itself?
I’m interested in hearing from anyone who has experience managing large or high-traffic instances and what their workflow looks like for maintenance.
If you have a two container install, the new container builds while the old one runs. Downtime is just the amount of time it takes to launch the new container. The only issue is that you need enough ram to build a container while the other runs.
If you want zero down time then you need a load balancer that keeps the old container running until the new one has fully started. Then you shut down the old container and do the post update migrations.
Discourse is so stable this is pretty unnecessary for most installs (but I guess you might consider it for very high availability requirements or if you are hosting others?!)
I don’t think I’ve had a single outage in 7 years due to a production “glitch” …
The riskiest moments in a Discourse’s life is always at rebuild.
the two container setup gives you the ability to bootstrap a new build before committing to it though that won’t catch some runtime errors of course.
The issue is that if your migrations have run, you might need to commit to the new build and so you would usually try to track down and fix the source of those errors rather than roll back.