Race condition in generation of desktop.css

(Leon Torres) #1

We have a multi-server deployment of discourse behind a LB. We use rsyncd to keep the /uploads/stylesheet-cache/ directory in sync. However sometimes when we do deployments our users will get 404 when requesting desktop_${digest}.css.

We poked around and discovered that this happens because the digest function in discourse_stylesheets.rb is called simultaneously on both our servers from two different requests. Part of the digest is the last modified time of the most recent stylesheet in /app/assets/stylesheets/**/*.*css. Since rsyncd can’t guarantee instant file synchronization, this may lead to different digest results, and hence our users will sometimes get 404 on new sessions.

Maybe there should be a better digest synchronization than relying on most recent mtime of a local directory? Otherwise it seems NFS is our only risk-free solution, but that has its own set of problems.

(Sam Saffron) #2

This is not a bug, or at least not our bug.

The recommendation is to build a base container and then distribute it to multiple hosts in a multi host environment, and use something like gluster or cyph or nfs for the uploads directory.

We have no internal mechanism for cross machine file replication, on complex setups you need to take care of that.

(Leon Torres) #3

OK thanks for the clarification. This makes sense to solve at the system level.