Faster rebuilds?

If rebuild time is a huge concern to you I would strongly recommend running a dedicated data container

That way you can bootstrap while your existing site is up and running, meaning you only deal with a few seconds of outage when upgrading via launcher

Bundle install is usually very fast when we just ship an image, I plan on having more regular updates to help out here

Assets precompile is the slowest part, we are cosidering pre-bundling assets or possibly adding a caching server or even more aggressive image updates, but nothing is a trivially easy solution

7 Likes

Is there any migration documentation about how to change to this configuration? I thought I saw something at one point but can’t find it in #howto now.

This one. Noted some steps is out-dated:

  1. Use enter to get into a container.
  2. DISCOURSE_DB_HOST: data and DISCOURSE_REDIS_HOST: data
  • link to a container is prefered than IP now.
  • define links block as follows:
links:
  - link:
      name: data
      alias: data
3 Likes

if we don’t stop forum during rebuild (make it read-only if necessary), it will be better?

Can not be done in a single container setup without major headaches… part of the bootstrap process is spawning the db, you can’t have two instances of a database running at the same time.

2 Likes

How does this work when using a separate data container? Does it run migrations on the data that the old site is still using, potentially breaking the site until the web container is upgraded if the migrations are incompatible?

5 Likes

Yes, but… We are very careful to keep migrations compatible data wise, even have a pretty elaborate scheme that delays dropping columns when we move them to new tables

5 Likes

Should this process have copied my data & configuration over? I ran through the steps but got a clean/“fresh” site, so rolled back to the single-container version.

I did a restore from a backup made just before the process, and it seemed to restore everything. So probably this just needs to be made clear. :slightly_smiling:

1 Like

I flagged it to ask for wikifing the topic… Let’s amend it :wink:

PS: Don’t know whether it’s a right way to use flag…

Yes, that’s the right way to use the notify moderator feature :wink:

2 Likes

Can you say a bit more about how this process works in your mind?

After switching to the multi-container scenario, I just did a ./launcher rebuild web_only but my downtime was the same. Clearly I need more coffee still this morning.

./launcher bootstrap web_only

Start outage

./launcher destroy web_only
./launcher start web_only

End outage

4 Likes

Is there any way to skip the whole uglify and compressing of assets & locales during bootstrap if the assets already exist?

Maybe it would make sense to have a dedicated assets volume and/or a comparison of checksums of input files and output filenames before a recompile is attempted.

Or utilising the cache of a docker image build on bootstrap, which could cache the whole git pull / install process unless git repo changes ( https://stackoverflow.com/a/39278224 )

4 Likes

Could checksums be used here @sam or @falco?

1 Like

Yes, I discussed this with @eviltrout in the past, we would need a post build step, an asset server and to customize asset precompile

I estimate it would take less than 2 days to setup

6 Likes

In the event of a failure in the start method, how can you roll back to the previous container?

I must admit this has been a bit of a headache for me as well.

I just went https (smooth process, great) and decided to change the dated “out” email at the same time (basically gmail -> gandi). 4h later and umpteen rebuilds later I’m just reverting to the dated one (at least for today). 10 minutes per rebuild is not fun. Not to mention the time figuring out how to cleanup when I run out of space. I’ll be faster next time :wink: .

Do all the settings in app.yml really require a rebuild? One would imagine changing email settings would just change something in one place.

If you change something in env: or expose: then you can just do ./launcher restart app

3 Likes

Ok, I’m sure there is somewhere where I can figure out what belongs to which area?

I really try not to ask until I’ve investigated (read: suffered) but this time I apparently/probably should have…

Edit: Ok, looking at app.yml gave me a clear answer, never mind :smiley: .

Friday and 7h of configuring is not good for anyone. Luckily I’m not 15 anymore, 24h configuring…

1 Like

Rebuilds are about 25% faster now as we finally skip needlessly recompiling locales.

Not sure what else we could do to make it faster.

5 Likes