Faster rebuilds?

Installation
1

Hey guys, I wonder if there is a way to run ./launcher rebuild app a bit faster. Right now it takes 6-9 minutes to do the whole stuff and i’d like to skip some stuff that looks a bit useless:

These three commands takes a lot of time (about 20-30-60 sec each)

  • cd /var/www/discourse && chown -R discourse /var/www/discourse
  • cd /var/www/discourse && su discourse -c 'bundle install --deployment --verbose --without test --without development'
  • cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'

Compiling assets for each locale is also take about a minute, uglifying js is also takes some time.

Aaaaanyhooow, I was hoping there is a way of semi-rebuilding (?!) the app a bit faster than that. I understand the need of running all of these when you’re installing an update or a new plugin, but sometimes I only need a very quick „restart” for more… special cases:

  • update of a SSL cert
  • the other day i tried to make it play nice with other sites (following this article) and i had to rebuild couple of times.

PS: time ./launcher rebuild app --skip-prereqs is giving only negligible improvements:

real    5m27.461s
user    0m1.027s
sys     0m0.652s

Thanks!

5 Likes
2

If rebuild time is a huge concern to you I would strongly recommend running a dedicated data container

That way you can bootstrap while your existing site is up and running, meaning you only deal with a few seconds of outage when upgrading via launcher

Bundle install is usually very fast when we just ship an image, I plan on having more regular updates to help out here

Assets precompile is the slowest part, we are cosidering pre-bundling assets or possibly adding a caching server or even more aggressive image updates, but nothing is a trivially easy solution

8 Likes
3

Is there any migration documentation about how to change to this configuration? I thought I saw something at one point but can’t find it in howto now.

4

This one. Noted some steps is out-dated:

  1. Use enter to get into a container.
  2. DISCOURSE_DB_HOST: data and DISCOURSE_REDIS_HOST: data
  • link to a container is prefered than IP now.
  • define links block as follows:
links:
  - link:
      name: data
      alias: data
3 Likes
5

if we don’t stop forum during rebuild (make it read-only if necessary), it will be better?

2 Likes
6

Can not be done in a single container setup without major headaches… part of the bootstrap process is spawning the db, you can’t have two instances of a database running at the same time.

3 Likes
7

How does this work when using a separate data container? Does it run migrations on the data that the old site is still using, potentially breaking the site until the web container is upgraded if the migrations are incompatible?

5 Likes
8

Yes, but… We are very careful to keep migrations compatible data wise, even have a pretty elaborate scheme that delays dropping columns when we move them to new tables

5 Likes
9

Should this process have copied my data & configuration over? I ran through the steps but got a clean/“fresh” site, so rolled back to the single-container version.

I did a restore from a backup made just before the process, and it seemed to restore everything. So probably this just needs to be made clear. :slightly_smiling:

1 Like
10

I flagged it to ask for wikifing the topic… Let’s amend it :wink:

PS: Don’t know whether it’s a right way to use flag…

11

Yes, that’s the right way to use the notify moderator feature :wink:

2 Likes
12

Can you say a bit more about how this process works in your mind?

After switching to the multi-container scenario, I just did a ./launcher rebuild web_only but my downtime was the same. Clearly I need more coffee still this morning.

13 
./launcher bootstrap web_only

Start outage

./launcher destroy web_only
./launcher start web_only

End outage

9 Likes
14

Is there any way to skip the whole uglify and compressing of assets & locales during bootstrap if the assets already exist?

Maybe it would make sense to have a dedicated assets volume and/or a comparison of checksums of input files and output filenames before a recompile is attempted.

Or utilising the cache of a docker image build on bootstrap, which could cache the whole git pull / install process unless git repo changes ( https://stackoverflow.com/a/39278224 )

4 Likes
15

Could checksums be used here @sam or @falco?

2 Likes
16

Yes, I discussed this with @eviltrout in the past, we would need a post build step, an asset server and to customize asset precompile

I estimate it would take less than 2 days to setup

7 Likes
17

In the event of a failure in the start method, how can you roll back to the previous container?

18

I must admit this has been a bit of a headache for me as well.

I just went https (smooth process, great) and decided to change the dated “out” email at the same time (basically gmail -> gandi). 4h later and umpteen rebuilds later I’m just reverting to the dated one (at least for today). 10 minutes per rebuild is not fun. Not to mention the time figuring out how to cleanup when I run out of space. I’ll be faster next time :wink: .

Do all the settings in app.yml really require a rebuild? One would imagine changing email settings would just change something in one place.

19

If you change something in env: or expose: then you can just do ./launcher restart app

4 Likes
20

Ok, I’m sure there is somewhere where I can figure out what belongs to which area?

I really try not to ask until I’ve investigated (read: suffered) but this time I apparently/probably should have…

Edit: Ok, looking at app.yml gave me a clear answer, never mind :smiley: .

Friday and 7h of configuring is not good for anyone. Luckily I’m not 15 anymore, 24h configuring…

2 Likes