Rebuild app has stuck

Hi,

I was prompted to refresh the launcher and run ./launcher rebuild app for the latest version upgrade.

It’s now got stuck here:

155:M 02 Aug 08:34:46.052 * 10 changes in 300 seconds. Saving...
155:M 02 Aug 08:34:46.054 * Background saving started by pid 2373
2373:C 02 Aug 08:34:46.439 * DB saved on disk
2373:C 02 Aug 08:34:46.439 * RDB: 36 MB of memory used by copy-on-write
155:M 02 Aug 08:34:46.456 * Background saving terminated with success

It’s been 30 minutes, and nothing has changed. The machine docker is running on (an Amazon instance) is running at 99.9% idle in top.

The last part of ./launcher logs app is:

229:M 02 Aug 08:18:55.087 * 10 changes in 300 seconds. Saving...
229:M 02 Aug 08:18:55.088 * Background saving started by pid 22525
22525:C 02 Aug 08:18:55.371 * DB saved on disk
22525:C 02 Aug 08:18:55.372 * RDB: 31 MB of memory used by copy-on-write
229:M 02 Aug 08:18:55.389 * Background saving terminated with success
229:M 02 Aug 08:23:56.044 * 10 changes in 300 seconds. Saving...
229:M 02 Aug 08:23:56.044 * Background saving started by pid 22911
22911:C 02 Aug 08:23:56.367 * DB saved on disk
22911:C 02 Aug 08:23:56.368 * RDB: 31 MB of memory used by copy-on-write
229:M 02 Aug 08:23:56.445 * Background saving terminated with success
229:M 02 Aug 08:28:57.006 * 10 changes in 300 seconds. Saving...
229:M 02 Aug 08:28:57.007 * Background saving started by pid 23268
23268:C 02 Aug 08:28:57.528 * DB saved on disk
23268:C 02 Aug 08:28:57.529 * RDB: 25 MB of memory used by copy-on-write
229:M 02 Aug 08:28:57.604 * Background saving terminated with success
Shutting Down
run-parts: executing /etc/runit/3.d/01-nginx
ok: down: nginx: 0s, normally up
run-parts: executing /etc/runit/3.d/02-unicorn
exiting
ok: down: unicorn: 1s, normally up
run-parts: executing /etc/runit/3.d/10-redis
229:signal-handler (1501662563) Received SIGTERM scheduling shutdown...
229:M 02 Aug 08:29:23.364 # User requested shutdown...
229:M 02 Aug 08:29:23.364 * Saving the final RDB snapshot before exiting.
229:M 02 Aug 08:29:23.684 * DB saved on disk
229:M 02 Aug 08:29:23.684 # Redis is now ready to exit, bye bye...
ok: down: redis: 0s, normally up
run-parts: executing /etc/runit/3.d/99-postgres
2017-08-02 08:29:23.718 UTC [225] LOG:  received smart shutdown request
2017-08-02 08:29:23.721 UTC [255] LOG:  autovacuum launcher shutting down
timeout: run: postgres: (pid 225) 2929742s, want down, got TERM
run-parts: /etc/runit/3.d/99-postgres exited with return code 1
When using programs that use GNU Parallel to process data for publication please cite:

  O. Tange (2011): GNU Parallel - The Command-Line Power Tool,
  ;login: The USENIX Magazine, February 2011:42-47.

This helps funding further development; and it won't cost you a cent.
Or you can get GNU Parallel without this requirement by paying 10000 EUR.

To silence this citation notice run 'parallel --bibtex' once or use '--no-notice'.

ok: down: nginx: 9s, normally up
ok: down: cron: 0s, normally up
ok: down: redis: 8s, normally up

Help please!

I got bored of waiting and rebooted the host. ^C wouldn’t stop the launcher process.

Once back up, I relaunched the rebuild app, this time it worked. :man_shrugging:

6 Likes