Launcher Rebuild failing at “'bundle exec rake themes:update assets:precompile' ”

Divyaanand_Sinha · December 15, 2020, 3:33pm

I trying to build a new discourse docker image however it seems like its failing at the migrate stage.

    I, [2020-12-15T15:02:28.688540 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'

    I, [2020-12-15T15:04:19.530701 #1] INFO -- :

    I, [2020-12-15T15:04:19.530966 #1] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake themes:update assets:precompile'

    Purging temp files

    Bundling assets

    I, [2020-12-15T15:04:36.448680 #374] INFO -- : Writing /var/www/discourse/public/assets/vendor-bdb3fefe8aefde7bf6aef331a6edd24482502b370ed9f6f299836c72ee6a843c.js

    I, [2020-12-15T15:04:43.818563 #374] INFO -- : Writing /var/www/discourse/public/assets/admin-073db5e5695d9f3834b9ad41d66ef30a02b9ca85fc9c95f272257ea9c7a67f21.js

    I, [2020-12-15T15:04:43.822864 #374] INFO -- : Writing /var/www/discourse/public/assets/browser-detect-115ab5953de1b5bb122bfb26b757f5391dd8d1d2aef2b81baf7b59aee99d9f34.js
.
.
.
.
.
.
.
.
.
.
.

    35:M 15 Dec 2020 15:06:27.009 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:27.010 * Background saving started by pid 1066

    1066:C 15 Dec 2020 15:06:27.010 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:27.110 # Background saving error

    35:M 15 Dec 2020 15:06:33.020 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:33.020 * Background saving started by pid 1067

    1067:C 15 Dec 2020 15:06:33.020 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:33.121 # Background saving error

    35:M 15 Dec 2020 15:06:39.030 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:39.031 * Background saving started by pid 1068

    1068:C 15 Dec 2020 15:06:39.031 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:39.131 # Background saving error

    gzip -f -c -9 /var/www/discourse/public/assets/application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js > /var/www/discourse/public/assets/application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js.gz

    brotli -f --quality=11 /var/www/discourse/public/assets/application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js --output=/var/www/discourse/public/assets/application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js.br

    35:M 15 Dec 2020 15:06:45.041 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:45.042 * Background saving started by pid 1072

    1072:C 15 Dec 2020 15:06:45.042 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:45.142 # Background saving error

    Done compressing application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js : 25.87 secs

    Done compressing all JS files : 83.62 secs

    Start compiling CSS: 2020-12-15 15:06:49 UTC

    Compiling css for default 2020-12-15 15:06:49 UTC

    precompile target: desktop Dark

    35:M 15 Dec 2020 15:06:51.051 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:51.051 * Background saving started by pid 1074

    1074:C 15 Dec 2020 15:06:51.051 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:51.151 # Background saving error

    rake aborted!

    Redis::CommandError: ERR Error running script (call to f_45fe64238f6598e6c505af0aad74b4e178e891f4): @user_script:14: @user_script: 14: -MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis/client.rb:132:in `call'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.2.0/lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:2585:in `block in _eval'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `block in synchronize'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:69:in `synchronize'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:2584:in `_eval'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/redis-4.2.5/lib/redis.rb:2636:in `evalsha'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/backends/redis.rb:425:in `cached_eval'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/backends/redis.rb:144:in `publish'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus.rb:382:in `publish'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/distributed_cache.rb:80:in `publish'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/distributed_cache.rb:90:in `set'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/message_bus-3.3.4/lib/message_bus/distributed_cache.rb:135:in `[]='

    /var/www/discourse/app/models/theme.rb:142:in `get_set_cache'

    /var/www/discourse/app/models/theme.rb:158:in `components_for'

    /var/www/discourse/app/models/theme.rb:185:in `block in transform_ids'

    /var/www/discourse/app/models/theme.rb:142:in `get_set_cache'

    /var/www/discourse/app/models/theme.rb:177:in `transform_ids'

    /var/www/discourse/app/models/theme.rb:371:in `list_baked_fields'

    /var/www/discourse/app/models/theme.rb:367:in `resolve_baked_field'

    /var/www/discourse/lib/stylesheet/manager.rb:423:in `color_scheme_digest'

    /var/www/discourse/lib/stylesheet/manager.rb:361:in `digest'

    /var/www/discourse/lib/stylesheet/manager.rb:339:in `stylesheet_filename'

    /var/www/discourse/lib/stylesheet/manager.rb:300:in `source_map_filename'

    /var/www/discourse/lib/stylesheet/manager.rb:251:in `compile'

    /var/www/discourse/lib/stylesheet/manager.rb:169:in `block (2 levels) in precompile_css'

    /var/www/discourse/lib/stylesheet/manager.rb:162:in `each'

    /var/www/discourse/lib/stylesheet/manager.rb:162:in `block in precompile_css'

    /var/www/discourse/lib/stylesheet/manager.rb:161:in `each'

    /var/www/discourse/lib/stylesheet/manager.rb:161:in `precompile_css'

    /var/www/discourse/lib/tasks/assets.rake:54:in `block (2 levels) in <main>'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:76:in `with_connection'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:86:in `each_connection'

    /var/www/discourse/lib/tasks/assets.rake:47:in `block in <main>'

    /var/www/discourse/lib/tasks/assets.rake:331:in `block in <main>'

    /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rake-13.0.1/exe/rake:27:in `<top (required)>'

    /usr/local/bin/bundle:23:in `load'

    /usr/local/bin/bundle:23:in `<main>'

    Tasks: TOP => assets:precompile:css

    (See full trace by running task with --trace)

    I, [2020-12-15T15:06:53.012968 #1] INFO -- : Downloading MaxMindDB...

    Compressing Javascript and Generating Source Maps

    I, [2020-12-15T15:06:53.013385 #1] INFO -- : Terminating async processes

    I, [2020-12-15T15:06:53.013424 #1] INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 35

    35:signal-handler (1608044813) Received SIGTERM scheduling shutdown...

    35:M 15 Dec 2020 15:06:53.054 # User requested shutdown...

    35:M 15 Dec 2020 15:06:53.054 * Saving the final RDB snapshot before exiting.

    35:M 15 Dec 2020 15:06:53.054 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:53.054 # Error trying to save the DB, can't exit.

    35:M 15 Dec 2020 15:06:53.054 # SIGTERM received but errors trying to shut down the server, check the logs for more information

    35:M 15 Dec 2020 15:06:57.060 * 10 changes in 300 seconds. Saving...

    35:M 15 Dec 2020 15:06:57.060 * Background saving started by pid 1078

    1078:C 15 Dec 2020 15:06:57.061 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:57.161 # Background saving error

    I, [2020-12-15T15:07:03.014092 #1] INFO -- : exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid:35 did not terminate cleanly, forcing termination!

    FAILED

    --------------------

    Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake themes:update assets:precompile' failed with return #<Process::Status: pid 372 exit 1>

    Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'

    exec failed with the params {"cd"=>"$home", "hook"=>"assets_precompile", "cmd"=>["su discourse -c 'bundle exec rake themes:update assets:precompile'"]}

    be9c92f00a4814e5e9ebcdb06f25232b03f7522390b97c157085c2f7ef7f3387

    ** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.

    In  short I am seeing this error

    68:C 15 Dec 2020 15:06:39.031 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:39.131 # Background saving error

    35:M 15 Dec 2020 15:06:51.051 * Background saving started by pid 1074

    1074:C 15 Dec 2020 15:06:51.051 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

    35:M 15 Dec 2020 15:06:51.151 # Background saving error

    rake aborted!

    Redis::CommandError: ERR Error running script (call to f_45fe64238f6598e6c505af0aad74b4e178e891f4): @user_script:14: @user_script: 14: -MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error.

my app.yaml file looks like this

templates:
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/redis.template.yml"
env:
  LANG: en_US.UTF-8
  UNICORN_WORKERS: 8
  DISCOURSE_DB_USERNAME: discourse
  DISCOURSE_DB_PASSWORD: "xxxxx"
  DISCOURSE_DB_HOST: "xxxxx"
  DISCOURSE_REDIS_HOST: 127.0.0.1
  DISCOURSE_REDIS_PORT: "6379"
  DISCOURSE_DB_NAME:  discourse_prod_2
  DISCOURSE_DEVELOPER_EMAILS: 'xxxxxxx'
  DISCOURSE_HOSTNAME: 'localhost'
  DISCOURSE_DB_PORT: "5301"  
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - mkdir -p plugins
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-solved.git
          - git clone https://github.com/discourse/discourse-voting.git
          - git clone https://github.com/discourse/discourse-slack-official.git
          - git clone https://github.com/discourse/discourse-assign.git
run:
  - exec:
      cd: /var/www/discourse
      cmd:
        - sed -i 's/GlobalSetting.serve_static_assets/true/' config/environments/production.rb

I have hidden the password,hostname and email for security purposes. For redis I am running a separate docker container. I am able to connect to it on localhost and the given port.

Please let me know if I am missing something.

Falco · December 15, 2020, 3:57pm

If you are running in a separate container you should remove the line

- "templates/redis.template.yml"

from your app.yml.

Your external Redis is either misconfigured or under heavy I/O wait pressure and it’s causing this problem. You will need to fix that.

Divyaanand_Sinha · December 15, 2020, 4:45pm

@Falco Even when i use the default redis by removing the redis host and port from the file, I get the same error

sets',url='/assets/application-42d4413fabab7a4b12e0c25fb01f736af560aa283089071a8493f37c9b042a4a.js.map'"

35:M 15 Dec 2020 16:43:25.078 * 10 changes in 300 seconds. Saving...

35:M 15 Dec 2020 16:43:25.079 * Background saving started by pid 1064

1064:C 15 Dec 2020 16:43:25.079 # Failed opening the RDB file dump.rdb (in server root dir /pups) for saving: Permission denied

35:M 15 Dec 2020 16:43:25.179 # Background saving error 

Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake themes:update assets:precompile' failed with return #<Process::Status: pid 371 exit 1>

Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'

exec failed with the params {"cd"=>"$home", "hook"=>"assets_precompile", "cmd"=>["su discourse -c 'bundle exec rake themes:update assets:precompile'"]}

d514879d2a044425c1e7d833dafb8196ae0a3026aa75119512349188531b2a5d

Falco · December 15, 2020, 6:01pm

You have broken file permissions. Redis can’t open it’s own dump file.

Divyaanand_Sinha · December 15, 2020, 6:05pm

@Falco but this happens inside the discourse docker while it is building the image. How can I change it?

Falco · December 15, 2020, 7:47pm

Have you added back the redis template to the app.yml ?

That file lives in /var/discourse/shared/standalone/redis_data/dump.rdb with 644 access and owned by UID 106 \ GID 110.

Sounds like you manually changed permissions on this file and potentially others.

Divyaanand_Sinha · December 15, 2020, 8:44pm

I haven’t even built the image yet, so I still don’t understand how it can get changed. I have the redis.template.yml added to my file.

templates:
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
  - "templates/redis.template.yml"
env:
  LANG: en_US.UTF-8
  UNICORN_WORKERS: 8
  DISCOURSE_DB_USERNAME: discourse
  DISCOURSE_DB_PASSWORD: "xxxxx"
  DISCOURSE_DB_HOST: "xxxxx"
  DISCOURSE_DB_NAME:  discourse_prod_2
  DISCOURSE_DEVELOPER_EMAILS: 'xxxxxxx'
  DISCOURSE_HOSTNAME: 'localhost'
  DISCOURSE_DB_PORT: "5301"  
hooks:
  after_code:
    - exec:
        cd: $home/plugins
        cmd:
          - mkdir -p plugins
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-solved.git
          - git clone https://github.com/discourse/discourse-voting.git
          - git clone https://github.com/discourse/discourse-slack-official.git
          - git clone https://github.com/discourse/discourse-assign.git
run:
  - exec:
      cd: /var/www/discourse
      cmd:
        - sed -i 's/GlobalSetting.serve_static_assets/true/' config/environments/production.rb

However, I am running ./launcher rebuild as a root user. Will that have any affect on this?

Falco · December 15, 2020, 8:58pm

You will now have to manually fix the permissions of the affected file.

Divyaanand_Sinha · December 16, 2020, 11:32am

Issue resolved after building using separate containers for postgres and redis and removing the "templates/redis.template.yml" line from the app.yml

system · January 15, 2021, 11:32am

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.

Topic		Replies	Views
Failed to rebuild: no such file /usr/local/etc/ImageMagick-6/policy.xml Installation	4	1019	November 23, 2020
Error when "./discourse rebuild app" due to theme error Installation	6	1649	December 1, 2021
Build failed for specific commit hash Installation	9	618	April 12, 2023
Updating failed, unable to rebuild Installation	4	1063	July 9, 2021
HELP: Discourse Update broke app -- fails to rebuild Installation	15	1817	January 10, 2023

Launcher Rebuild failing at “'bundle exec rake themes:update assets:precompile' ”

Related topics