I can't start discourse on CentOS

[502647092 (MiaoWoo)]

System Info:

LSB Version:    :core-4.1-amd64:core-4.1-noarch:cxx-4.1-amd64:cxx-4.1-noarch:desktop-4.1-amd64:desktop-4.1-noarch:languages-4.1-amd64:languages-4.1-noarch:printing-4.1-amd64:printing-4.1-noarch
Distributor ID: CentOS
Description:    CentOS Linux release 7.3.1611 (Core)
Release:        7.3.1611
Codename:       Core

Docker Info

[root@sz discourse]# docker info
Containers: 1
 Running: 1
 Paused: 0
 Stopped: 0
Images: 7
Server Version: 1.12.6
Storage Driver: overlay
 Backing Filesystem: xfs
Logging Driver: journald
Cgroup Driver: systemd
Plugins:
 Volume: local
 Network: null bridge host overlay
Swarm: inactive
Runtimes: runc docker-runc
Default Runtime: docker-runc
Security Options: seccomp
Kernel Version: 3.10.0-514.10.2.el7.x86_64
Operating System: CentOS Linux 7 (Core)
OSType: linux
Architecture: x86_64
Number of Docker Hooks: 2
CPUs: 2
Total Memory: 15.38 GiB
Name: sz
ID: SR7C:JIH6:OGE4:55KT:CJBY:CUCL:YVQL:HIOH:ZZLB:UVBA:JCNE:53X2
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8
Registries: docker.io (secure)

ERR LOG

I, [2017-05-12T01:35:20.621105 #14]  INFO -- : > gem sources --add https://gems.ruby-china.org/ --remove https://rubygems.org/
I, [2017-05-12T01:35:24.277753 #14]  INFO -- : https://gems.ruby-china.org/ added to sources
https://rubygems.org/ removed from sources

I, [2017-05-12T01:35:24.278166 #14]  INFO -- : > cd /var/www/discourse && gem update bundler
I, [2017-05-12T01:35:27.143341 #14]  INFO -- : Updating installed gems
Nothing to update

I, [2017-05-12T01:35:27.143465 #14]  INFO -- : > cd /var/www/discourse && chown -R discourse /var/www/discourse
chown: cannot access '/var/www/discourse/.mention-bot': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/components/customize-link.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/components/disable-custom-stylesheets.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/controllers/admin-customize-css-html-show.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/controllers/modals/change-site-customization-details.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/controllers/modals/delete-site-customization-details.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/models/site-customization.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/routes/admin-customize-css-html-show.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/routes/admin-customize-css-html.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/templates/components/customize-link.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/templates/customize-css-html-index.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/templates/customize-css-html-show.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/templates/customize-css-html.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/admin/templates/modal/site-customization-change.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/components/auto-close-form.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/components/check-mark.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/components/json-file-uploader.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/components/topic-closing.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/controllers/edit-topic-auto-close.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/controllers/upload-customization.js.es6': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/templates/components/auto-close-form.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/templates/components/check-mark.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/templates/components/json-file-uploader.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/templates/modal/edit-topic-auto-close.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/javascripts/discourse/templates/modal/upload-customization.hbs': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/badges.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/banner.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/buttons.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/date-picker.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/keyboard_shortcuts.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/common/components/navs.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/embed.css.scss': No such file or directory
chown: cannot access '/var/www/discourse/app/assets/stylesheets/vendor/sweetalert.css': No such file or directory
chown: cannot access '/var/www/discourse/app/controllers/admin/site_customizations_controller.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/controllers/site_customizations_controller.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/jobs/onceoff/grand_first_reply_by_email.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/jobs/regular/create_thumbnails.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/jobs/scheduled/enqueue_mailing_list_emails.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/models/site_customization.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/models/warning.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/serializers/site_customization_serializer.rb': No such file or directory
chown: cannot access '/var/www/discourse/app/views/user_notifications/mailing_list.html.erb': No such file or directory
chown: cannot access '/var/www/discourse/app/views/user_notifications/mailing_list.text.erb': No such file or directory
chown: cannot access '/var/www/discourse/app/views/users/account_created.html.erb': No such file or directory
chown: cannot access '/var/www/discourse/config/initializers/100-sprockets.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/email/html_cleaner.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/freedom_patches/resolve.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/sass/discourse_safe_sass_importer.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/sass/discourse_sass_compiler.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/sass/discourse_sass_importer.rb': No such file or directory
chown: cannot access '/var/www/discourse/lib/sass/discourse_stylesheets.rb': No such file or directory
chown: cannot access '/var/www/discourse/public/javascripts/jquery.magnific-popup-min.js': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/discourse_sass_compiler_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/discourse_stylesheets_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/step_updater_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/wizard_builder_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/wizard_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/components/wizard_step_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/controllers/admin/site_customizations_controller_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/controllers/site_customizations_controller_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/jobs/close_topic_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/jobs/enqueue_mailing_list_emails_spec.rb': No such file or directory
chown: cannot access '/var/www/discourse/spec/models/site_customization_spec.rb': No such file or directory
I, [2017-05-12T01:35:29.491952 #14]  INFO -- : 
I, [2017-05-12T01:35:29.492936 #14]  INFO -- : Terminating async processes
I, [2017-05-12T01:35:29.493008 #14]  INFO -- : Sending INT to HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main pid: 43
I, [2017-05-12T01:35:29.493062 #14]  INFO -- : Sending TERM to exec chpst -u redis -U redis /usr/bin/redis-server /etc/redis/redis.conf pid: 156
2017-05-12 01:35:29 UTC [43-2] LOG:  received fast shutdown request
2017-05-12 01:35:29 UTC [43-3] LOG:  aborting any active transactions
2017-05-12 01:35:29 UTC [50-2] LOG:  autovacuum launcher shutting down
156:signal-handler (1494552929) Received SIGTERM scheduling shutdown...
2017-05-12 01:35:29 UTC [47-1] LOG:  shutting down
156:M 12 May 01:35:29.550 # User requested shutdown...
156:M 12 May 01:35:29.550 * Saving the final RDB snapshot before exiting.
156:M 12 May 01:35:30.680 * DB saved on disk
156:M 12 May 01:35:30.680 # Redis is now ready to exit, bye bye...
2017-05-12 01:35:30 UTC [47-2] LOG:  database system is shut down


FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && chown -R discourse /var/www/discourse failed with return #<Process::Status: pid 292 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"web", "cmd"=>["gem update bundler", "chown -R discourse $home"]}
8e2d407b64c3f61aaf9597d79dd3e97453f512a30f2d8779245bfa6432f8f837
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

[kraml (Kai Liu)]

This problem has nothing to do with China…

Check this post.

Bootstrap fails on CentOS 7

Most probably it’s because the backing xfs filesystem was not created with ftype=1 option. This is very common if you let the OS installer format the / fs and you just have /var/lib/docker on /. On newer docker verision (1.13?) with docker info you can see support d_type = false and that indicates your xfs was created with ftype=0 instead of the required ftype=1. This will cause weird issues if you use overlayfs on top of it. See below issues for detail: https://github.com/docker/docker/issue…

[502647092 (MiaoWoo)]

Thank you
I remount new partition
/dev/sda3 on /var/lib/docker type xfs (rw,relatime,attr2,inode64,noquota)

then run ./launcher rebuild app

have an another error

I, [2017-05-12T02:30:44.433734 #14]  INFO -- : > /root/upgrade_postgres
I, [2017-05-12T02:30:44.438227 #14]  INFO -- :
I, [2017-05-12T02:30:44.438392 #14]  INFO -- : > rm /root/upgrade_postgres
I, [2017-05-12T02:30:44.441431 #14]  INFO -- :
I, [2017-05-12T02:30:44.442042 #14]  INFO -- : Replacing data_directory = '/var/lib/postgresql/9.5/main' with data_directory = '/shared/postgres_data' in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.442863 #14]  INFO -- : Replacing (?-mix:#?listen_addresses *=.*) with listen_addresses = '*' in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.443175 #14]  INFO -- : Replacing (?-mix:#?synchronous_commit *=.*) with synchronous_commit = $db_synchronous_commit in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.443644 #14]  INFO -- : Replacing (?-mix:#?shared_buffers *=.*) with shared_buffers = $db_shared_buffers in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.443991 #14]  INFO -- : Replacing (?-mix:#?work_mem *=.*) with work_mem = $db_work_mem in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.444440 #14]  INFO -- : Replacing (?-mix:#?default_text_search_config *=.*) with default_text_search_config = '$db_default_text_search_config' in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.445620 #14]  INFO -- : > install -d -m 0755 -o postgres -g postgres /shared/postgres_backup
I, [2017-05-12T02:30:44.448525 #14]  INFO -- :
I, [2017-05-12T02:30:44.449546 #14]  INFO -- : Replacing (?-mix:#?max_wal_senders *=.*) with max_wal_senders = $db_max_wal_senders in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.499680 #14]  INFO -- : Replacing (?-mix:#?wal_level *=.*) with wal_level = $db_wal_level in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.500683 #14]  INFO -- : Replacing (?-mix:#?checkpoint_segments *=.*) with checkpoint_segments = $db_checkpoint_segments in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.501851 #14]  INFO -- : Replacing (?-mix:#?logging_collector *=.*) with logging_collector = $db_logging_collector in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.502813 #14]  INFO -- : Replacing (?-mix:#?log_min_duration_statement *=.*) with log_min_duration_statement = $db_log_min_duration_statement in /etc/postgresql/9.5/main/postgresql.conf
I, [2017-05-12T02:30:44.503941 #14]  INFO -- : Replacing (?-mix:^#local +replication +postgres +peer$) with local replication postgres  peer in /etc/postgresql/9.5/main/pg_hba.conf
I, [2017-05-12T02:30:44.504781 #14]  INFO -- : Replacing (?-mix:^host.*all.*all.*127.*$) with host all all 0.0.0.0/0 md5 in /etc/postgresql/9.5/main/pg_hba.conf
I, [2017-05-12T02:30:44.505135 #14]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/9.5/bin/postmaster -D /etc/postgresql/9.5/main
I, [2017-05-12T02:30:44.508061 #14]  INFO -- : > sleep 5
2017-05-12 02:30:44 UTC [43-1] LOG:  could not open configuration file "/etc/postgresql/9.5/main/pg_hba.conf": Permission denied
2017-05-12 02:30:44 UTC [43-2] FATAL:  could not load pg_hba.conf
I, [2017-05-12T02:30:49.514593 #14]  INFO -- :
I, [2017-05-12T02:30:49.514885 #14]  INFO -- : > su postgres -c 'createdb discourse' || true
createdb: could not connect to database template1: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2017-05-12T02:30:49.584067 #14]  INFO -- :
I, [2017-05-12T02:30:49.584354 #14]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2017-05-12T02:30:49.645721 #14]  INFO -- :
I, [2017-05-12T02:30:49.646604 #14]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse todiscourse;"' || true
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2017-05-12T02:30:49.700583 #14]  INFO -- :
I, [2017-05-12T02:30:49.701330 #14]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
psql: could not connect to server: No such file or directory
        Is the server running locally and accepting
        connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2017-05-12T02:30:49.756242 #14]  INFO -- :
I, [2017-05-12T02:30:49.756928 #14]  INFO -- : Terminating async processes


FAILED
--------------------
Pups::ExecError: su postgres -c 'psql discourse -c "alter schema public owner to discourse;"' failed with return #<Process::Status: pid 69 exit 2>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "su postgres -c 'psql $db_name -c \"alter schema public owner to $db_user;\"'"
6bd432f7a7decf0cce67f34eb08e132209408d06c08e235266aa7682b947a748
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

[kraml (Kai Liu)]

Just check to ensure, you did check with xfs_info command that the new /var/lib/docker fs is created correctly with d_type support, right?

The error log looks related to permission problem, which seems nobody have reported on meta. As it is related to file operation I still suspect it is the same problem.

[502647092 (MiaoWoo)]

i run this command

mv /var/lib/docker /var/lib/docker-bak
mkfs.xfs -n ftype=1 /dev/sda3 -f
mount -t xfs /dev/sda3 /var/lib/docker
cp /var/lib/docker-bak/* /var/lib/docker -R
./launcher bootstrap app

how to check /var/lib/docker fs is created correctly with d_type support

[kraml (Kai Liu)]

If you have the -n ftype=1 parameter for mkfs.xfs command then it is fine.

You don’t need to (and probably shouldn’t) move the old files under /var/lib/docker back to the new fs. Just start docker and it will create necessary files/dirs. If you have other containers need to be restored, that should be done with docker export/load, not raw file operation.

BTW, you did stop docker before these commands and start it again after, right?

[502647092 (MiaoWoo)]

i run service docker restart after those command
Let me have a try rm /var/lib/docker and rerun it

[502647092 (MiaoWoo)]

the installation able to continue running
Think you
If there are any more problem, I’ll ask you