The discussion revolves around errors encountered while processing PostgreSQL 9.5 when bootstrapping a Discourse instance. The original poster, CamilleRoux, is using a PostgreSQL 9.3 template due to issues with the LANG value, but is unable to bootstrap.
Falco suggests updating the templates to use the latest PostgreSQL version, but CamilleRoux encounters errors when trying to upgrade. Falco then proposes a workaround involving editing the app.yml file, taking a backup, and restoring the instance.
sphyden is migrating a Discourse instance from CentOS 6 to CentOS 7 and encounters similar issues. Falco suggests using the rails console to logout all users simultaneously, but sphyden still encounters login loop issues.
After trying various solutions, sphyden is able to get a functioning version of Discourse working by copying the entire /var/docker directory over, but is still unable to update Discourse due to PostgreSQL upgrade errors. sam suggests following the manual backup/restore steps to resolve the issue.
Here is what I get when I execute ./launcher rebuild app
Ensuring launcher is up to date
Fetching origin
Launcher is up-to-date
Stopping old container
+ /usr/bin/docker stop -t 10 app
app
cd /pups && git pull && /pups/bin/pups --stdin
Already up-to-date.
I, [2016-11-08T16:42:43.632690 #15] INFO -- : Loading --stdin
I, [2016-11-08T16:42:43.641225 #15] INFO -- : > apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
LANGUAGE = (unset),
LC_ALL = (unset),
LANG = "fr_FR.UTF-8"
are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").
E: Sub-process /usr/bin/dpkg returned an error code (1)
I, [2016-11-08T16:42:45.908508 #15] INFO -- : Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be REMOVED:
postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 26.1 MB disk space will be freed.
(Reading database ... 34341 files and directories currently installed.)
Removing postgresql-contrib-9.5 (9.5.5-1.pgdg16.04+1) ...
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/9.5/man/man1/postmaster.1.gz because link group postmaster.1.gz is broken
update-alternatives: error: error creating symbolic link '/usr/share/man/man1/postmaster.1.gz.dpkg-tmp': No such file or directory
dpkg: error processing package postgresql-contrib-9.5 (--remove):
subprocess installed pre-removal script returned error exit status 2
dpkg: postgresql-9.5: dependency problems, but removing anyway as you requested:
postgresql-contrib-9.5 depends on postgresql-9.5 (= 9.5.5-1.pgdg16.04+1).
Removing postgresql-9.5 (9.5.5-1.pgdg16.04+1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of stop.
Removing postgresql-client-9.5 (9.5.5-1.pgdg16.04+1) ...
Processing triggers for postgresql-common (177.pgdg16.04+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Errors were encountered while processing:
postgresql-contrib-9.5
FAILED
--------------------
Pups::ExecError: apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 failed with return #<Process::Status: pid 17 exit 100>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5"
4a144bfb01e96e35da31aa0dae839f3e344cc9e0b667e20e6ae355f36251c673
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
I choose the 9.3 because there was some bug with the LANG value.
Now if I’m using templates/postgres.template.yml, I get the following message in logs:
Failure, exiting
-------------------------------------------------------------------------------------
UPGRADE OF POSTGRES FAILED
You are going to need to export your data and import into a clean instance:
In containers/app.yml: Change "templates/postgres.template.yml" TO "templates/postgres.9.3.template.yml"
Run ./launcher rebuild app again
When your instance is running:
Run ./launcher enter app
Run cd /shared/postgres_backup && sudo -u postgres pg_dump discourse > backup.db
Undo the postgres template in your container config
Run: ./launcher stop app
Run: sudo mv /var/discourse/shared/standalone/postgres_data /var/discourse/shared/standalone/postgres_data_old
Run: ./launcher rebuild app
Run: ./launcher enter app
Run: cd /shared/postgres_backup
Run: sv stop unicorn
Run: sudo -iu postgres dropdb discourse
Run: sudo -iu postgres createdb discourse
Run: sudo -iu postgres psql discourse < backup.db
Run: exit
Run: ./launcher rebuild app
FAILED
--------------------
Pups::ExecError: /root/upgrade_postgres failed with return #<Process::Status: pid 39 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "/root/upgrade_postgres"
ebd56652d59e7b6edf01ea0d356cf73719b37fc918a87d6f6db292bf18e6aa68
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
But, because rebuild is failing with the template postgres.9.3.template.yml, I don’t know what to do…
Then edit app.yml, removing postgres version from it. After the editing the first lines should look like this:
## this is the all-in-one, standalone Discourse Docker container template
##
## After making changes to this file, you MUST rebuild
## /var/discourse/launcher rebuild app
##
## BE *VERY* CAREFUL WHEN EDITING!
## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT!
## visit http://www.yamllint.com/ to validate this file as needed
templates:
- "templates/postgres.template.yml"
- "templates/redis.template.yml"
- "templates/web.template.yml"
- "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
...
Then ./launcher rebuild app, create an admin, upload a backup and restore. Just make sure to have plenty of space.
This procedure isn’t the fastest, but will work to restore a broken Discourse from any point to a usable and working install.
I am migrating my companies instance of discourse from CentOS 6 to CentOS 7 host. I can no longer update Discourse on Cent 6 because there is no new Docker versions. The first thing that I tried is what you mention above. I can import my backup, and successfully restore my data. The catch is I am no longer able to log into the site. Its in some weird state that the new instance thinks that I am logged in from the old instance, causing a log in loop on the new instance. I tried to use the api gem to force log myself out, but you need admin priv’s to do this. I cannot log in, in any form or fashion on the new instance to use the log out API call.
After that, I decided to attempt to simply stand up a new instance, and sync my standalone dir. on top of the new one, considering you offer a psql 9.3 template. Unfortunately I cannot bootstrap with the 9.3 template due to the apt-get errors mentioned above. If I bootstrap with the vanilla postgres template, I get schema errors because my old instance was running on 9.3.
I guess my next option will be to import the old Docker container via registry, and sync over the entire old “var/docker” directory to my new server, then fingers crossed when I attempt to actually update the instance.
Please let me know if there is anything else I can try here.
I just stood up a new instance of Discourse, imported and restored to a backup of my production instance via the web Admin panel. At that point, all of my data shows up and the forum looks as it should, BUT I get stuck in the log in look explained earlier.
I tried the rails command quoted above, with the following return:
After that, I attempt to log in to the forum via web again, and again get the log in loop. It successfully authenticates, but never acts as If i am logged in.
I have SSL enabled on my production instance, but not the current test instance. The web.ssl.template is being used in production, and commented out in test.
I will work on getting the site opened up. In the mean time, I did find this in my production log, from attempted log in’s:
Started GET "/session/csrf?_=1481229888046" for ip at 2016-12-08 20:46:39 +0000
Processing by SessionController#csrf as */*
Parameters: {"_"=>"1481229888046"}
Completed 200 OK in 1ms (Views: 0.2ms | ActiveRecord: 0.0ms)
Started POST "/session" for ip at 2016-12-08 20:46:39 +0000
Processing by SessionController#create as */*
Parameters: {"login"=>"chu@phmgmt.com", "password"=>"[FILTERED]"}
Can't verify CSRF token authenticity
Rendered text template (0.0ms)
Filter chain halted as :verify_authenticity_token rendered or redirected
Completed 403 Forbidden in 1ms (Views: 0.3ms | ActiveRecord: 0.0ms)
I ended up getting Discourse over to my Cent 7 server, by pushing the current container to our registry, and copying the entire /var/docker directory over. I am able to get a functioning version of our Discourse working that way. I am still unable to update discourse due to the following error:
Pups::ExecError: /root/upgrade_postgres failed with return #<Process::Status: pid 40 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "/root/upgrade_postgres"
e68c256f43b7101b5c7003b8db31b80ac83e845999e9f15bec369592d9d04649
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one