Backup restore failing on clean dev docker env: FATAL: Peer authentication failed for user “postgres”

koen360 · November 3, 2021, 9:22pm

Hi over in the beginners guide to install discourse for development using docker some of the comments mention this issue which I think is actually a bug that should be raised as such here. Correct me if I am wrong.

I tend to create a copy of production to test major config changes and have many times created a clean slate like this before. However now the backups fail with this error conerning the peer authentication failing.

Just to provide context on what I am doing (wrong potentially)

delete the local discourse directory altogether
stop and remove any containers
start fresh git cloning discourse and also the plugins I use directly after in the plugins directory
do the boot_dev --init incantations, bundle install if needed, migrate database if needed.
with the dummy admin account i log in, upload the backup I downloaded from my live site
hit restore backup.

Let me know if I can provide anymore info. The backup logs are below. And oh yes, I checked the pg_hba.conf in data/postgres/ had all the authentication methods set to “trust”.

[2021-11-03 20:55:35] 'koen' has started the restore!
[2021-11-03 20:55:35] Marking restore as running...
[2021-11-03 20:55:35] Making sure /src/tmp/restores/default/2021-11-03-205535 exists...
[2021-11-03 20:55:35] Copying archive to tmp directory...
[2021-11-03 20:55:35] Extracting dump file...
[2021-11-03 20:55:36] Validating metadata...
[2021-11-03 20:55:36]   Current version: 20211022154420
[2021-11-03 20:55:36]   Restored version: 20211022154420
[2021-11-03 20:55:36] Enabling readonly mode...
[2021-11-03 20:55:36] Pausing sidekiq...
[2021-11-03 20:55:36] Waiting up to 60 seconds for Sidekiq to finish running jobs...
[2021-11-03 20:55:42] Creating missing functions in the discourse_functions schema...
[2021-11-03 20:55:42] Restoring dump file... (this may take a while)
[2021-11-03 20:55:42] psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"
[2021-11-03 20:55:42] EXCEPTION: psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"
[2021-11-03 20:55:42] /src/lib/backup_restore/database_restorer.rb:92:in `restore_dump'
/src/lib/backup_restore/database_restorer.rb:26:in `restore'
/src/lib/backup_restore/restorer.rb:51:in `run'
/src/script/spawn_backup_restore.rb:23:in `restore'
/src/script/spawn_backup_restore.rb:36:in `block in <main>'
/src/script/spawn_backup_restore.rb:4:in `fork'
/src/script/spawn_backup_restore.rb:4:in `<main>'
[2021-11-03 20:55:42] Trying to rollback...
[2021-11-03 20:55:42] There was no need to rollback
[2021-11-03 20:55:42] Cleaning stuff up...
[2021-11-03 20:55:42] Dropping functions from the discourse_functions schema...
[2021-11-03 20:55:42] Removing tmp '/src/tmp/restores/default/2021-11-03-205535' directory...
[2021-11-03 20:55:42] Unpausing sidekiq...
[2021-11-03 20:55:42] Marking restore as finished...
[2021-11-03 20:55:42] Notifying 'koen' of the end of the restore...
[2021-11-03 20:55:46] Finished!

koen360 · November 8, 2021, 2:44pm

Hi all, just wondering, does this classify as a bug and will it be accepeted as such? Should I write it up differently or add more info? What’s the next step is all I am wondering, I am not pushing for any resolution date whatsoever.

koen360 · November 29, 2021, 8:02pm

Hi all, couple of weeks down the line I thought I’d give another shot. So went all cleanslate again, set up and logged into the fresh development environment on my local machine. Then added the plugins to match my PROD env. Pulled a back-up from the live site and try to restore that on my DTA environment. Alas still the same error.

Can anyone reproduce this? I have one upon a time had the native development environment running but then got rid of that when I noticed the ease of use of the docker setup. Could something be biting me from that legacy? Something wrong stemming from the former local postgres database platform? Can that even bite each other if the clean slate runs from docker? Just conjecturing. Can anyone point me in the right direction or the right people? Taking a copy of the live site for testing purposes is a pretty important use case I would imagine? Anything I can do to provide more info? Running the auto test thing as we speak but that doesn’t yield anything either…

[2021-11-29 19:43:39] 'koen' has started the restore!
[2021-11-29 19:43:39] Marking restore as running...
[2021-11-29 19:43:39] Making sure /src/tmp/restores/default/2021-11-29-194339 exists...
[2021-11-29 19:43:39] Copying archive to tmp directory...
[2021-11-29 19:43:39] Extracting dump file...
[2021-11-29 19:43:40] Validating metadata...
[2021-11-29 19:43:40]   Current version: 20211124161346
[2021-11-29 19:43:40]   Restored version: 20211123033311
[2021-11-29 19:43:40] Enabling readonly mode...
[2021-11-29 19:43:40] Pausing sidekiq...
[2021-11-29 19:43:40] Waiting up to 60 seconds for Sidekiq to finish running jobs...
[2021-11-29 19:43:46] Creating missing functions in the discourse_functions schema...
[2021-11-29 19:43:46] Restoring dump file... (this may take a while)
[2021-11-29 19:43:47] psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"
[2021-11-29 19:43:47] EXCEPTION: psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"

[2021-11-29 19:43:47] /src/lib/backup_restore/database_restorer.rb:92:in `restore_dump'
/src/lib/backup_restore/database_restorer.rb:26:in `restore'
/src/lib/backup_restore/restorer.rb:51:in `run'
/src/script/spawn_backup_restore.rb:23:in `restore'
/src/script/spawn_backup_restore.rb:36:in `block in <main>'
/src/script/spawn_backup_restore.rb:4:in `fork'
/src/script/spawn_backup_restore.rb:4:in `<main>'
[2021-11-29 19:43:47] Trying to rollback...
[2021-11-29 19:43:47] There was no need to rollback
[2021-11-29 19:43:47] Cleaning stuff up...
[2021-11-29 19:43:47] Dropping functions from the discourse_functions schema...
[2021-11-29 19:43:47] Removing tmp '/src/tmp/restores/default/2021-11-29-194339' directory...
[2021-11-29 19:43:47] Unpausing sidekiq...
[2021-11-29 19:43:47] Marking restore as finished...
[2021-11-29 19:43:47] Notifying 'koen' of the end of the restore...
[2021-11-29 19:43:51] Finished!

Koen

koen360 · November 29, 2021, 8:36pm

Well, d/rake autospec did yield 14 failures, the first 6 of which seem to be related to the PEER authentication issue.

Can’t make heads or tails of it but thought I’d post it here for when someone is able to investigate. Looks like the same error but with different Postgres versions everytime.

Hope this helps

Failures:

  1) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 9.5
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:61:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

  2) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 9.3
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:51:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

  3) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 12
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:74:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

  4) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 11
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:70:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

  5) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 9.5.5
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:56:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

  6) BackupRestore::DatabaseRestorer#restore with real psql restores from PostgreSQL 10
     Failure/Error: raise DatabaseRestoreError.new("psql failed: #{last_line}") if Process.last_status&.exitstatus != 0
     
     BackupRestore::DatabaseRestoreError:
       psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"                                                                                        
     # ./lib/backup_restore/database_restorer.rb:92:in `restore_dump'
     # ./lib/backup_restore/database_restorer.rb:26:in `restore'
     # ./spec/lib/backup_restore/shared_context_for_backup_restore.rb:59:in `execute_stubbed_restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:41:in `restore'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:45:in `expect_restore_to_work'
     # ./spec/lib/backup_restore/database_restorer_spec.rb:66:in `block (4 levels) in <main>'
     # ./spec/rails_helper.rb:284:in `block (2 levels) in <top (required)>'

mottosso · March 28, 2022, 10:52am

Also seeing this, same circumstances. Both via the UI and terminal.

Full output below. This is running from within WSL2 on Windows.

Any ideas?

$ d/discourse restore ragdoll-forums-2022-03-27-000005-v20220215015538.tar.gz
Starting restore: ragdoll-forums-2022-03-27-000005-v20220215015538.tar.gz
[STARTED]
'system' has started the restore!
Marking restore as running...
Making sure /src/tmp/restores/default/2022-03-28-104731 exists...
Copying archive to tmp directory...
Unzipping archive, this may take a while...
Extracting dump file...
Validating metadata...
  Current version: 20220325064954
  Restored version: 20220215015538
Enabling readonly mode...
Pausing sidekiq...
Waiting up to 60 seconds for Sidekiq to finish running jobs...
Creating missing functions in the discourse_functions schema...
Restoring dump file... (this may take a while)
psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"
EXCEPTION: psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL:  Peer authentication failed for user "postgres"
/src/lib/backup_restore/database_restorer.rb:92:in `restore_dump'
/src/lib/backup_restore/database_restorer.rb:26:in `restore'
/src/lib/backup_restore/restorer.rb:51:in `run'
script/discourse:149:in `restore'
/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'
/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'
script/discourse:290:in `<main>'
Trying to rollback...
There was no need to rollback
Cleaning stuff up...
Dropping functions from the discourse_functions schema...
Removing tmp '/src/tmp/restores/default/2022-03-28-104731' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[FAILED]
Restore done.
Failed to report error: can't alloc thread 0   PostSearchData Upsert (1.1ms)  INSERT INTO "post_search_data" ("raw_data","post_id","locale","version","search_data","private_message") VALUES ('The restore has failed. Here’s the log: [2022-03-28 10:47:31] [STARTED] [2022-03-28 10:47:31] ''system'' has started the restore! [2022-03-28 10:47:31] Marking restore as running... [2022-03-28 10:47:31] Making sure /src/tmp/restores/default/2022-03-28-104731 exists... [2022-03-28 10:47:31] Copying archive to tmp directory... [2022-03-28 10:47:31] Unzipping archive, this may take a while... [2022-03-28 10:47:31] Extracting dump file... [2022-03-28 10:47:32] Validating metadata... [2022-03-28 10:47:32] Current version: 20220325064954 [2022-03-28 10:47:32] Restored version: 20220215015538 [2022-03-28 10:47:32] Enabling readonly mode... [2022-03-28 10:47:32] Pausing sidekiq... [2022-03-28 10:47:32] Waiting up to 60 seconds for Sidekiq to finish running jobs... [2022-03-28 10:47:38] Creating missing functions in the discourse_functions schema... [2022-03-28 10:47:38] Restoring dump file... (this may take a while) [2022-03-28 10:47:38] psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL : Peer authentication failed for user "postgres" [2022-03-28 10:47:38] EXCEPTION: psql failed: psql: error: connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed: FATAL: Peer authentication failed for user "postgres" [2022-03-28 10:47:38] /src/lib/backup_restore/database_restorer.rb:92:in `restore_dump'' /src/lib/backup_restore/database_restorer.rb:26:in `restore'' /src/lib/backup_restore/restorer.rb:51:in `run'' script/discourse:149:in `restore'' /usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb:27:in `run'' /usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb:127:in `invoke_command'' /usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb:392:in `dispatch'' /usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb:485:in `start'' script/discourse:290:in ` < main > '' [2022-03-28 10:47:38] Trying to rollback... [2022-03-28 10:47:38] There was no need to rollback [2022-03-28 10:47:38] Cleaning stuff up... [2022-03-28 10:47:38] Dropping functions from the discourse_functions schema... [2022-03-28 10:47:38] Removing tmp ''/src/tmp/restores/default/2022-03-28-104731'' directory... [2022-03-28 10:47:38] Unpausing sidekiq... [2022-03-28 10:47:38] Marking restore as finished... [2022-03-28 10:47:38] Notifying ''system'' of the end of the restore...', 17, 'en', 4, '''-03'':12,19,30,40,50,61,74,83,91,100,109,118,126,143,157,171,193,218,262,271,283,292,305,315,323,333 ''-28'':13,20,31,41,51,62,75,84,92,101,110,119,127,144,158,172,194,219,263,272,284,293,306,316,324,334 ''/src/lib/backup_restore/database_restorer.rb'':223,228 ''/src/lib/backup_restore/restorer.rb'':232 ''/src/tmp/restores/default/2022-03-28-104731'':47,312 ''/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor.rb'':249 ''/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/base.rb'':253 ''/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/command.rb'':240 ''/usr/local/lib/ruby/gems/2.7.0/gems/thor-1.2.1/lib/thor/invocation.rb'':244 ''/var/run/postgresql/.s.pgsql.5432'':183,208 ''10'':14,21,32,42,52,63,76,85,93,102,111,120,128,145,159,173,195,220,264,273,285,294,307,317,325,335 ''127'':245 ''149'':237 ''2022'':11,18,29,39,49,60,73,82,90,99,108,117,125,142,156,170,192,217,261,270,282,291,304,314,322,332 ''20220215015538'':107 ''20220325064954'':98 ''26'':229 ''27'':241 ''290'':258 ''31'':16,23,34,44,54,65,78 ''32'':87,95,104,113,122,130 ''38'':147,161,175,197,222,266,275,287,296,309,319,327,337 ''392'':250 ''47'':15,22,33,43,53,64,77,86,94,103,112,121,129,146,160,174,196,221,265,274,286,295,308,318,326,336 ''485'':254 ''51'':233 ''60'':134 ''92'':224 ''archiv'':56,67 ''authent'':187,212 ''clean'':288 ''command'':248 ''connect'':178,203 ''copi'':55 ''creat'':148 ''current'':96 ''directori'':59,313 ''discours'':153,301 ''dispatch'':252 ''drop'':297 ''dump'':80,163,227 ''enabl'':114 ''end'':342 ''error'':177,202 ''except'':198 ''exist'':48 ''extract'':79 ''fail'':2A,6,184,188,200,209,213 ''fatal'':185,210 ''file'':81,164 ''finish'':139,331 ''function'':150,154,298,302 ''invok'':247 ''job'':141 ''log'':10 ''main'':260 ''make'':45 ''mark'':35,328 ''may'':69,166 ''metadata'':89 ''miss'':149 ''mode'':116 ''need'':279 ''notifi'':338 ''paus'':123 ''peer'':186,211 ''postgr'':191,216 ''psql'':176,199,201 ''readon'':115 ''remov'':310 ''restor'':1A,4,28,36,105,162,226,231,239,329,345 ''rollback'':269,281 ''run'':38,140,235,243 ''schema'':155,303 ''script/discourse'':236,257 ''second'':135 ''server'':180,205 ''sidekiq'':124,137,321 ''socket'':182,207 ''start'':17,26,256 ''stuff'':289 ''sure'':46 ''system'':24,339 ''take'':70,167 ''tmp'':58,311 ''tri'':267 ''unpaus'':320 ''unzip'':66 ''user'':190,215 ''valid'':88 ''version'':97,106 ''wait'':131 ', TRUE) ON CONFLICT ("post_id") DO UPDATE SET "raw_data"=excluded."raw_data","locale"=excluded."locale","version"=excluded."version","search_data"=excluded."search_data","private_message"=excluded."private_message" RETURNING "post_id"

thoka · March 19, 2023, 10:05pm

I have had the same problems.
Finally I inserted the following as a workaround

   if Rails.env.development?
      username = "discourse"
      password = "discourse"
      config["host"] = "localhost"
      config["port"] = 5432
      config["database"] = "discourse_development"
    end

before

  DatabaseConfiguration.new(

into /lib/backup_restore.rb.

I also had to set the password for user discourse via

d/psql -c "ALTER USER discourse WITH PASSWORD 'discourse';"

There might be nicer ways to achieve this.

Luicid · May 31, 2023, 9:06pm

Thank you so much! It really helped!

thoka · July 17, 2023, 5:01pm

This should be considered a bug, IMHO.

gerhard · July 24, 2023, 4:58pm

Low priority bug as it doesn’t affect a production site. And I don’t think anyone from the team currently uses the docker environment for development. I’ve recategorized the topic and put the pr-welcome tag on it.

thoka · April 4, 2024, 7:57pm

This works for me:

But there are several other ways to fix this to choose from:

add env setting USER=discourse to the docker boot image
add -e USER=discourse \ to d\exec

add this to backup_restore.rb

  if Rails.env.development?
    username = ENV["USER"] || "discourse"
  end

In addition, there is a new problem when executing d/discourse in the docker dev environment:

$ d/discourse backup
<internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require': cannot load such file -- thor (LoadError)
        from <internal:/usr/local/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:86:in `require'
        from script/discourse:4:in `<main>'

while d/bundle exec script/discourse backup works (see patch).

Topic		Replies	Views
Backup failed due to PG/SQL errors installation	9	1982	December 17, 2020
Updating failed, unable to rebuild installation	4	945	July 9, 2021
Restore failing- possible Data Explorer issue installation	21	1973	December 9, 2019
Discourse update error installation	37	1630	March 1, 2023
What is this error? installation	3	382	April 15, 2023

Backup restore failing on clean dev docker env: FATAL: Peer authentication failed for user “postgres”

Related Topics