Lots of problems: Logged in staff cannot reach topics, manual upgrade fails, user cards not opening, default login not working

support
1

Hello. I have posted a similar problem before, which was not completely resolved. Currently there are some breaking problems going on with our forums.

  1. Upgrades from command line fail.

    1. I have tried removing all the extensions but it still did not work.

    2. I have created a new instance, copied the backup of the old one and restored via command line. It ended with some errors:

      ERROR:  could not create unique index "index_incoming_referers_on_path_and_incoming_domain_id"
DETAIL:  Key (path, incoming_domain_id)=(/m/search, 18) is duplicated.
EXCEPTION: psql failed: DETAIL:  Key (path, incoming_domain_id)=(/m/search, 18) is duplicated.
/var/www/discourse/lib/backup_restore/database_restorer.rb:87:in `restore_dump'
/var/www/discourse/lib/backup_restore/database_restorer.rb:26:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:51:in `run'
script/discourse:145:in `restore'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.1.0/lib/thor/command.rb:27:in `run'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.1.0/lib/thor/invocation.rb:127:in `invoke_command'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.1.0/lib/thor.rb:392:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.1.0/lib/thor/base.rb:485:in `start'
script/discourse:286:in `<top (required)>'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli.rb:494:in `exec'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/exe/bundle:49:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/lib/bundler/friendly_errors.rb:130:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.7/exe/bundle:37:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Trying to rollback...
Rolling back...
Cleaning stuff up...
Dropping functions from the discourse_functions schema...
Removing tmp '/var/www/discourse/tmp/restores/default/2021-08-30-124350' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[FAILED]
Restore done.

    3. I then run a ./launcher rebuild app to see if it works but got similar error messages:

      rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

ERROR:  unsupported Unicode escape sequence
DETAIL:  \u0000 cannot be converted to text.

      and

      FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with 
return #<Process::Status: pid 4162 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"db_migrate", "cmd"=>["su discourse -c 'bundle exec rake db:migrate'"]}
c618c071d6a0240d55ce704cedd63d2ee4cf1df589810d2130049a78b69f327e

    4. I also ran discourse-doctor to no avail.

  2. We cannot login normally with a username and password, it gives an error.
    1. Only other methods of logins work (GitHub, Google, etc.)
  3. User cards are not showing for users. If you click a user’s name, the card tries to pop up for an instant but it cannot.
    1. When you visit the user’s profile page (by opening in a new tab for example) it gives an “Oops, this page does not exist” error.
    2. When a user tries to view their preferences, messages, etc. they are also presented with an “Oops” error page.
    3. We can see user’s other settings via the admin interface but cannot perform changes to settings in for example interface or user avatar.
  4. Admins cannot view topics or the main page.
    1. The site can be viewed normally when logged out.
    2. Other users can login and interact with the forum (create topics, replies, etc.).
    3. If we login as an admin, we are greeted with an error message:

      Oops

      The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.

      Detailed information about the error was logged, and an automatic notification generated. We’ll take a look at it.

      No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site’s feedback category.

Any help is appreciated. If you had any idea how to fix these, or a way to create a new instance with a working backup please let me know.

2

See this for a similar (if different) problem: Duplicate data explorer key blocking rebuild

Check for associated data integrity issues in your current database, fix them, then create backup.

2 Likes
3

@merefield, I don’t have comprehensive knowledge so I checked using Data Explorer and there is an incoming_domain_id column in incoming_referers table of type integer as a foreign key to incoming_domains. Should I be working on this? If so, how do I check that?

4

This looks so messy that if you are not confident and the data is critical, I would get an experienced SA to help you recover everything #marketplace

1 Like
5

Did you figure out the issues?