Migrating Discourse from one DigitalOcean droplet to another without downtime

mholt · December 29, 2020, 7:49pm

We’re moving to a new DigitalOcean droplet and tried using the marketplace image. When running the setup script, it fails early on because our domain name is still pointing to our current instance in production.

I need to get this new installation up and running so I can restore its backup and then update the DNS records after that.

Error is:

Checking your domain name . . .
WARNING: Port 443 of computer does not appear to be accessible using hostname:  x
WARNING: Connection to x (port 80) also fails.

This suggests that x resolves to some IP address that does not reach this 
machine where you are installing discourse.

The first thing to do is confirm that x resolves to the IP address of this server.
You usually do this at the same place you purchased the domain.

If you are sure that the IP address resolves correctly, it could be a firewall issue.
A web search for "open ports YOUR CLOUD SERVICE" might help.

The domain name does in fact respond on ports 80 and 443, so this error message appears to be in error as well.

jomaxro · December 29, 2020, 8:37pm

Hey Matt,

We (the Discourse team) don’t manage the DO marketplace image, so I’m afraid we’re going to be of limited help resolving that specific issue.

mholt · December 29, 2020, 8:41pm

But you do manage this, yes?

github.com

discourse/discourse_docker/blob/master/discourse-setup#L55


check_IP_match() {
  HOST="$1"
  echo
  echo Checking your domain name . . .
  connect_to_port $HOST 443; ec=$?
  case $ec in
    0)
      echo "Connection to $HOST succeeded."
      ;;
    1)
      echo "WARNING: Port 443 of computer does not appear to be accessible using hostname:  $HOST."
      if connect_to_port $HOST 80; then
        echo
        echo SUCCESS: A connection to port 80 succeeds!
        echo This suggests that your DNS settings are correct,
        echo but something is keeping traffic to port 443 from getting to your server.
        echo Check your networking configuration to see that connections to port 443 are allowed.
      else
        echo "WARNING: Connection to http://$HOST (port 80) also fails."
        echo
        echo "This suggests that $HOST resolves to some IP address that does not reach this "

Even the manual install instructions have this step.

Surely I can’t be the first person to do this. How are people doing it?

jomaxro · December 29, 2020, 8:48pm

Yes, we do manage that. I did not look at the code, I assumed the check was coming from the marketplace image.

./discourse-setup is intended as a simple way of setting up Discourse - avoiding the need to manually edit a text file when spinning up a new Discourse site. Your use case isn’t a “typical” one that the setup script handles.

In your case, your best bet would likely be to copy containers/app.yml file from your current server to the new one. Alternatively you can manually edit the file yourself as suggested on lines 75/76:

github.com

discourse/discourse_docker/blob/master/discourse-setup#L75


      echo "This suggests that $HOST resolves to some IP address that does not reach this "
      echo machine where you are installing discourse.
    fi
    echo
    echo "The first thing to do is confirm that $HOST resolves to the IP address of this server."
    echo You usually do this at the same place you purchased the domain.
    echo
    echo If you are sure that the IP address resolves correctly, it could be a firewall issue.
    echo A web search for  \"open ports YOUR CLOUD SERVICE\" might help.
    echo
    echo This tool is designed only for the most standard installations. If you cannot resolve
    echo the issue above, you will need to edit containers/app.yml yourself and then type
    echo
    echo                   ./launcher rebuild app
    echo
    exit 1
    ;;
  2)
    echo "Continuing without port check."
    ;;
esac

mholt · December 29, 2020, 9:00pm

Where can I find the default app.yml? I want to start with a fresh default installation.

Also, how do I get the server up and running without the setup script? Accessing the IP address is still unresponsive since I can’t run the setup script.

jomaxro · December 29, 2020, 9:07pm

Default can be found at samples/standalone.yml. Also on GitHub.

Assuming you’re following the official install guide, after the commands in “Install Discourse”, you’ll want to do the following:

Copy the default yaml file from samples to containers:

cp samples/standalone.yml containers/app.yml

Edit the file by hand:

nano containers/app.yml

Bootstrap and start Discourse:

./launcher rebuild app

mholt · December 29, 2020, 9:24pm

Thank you, that gets me closer.

But now I cannot import the backup because I cannot activate my temporary admin account:

(6) Refused to load the script ‘’ because it violates the following Content Security Policy directive: "script-src ". Note that ‘script-src-elem’ was not explicitly set, so ‘script-src’ is used as a fallback.

Is there a direct way to restore from a backup or disable the CSP until then?

jomaxro · December 29, 2020, 9:33pm

SSH to your server, then:

cd /var/discourse
sudo ./launcher enter app
rails c
SiteSetting.content_security_policy = false
exit
exit

Note that I’d try the backup restore from CLI first - it solves the actual issue you have (restore a backup) vs the current roadblock (CSP).

mholt · December 29, 2020, 9:45pm

Thanks for the pointers.

Running restore, I get:

ERROR:  could not create unique index "index_incoming_referers_on_path_and_incoming_domain_id"
DETAIL:  Key (path, incoming_domain_id)=(/s/free+proxy+hideip.me, 1009) is duplicated.
EXCEPTION: psql failed: DETAIL:  Key (path, incoming_domain_id)=(/s/free+proxy+hideip.me, 1009) is duplicated.
/var/www/discourse/lib/backup_restore/database_restorer.rb:87:in `restore_dump'
/var/www/discourse/lib/backup_restore/database_restorer.rb:26:in `restore'
/var/www/discourse/lib/backup_restore/restorer.rb:51:in `run'
script/discourse:143:in `restore'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.0.1/lib/thor/command.rb:27:in `run'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.0.1/lib/thor/invocation.rb:127:in `invoke_command'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.0.1/lib/thor.rb:392:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/thor-1.0.1/lib/thor/base.rb:485:in `start'
script/discourse:284:in `<top (required)>'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli/exec.rb:63:in `load'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli/exec.rb:63:in `kernel_load'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli/exec.rb:28:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli.rb:497:in `exec'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/vendor/thor/lib/thor/command.rb:27:in `run'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/vendor/thor/lib/thor/invocation.rb:127:in `invoke_command'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/vendor/thor/lib/thor.rb:392:in `dispatch'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli.rb:30:in `dispatch'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/vendor/thor/lib/thor/base.rb:485:in `start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/cli.rb:24:in `start'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/exe/bundle:49:in `block in <top (required)>'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/lib/bundler/friendly_errors.rb:130:in `with_friendly_errors'
/usr/local/lib/ruby/gems/2.7.0/gems/bundler-2.2.3/exe/bundle:37:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Trying to rollback...
Rolling back...
Cleaning stuff up...
Dropping functions from the discourse_functions schema...
Removing tmp '/var/www/discourse/tmp/restores/default/2020-12-29-214249' directory...
Unpausing sidekiq...
Marking restore as finished...
Notifying 'system' of the end of the restore...
Finished!
[FAILED]
Restore done.

pfaffman · December 30, 2020, 12:49am

Looks like you have a corrupt index. Have you run an upgrade on the existing instance? There’s a chance that help.

There is a topic somewhere with instructions for copying over the raw database (and let’s encrypt) files from the old instance. That’s probably what I would do.

mholt · December 30, 2020, 12:46am

What’s rub run?

Upgrades on the existing instance keep having errors (different errors than this one) so that’s why I’m moving to a new instance.

mholt · December 30, 2020, 12:46am

Do you have a link to this?

pfaffman · December 30, 2020, 12:50am

Transferring backups via rsync and cronperhaps