Upgrade from Ubuntu 12.04 to 14.04 on Live System?


(Craig Oda) #1

Has anyone upgraded from 12.04 to 14.04 on a live system? I am running 0.9.9.12

I have all the updates applied for 12.04 running on Digital Ocean. Although the system is running fine with 12.04, It’s tempting to run a sudo do-release-upgrade and see what happens. Anyone do this?

It’s a low traffic system and I can take it offline if I need to.


(Manthan Mallikarjun) #2

This is probably how I would do it.

Take it offline, make a snapshot, make another server with the snapshot, upgrade that server, create a backup of the old server through the admin panel, upload it to the new server, and then change your DNS.


(Jeff Atwood) #3

I say do it! Who wants to live forever??


(Sam Saffron) #4

According to his sex tape, Will Arnett


(Tuan Anh Tran) #5

DO makes it extremely easy on this. Make a snapshot, do whatever you want on it. If it works as expected, update dns.


(Craig Oda) #6

@nahtnam thanks for the tip. I am new to this snapshot concept. It’ll be a good learning experience. When you say, create a backup of the old server through the admin panel, is this the admin panel of Discourse of of Digital Ocean? I haven’t created a backup of Discourse yet and am intrigued by this new challenge. Changing the DNS is no problem.

I’ve been updating 12.04 from the command line, breaking things without a backup, and then going in to fix it. This is useful for learning, but I realize that I now have usable content on the site that I want to preserve.

thanks for the help.


(Wint) #7

I would guess the Digital Ocean panel. You can create snapshots of the machines so as to restore them quickly if something goes wrong.


(Manthan Mallikarjun) #8

Hello!

Let me explain this in a little bit more detail.

  1. Take your old server (ill call it server1 for short) offline, and make a snapshot of it with Digital Ocean.
  2. Start up another server (server2) using that snapshot.
  3. Upgrade server2 to the latest os
  4. If all went well, go back to server1, and create a backup through the Discourse gui (http://domain.com/admin/backups).
  5. Download that backup to your computer and upload it to server2 and make double check everything.
  6. Update the DNS settings and delete server1.

I hope this clears things up!


(Kane York) #9

And, for now make sure to always run a ./launcher restart app after performing a restore, so that your settings are fully propagated everywhere.

If someone can make a community PR to have the restore process do that automatically, that would be fantastic.

EDIT: I now see that @nahtnam didn’t actually say to do a Discourse restore, but it’s something to keep in mind anyways.


(Craig Oda) #10

Thanks to everyone for their help. This worked perfectly. The community support for Discourse is amazing.

Although I did create an image and restore it, per the great instructions from @nahtnam, I think that it would work with a simple upgrade command on the live system:

sudo do-release-upgrade -d 

The data transfer speed is so fast when getting it from the Digital Ocean mirror. I guess it’s just going over gig-e. Those SSD disks are really nice. I’ll keep tinkering around with Discourse. It’s an interesting hobby and I continue to learn new things, such as how to use the tools on Digital Ocean, which is a pretty cool service itself.

My next project will be to figure out how to get Discourse to automatically start when I reboot the server. I’m manually going in and doing a ./launcher start app

Well, I’ll save that for another day.

Thanks again for all the help.


(Jeff Atwood) #11

There was a bug in older versions of Docker where Discourse would not auto start, but it does now on a clean install on Digital Ocean – auto start is handled by Docker out of the box on a new install.

edit: but it does appear that OS and Docker upgrades interfere with this auto-start.


(Manthan Mallikarjun) #12

There will always be some downtime, but it all depends on how big your forum is. If it holds millions of users, you would probably do it my way, but if its just 100 users, sudo do-release-upgrade -d would work just as fine! :slight_smile:

I created another topic about getting discourse to start on reboot. Check it out here: Discourse Meta


(Craig Oda) #13

@nahtnam thanks for your help. I really like your approach because I have snapshot and backup to fall back to in case the update fails. I feel that is a real risk for the update to fail.

Also, thanks for starting the topic on getting Discourse to start on reboot. I’ve followed Jeff’s test and appreciate the the CEO of the company has time to personally try these things out. I’ll wait until there’s resolution.

BTW, I wish I was managing a Discourse forum of a million users. Sadly, the hobby forum I run only has a handful of users. I haven’t convinced any of my paying clients to bite the bullet and dump Lithium or Jive in order to move to Discourse. I sent out another proposal to a well-known company last week, but no go.

The good news is that with 14.04 on my system, I’m now feeling very modern and hip. I’m ready for the weekend!


(Jeff Atwood) #14

I just upgraded http://discourse.codinghorror.com from Ubuntu Server 12.04 LTS to Ubuntu Server 14.04 LTS.

It did work, but it’s not exactly something I would recommend… you still get asked a number of pretty confusing questions that even I was hesitant to answer.

I did decline overwriting sysctl.conf with the Ubuntu 14.04 LTS version for example, and I’m not sure if that was the right call.

I also had to manually start Docker / Discourse after rebooting, since it appears that many kinds of upgrades (Docker, OS) break the auto-start a default install of Discourse sets up somehow.

So:

  • Yes, the upgrade does work on a live system

  • However, it is fairly confusing

  • It might be better to just spin up a new droplet, install Discourse again from scratch, and restore a database backup.


(Craig Oda) #15

Wow, I’m impressed that you’re a hard worker and care enough to do this yourself. I’m sure it will lead your business to success.

I also declined overwriting sysctl.conf, so at the moment, no one knows what happens if you overwrite sysctl.conf.

Now that I think about it, I should have simply exported the Discourse data from a backup and then imported it into a brand new 14.04 system. At the time I started the upgrade project, I didn’t know about the backup feature (or knew, but it wasn’t top of mind).

I’m a little paranoid about the security updates since I suspect that one of my Digital Ocean droplets got hacked into several months ago. I wanted to do the upgrade to 14.04 since that is what my workstation and laptop are running and I assume that the primary target for Discourse development will be 14.04 LTS going forward.


(Sam Saffron) #16

Turns out this is nothing to do with the OS upgrade … what happened was that auto-start broke on Docker 1.1.0 and continues to be broke on 1.1.1

You can track the progress of this issue here: docker 1.1 does not restart running containers · Issue #7132 · moby/moby · GitHub

To revert to an old working docker run

apt-get install lxc-docker-1.0.1
reboot
cd /var/docker
./launcher rebuild app

At this point reboots will start working as expected, hoping this issue is resolved on dockers end in short order.

Update:

Expecting a fix in the next couple of days:


(BDub) #17

@nahtnam thanks for the step-by-step… this was really easy on Digital Ocean, and via SSH. FWIW I upgraded my server with the sudo do-release-upgrade -d command, basically said YES to every question except for the sysctl.conf question which I answered NO (keep the default, recommend option). Even though I have a slow and small forum, I performed your method and everything worked perfectly, thanks!

Definitely easier in my opinion to just work from the Snapshot per @nahtnam’s instructions :wink: Also, does a database backup restore all of your container edits as well? If not then definitely stick with the snapshot.


(Lee_Ars) #18

@codinghorror and @sam, what say you guys on a distro upgrade from 12.04 to 14.04, now that the .1 release is out? I’d love to push the button, if for nothing else than to escape the “New release ‘14.04.1 LTS’ available” MOTD every time I log into the server, but not if it’s going to implode after the upgrade.


(Sam Saffron) #19

Should be fine, I upgraded quite a few systems from 12.04 to 14.04


(Arpit Jalan) #20

I just upgraded a server from Ubuntu 12.04 to Ubuntu 14.04 and now the rebuild is failing with this error:

I, [2017-05-05T10:48:25.831302 #14]  INFO -- : > apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
E: Sub-process /usr/bin/dpkg returned an error code (1)
I, [2017-05-05T10:48:28.455627 #14]  INFO -- : Reading package lists...
Building dependency tree...
Reading state information...
The following packages will be REMOVED:
  postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5
0 upgraded, 0 newly installed, 3 to remove and 0 not upgraded.
After this operation, 26.1 MB disk space will be freed.
(Reading database ... 34533 files and directories currently installed.)
Removing postgresql-contrib-9.5 (9.5.5-1.pgdg16.04+1) ...
update-alternatives: warning: forcing reinstallation of alternative /usr/share/postgresql/9.5/man/man1/postmaster.1.gz because link group postmaster.1.gz is broken
update-alternatives: error: error creating symbolic link '/usr/share/man/man1/postmaster.1.gz.dpkg-tmp': No such file or directory
dpkg: error processing package postgresql-contrib-9.5 (--remove):
 subprocess installed pre-removal script returned error exit status 2
dpkg: postgresql-9.5: dependency problems, but removing anyway as you requested:
 postgresql-contrib-9.5 depends on postgresql-9.5 (= 9.5.5-1.pgdg16.04+1).

Removing postgresql-9.5 (9.5.5-1.pgdg16.04+1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of stop.
Removing postgresql-client-9.5 (9.5.5-1.pgdg16.04+1) ...
Processing triggers for postgresql-common (178.pgdg16.04+1) ...
Building PostgreSQL dictionaries from installed myspell/hunspell packages...
Removing obsolete dictionary files:
Errors were encountered while processing:
 postgresql-contrib-9.5



FAILED
--------------------
Pups::ExecError: apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 failed with return #<Process::Status: pid 17 exit 100>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5"
f20104f79ba40d6a171f6de54ebab510c0cb29a31244425c8361ae4e9917be40
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

Looking at logs, seems like apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 is failling.

Just out of curiosity I tried commenting out apt-get remove -y postgresql-9.5 postgresql-client-9.5 postgresql-contrib-9.5 from templates/postgres.9.3.template.yml. And now the rebuild fails with this error:

I, [2017-05-05T10:51:09.365549 #14]  INFO -- : > apt-get update && apt-get install -y postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://apt.postgresql.org/pub/repos/apt xenial-pgdg InRelease: Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://archive.ubuntu.com/ubuntu xenial InRelease: Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: https://deb.nodesource.com/node_6.x xenial InRelease: Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://archive.ubuntu.com/ubuntu xenial-updates InRelease: Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://archive.ubuntu.com/ubuntu xenial-security InRelease: Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial/InRelease  Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-updates/InRelease  Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/xenial-security/InRelease  Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch https://deb.nodesource.com/node_6.x/dists/xenial/InRelease  Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Failed to fetch http://apt.postgresql.org/pub/repos/apt/dists/xenial-pgdg/InRelease  Could not execute 'apt-key' to verify signature (is gnupg installed?)
W: Some index files failed to download. They have been ignored, or old ones used instead.
E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/main/p/postgresql-9.3/postgresql-client-9.3_9.3.15-1.pgdg16.04+1_amd64.deb  404  Not Found [IP: 174.143.35.246 80]

E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/main/p/postgresql-9.3/postgresql-9.3_9.3.15-1.pgdg16.04+1_amd64.deb  404  Not Found [IP: 174.143.35.246 80]

E: Failed to fetch http://apt.postgresql.org/pub/repos/apt/pool/main/p/postgresql-9.3/postgresql-contrib-9.3_9.3.15-1.pgdg16.04+1_amd64.deb  404  Not Found [IP: 174.143.35.246 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
I, [2017-05-05T10:51:13.329180 #14]  INFO -- : Get:1 http://apt.postgresql.org/pub/repos/apt xenial-pgdg InRelease [41.4 kB]
Err:1 http://apt.postgresql.org/pub/repos/apt xenial-pgdg InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Hit:2 http://archive.ubuntu.com/ubuntu xenial InRelease
Err:2 http://archive.ubuntu.com/ubuntu xenial InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Get:3 https://deb.nodesource.com/node_6.x xenial InRelease [4,634 B]
Err:3 https://deb.nodesource.com/node_6.x xenial InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Get:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease [102 kB]
Err:4 http://archive.ubuntu.com/ubuntu xenial-updates InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Get:5 http://archive.ubuntu.com/ubuntu xenial-security InRelease [102 kB]
Err:5 http://archive.ubuntu.com/ubuntu xenial-security InRelease
  Could not execute 'apt-key' to verify signature (is gnupg installed?)
Fetched 250 kB in 0s (371 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following additional packages will be installed:
  libossp-uuid16
Suggested packages:
  uuid locales-all postgresql-doc-9.3 libdbd-pg-perl
The following NEW packages will be installed:
  libossp-uuid16 postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3
0 upgraded, 4 newly installed, 0 to remove and 0 not upgraded.
Need to get 5,002 kB of archives.
After this operation, 23.9 MB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu xenial/universe amd64 libossp-uuid16 amd64 1.6.2-1.5build2 [28.6 kB]
Err:2 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 postgresql-client-9.3 amd64 9.3.15-1.pgdg16.04+1
  404  Not Found [IP: 174.143.35.246 80]
Err:3 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 postgresql-9.3 amd64 9.3.15-1.pgdg16.04+1
  404  Not Found [IP: 174.143.35.246 80]
Err:4 http://apt.postgresql.org/pub/repos/apt xenial-pgdg/main amd64 postgresql-contrib-9.3 amd64 9.3.15-1.pgdg16.04+1
  404  Not Found [IP: 174.143.35.246 80]
Fetched 28.6 kB in 0s (78.4 kB/s)



FAILED
--------------------
Pups::ExecError: apt-get update && apt-get install -y postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3 failed with return #<Process::Status: pid 17 exit 100>
Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'
exec failed with the params "apt-get update && apt-get install -y postgresql-9.3 postgresql-client-9.3 postgresql-contrib-9.3"
893bf5ef82d1530829ecc5e4091e9b09bc8deba9f87d3d7ed3ecf68942e70c0a
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

The postgres template in use is templates/postgres.9.3.template.yml.

Seems like I will have to reset/create a new droplet and install Discourse from scratch?