docker rm -f aa6b422d88ca
docker rm -f 1fba0860cbc3
./launcher cleanup
./launcher rebuild data
./launcher rebuild web_only
But really the root cause here is that you are just way too tight on disk space. I would recommend doubling the size /dev/mappper/vg-lv_root
The last step (rebuild web_only) fails:
Bundle complete! 110 Gemfile dependencies, 201 gems now installed.
Gems in the group development were not installed.
Bundled gems are installed into `./vendor/bundle`
I, [2018-08-31T00:50:57.518677 #13] INFO -- : > cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate'
rake aborted!
Gem::LoadError: can't activate public_suffix-2.0.5, already activated public_suffix-3.0.2
/var/www/discourse/lib/plugin_gem.rb:18:in `load'
/var/www/discourse/lib/plugin/instance.rb:501:in `gem'
/var/www/discourse/plugins/discourse-sync-to-dropbox/plugin.rb:7:in `activate!'
/var/www/discourse/lib/plugin/instance.rb:431:in `instance_eval'
/var/www/discourse/lib/plugin/instance.rb:431:in `activate!'
lib/discourse.rb:164:in `block in activate_plugins!'
lib/discourse.rb:161:in `each'
lib/discourse.rb:161:in `activate_plugins!'
/var/www/discourse/config/application.rb:223:in `<class:Application>'
/var/www/discourse/config/application.rb:39:in `<module:Discourse>'
/var/www/discourse/config/application.rb:38:in `<top (required)>'
/var/www/discourse/Rakefile:5:in `require'
/var/www/discourse/Rakefile:5:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.5.0/gems/rake-12.3.1/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
(See full trace by running task with --trace)
I, [2018-08-31T00:51:18.343389 #13] INFO -- : gem install geocoder -v 1.4.4 -i /var/www/discourse/plugins/discourse-locations/gems/2.5.1 --no-document --ignore-dependencies
Successfully installed geocoder-1.4.4
1 gem installed
gem install public_suffix -v 2.0.5 -i /var/www/discourse/plugins/discourse-sync-to-dropbox/gems/2.5.1 --no-document --ignore-dependencies
Successfully installed public_suffix-2.0.5
1 gem installed
FAILED
--------------------
Pups::ExecError: cd /var/www/discourse && su discourse -c 'bundle exec rake db:migrate' failed with return #<Process::Status: pid 507 exit 1>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params {"cd"=>"$home", "hook"=>"bundle_exec", "cmd"=>["su discourse -c 'bundle install --deployment --verbose --without test --without development --retry 3 --jobs 4'", "su discourse -c 'bundle exec rake db:migrate'", "su discourse -c 'bundle exec rake assets:precompile'"]}
36b568dc13935b0491b99afd4763776899dad8d57a1df149633b9c7c02fd3c0b
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one
Trying without the dropbox plugin now…
Hmmmm … are you on tests passed… stable or beta?
Also, remove discourse-sync-to-dropbox asap. It looks like it is not working at the moment.
Phew! After removing the dropbox plugin my site is finally back up and running. Thanks a lot to everyone for your support!
Since I never changed that, I guess I’m on tests-passed.
I guess you’re right, but how does that relate to discourse’s minimum requirements of 10 GB HDD?
10GB is ambitious and absolutely not the case if you are running seperate data and web containers.
Ah, I see. I don’t think that was mentioned anywhere. I’ll check tomorrow where that information might be usefully added.
However, the situation seems to look much brighter now:
$ df -h
Filesystem Size Used Avail Use% Mounted on
udev 476M 0 476M 0% /dev
tmpfs 100M 12M 88M 12% /run
/dev/mapper/vg-lv_root 19G 8.4G 9.0G 49% /
tmpfs 497M 1.4M 495M 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 497M 0 497M 0% /sys/fs/cgroup
/dev/sda1 461M 160M 278M 37% /boot
tmpfs 100M 0 100M 0% /run/user/1000
Not sure where all the space came from but it looks good, doesn’t it? Do you still think I need to upgrade the server?
The data container was using the old image, so you had a situation where:
“data container” was saying HEY I need the old Discourse base image
“web container” was saying I need the new image
Not sure where all the space came from but this is a chunk of it.
How did that happen?
From my side, I can say that I started an update via /admin/upgrade which failed, presumably due to a lack of space. Since the website was down, I tried to rebuild via ssh and that’s when the whole misery started…
Regardless of this specific situation, I wonder if the web-updater could not handle lack of disk space better. Rather than crashing the site, could it not just exit before it’s too late?
Because you hadn’t upgraded the data container.
It could probably do a disk space check before it started, and refuse to proceed if there wasn’t enough space.
Another question:
Where did you get these IDs from:
Those are the container IDs (first column in the docker ps output).
This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.
I still don’t quite understand this. If I’m supposed to upgrade the data conatainer too, why doesn’t it appear in these instructions?
And isn’t the whole point of the two container setup that I don’t have to rebuild the data container? Well, I guess, not, but if someone could help me understand what the proper procedure for a two container setup is when /admin/upgrade/ tells me to do
cd /var/discourse/
git pull
./launcher rebuild appHence the command you quoted is talking about the web_only container… not the data container.
2 container setup is not something we detect in the web upgrader. The commands you want to run are exactly the ones you quoted…
It’s assumed that if you can set up a two container install that you can figure out the nuances. Usually you just do the bootstrap/destroy/start sequence.
Database upgrades don’t happen often and they are typically not required when they are first available. They are sometimes required. When they are you can just do a rebuild on the data container. (twice for the last upgrade)
Yet, things went wrong because I did exactly that:
So I guess this was a situation where a database upgrade was required.
Bur how do I know whether a database upgrade is required?
You see what version of the database you’re running and look at the database templates to see what version they are upgrading to.
One way to see what version you’re running is
./launcher enter data
psql --version
There are a bunch of topics about the last database upgrade, about 6 months ago (but it might have been 3 or 9). As much as I see you here in surprised you missed it!
And if disk space is a problem then the database upgrade will be challenging because you need at least 2x free space since you’ll be making another copy of the database to upgrade it.
Haha, well, I may have missed it or I may have seen it and scrolled past it because I had no reason to be interested in this kind of low level stuff. Database? It’s like the road when driving a car: you don’t really think about it until it gets bad.
It even took me a while to find what you (probably) where referring to now:
But on the way, I found this useful specification of what “not often” means:
To summary my lesson from this topic:
The confusion around the original issue was due to a combination of rather rare circumstances which meant that all standard instructions failed for various reasons:
Points 1 and 3 are still true but probably not nr. 2. Yet, I still had problems doing the web_only upgrade. So the db upgrade probably worsened things but it seems it was a mistake to run the two-container setup on 20GB.
That seems true. I won’t do it with less than a 2GB-50GB-$10/month Digital Ocean droplet. But it’s hard to get by on 20GB SSD at all. And unless this is your Discourse for your family who you can’t convince to use Discourse to use Discourse at all (maybe I’m projecting), even 25GB is tough.
Glad you got it sorted (you did, right?)