Fail to upgrade - Make sure that `gem install json -v '1.8.2'` succeeds before bundling

installation
1

I try to update from Version 1.2.0.beta5 to 1.2.0.beta6 using Discourse upgrade feature in admin control panel. However, it failed to execute the upgrading.

Here is the log:

$ cd /var/www/discourse && git fetch && git reset --hard HEAD@{upstream}
Checking out files:  22% (1570/7114)   
Checking out files:  23% (1637/7114)   
Checking out files:  24% (1708/7114)   
Checking out files:  25% (1779/7114)   
Checking out files:  26% (1850/7114)   
Checking out files:  27% (1921/7114)   
Checking out files:  28% (1992/7114)   
Checking out files:  29% (2064/7114)   
Checking out files:  30% (2135/7114)   
Checking out files:  31% (2206/7114)   
Checking out files:  32% (2277/7114)   
Checking out files:  33% (2348/7114)   
Checking out files:  34% (2419/7114)   
Checking out files:  35% (2490/7114)   
Checking out files:  36% (2562/7114)   
Checking out files:  37% (2633/7114)   
Checking out files:  38% (2704/7114)   
Checking out files:  39% (2775/7114)   
Checking out files:  40% (2846/7114)   
Checking out files:  41% (2917/7114)   
Checking out files:  42% (2988/7114)   
Checking out files:  43% (3060/7114)   
Checking out files:  44% (3131/7114)   
Checking out files:  44% (3166/7114)   
Checking out files:  45% (3202/7114)   
Checking out files:  46% (3273/7114)   
Checking out files:  47% (3344/7114)   
Checking out files:  48% (3415/7114)   
Checking out files:  49% (3486/7114)   
Checking out files:  50% (3557/7114)   
Checking out files:  51% (3629/7114)   
Checking out files:  52% (3700/7114)   
Checking out files:  53% (3771/7114)   
Checking out files:  54% (3842/7114)   
Checking out files:  55% (3913/7114)   
Checking out files:  56% (3984/7114)   
Checking out files:  57% (4055/7114)   
Checking out files:  58% (4127/7114)   
Checking out files:  59% (4198/7114)   
Checking out files:  60% (4269/7114)   
Checking out files:  61% (4340/7114)   
Checking out files:  62% (4411/7114)   
Checking out files:  63% (4482/7114)   
Checking out files:  64% (4553/7114)   
Checking out files:  65% (4625/7114)   
Checking out files:  66% (4696/7114)   
Checking out files:  67% (4767/7114)   
Checking out files:  68% (4838/7114)   
Checking out files:  69% (4909/7114)   
Checking out files:  70% (4980/7114)   
Checking out files:  71% (5051/7114)   
Checking out files:  72% (5123/7114)   
Checking out files:  73% (5194/7114)   
Checking out files:  74% (5265/7114)   
Checking out files:  75% (5336/7114)   
Checking out files:  75% (5360/7114)   
Checking out files:  76% (5407/7114)   
Checking out files:  77% (5478/7114)   
Checking out files:  78% (5549/7114)   
Checking out files:  79% (5621/7114)   
Checking out files:  80% (5692/7114)   
Checking out files:  81% (5763/7114)   
Checking out files:  82% (5834/7114)   
Checking out files:  83% (5905/7114)   
Checking out files:  84% (5976/7114)   
Checking out files:  85% (6047/7114)   
Checking out files:  86% (6119/7114)   
Checking out files:  87% (6190/7114)   
Checking out files:  88% (6261/7114)   
Checking out files:  89% (6332/7114)   
Checking out files:  90% (6403/7114)   
Checking out files:  91% (6474/7114)   
Checking out files:  92% (6545/7114)   
Checking out files:  93% (6617/7114)   
Checking out files:  93% (6687/7114)   
Checking out files:  94% (6688/7114)   
Checking out files:  95% (6759/7114)   
Checking out files:  96% (6830/7114)   
Checking out files:  97% (6901/7114)   
Checking out files:  98% (6972/7114)   
Checking out files:  99% (7043/7114)   
Checking out files: 100% (7114/7114)   
Checking out files: 100% (7114/7114), done.
HEAD is now at 16465ed FEATURE: search your bookmarks using in:bookmarks
********************************************************
*** Please be patient, next steps might take a while ***
********************************************************
$ bundle install --deployment --without test --without development
Fetching source index from https://rubygems.org/
Using rake 10.4.2
Using CFPropertyList 2.2.8
Using i18n 0.6.11

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    /usr/local/bin/ruby -r ./siteconf20150205-26262-1dc3px6.rb extconf.rb
Cannot allocate memory - /usr/local/bin/ruby -r ./siteconf20150205-26262-1dc3px6.rb extconf.rb  2>&1

Gem files will remain installed in /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/json-1.8.2 for inspection.
Results logged to /var/www/discourse/vendor/bundle/ruby/2.0.0/extensions/x86_64-linux/2.0.0-static/json-1.8.2/gem_make.out
An error occurred while installing json (1.8.2), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.2'` succeeds before bundling.

What should I do to upgrade?

Thanks

2

you need more memory … you are running tight … recommend

cd /var/discourse
git pull
./launcher rebuild app
1 Like
3

Actually this is the second time I face this problem. Rebuild app saved me for the last time. I am using DO droplet

1GB Ram 30GB SSD Disk Singapore 1 Ubuntu Ubuntu 14.04 x64

Should I upgrade or just rebuild app every time I need upgrade Discourse?

Here is my Site Statistics

4

You can not expect /admin/upgrade to work properly if you are memory constrained. Did you configure swap space at least?

5

Yes, I did.

  System load:  0.26               Processes:              109
  Usage of /:   43.5% of 29.40GB   Users logged in:        0
  Memory usage: 84%                IP address for eth0:    xxx
  Swap usage:   59%                IP address for docker0: xxx
6

You have 3 options:

  1. Increase swap size
  2. Upgrade to a 2GB instance
  3. Only upgrade via ./launcher rebuild app
7

The current swap size is 1GB. Does it help if I increase swap size to 2 GB?

swapon -s
Filename                                Type            Size    Used    Priority
/swapfile                               file            1048572 77896   -1
8

Yes it would alleviate pressure during upgrades via web (note, you will be swapping like crazy so performance during upgrade window will be affected and upgrade window will be long)

2 Likes
9

Is there anything else running on the server other than Discourse?

10

Only Discourse.

I just deleted the old 1GB swapfile and added new 2GB swapfile

             total       used       free     shared    buffers     cached
Mem:          994M       928M        65M       100M       1.4M       124M
-/+ buffers/cache:       802M       191M
Swap:         2.0G        93M       1.9G

I will wait for the next upgrade to test.

11

It is unusual, I monitor a lot of 1gb instances and I have never seen out of memory on upgrades.

Did you set number of unicorn processes to 2 during the setup, and the postgres db buffers to 128mb? This is all part of app.yml.

12

At first, it was good to upgrade using web upgrade. I have upgraded several times before the last 2 releases 1.2.0.beta5 and 1.2.0.beta6.

I did

UNICORN_WORKERS: 2
db_shared_buffers: "128MB"
13

Very strange. I wonder what pushed it over the edge.

14

Do you think the available memory at the time I did upgrade was low?

  System load:  0.26               Processes:              109
  Usage of /:   43.5% of 29.40GB   Users logged in:        0
  Memory usage: 84%                IP address for eth0:    xxx
  Swap usage:   59%                IP address for docker0: xxx
15

I would recommend upgrading to a 2GB instance anyway. I have Discourse and 10 small wordpress sites on my server and I’m capping out at 70% memory usage (at the moment). I think the 2gb instance gives you a lot more flexibility.

EDIT: It’s also worth mentioning that I also use memcached on that server as well.

1 Like
16

1GB should really work fine for what you have.

2 Likes
17

I’d love to update to everyone that the new update to 1.2.0.beta8 via web was successful this time.

@codinghorror Do you think 2 GB swap solved this problem?
@sam The forum’s performance was not so bad and upgrade window was fine too. I found that it was even faster than before.

18

Nah, it was probably this: https://meta.discourse.org/tags/memory-leak-feb-09

1 Like