Fail to upgrade - Make sure that `gem install json -v '1.8.2'` succeeds before bundling


(Lê Trần Đạt) #1

I try to update from Version 1.2.0.beta5 to 1.2.0.beta6 using Discourse upgrade feature in admin control panel. However, it failed to execute the upgrading.

Here is the log:

$ cd /var/www/discourse && git fetch && git reset --hard HEAD@{upstream}
Checking out files:  22% (1570/7114)   
Checking out files:  23% (1637/7114)   
Checking out files:  24% (1708/7114)   
Checking out files:  25% (1779/7114)   
Checking out files:  26% (1850/7114)   
Checking out files:  27% (1921/7114)   
Checking out files:  28% (1992/7114)   
Checking out files:  29% (2064/7114)   
Checking out files:  30% (2135/7114)   
Checking out files:  31% (2206/7114)   
Checking out files:  32% (2277/7114)   
Checking out files:  33% (2348/7114)   
Checking out files:  34% (2419/7114)   
Checking out files:  35% (2490/7114)   
Checking out files:  36% (2562/7114)   
Checking out files:  37% (2633/7114)   
Checking out files:  38% (2704/7114)   
Checking out files:  39% (2775/7114)   
Checking out files:  40% (2846/7114)   
Checking out files:  41% (2917/7114)   
Checking out files:  42% (2988/7114)   
Checking out files:  43% (3060/7114)   
Checking out files:  44% (3131/7114)   
Checking out files:  44% (3166/7114)   
Checking out files:  45% (3202/7114)   
Checking out files:  46% (3273/7114)   
Checking out files:  47% (3344/7114)   
Checking out files:  48% (3415/7114)   
Checking out files:  49% (3486/7114)   
Checking out files:  50% (3557/7114)   
Checking out files:  51% (3629/7114)   
Checking out files:  52% (3700/7114)   
Checking out files:  53% (3771/7114)   
Checking out files:  54% (3842/7114)   
Checking out files:  55% (3913/7114)   
Checking out files:  56% (3984/7114)   
Checking out files:  57% (4055/7114)   
Checking out files:  58% (4127/7114)   
Checking out files:  59% (4198/7114)   
Checking out files:  60% (4269/7114)   
Checking out files:  61% (4340/7114)   
Checking out files:  62% (4411/7114)   
Checking out files:  63% (4482/7114)   
Checking out files:  64% (4553/7114)   
Checking out files:  65% (4625/7114)   
Checking out files:  66% (4696/7114)   
Checking out files:  67% (4767/7114)   
Checking out files:  68% (4838/7114)   
Checking out files:  69% (4909/7114)   
Checking out files:  70% (4980/7114)   
Checking out files:  71% (5051/7114)   
Checking out files:  72% (5123/7114)   
Checking out files:  73% (5194/7114)   
Checking out files:  74% (5265/7114)   
Checking out files:  75% (5336/7114)   
Checking out files:  75% (5360/7114)   
Checking out files:  76% (5407/7114)   
Checking out files:  77% (5478/7114)   
Checking out files:  78% (5549/7114)   
Checking out files:  79% (5621/7114)   
Checking out files:  80% (5692/7114)   
Checking out files:  81% (5763/7114)   
Checking out files:  82% (5834/7114)   
Checking out files:  83% (5905/7114)   
Checking out files:  84% (5976/7114)   
Checking out files:  85% (6047/7114)   
Checking out files:  86% (6119/7114)   
Checking out files:  87% (6190/7114)   
Checking out files:  88% (6261/7114)   
Checking out files:  89% (6332/7114)   
Checking out files:  90% (6403/7114)   
Checking out files:  91% (6474/7114)   
Checking out files:  92% (6545/7114)   
Checking out files:  93% (6617/7114)   
Checking out files:  93% (6687/7114)   
Checking out files:  94% (6688/7114)   
Checking out files:  95% (6759/7114)   
Checking out files:  96% (6830/7114)   
Checking out files:  97% (6901/7114)   
Checking out files:  98% (6972/7114)   
Checking out files:  99% (7043/7114)   
Checking out files: 100% (7114/7114)   
Checking out files: 100% (7114/7114), done.
HEAD is now at 16465ed FEATURE: search your bookmarks using in:bookmarks
********************************************************
*** Please be patient, next steps might take a while ***
********************************************************
$ bundle install --deployment --without test --without development
Fetching source index from https://rubygems.org/
Using rake 10.4.2
Using CFPropertyList 2.2.8
Using i18n 0.6.11

Gem::Ext::BuildError: ERROR: Failed to build gem native extension.

    /usr/local/bin/ruby -r ./siteconf20150205-26262-1dc3px6.rb extconf.rb
Cannot allocate memory - /usr/local/bin/ruby -r ./siteconf20150205-26262-1dc3px6.rb extconf.rb  2>&1

Gem files will remain installed in /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/json-1.8.2 for inspection.
Results logged to /var/www/discourse/vendor/bundle/ruby/2.0.0/extensions/x86_64-linux/2.0.0-static/json-1.8.2/gem_make.out
An error occurred while installing json (1.8.2), and Bundler cannot continue.
Make sure that `gem install json -v '1.8.2'` succeeds before bundling.

What should I do to upgrade?

Thanks


(Sam Saffron) #2

you need more memory … you are running tight … recommend

cd /var/discourse
git pull
./launcher rebuild app

(Lê Trần Đạt) #3

Actually this is the second time I face this problem. Rebuild app saved me for the last time. I am using DO droplet

1GB Ram 30GB SSD Disk Singapore 1 Ubuntu Ubuntu 14.04 x64

Should I upgrade or just rebuild app every time I need upgrade Discourse?

Here is my Site Statistics


(Sam Saffron) #4

You can not expect /admin/upgrade to work properly if you are memory constrained. Did you configure swap space at least?


(Lê Trần Đạt) #5

Yes, I did.

  System load:  0.26               Processes:              109
  Usage of /:   43.5% of 29.40GB   Users logged in:        0
  Memory usage: 84%                IP address for eth0:    xxx
  Swap usage:   59%                IP address for docker0: xxx

(Sam Saffron) #6

You have 3 options:

  1. Increase swap size
  2. Upgrade to a 2GB instance
  3. Only upgrade via ./launcher rebuild app

(Lê Trần Đạt) #7

The current swap size is 1GB. Does it help if I increase swap size to 2 GB?

swapon -s
Filename                                Type            Size    Used    Priority
/swapfile                               file            1048572 77896   -1

(Sam Saffron) #8

Yes it would alleviate pressure during upgrades via web (note, you will be swapping like crazy so performance during upgrade window will be affected and upgrade window will be long)


(Jeff Atwood) #9

Is there anything else running on the server other than Discourse?


(Lê Trần Đạt) #10

Only Discourse.

I just deleted the old 1GB swapfile and added new 2GB swapfile

             total       used       free     shared    buffers     cached
Mem:          994M       928M        65M       100M       1.4M       124M
-/+ buffers/cache:       802M       191M
Swap:         2.0G        93M       1.9G

I will wait for the next upgrade to test.


(Jeff Atwood) #11

It is unusual, I monitor a lot of 1gb instances and I have never seen out of memory on upgrades.

Did you set number of unicorn processes to 2 during the setup, and the postgres db buffers to 128mb? This is all part of app.yml.


(Lê Trần Đạt) #12

At first, it was good to upgrade using web upgrade. I have upgraded several times before the last 2 releases 1.2.0.beta5 and 1.2.0.beta6.

I did

UNICORN_WORKERS: 2
db_shared_buffers: "128MB"

(Jeff Atwood) #13

Very strange. I wonder what pushed it over the edge.


(Lê Trần Đạt) #14

Do you think the available memory at the time I did upgrade was low?

  System load:  0.26               Processes:              109
  Usage of /:   43.5% of 29.40GB   Users logged in:        0
  Memory usage: 84%                IP address for eth0:    xxx
  Swap usage:   59%                IP address for docker0: xxx

(Tom Wells) #15

I would recommend upgrading to a 2GB instance anyway. I have Discourse and 10 small wordpress sites on my server and I’m capping out at 70% memory usage (at the moment). I think the 2gb instance gives you a lot more flexibility.

EDIT: It’s also worth mentioning that I also use memcached on that server as well.


(Jeff Atwood) #16

1GB should really work fine for what you have.


(Lê Trần Đạt) #17

I’d love to update to everyone that the new update to 1.2.0.beta8 via web was successful this time.

@codinghorror Do you think 2 GB swap solved this problem?
@sam The forum’s performance was not so bad and upgrade window was fine too. I found that it was even faster than before.


(Kane York) #18

Nah, it was probably this: Discourse Meta


(Jeff Atwood) #19