# Multisite upgrade failed due to missing database columns

**URL:** https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239
**Category:** Bug
**Created:** [14 Dicembre 2016, 2:41pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239 "2016-12-14T14:41:35Z")
**Posts on this page:** 12
**Page:** 1

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [14 Dicembre 2016, 2:41pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/1 "2016-12-14T14:41:35Z")

</div>

Doing a `./launcher rebuild discourse-web.yml` I got this error:

```plaintext
Compiling css for kulturladen
rake aborted!
ActiveRecord::StatementInvalid: PG::UndefinedColumn: ERROR: column "uploaded_background_id" does not exist
LINE 1: ...categories"."updated_at" FROM "categories" WHERE (uploaded_b...
                                                             ^
: SELECT "categories"."updated_at" FROM "categories" WHERE (uploaded_background_id IS NOT NULL) ORDER BY updated_at desc LIMIT 1
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/pg.rb:90:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/patches/db/pg.rb:90:in `async_exec'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `block in exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:484:in `block in log'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract_adapter.rb:478:in `log'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:592:in `exec_no_cache'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:584:in `execute_and_clear'
/var/www/discourse/lib/freedom_patches/fast_pluck.rb:41:in `select_raw'
/var/www/discourse/lib/freedom_patches/fast_pluck.rb:79:in `pluck'
/var/www/discourse/app/models/category.rb:108:in `last_updated_at'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:70:in `block in last_updated_at'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation.rb:302:in `scoping'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/relation/delegation.rb:70:in `last_updated_at'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:163:in `digest'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:153:in `stylesheet_filename'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:129:in `stylesheet_fullpath'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:80:in `compile'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:39:in `block in compile'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:36:in `synchronize'
/var/www/discourse/lib/sass/discourse_stylesheets.rb:36:in `compile'
/var/www/discourse/lib/tasks/assets.rake:51:in `block (3 levels) in <top (required)>'
/var/www/discourse/lib/tasks/assets.rake:50:in `each'
/var/www/discourse/lib/tasks/assets.rake:50:in `block (2 levels) in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rails_multisite-1.0.6/lib/rails_multisite/connection_management.rb:126:in `block in each_connection'

/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rails_multisite-1.0.6/lib/rails_multisite/connection_management.rb:124:in `each'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rails_multisite-1.0.6/lib/rails_multisite/connection_management.rb:124:in `each_connection'
/var/www/discourse/lib/tasks/assets.rake:44:in `block in <top (required)>'
/var/www/discourse/lib/tasks/assets.rake:130:in `block in <top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rake-11.2.2/exe/rake:27:in `<top (required)>'
/usr/local/bin/bundle:22:in `load'
/usr/local/bin/bundle:22:in `<main>'
PG::UndefinedColumn: ERROR: column "uploaded_background_id" does not exist
LINE 1: ...categories"."updated_at" FROM "categories" WHERE (uploaded_b...

```

I’m at the _version: tests-passed_ and using a multi site installation with separate web and data containers.

Looks like the database column `` got added 2 days ago:

> <https://github.com/discourse/discourse/blob/53086fdb988d157cacbda1f52935f7ceda14812e/db/migrate/20161202034856_add_uploads_to_categories.rb>

Is there any way to upgrade the database scheme manually so I get my sites up again?

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [14 Dicembre 2016, 3:23pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/2 "2016-12-14T15:23:18Z")

</div>

Did you run the migrations on your data container before rebuilding the web container?

---

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [14 Dicembre 2016, 3:25pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/3 "2016-12-14T15:25:40Z")

</div>

No because I thought that is taken care of in the web containers YAML file by this hook:

```plaintext
  after_bundle_exec:
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

```

If not: how am I supposed to migrate the data container?

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [14 Dicembre 2016, 3:30pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/4 "2016-12-14T15:30:35Z")

</div>

Does rebuilding again fail as well?

Otherwise, we’re need to check if the migrations ran by running the following query in Postgres

`SELECT * FROM schema_migration_details WHERE name = 'AddUploadsToCategories';`

---

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [14 Dicembre 2016, 3:36pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/5 "2016-12-14T15:36:59Z")

</div>

Rebuilding again fails as well. Did it 4 times.

The SELECT statement yields 0 records:

```plaintext
discourse_kulturladen=# SELECT * FROM schema_migration_details WHERE name = 'AddUploadsToCategories';
 id | version | name | hostname | git_version | rails_version | duration | direction | created_at
----+---------+------+----------+-------------+---------------+----------+-----------+------------
(0 rows)

```

---

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [14 Dicembre 2016, 3:38pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/6 "2016-12-14T15:38:13Z")

</div>

BTW: At the default multi site the migration did run:

```plaintext
discourse=# SELECT * FROM schema_migration_details WHERE name = 'AddUploadsToCategories';
 id | version | name | hostname | git_version | rails_version | duration | direction | created_at
-----+----------------+------------------------+--------------+------------------------------------------+---------------+----------+-----------+----------------------------
 695 | 20161202034856 | AddUploadsToCategories | d1bcdd8eb4d7+| 20f9aad539bc4929f9149ddf989fc8abb69329d3 | 4.2.7.1 | 129 | up | 2016-12-14 14:10:12.342404
     | | | | | | | |
(1 row)

```

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [14 Dicembre 2016, 4:10pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/7 "2016-12-14T16:10:03Z")

</div>

Try running multisite:migrate manually.

---

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [14 Dicembre 2016, 4:30pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/8 "2016-12-14T16:30:01Z")

</div>

I tried from within the data container. It’s code stays at 9th of Dec though (02b21a26ddfe48536251276c235c6c9ac9a7227a).

I probably need to run it from the web container which I can not access because the rebuild failed.

---

<div class="post-metadata">

### Author: ![tgxworld](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/tgxworld/32/106117_2.png) [@tgxworld](https://meta.discourse.org/u/tgxworld)
#### Post date: [15 Dicembre 2016, 2:30am UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/9 "2016-12-15T02:30:40Z")

</div>

Hmm ok I think I see where the problem is.

> <https://github.com/discourse/discourse_docker/blob/f92ac1191a1a86526a6e92c7ddbad62d44550212/templates/web.template.yml#L178-L184>

You’re only running `multisite:migrate` after the `bundle exec` hook but during the hook, assets are already being precompiled when the multisite DB hasn’t been migrated yet. To work around the problem, can you change your template to something like

```plaintext
  before_bundle_exec:
    - exec: su discourse -c 'bundle install --deployment --verbose --without test --without development'
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

```

---

<div class="post-metadata">

### Author: ![fuerst](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/fuerst/32/120729_2.png) [@fuerst](https://meta.discourse.org/u/fuerst)
#### Post date: [15 Dicembre 2016, 9:14am UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/10 "2016-12-15T09:14:14Z")

</div>

You are right - precompiling assets started before database migration. Adding your code above to the `before_bundle_exec` hook worked. Thanks a lot for your help!

I had to modify it slightly to get it executed properly. My hook for multi site looks like this now:

```plaintext
  before_bundle_exec:
    - file:
        path: $home/config/multisite.yml
        contents: |
         # (your multisite.yml config here)
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle install --deployment --verbose --without test --without development
    - exec: cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate

```

Should I consider this as a one time workaround or can this two `exec` stay forever in my template file?

---

<div class="post-metadata">

### Author: ![Sander78](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sander78/32/296615_2.png) [@Sander78](https://meta.discourse.org/u/Sander78)
#### Post date: [10 Gennaio 2017, 3:28pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/11 "2017-01-10T15:28:12Z")

</div>

I’m glad I found this topic. Had the same problem. But even now I have an error:

```plaintext
INFO -- : Running `bundle install --deployment --verbose --without "development"` with bundler 1.13.7
The --deployment flag requires a Gemfile.lock. Please make sure you have checked
your Gemfile.lock into version control before deploying.

```

So it seems it’s missing that `Gemfile.lock` file.

In the end I fixed it by running the older version of Discourse I had, then:

- enter it with `launcher enter app`
- `cd db/migrations`
- paste the contents of the migration file above into a new file
- `cd /var/www/discourse && sudo -E -u discourse bundle exec rake multisite:migrate`

Now you can upgrade as usual with `launcher rebuild app`.

---

<div class="post-metadata">

### Author: ![codinghorror](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/codinghorror/32/110067_2.png) [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [13 Settembre 2017, 5:40pm UTC](https://meta.discourse.org/t/multisite-upgrade-failed-due-to-missing-database-columns/54239/12 "2017-09-13T17:40:44Z")

</div>


