Docker base image with PostgreSQL 10 released

tgxworld · April 4, 2018, 7:08am

launcher has been updated to use the new Docker base image and postgres.template.yml which would use PostgreSQL 10 by default instead of Postgres 9.5.

As long as your container’s config is using the default postgres.template.yml, the upgrade of the PostgreSQL data directory would happen automatically the next time you do a ./launcher rebuild. In the event that the PostgreSQL data directory fails to upgrade, please follow the instructions printed by ./launcher rebuild or you may refer to the instructions in this template. If following the instructions does not work for you, please post the full log of ./launcher rebuild here so that we can help you restore your instance into a working state.

blattersturm · April 14, 2018, 6:08pm

I, [2018-04-14T18:05:10.944387 #15]  INFO -- : Upgrading PostgreSQL from version 9.5 to 10
WARNING: Upgrading PostgresSQL would require an addtional 19M of disk space
Please free up some space, or expand your disk, before continuing.

Earlier I got an error saying ‘requiring addtional(sic) 21M of disk space’, I made 2GB of additional space, and now it still wants 19 MB.

How much disk space does this process need? In addition, I can’t easily expand disk space on this physical server whatsoever, nor free up much more either (uploads/data use practically all disk space).

codinghorror · April 14, 2018, 7:17pm

Is this a bug in the space check @tgxworld?

tgxworld · April 14, 2018, 11:35pm

Can you provide me with the output of df -h?

pfaffman · April 23, 2018, 10:03pm

If data and web containers are separate is something like this going to work? And is there anything special if it’s multisite?

./launcher rebuild data
./launcher rebuild multi

Also, I don’t know if you ever got an answer on this so I partitioned a disk in 100, 1000, 10000, 100000 MB.

df -h *
Filesystem      Size  Used Avail Use% Mounted on
/dev/sdd4        96G   60M   92G   1% /media/pfaffman/1303298c-1357-4534-a616-ffc24f4d35dd
/dev/sdd1        93M  1.6M   85M   2% /media/pfaffman/14d764e1-d27d-4961-91e8-815fd0b84be9
/dev/sdd2       969M  1.3M  902M   1% /media/pfaffman/4b61b56c-8730-420a-b3b9-9e50e273e3a2
/dev/sdd3       9.5G   22M  9.0G   1% /media/pfaffman/b3bb897d-57e9-403f-ab96-fe89386c8592
/dev/sdb3        46G   26G   19G  59% /
/dev/sr0         49M   49M     0 100% /media/pfaffman/SAMSUNG SSD

And here it is without -h

pfaffman@balloon:/media/pfaffman$ df  *
Filesystem     1K-blocks     Used Available Use% Mounted on
/dev/sdd4      100660656    61044  95463228   1% /media/pfaffman/1303298c-1357-4534-a616-ffc24f4d35dd
/dev/sdd1          95054     1550     86336   2% /media/pfaffman/14d764e1-d27d-4961-91e8-815fd0b84be9
/dev/sdd2         991512     1264    922664   1% /media/pfaffman/4b61b56c-8730-420a-b3b9-9e50e273e3a2
/dev/sdd3        9948012    22488   9397140   1% /media/pfaffman/b3bb897d-57e9-403f-ab96-fe89386c8592
/dev/sdb3       47930248 26550824  18921620  59% /
/dev/sr0           49780    49780         0 100% /media/pfaffman/SAMSUNG SSD

Here’s what someone (I don’t think it was me) did for the space check in discourse-setup

  free_disk="$(df /var | tail -n 1 | awk '{print $4}')"

ixeft · April 23, 2018, 10:11pm

Other problem with this new base image :

We noticed that ./launcher rebuild with postgres 10 fail when the variable LANG is setted to fr_FR.UTF8 in app.yml. postgres just won’t boot up.

It wasn’t the case with postgres 9.5, but maybe we shouldn’t have changed the locale (in this case, it might not be useful to put in the sample app.yml this env variable)

sam · April 23, 2018, 11:14pm

@tgxworld can you give this a shot?

tgxworld · April 23, 2018, 11:36pm

Is there somewhere using the -h option? I’m not sure why this is a concern.

As long as you’re using the postgres.template.yml, the upgrade will work.

pfaffman · April 23, 2018, 11:40pm

I thought that you were using -h at some point, because

But maybe you fixed the error described above:(that suggested there was a problem detecting disk space correctly) already and I missed it. Sorry if my post was confusing and not helpful.

tgxworld · April 23, 2018, 11:50pm

I wanted to see the user’s disk space usage and -h makes it easier to read for me. The source code always has the answer

github.com

discourse/discourse_docker/blob/56032fc353419695f0ddae20884f83da61dccf88/templates/postgres.template.yml#L64


      
          
          - file:
             path: /root/upgrade_postgres
             chmod: "+x"
             contents: |
               #!/bin/bash
               PG_MAJOR_OLD=`cat /shared/postgres_data/PG_VERSION`
          
               if [ ! "10" = "$PG_MAJOR_OLD" ]; then
                 echo Upgrading PostgreSQL from version ${PG_MAJOR_OLD} to 10
                 free_disk=$(df /var | tail -n 1 | awk '{print $4}')
                 required=$(($(du -s /shared/postgres_data | awk '{print $1}') * 2))
          
                 if [ "$free_disk" -lt "$required" ]; then
                  echo "WARNING: Upgrading PostgresSQL would require an addtional $(numfmt --to=si $(($required - $free_disk))) of disk space"
                  echo "Please free up some space, or expand your disk, before continuing."
                  echo ''
                  echo 'To avoid upgrading change "templates/postgres.template.yml" TO "templates/postgres.9.5.template.yml" in containers/app.yml'
                  exit 1
                 fi

tgxworld · April 26, 2018, 2:38am

Hmm that is a good point actually. @sam Do you know if Discourse search would be affected if the LANG is configured differently? Otherwise, I think we should just bake the ENV into postgres.template.yml.

sam · April 26, 2018, 2:40am

Yeah you basically want fulltext search to be configured with the right defaults otherwise the tokenizer does not work right and search results suffer. Changing this after the fact is real hard.

tgxworld · April 26, 2018, 3:47am

You can try rebuilding again with

https://github.com/discourse/discourse_docker/commit/d4eb8c3f5b73c0e46cd7ad784567fe811641cd50

but do note that the upgrade will fail if you change the locale when trying to upgrade. You’ll then need to upgrade postgres via pg_dump, there will be instructions printed when the upgrade fails.

ixeft · April 26, 2018, 10:42am

Everything went well when rebuilding with fr_FR locale, thank for the fix

So I assume that the search will be less good for the few days we used the wrong local ? Is that the only downside ? Good to have this information about the LANG variable.

sam · April 26, 2018, 10:45am

I think we need to test this cause we changed stuff over the years and it may no longer be a big issue. The bottom line is that we want pg using french stop words and french snowball when tokenizing for search, now I think we have a bunch of internal changed that help force things but we may have missed some spots. Setting to fr_FR is safest but it may not be needed.

pfaffman · June 1, 2018, 10:30pm

I found what might have been the problem above. I did an upgrade of a huge database on a small disk yesterday and that free_disk check fails if /shared isn’t the same partition as /.

https://github.com/discourse/discourse_docker/pull/394

snakeninny · June 23, 2018, 2:38pm

My website is not accessible after ./launcher rebuild app and here’s the full log. Thank you!

Falco · June 23, 2018, 3:03pm

The rebuild is looking good. Are you sure it’s not your reverse proxy?

Because Discourse is definitely up and running http://bbs.iosre.com:30080

snakeninny · June 23, 2018, 3:19pm

I’m not sure; how do I diagnose that? Before rebuild, everything worked as expected though

Mittineague · June 23, 2018, 7:15pm

Did you rebuild twice? On my localhost a first rebuild updated Postgres, and a second was needed to get things copacetic.

Topic		Replies	Views
Stuck in a loop of freeing up space and filling it up again when rebuilding Installation server-resources	33	3635	June 8, 2024
Not enough disk space for upgrade when Postgres is upgraded Installation server-resources	8	930	June 8, 2024
Failed upgrade from 2.5.0beta4 to 2.5.0beta5 Installation	6	1007	June 29, 2020
When rebuild, Discourse removes PostGreSQL 10 and reinstalls version 9.5 Installation	6	945	May 8, 2020
Upgrade failed due to unclean database shutdown Installation	21	4164	November 9, 2017

Docker base image with PostgreSQL 10 released

Related topics