Rebuilding container fails - with yaml error, but app.yml has no errors


(AstonJ) #1

Rebuilding the container says:

(<unknown>): could not find expected ‘:’ while scanning a simple key at line 50 column 1

this is line 50:

#UNICORN_WORKERS: 3

Running the file through http://www.yamllint.com says the file is valid. Here is the full file.

Here is the full output from the terminal:

Any ideas what’s going on please?


#2

This line should be uncommented I think :thinking:

params:
  db_default_text_search_config: "pg_catalog.english"

  ## Set db_shared_buffers to a max of 25% of the total memory.
  ##
  ## On 1GB installs set to 128MB (to leave room for other processes)
  ## on a 4GB instance you may raise to 1GB
  db_shared_buffers: "256MB"
  #
  ## Set higher on large instances it defaults to 10MB, for a 3GB install 40MB is a good default
  ## this improves sorting performance, but adds memory usage per-connection
  #db_work_mem: "40MB"
  #
  ## Which Git revision should this container use? (default: tests-passed)
  #version: tests-passed

env:
  LANG: en_US.UTF-8
  DISCOURSE_DEFAULT_LOCALE: en

  ## TODO: How many concurrent web requests are supported?
  ## With 2GB we recommend 3-4 workers, with 1GB only 2
  UNICORN_WORKERS: 3

(AstonJ) #3

I tried that… get the same unfortunately :frowning:


(Jay Pfaffman) #4

Try this

git pull
./discourse-doctor 

And see if that sheds any light


(Jeff Atwood) #5

Does doctor do YAML validation? It’s not a bad idea.


(Jay Pfaffman) #6

Not yet. It’s on the list for the future. I’m somewhat of the opinion that if you edit the yml file yourself, then you’re responsible for formatting, but agree that if we could easily integrate yml lint that would be pretty great.

I’m almost certain that this particular problem is with Docker, however.


(AstonJ) #7

Ok here is the output from that:

#./discourse-doctor 
DISCOURSE DOCTOR Sun 15 Jul 14:05:58 BST 2018
OS: Linux rock.myhost.net 3.10.0-514.21.1.el7.x86_64 #1 SMP Thu May 25 17:04:51 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux


Found containers/app.yml

==================== YML SETTINGS ====================
DISCOURSE_HOSTNAME=myforum1.com
SMTP_ADDRESS=mail.myhost.net
DEVELOPER_EMAILS=my@email.net
SMTP_PASSWORD=mypass
SMTP_PORT=25
SMTP_USER_NAME=contact@myforum1.com
LETSENCRYPT_ACCOUNT_EMAIL=

==================== DOCKER INFO ====================
which: no docker.io in (/opt/rubies/ruby-2.2.2/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin)
DOCKER VERSION: Docker version 17.05.0-ce, build 89658be

DOCKER PROCESSES (docker ps -a)

CONTAINER ID        IMAGE                  COMMAND             CREATED             STATUS              PORTS                                        NAMES
7901c9a3a122        local_discourse/app2   "/sbin/boot"        6 weeks ago         Up 12 hours         0.0.0.0:2223->22/tcp, 0.0.0.0:8889->80/tcp   app2
c26d73c29ad4        local_discourse/app    "/sbin/boot"        21 months ago       Up 6 weeks          0.0.0.0:2222->22/tcp, 0.0.0.0:8888->80/tcp   app
8a31761e4d98        local_discourse/app3   "/sbin/boot"        2 years ago         Up 6 weeks          0.0.0.0:2224->22/tcp, 0.0.0.0:8890->80/tcp   app3

7901c9a3a122        local_discourse/app2   "/sbin/boot"        6 weeks ago         Up 12 hours         0.0.0.0:2223->22/tcp, 0.0.0.0:8889->80/tcp   app2
c26d73c29ad4        local_discourse/app    "/sbin/boot"        21 months ago       Up 6 weeks          0.0.0.0:2222->22/tcp, 0.0.0.0:8888->80/tcp   app
8a31761e4d98        local_discourse/app3   "/sbin/boot"        2 years ago         Up 6 weeks          0.0.0.0:2224->22/tcp, 0.0.0.0:8890->80/tcp   app3

Discourse container app is running


==================== PLUGINS ====================
          - git clone https://github.com/discourse/docker_manager.git
          - git clone https://github.com/discourse/discourse-tagging.git
          - git clone https://github.com/discourse/discourse-solved.git

No non-official plugins detected.

See https://github.com/discourse/discourse/blob/master/lib/plugin/metadata.rb for the official list.

========================================
Discourse 1.9.0.beta1
Discourse version at myforum1.com: Discourse 1.9.0.beta1 
Discourse version at localhost: NOT FOUND
==================== DNS PROBLEM ====================
This server reports NOT FOUND, but myforum1.com reports Discourse 1.9.0.beta1 .
This suggests that you have a DNS problem or that an intermediate proxy is to blame.
If you are using Cloudflare, or a CDN, it may be improperly configured.


==================== MEMORY INFORMATION ====================
RAM (MB): 49286

              total        used        free      shared  buff/cache   available
Mem:          48131        6713        1973        3044       39444       37797
Swap:         24559         683       23876

==================== DISK SPACE CHECK ====================
---------- OS Disk Space ----------
df: ‘/var/discourse’: No such file or directory
Filesystem      Size  Used Avail Use% Mounted on
/dev/md2        1.8T  143G  1.6T   9% /
/dev/md2        1.8T  143G  1.6T   9% /var/lib/docker/devicemapper
/dev/md2        1.8T  143G  1.6T   9% /

==================== DISK INFORMATION ====================

Disk /dev/sdb: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x00000000

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1            2048    50333696    25165824+  fd  Linux raid autodetect
/dev/sdb2        50335744    51384320      524288+  fd  Linux raid autodetect
/dev/sdb3        51386368  3907027120  1927820376+  fd  Linux raid autodetect

Disk /dev/sda: 2000.4 GB, 2000398934016 bytes, 3907029168 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0x0000b2ab

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1            2048    50333696    25165824+  fd  Linux raid autodetect
/dev/sda2        50335744    51384320      524288+  fd  Linux raid autodetect
/dev/sda3        51386368  3907027120  1927820376+  fd  Linux raid autodetect

Disk /dev/md0: 25.8 GB, 25752895488 bytes, 50298624 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/md1: 536 MB, 536543232 bytes, 1047936 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/md2: 1974.0 GB, 1973953691648 bytes, 3855378304 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes


Disk /dev/mapper/docker-9:2-29884778-pool: 107.4 GB, 107374182400 bytes, 209715200 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes


Disk /dev/mapper/docker-9:2-29884778-4db9bec1c03bdcdd414144dc2a9aa25bd62a821b582362200d486e5b732577a7: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes


Disk /dev/mapper/docker-9:2-29884778-9e7e394ed38af84802bf64df0f8fc607ef04cba001b88df686836e053a0fe3c4: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes


Disk /dev/mapper/docker-9:2-29884778-918ef45791b35647e6b3526b21b54b32536c080044e7d749d816b99532eb430a: 10.7 GB, 10737418240 bytes, 20971520 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 65536 bytes / 65536 bytes

==================== END DISK INFORMATION ====================

==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Or just send a test message to yourself.
Email address for mail test? ('n' to skip) [my@email.net]: 
Sending mail to my@email.net. . . 
unknown shorthand flag: 'w' in -w
See 'docker exec --help'.

Replacing: SMTP_PASSWORD
Replacing: LETSENCRYPT_ACCOUNT_EMAIL
Replacing: DEVELOPER_EMAILS
Replacing: DISCOURSE_DB_PASSWORD
Replacing: Sending mail to

==================== DONE! ====================
unknown shorthand flag: 'w' in -w
See 'docker exec --help'.
The output of this program may be available at http://myforum1.com/MWRhZDczYTJlMTVlZWJj.txt
You should inspect that file carefully before sharing the URL.

You can examine the output of this script with 
LESS=-Ri less /tmp/discourse-debug.txt

BUT FIRST, make sure that you know the first three commands below!!!

Commands to know when viewing the file with the above command (called 'less'): 
q              -- quit
/error<ENTER>  -- search for the word 'error'
n              -- search for the next occurrence
g              -- go to the beginning of the file
f              -- go forward a page
b              -- go back a page
G              -- go to the end of the file

Could the fact that I have three discourse instances by an issue?

app is the original Discourse forum (fairly quiet)
app2 is the main Discourse forum (fairly busy)
app3 I never went live with (so should maybe just delete that one)

I also did a rebuild a little while back, and I don’t think I changed anything then apart from using the Postgres 9 template as it reported I did not have enough disk space to upgrade to Postgres 10. That meant this time round I had to:

#git pull
remote: Counting objects: 53, done.
remote: Compressing objects: 100% (4/4), done.
remote: Total 53 (delta 34), reused 34 (delta 34), pack-reused 15
Unpacking objects: 100% (53/53), done.
From https://github.com/discourse/discourse_docker
fb62e7a..24b887f master -&gt; origin/master
Updating fb62e7a..24b887f
error: Your local changes to the following files would be overwritten by merge:
templates/postgres.template.yml
Please, commit your changes or stash them before you can merge.
Aborting

So I just committed the changes. That is the only thing I think had changed :confused:


(Jay Pfaffman) #8

Well, it certainly means that discourse-doctor isn’t likely to be able to help on 2 of the 3 installs.

Which one is the broken one?

Also, you need to

git stash
git pull

to get latest

You’ll need to see elsewhere (perhaps in the output of the postgres upgrade script) to see the recommended way to put off the postgres upgrade.


(AstonJ) #9

app2 is the one that’s broken (or at least that is the one I was trying to rebuild after a git pull).

I didn’t do git stash just added the changes and committed them (the only difference was renaming the postgres9 file to postgres - as per the instructions in the terminal because apparently there wasn’t enough room to upgrade to pg10 - but this was done several weeks ago when I last upgraded the forum, and where everything worked fine then).

Currently the forum is online as I just started the existing container.

I could delete the app3 file and then try to upgrade the original discourse instance (app) - and then see what happens?


(Jay Pfaffman) #10

For me it’s too complicated a scenario to offer much more advice here.

I’d solve the space problems and do the full upgrade. I can’t guess how the multiple instances are affecting each other (or not).


(AstonJ) #11

It’s not even getting to the postgres upgrade notice at this point - so I am guessing this may not be related to that.

Is there a way to do a force or clean git pull or something - so that it ignores the change I made to the postgres file and pulls in everything from the official repo? Might be worth trying it from scratch again (I’m fairly sure the only file I changed was the postgres file, but maybe there were other changes it told me to make as well…)


(Jay Pfaffman) #12

https://git-scm.com/docs/git-stash


(AstonJ) #13

When I do that, then git pull it says everything is up to date, however when I go to the postgres.template.yml file it still has the contents of the postgres.9.5.template.yml file.

Should the latest version of that file been fetched with git pull?


(Jay Pfaffman) #14

If you committed changes, then you need to switch back to the master branch, or switch back before your changes.


(AstonJ) #15

Ah thanks. So in case it helps anyone in future:

git log

Which will show you your latest commit’s then do the following for whichever commit you want to revert to, thus deleting changes after that:

git reset --hard 24b887f2db1797d97f3643d32113eac70283176c

Where the last element is the commit, so eg from:

commit 24b887f2db1797d97f3643d32113eac70283176c
Author: Jay Pfaffman <pfaffman@gmail.com>
Date:   Thu Jul 12 18:05:48 2018 +0200

    FEATURE: Add the doctor (#396)

Ok so now the PG file is updated to the latest version… I’ll try to do the upgrade tonight when the forum is quiet, however I’m pretty sure I got the same error as reported in this:

My DB backup is 3.5GB in size, and my disk reports:

/dev/md2        1.8T  144G  1.6T   9% /
devtmpfs         24G     0   24G   0% /dev
tmpfs            24G     0   24G   0% /dev/shm
tmpfs            24G  2.4G   22G  11% /run
tmpfs            24G     0   24G   0% /sys/fs/cgroup
/dev/md1        488M  437M   26M  95% /boot
/dev/dm-1       9.8G  2.8G  6.5G  30% /var/lib/docker/devicemapper/mnt/4db9bec1c03bdcdd414144dc2a9aa25bd62a821b582362200d486e5b732577a7
/dev/dm-2       9.8G  1.8G  7.5G  19% /var/lib/docker/devicemapper/mnt/9e7e394ed38af84802bf64df0f8fc607ef04cba001b88df686836e053a0fe3c4
shm              64M  4.0K   64M   1% /var/lib/docker/containers/c26d73c29ad4c6756e23aeb0cb7b4f6f6dcb3656e128912f1848a9b18796971b/shm
shm              64M  4.0K   64M   1% /var/lib/docker/containers/8a31761e4d98ba36838244d437a31168d9ef2d9b3a2ecf7544581e8f680b082e/shm
/dev/dm-3       9.8G  2.6G  6.7G  29% /var/lib/docker/devicemapper/mnt/918ef45791b35647e6b3526b21b54b32536c080044e7d749d816b99532eb430a
shm             512M  4.0K  512M   1% /var/lib/docker/containers/7901c9a3a1225208d10dbb75b4ad6928bdaf1587ccfc0d596a3bc799d543e71c/shm
tmpfs           4.8G     0  4.8G   0% /run/user/0

(Shall I post in that announcement instead?)


(AstonJ) #16

Just to report that the PG upgrade worked without issue this time. I did get a container cannot be removed as already marked for deletion error but that was rectified by rebooting the box.

Apart from a few CSS breaks, all seems fine now - thanks! :+1: