Fatal error when trying to start Docker (Oracle VM)

Meathead40 · June 11, 2021, 4:16am

Hi all - new to Discourse (looks awesome!) and although I have a some general technical skill, I think I’d classify myself as a relative noob in the docker/linux VM world.

I have a free oracle VM running Oracle Linux. I’ve completed the steps here (https://blogs.oracle.com/developers/install-run-discourse-for-free-in-the-oracle-cloud) and am running the ./discourse-setup step.

I am consistently getting the error message below… tried running the discourse doctor but to no avail. I’m hoping someone can point me in the right direct as to what might be going on here. Much thanks in advance! -Tim

[2021-06-11T04:09:29.733935 #1]  INFO -- : > HOME=/var/lib/postgresql USER=postgres exec chpst -u postgres:postgres:ssl-cert -U postgres:postgres:ssl-cert /usr/lib/postgresql/13/bin/postmaster -D /etc/postgresql/13/main
I, [2021-06-11T04:09:29.735773 #1]  INFO -- : > sleep 5
2021-06-11 04:09:30.320 GMT [54] LOG:  0 8kB is outside the valid range for parameter "shared_buffers" (16 .. 1073741823)
2021-06-11 04:09:30.322 UTC [54] FATAL:  configuration file "/etc/postgresql/13/main/postgresql.conf" contains errors
I, [2021-06-11T04:09:34.739847 #1]  INFO -- : 
I, [2021-06-11T04:09:34.740097 #1]  INFO -- : > su postgres -c 'createdb discourse' || true
createdb: error: could not connect to database template1: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2021-06-11T04:09:34.860266 #1]  INFO -- : 
I, [2021-06-11T04:09:34.860745 #1]  INFO -- : > su postgres -c 'psql discourse -c "create user discourse;"' || true
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2021-06-11T04:09:35.023423 #1]  INFO -- : 
I, [2021-06-11T04:09:35.023810 #1]  INFO -- : > su postgres -c 'psql discourse -c "grant all privileges on database discourse to discourse;"' || true
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2021-06-11T04:09:35.137806 #1]  INFO -- : 
I, [2021-06-11T04:09:35.138325 #1]  INFO -- : > su postgres -c 'psql discourse -c "alter schema public owner to discourse;"'
psql: error: could not connect to server: No such file or directory
	Is the server running locally and accepting
	connections on Unix domain socket "/var/run/postgresql/.s.PGSQL.5432"?
I, [2021-06-11T04:09:35.257190 #1]  INFO -- : 
I, [2021-06-11T04:09:35.257476 #1]  INFO -- : Terminating async processes


FAILED
--------------------
Pups::ExecError: su postgres -c 'psql discourse -c "alter schema public owner to discourse;"' failed with return #<Process::Status: pid 80 exit 2>
Location of failure: /pups/lib/pups/exec_command.rb:112:in `spawn'
exec failed with the params "su postgres -c 'psql $db_name -c \"alter schema public owner to $db_user;\"'"
14ef6216494c846091ea6ce48143e2f25018b9d2579b6d4d0021d605f7f5e145
** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one.

stefanino · August 2, 2021, 2:41pm

Hi @Meathead40
I’m experiencing possibly the same issue.

I’ve managed to setup Discourse on Oracle Free back in June. Now I’ve re run the setup script to change a couple of parameters related to the email settings.
The first error I can trace back to in the log is FATAL: configuration file "/etc/postgresql/13/main/postgresql.conf" contains errors
but in my case that file doesn’t exists. Additionally the postgresql service is not running (well not even available as a service)

Did you manage to solve this issue? Anyone else experienced the same?

cheers
Stef

Falco · August 2, 2021, 6:13pm

Can you share the output of free -m --si ?

Ed_S · August 2, 2021, 6:32pm

Did you look inside the container? Did you also have the log entry about shared_buffers being too small?

In my case:

# /var/discourse/launcher enter app
# egrep shared_buffers /etc/postgresql/13/main/postgresql.conf
shared_buffers = 128MB
exit

stefanino · August 3, 2021, 7:33am

Hi @Falco , this is the output of the command:
total used free shared buff/cache available
Mem: 703 359 86 7 257 207
Swap: 8388 246 8142

stefanino · August 3, 2021, 7:34am

Hi @Ed_S , I do get the following message:

/var/discourse/launcher enter app
Cannot connect to the docker daemon - verify it is running and you have access

Ed_S · August 3, 2021, 8:37am

You should see the same file here:
/var/discourse/shared/standalone/postgres_data/postgresql.conf

Ideally, just before the log entry which says there’s an error in the config, you’ll see a line describing what the error is.

Maybe you’ll find the log at:
/var/discourse/shared/standalone/log/var-log/postgres/current

stefanino · August 3, 2021, 9:06am

thanks for the suggestions. I managed to collect more info.
This is from the log, around the time of the error (previous messages are from days before):

2021-08-02 13:33:16.980 UTC [2419] LOG:  received smart shutdown request
2021-08-02 13:33:28.273 UTC [2419] LOG:  background worker "logical replication launcher" (PID 2442) exited with exit code 1
2021-08-02 13:33:28.344 UTC [2437] LOG:  shutting down
2021-08-02 13:33:28.552 UTC [2419] LOG:  database system is shut down

I also got the shared buffer configuration:

shared_buffers = 128MB     # min 128kB
#wal_buffers = -1            # min 32kB, -1 sets based on shared_buffers

Ed_S · August 3, 2021, 9:31am

Hmm, can we backtrack a bit: wherever you see that FATAL error, what few lines precede it?

stefanino · August 3, 2021, 10:34am

sure, and thanks again for the support.

I’m running the setup on an existing installation for the purpose of changing few configuration parameters. The first part of the script detects the following:

sudo ./discourse-setup 
which: no docker.io in (/sbin:/bin:/usr/sbin:/usr/bin)
which: no docker.io in (/sbin:/bin:/usr/sbin:/usr/bin)
The configuration file containers/app.yml already exists!

. . . reconfiguring . . .


Saving old file as app.yml.2021-08-03-102829.bak
Stopping existing container in 5 seconds or Control-C to cancel.
+ /bin/docker stop -t 30 app
app

Found 0GB of memory and 1 physical CPU cores
setting db_shared_buffers = 0MB
setting UNICORN_WORKERS = 0
containers/app.yml memory parameters updated.

continuing passing on all the configuration parameters and finally building… this should be the clue you’re looking for

2021-08-03 10:30:37.709 GMT [55] LOG:  0 8kB is outside the valid range for parameter "shared_buffers" (16 .. 1073741823)
2021-08-03 10:30:37.713 UTC [55] FATAL:  configuration file "/etc/postgresql/13/main/postgresql.conf" contains errors

Ed_S · August 3, 2021, 10:57am

That’s not a good sign! That number should come directly from the output of

free -g --si | awk ' /Mem:/  {print $2} '

That’s reporting 703 MByte, which is (officially) too small for discourse to work well. If you want to live dangerously (and have no support!) you could edit the magic number 990 in

github.com

discourse/discourse_docker/blob/master/discourse-setup#L152


      
            echo `uname -s`
          }
          
          ##
          ## OS X available memory
          ##
          check_osx_memory() {
            echo `free -m | awk '/Mem:/ {print $2}'`
          }
          
          ##
          ## Linux available memory
          ##
          check_linux_memory() {
            ## some VMs report just under 1GB of RAM, so
            ## make an exception and allow those with more
            ## than 989MB
            mem=`free -m --si | awk ' /Mem:/ {print $2}'`
            if [ "$mem" -ge 990 -a "$mem" -lt 1000 ]; then
              echo 1
            else

I think you need a bigger instance with more RAM.

stefanino · August 3, 2021, 11:12am

yeah. till now the instance was working fine and was installed disabling the memory check already. Somehow related to this post Discourse won't install because I have 960MB of ram and not 1GB - Discourse System Administration - Unix Linux Community

the config file still has the check disabled but I guess it gets overridden? I will try modifying the value as you indicated and report what happens

Ed_S · August 3, 2021, 11:33am

Hmm, much depends on what you did there.

stefanino · August 3, 2021, 12:56pm

I managed to complete configuration and rebuild. The steps have been:

comment out the memory check (#check_disk_and_memory) in /var/discourse/discourse-setup (not sure if this is needed)
run the command sudo ./discourse-setup from the /var/discourse folder (this will generate the file /var/discourse/containers/app.yml and try to go on with the build, which will fail as described above)
edit the app.yml to explicitly set db_shared_buffers: “128MB” and UNICORN_WORKERS: 1
launch a rebuild with sudo ./launcher rebuild app (from /var/discourse folder)

I know this is on the edge of what the requirements are, but I’ll keep a close eye on how the instance will behave.

thanks for the support

Ed_S · August 3, 2021, 1:09pm

Glad you’re sorted. On the linked thread, and the thread here that it refers to, it feels to me far preferable to change the “magic forgiveness” number of 990 to the number that represents your own system. “Disabling the memory check” is actually causing trouble.

It’s clear to me that the Discourse team need to set a lower limit of some value, to draw a line between supported and unsupported configurations, and they’ve formally set it at 1G, with a relaxation to 990. But 960 seems pretty close to 990 to me, and arises for similar reasons. On the other hand, 703 seems a long way different!

stefanino · August 3, 2021, 2:41pm

Hi Ed,
I agree I will change the number. I’m quite surprised of such a low memory. Oracle (Free tier) instance specs indicates 1GB memory, yet the free comes up with kinda 60-70% of it. I’m a bit confused and don’t know the reason for that

Ed_S · August 3, 2021, 2:59pm

It’s been mentioned before - I think Oracle is being naughty, giving you much less than 1G which they round up when describing it:

Edit: see the linked blog:

Perhaps most important missing instruction is don’t use the default server image . Discourse requires 1GB of RAM (or there abouts) and the Oracle Linux image doesn’t leave enough memory for some reason. I don’t know whether CentOS will work, but the Ubuntu image is fine. Just be sure to pick the full install2 rather than the “Minimal” install

jericson · August 3, 2021, 6:41pm

I suspect the default Oracle Linux includes a bunch of stuff that’s not needed to install Discourse. Presumably its main use case is hosting Oracle database servers. Fortunately, the Ubuntu image works fine. My test site/comment host is still running. (Not much activity, though.)

Topic		Replies	Views
Discourse setup on local server: PostgreSQL and setup script issues Installation sql-query	10	1105	January 3, 2024
Discourse Bad Gateway after reboot Installation	16	2021	August 1, 2020
Can't install Discourse with only 10 GB disk, run out of space Installation docker , server-resources	42	12987	September 6, 2017
Error while doing discourse setup Installation	11	2953	October 6, 2017
Self hosting for free (with caveats!) with Oracle Cloud Hosting	106	16415	February 22, 2023

Fatal error when trying to start Docker (Oracle VM)

Related topics