PG::ConnectionBad causes site to temporarily crash

Support
1

In Error Logs, I’m intermittently receiving the following errors. It causes the site to be unreachable for 15-30 seconds until it returns.

Info Error Message:

PG::ConnectionBad (FATAL: the database system is in recovery mode) /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/pg-1.2.3/lib/pg.rb:58:in `initialize’

Env Error Message:

image

One really odd thing I’ve noticed looking at these logs is that it appears to be looking for the folder at /var/www/… but that folder does on exist on the server. Should it? Any help is appreciated.

1 Like
2

If the database is broken, then the site is necessarily broken. You need to figure out why. The most common reasons are lack of ram and disk space. My guess is RAM. How much ram do you have? What does free -h output?

That’s the path inside the container. If you ./launcher enter app you’ll see it there.

3 Likes
3

I think you are right. Here’s the free -h output.

image

131M free does not seem like a lot to me.

1 Like
4

Well, you have 646M available, and 4GB is enough that swap is not typically recommended. How big is your database? How much traffic do you have? Is anything other than Discourse running on the server?

2 Likes
5

Nothing besides discourse is running on the server.

To get the size I ran

./launcher enter app
rails db
SELECT pg_size_pretty(pg_database_size(current_database()));

The resulting output was 4650 MB (1 row)

As for traffic:
We had 1,200 pageviews yesterday, 12,400 in the last week. Approximately 160 users in the last week & about 80 concurrent users at any given moment.

1 Like
6

If you have the spare disk add swap to give yourself some temporary breathing space.

2 Likes
7

I slapped a GB of swap in there. We’ll see what happens during the heavy traffic tonight.

image

1 Like
8

You are the man. The 1 GB swap seems to have ended all crashing problems. I’m honestly glad it was a RAM issue & not a DB issue. I’m gonna increase my RAM amount on the server over the weekend. Thanks so much to all who helped in this thread!

3 Likes
9

After you do, re-run discourse-setup to have it reset the memory parameters.

3 Likes
10

I’m curious if they were tuned correctly to begin with. :man_shrugging:t2:

3 Likes
11

Honestly not sure. Before this we had done a rebuild & server restart after changing the hardware a while back. I would assume after a reboot any system changes would’ve been seen by Discourse.

After installing the swap file there were times when over 4 GB of “memory” was being used, so it appears that was definitely the issue in our busiest times which coincides with the time periods when the server crashed. We upgraded today & the site is running very well now.

3 Likes
12

But they aren’t. There are settings that control, e.g., how much RAM the database can use that are embedded in app.yml. You can edit them by hand or re-run discourse-setup.

2 Likes
13

I reran the discourse-setup function today as I hadn’t been aware of that before. It was new to me.

3 Likes
14

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.