PG throws could not resize shared memory segment error

installation
1

After upgrading (with ‘launcher rebuild app’ times two) to Postgres 10, we’ve started seeing errors like this in /logs:

Failed to handle exception in exception app middleware :
PG::DiskFull: ERROR: could not resize shared memory segment "/PostgreSQL.682207201" to 283432 bytes: No space left on device :
SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, …

Ideas? Looks like PG is unable to increase the size of a shared memory segment, can this be tuned somewhere?

3

This is due to the fact that docker by default restricts shared memory size to 64MB which makes pg10 run afoul of this limitation when doing a number of scatter/gather jobs in parallel:

Outside the container:
supermathie@host: ~ $ df -h /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
none             63G  1.2M   63G   1% /run/shm

Inside the container:
supermathie@host: ~ $ docker exec -it postgres-master df -h /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
shm              64M  8.0K   64M   1% /dev/shm

You should raise the limit when launching the postgres container by including the --shm-size=1g parameter.

We’ll look at fixing this in the launcher.

(or as the link suggests, disable parallelism in pg10 with max_parallel_workers_per_gather = 0)

8 Likes
5

Thanks. How would I do either of those? I searched, found this post (Starting Discourse on a Specific Docker Network) and following the example added the following line to my app.yml:

docker_args: "--shm-size=1g"

Then ran launcher rebuild app. That didn’t make any difference though, shm is still 64M inside the container.

Thanks,
Gunnar

6

You need to make sure that’s on your data container.

EDIT:

OK, adding the following:

docker_args: "--shm-size=512m"

to your app.yml (if all-in-one) or data.yml (if separate pg) containers should prevent this from happening. I’ve left the shm size at a conservative 512MB.

Or if you pull latest and relaunch it’ll now start the container with that shm size.

@tgxworld I’ve pushed a change that changes the launcher to always launch with 512MB SHM size under bootstrap or run. Should be safe.

9 Likes
7

Thanks for adding that to launcher, shm is now 512m inside our container (and yes, we have just the one).

Just FYI, for some reason, adding the

docker_args: "--shm-size=512m"

line to app.yml (and rebuilding) did nothing.

Thanks!
Gunnar

1 Like
8

Weird - it did when I tested it.

But that’s OK, launcher just takes care of it now.

3 Likes