PG throws could not resize shared memory segment error

After upgrading (with ‘launcher rebuild app’ times two) to Postgres 10, we’ve started seeing errors like this in /logs:

Failed to handle exception in exception app middleware :
PG::DiskFull: ERROR: could not resize shared memory segment "/PostgreSQL.682207201" to 283432 bytes: No space left on device :
SELECT "topics"."id" AS t0_r0, "topics"."title" AS t0_r1, "topics"."last_posted_at" AS t0_r2, …

Ideas? Looks like PG is unable to increase the size of a shared memory segment, can this be tuned somewhere?

This is due to the fact that docker by default restricts shared memory size to 64MB which makes pg10 run afoul of this limitation when doing a number of scatter/gather jobs in parallel:

Outside the container:
supermathie@host: ~ $ df -h /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
none             63G  1.2M   63G   1% /run/shm

Inside the container:
supermathie@host: ~ $ docker exec -it postgres-master df -h /dev/shm
Filesystem      Size  Used Avail Use% Mounted on
shm              64M  8.0K   64M   1% /dev/shm

You should raise the limit when launching the postgres container by including the --shm-size=1g parameter.

We’ll look at fixing this in the launcher.

(or as the link suggests, disable parallelism in pg10 with max_parallel_workers_per_gather = 0)

8 Likes

Thanks. How would I do either of those? I searched, found this post (Starting Discourse on a Specific Docker Network - #3 by dlynch158) and following the example added the following line to my app.yml:

docker_args: "--shm-size=1g"

Then ran launcher rebuild app. That didn’t make any difference though, shm is still 64M inside the container.

Thanks,
Gunnar

You need to make sure that’s on your data container.

EDIT:

OK, adding the following:

docker_args: "--shm-size=512m"

to your app.yml (if all-in-one) or data.yml (if separate pg) containers should prevent this from happening. I’ve left the shm size at a conservative 512MB.

Or if you pull latest and relaunch it’ll now start the container with that shm size.

@tgxworld I’ve pushed a change that changes the launcher to always launch with 512MB SHM size under bootstrap or run. Should be safe.

9 Likes

Thanks for adding that to launcher, shm is now 512m inside our container (and yes, we have just the one).

Just FYI, for some reason, adding the

docker_args: "--shm-size=512m"

line to app.yml (and rebuilding) did nothing.

Thanks!
Gunnar

1 Like

Weird - it did when I tested it.

But that’s OK, launcher just takes care of it now.

3 Likes

This topic was automatically closed after 2254 days. New replies are no longer allowed.