Docker shm_size Option in app.yml

Would it be possible to make the shm_size param an option in app.yml? I ran into this error today when trying to do a vacuum analyze:

ERROR: could not resize shared memory segment

It was trying to increase the shm usage to 1 GB and failed, as it’s only 512 MB out of the box. This param looks to be in the launcher script, but it might be nice to have this as an option (possibly commented out by default) in the .yml file.

You can pass custom docker args to override the default we provide with --docker-args.

I believe it’ll look something like ./launcher start app --docker-args "--shm-size=1024m"

7 Likes

This didn’t work when I tried it unfortunately. I think maybe the container needs to be rebuild to increase the size of the shared memory space? I had to change the value in the hard-coded areas in the launcher script and then rebuild the container for it to actually take effect.

This likely wouldn’t persist across updates/rebuilds either - you’d need to be manually changing it every time.

1 Like

What commands did you run? You’ll definitely have to rebuild the container for the --shm-size option to take effect.

2 Likes

All I did at first was stop the container, then run the launcher start command with that parameter. Then I manually edited the --shm-size params inside the launcher script itself and rebuilt the container. Then it adjusted the value correctly. If you override the param on the command line to rebuild the container, then won’t the change be lost if you perform an update/rebuild through the /admin/upgrade interface? That’s why I thought it might be best to make it a field in app.yml - then it would persist through rebuilds and code updates.

1 Like

Hmm I can see this being useful. @sam What do you think about this?

1 Like

That never rebuilds the container though, so it is safe.

Don’t we already have this per:

docker_args: .... in your yaml.

I’m not sure. Does that overwrite the launcher file itself? I see 3 places where the option is hardcoded:

We should fix this then, but the fix is very very complicated bash script. Feel free to give this a shot.

Our docker args getting function should be responsible for shm-size. Handling user overrides and setting a default.

2 Likes

Hey @Ghan

In the meantime (for testing purposes, see caveat below), you can change this directly with docker after the container is built, as follows:

  • Edit the /var/lib/docker/containers/$CONTAINER_ID/hostconfig.json file directly.
  • For example, change the value for ShmSize in the file above.
  • Stop and restart the container.

In our docker container hostconfig file, it looks like this:

"ShmSize":536870912,

HTH

Caveat: Some people have posted that you must make sure that docker service is stopped before you modify hostconfig.json (or the changes will be overwritten). However, I have not actually tested this. Obviously a launcher solution is better :slight_smile: