work_mem is used several times per query.
So a complex query (like suggested topics) can use 5~10 times that, and this for each query running and you are fast into swapping territory.
The allow memory per work that Jeff cited, is more on don’t set 10 unicorn workers if you have a 10CPU server with 1GB ram.