The lovely @joffreyjaffeux deployed an update today that should take care of this — you’ll need to be on the latest code, but when I tested, it worked like a charm.
After leaving a group chat, the user won’t be listed as a member, receive notifications for the group chat, or have access to the group chat directly (i.e. via URL) — until someone adds them back to the group.
Please take a look and let us know if you notice any issues!