There should be a small action post when a user removes themselves from a group discussion. Currently, there is only one if the author removes themselves.
It will be fixed, I’ll let you know when it’s done. Then you’ll be able to track this action, whether with a Data Explorer query or a custom plugin. 