When you are generating your SSO payload, you can set the add_groups field based on your user’s email, where add_groups is a comma delimited string of groups. I haven’t seen any documentation on this but the code is pretty clear:
That was our workaround to this issue and it has been working so far.