Users losing group status?

This seems unlikely. When the nonce is expired the user should not be updated in any way. SSO login just fails.

Are you sure that the add_groups and remove_groups parameters are being set correctly? You can check this in your logs by filtering for ‘User was logged on’ and looking at the ‘info’ tab.