# Managing group membership via authentication
**URL:** https://meta.discourse.org/t/managing-group-membership-via-authentication/175950
**Category:** Development
**Created:** [January 13, 2021, 6:50am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950 "2021-01-13T06:50:24Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 13, 2021, 6:50am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/1 "2021-01-13T06:50:24Z")
@david Continuing the discussion from
[https://github.com/discourse/discourse-openid-connect/pull/7#issuecomment-758082333](https://github.com/discourse/discourse-openid-connect/pull/7#issuecomment-758082333)
As you probably can guess, I’m also keen to see a stable solution to this problem, which is not specific to the openid-connect plugin. Let’s see if we can figure out a way forward
To pick up on the point you’ve raised.
> How do you decide which groups to add and remove the user from? And how does that affect manually adding/removing users from groups in Discourse itself?
>
> 1. a user authenticates via OIDC with groups `['group1', 'group2']`
> 2. In the discourse UI, the user is added to `group3`
> 3. later, the same user authenticates via OIDC with groups `['group1']`
>
> Inspecting this as a human, we can see that the final state should be `group1, group3` (group2 should be removed). But I don’t think there is enough state being tracked to make that decision programmatically. Similarly:
>
> 1. a user authenticates via OIDC with groups `['group1', 'group2']`
> 2. In the discourse UI, the user is removed from `group1`
> 3. later, the same user authenticates via OIDC with groups `['group1', 'group2']`
>
> Now what should happen? An admin has explicitely removed the user from `group1` , but OIDC just added them back. This is very confusing UX for the admin. We may need some way to identify groups as ‘externally managed’, and hide all the add/remove UI 🤔
I think there’s a few ways we could handle this issue (they’re non-exclusive)
### Group and token attribute identification
In any implementation, explicit identification of
1. which groups can have their membership managed via authentication; and
2. which attributes in an auth token govern group membership
should be required, whether that be via site settings, group settings or otherwise, and default should be “off”.
### Strict or permissive handling
Handling is “strict” if a user is removed when the group is absent from the identified token attribute. Handling is “permissive” if a user is not removed when the group is absent from the identified token attribute.
The strict/permissive state would be set on a per-group basis. There’s an example of how you could do this in a site setting here: [Handle groups by mattcg · Pull Request #7 · discourse/discourse-openid-connect · GitHub](https://github.com/discourse/discourse-openid-connect/pull/7/commits/1665bdccbb492788f64ba6844ad6f9ae1a41d95e). The same approach could be done via a group setting.
As you suggest, you could disable group membership addition / removal via the groups admin if the handling was “strict”.
### Membership source identification
You could also possibly store the source of the group membership in the `group_users` table to allow for a “mixed” approach within a group, i.e. an admin can’t remove memberships created via auth token.
The more I think about this, the more I’m thinking this should be done via group settings.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [January 13, 2021, 11:36am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/2 "2021-01-13T11:36:49Z")
Thanks @angus for getting this started! I know a lot of people are keen on this functionality!
> [@angus](#):
>
> In any implementation, explicit identification of
>
> 1. which groups can have their membership managed via authentication; and
> 2. which attributes in an auth token govern group membership
>
> should be required
Interesting! I had always imagined that groups would be auto-created during authentication, and that group name would be 1:1 with the group name on the identity provider. That’s how [DiscourseConnect](https://meta.discourse.org/t/13045?silent=true) works at the moment.
But I actually like this more explicit option a lot! It means that people’s Discourse instances won’t be polluted by unneeded groups from their identity provider, and it means that admins can customize group names to their liking. It has a lot of parity with our existing email-domain-based membership
> [@angus](#):
>
> ### Strict or permissive handling
>
> Handling is “strict” if a user is removed when the group is absent from the identified token attribute. Handling is “permissive” if a user is not removed when the group is absent from the identified token attribute.
>
> The strict/permissive state would be set on a per-group basis.
This sounds good from a technical point of view. My only worry is that it could be difficult to explain to users/admins. If we go with your idea of ‘explicit identification’ of auth-managed groups, I think we could just implement the ‘strict’ handling?
When a group is configured to be auth-managed, manually adding/removing is hidden behind a big warning, and it behaves like the ‘strict’ mode you described.
How does that sound?
Aside: we should also make sure that all automatic adding/removing of users is recorded in the group log. That will make it easier for everyone to understand what is happening and why.
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 13, 2021, 10:30pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/3 "2021-01-13T22:30:04Z")
> [@david](#):
>
> I had always imagined that groups would be auto-created during authentication, and that group name would be 1:1 with the group name on the identity provider. That’s how [DiscourseConnect](https://meta.discourse.org/t/13045?silent=true) works at the moment.
Yeah, I feel it’s better to be explicit, for v1 of this feature set at least. I haven’t come across a use case yet where auto-creation has been really needed, i.e. the group couldn’t just be set up and configured by the site admin prior to any claims being handled.
Perhaps we could move to auto-creation as an option after we do the explicit version?
In terms of Group Settings, it would be something like
**Settings Section** : Membership (i.e. the existing section)
**Settings Group Title** : Authentication Management
**Settings** :
- Service: List of authentication services. “All” would be an option. This setting would also function as an “enabled / disabled” state for this feature set. i.e. the default would be “None”.
- Claim: text input to identify id token claim. The “description” for this feature (perhaps in a post here on meta) would explain the supported formats, e.g. boolean, comma-delineated string etc.
- Mode: see below
> [@david](#):
>
> This sounds good from a technical point of view. My only worry is that it could be difficult to explain to users/admins. If we go with your idea of ‘explicit identification’ of auth-managed groups, I think we could just implement the ‘strict’ handling?
Yes, if there was a strict / permissive setting we’d have to explain it concisely. I think the way to approach that would be to focus on “addition” and “removal”. You could perhaps describe it like this
**Setting** : “Mode”
**Option 1** (permissive):
- Label: “Add Members”
- Description: “Allow members to be added to this group on authentication”
**Option 2** (strict):
- Label: “Add and Remove Members”
- Description: “Allow members to be both added and removed on authentication. This will disable manual membership controls.”
The reason I’m keen to keep the “permissive” option is that when working with clients on this kind of thing before, that is the most common use case, i.e.
- The main need is to allow for access to a group depending on a state in an external service
- The need to remove access depending on the state in the external service is more marginal, i.e. people do lose access and should be removed, but this is relatively less important
- There is often a desire to retain the ability to manually control membership in Discourse. When I’ve implemented the “strict” approach this has been “surprising” (i.e. “Why did person X lose membership?”) despite explaination(s) and it functioning (technically) as it should.
> [@david](#):
>
> Aside: we should also make sure that all automatic adding/removing of users is recorded in the group log. That will make it easier for everyone to understand what is happening and why.
Yes this is important, as folks will often wonder “why” someone is added or removed, particularly in strict mode, and we (i.e. “Discourse”) won’t have control over the veracity of the claims the external auth service is making.
Perhaps a new type of “Remove User” and “Add User” action that includes the auth service responsible for the action, i.e. “Remove User ([service name])”.
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 13, 2021, 10:44pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/4 "2021-01-13T22:44:43Z")
One other “gotcha” here is that we’ll need to be clear that this isn’t a cure-all for the use case of “I want a group’s memberships to be based on external service X” as people don’t authenticate that frequently, or at least, according to the needs of the standard use case of that type.
This (auth management) is a necessary piece of handling that kind use case, but to properly serve it you also need to set up event-based integrations, e.g. a webhook receiver (we have a private webhook receiver plugin designed for group management that I hope to open source at some point soonish).
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 21, 2021, 3:32am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/5 "2021-01-21T03:32:01Z")
@david How do we feel about this? If I worked up a PR would you be open to it?
---
### Author:  [@tobiaseigen](https://meta.discourse.org/u/tobiaseigen)
#### Post date: [January 21, 2021, 3:40pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/6 "2021-01-21T15:40:58Z")
This is valuable work you are doing here! 🌻
Just as a functionality comparison, thought I’d share what we do over at the Global Legal Empowerment Network, which uses wordpress and the discourse wp plugin. We have it set up so every time the user is updated in wordpress, it updates in discourse. This includes some special groups (e.g. core member, resource contributor, etc) as well as profile details. We added a hidden user field for “last updated” which helped with troubleshooting and making sure it was working properly.
We lock down those groups managed remotely so users cannot join or leave them when in discourse, but did not see the need to prevent group membership management by staff. I like the look of what you are trying above but it is a bit beyond me tbh.
We have some Discourse for Teams customers who use Okta actively for managing access to all their company apps. They set up roles there as well which are then supported for e.g. providing certain levels of accss to Microsoft Tableau, etc. Okta also is a directory so they manage user avatars, bio, location and other profile type info as well. They’d like to see this profile info updated in Teams externally from Okta as well.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [January 22, 2021, 8:13pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/7 "2021-01-22T20:13:07Z")
> [@angus](#):
>
> In terms of Group Settings…
We should try to keep the super-technical stuff out of the group settings if possible. Having to link to syntax documentation probably means it should be simplified, or relegated to the admin site settings. I think we should leave that part up to each auth plugin to implement, because it can vary so much.
Taking OIDC as an example, that plugin would add a new site setting `openid_connect_roles_claim`. If [using Okta](https://developer.okta.com/docs/guides/customize-tokens-groups-claim/add-groups-claim-org-as/), the admin would configure that as `groups`. I think string array is a pretty standard format for OIDC, but we could explore more complex options here if really necessary.
To receive this information, `Auth::Result` would be given a new `roles` attribute, which accepts a simple array of strings. Core will then take this (in `Auth::Result#apply_user_attributes!`), and load it into a new `UserAssociatedRoles` table with columns (`provider_name`, `user_id`, `role`). This `UserAssociatedRoles` table gives us the ‘membership source identification’ you mentioned in the OP.
I imagine the group settings looking something like this:
Role names would be prefixed with the `provider_name`. Autocomplete would be based on all the existing values in the `UserAssociatedRoles` table, but we would also accept non-autocompleted values in case the role hasn’t been seen by Discourse yet.
The beauty of having a complete `user_associated_roles` table in the database is that admins can do whatever they like with Discourse groups, and memberships will be instantly updated without users having to log in again.
> [@angus](#):
>
> There is often a desire to retain the ability to manually control membership in Discourse
That’s fair. I think it would be best to keep this as simple as possible, at least for v1, so I’d rather not have multiple ‘modes’. How about we have this as the default behaviour:
- Add members when they have a matching role from the auth provider
- Remove members when that role disappears
- Allow adding/removing in Discourse as well
- If an admin tries to remove a user which was added via an auth provider, show a warning “this user may be added again next time they log in”
> [@angus](#):
>
> When I’ve implemented the “strict” approach this has been “surprising” (i.e. “Why did person X lose membership?”)
I think we should try to be secure-by-default here, and remove members when they lose the role on the identity provider. With the membership log improvements, I hope it should be less confusing than the current status-quo.
For sites that really don’t want that, we could have a site setting like `remove_group_membership_when_auth_role_lost` (default true).
> [@angus](#):
>
> One other “gotcha” here is that we’ll need to be clear that this isn’t a cure-all for the use case of “I want a group’s memberships to be based on external service X” as people don’t authenticate that frequently, or at least, according to the needs of the standard use case of that type.
Yeah for sure. We also have this problem with other user metadata like name, avatar, etc. I think at some point soon we need to look at making a `sync_sso` equivalent for other auth plugins. That could be passed all the info which is normally passed via OIDC / SAML / etc., including this new ‘role’ information’. Probably a separate project though.
* * *
How does that all sound @angus? It is a **little** less flexible than separate strict/permissive modes, but I think having just one mode will make troubleshooting / docs / support much easier.
* * *
With that plan, here’s a high-level view of what it looks like from an admin perspective:
### Initial setup
1. Set up okta auth, and [enable the groups claim](https://developer.okta.com/docs/guides/customize-tokens-groups-claim/add-groups-claim-org-as/) on Okta’s end
2. In Discourse, set `openid_connect_roles_claim` to `groups`
### Setting up a new group
1. Create a discourse group as normal. Configure name / full-name / flair / etc. to whatever you like
2. Go to the ‘membership’ preferences, put your cursor in the SSO roles dropdown, and choose a role from the dropdown. If Discourse hasn’t seen someone log in with the role yet, you’ll have to enter it manually
3. Press save, and group membership will be instantly updated with the role info Discourse has cached from previous logins.
4. On future logins, any changes to identity provider roles will be reflected in the Discourse group
5. You can still add / remove users to the group in the native Discourse UI
6. If you later decide you do not want that IDP role to be associated with the group, you can remove it, and all users which were members via that role will be removed
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 25, 2021, 11:01pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/8 "2021-01-25T23:01:07Z")
> [@david](#):
>
> We should try to keep the super-technical stuff out of the group settings if possible. Having to link to syntax documentation probably means it should be simplified, or relegated to the admin site settings. I think we should leave that part up to each auth plugin to implement, because it can vary so much.
I agree.
> [@david](#):
>
> Taking OIDC as an example, that plugin would add a new site setting `openid_connect_roles_claim` . If [using Okta](https://developer.okta.com/docs/guides/customize-tokens-groups-claim/add-groups-claim-org-as/), the admin would configure that as `groups` . I think string array is a pretty standard format for OIDC, but we could explore more complex options here if really necessary.
hm yes, however there are cases in which supporting a boolean claim makes sense. But yes, I agree let’s keep it simple for now by just supporting a string array.
Doing it authenticator by authenticator, I guess we’d also have a `group_sync_enabled?` method in the `Auth::Authenticator` which would be overriden in individual authenticators, and in the generic authenticators upon the existence of a value in the relevant setting.
That actually raises an interesting question. This could also be used by the specific service authenticators, such as facebook (“groups”?), google (“groups”?) and discord (“roles”?). Not sure whether their id tokens would contain such info, but perhaps worth considering from a technical design perspective (i.e. the ability to add this down the track). Not a primary concern for v1 though I don’t think.
> [@david](#):
>
> To receive this information, `Auth::Result` would be given a new `roles` attribute, which accepts a simple array of strings. Core will then take this (in `Auth::Result#apply_user_attributes!` ), and load it into a new `UserAssociatedRoles` table with columns ( `provider_name` , `user_id` , `role` ). This `UserAssociatedRoles` table gives us the ‘membership source identification’ you mentioned in the OP.
Those mechanics sound fine, however I’m curious why we’ve switched to using “roles” instead of “groups” here. Not a huge issue, but want to be sure there’s not something I’m missing.
From a UX perspective I can see confusion arising out of that use of nomeclature (i.e. “roles” and “groups” at the same time), even if it’s grounded in a useful technical distinction. Also possible developer confusion if coming to this without this background context.
I like the seperate table, however perhaps we should have a foreign key association with `user_associated_accounts` instead of `provider_name`? It could be useful in things like cleanup operations. I guess the answer to that question is partly how much we want the group/role association to be interlinked with the associated account from a product perspective. Would there be any downside to linking the two now?
\*\* Edit, I guess we already have the association via the `user_id`.
> [@david](#):
>
> Role names would be prefixed with the `provider_name` . Autocomplete would be based on all the existing values in the `UserAssociatedRoles` table, but we would also accept non-autocompleted values in case the role hasn’t been seen by Discourse yet.
That sounds fine, however I’m just thinking that we also have this information on a provider basis by virtue of the site setting you proposed, which would also cover the case of when the role hasn’t been seen. Perhaps there’s a way to use `Discourse.authenticators` here, i.e. an in-memory list of providers’ and their claims?
> [@david](#):
>
> - Add members when they have a matching role from the auth provider
> - Remove members when that role disappears
> - Allow adding/removing in Discourse as well
> - If an admin tries to remove a user which was added via an auth provider, show a warning “this user may be added again next time they log in”
I think that’s a good compromise. Particularly if we also have the site setting mentioned below.
> [@david](#):
>
> For sites that really don’t want that, we could have a site setting like `remove_group_membership_when_auth_role_lost` (default true).
On a plugin (i.e. authenticator) basis? If so, I’m on board. That should service that need for v1.
Good summary of the user flow 👍 Pending the relatively minor suggestions I’ve made, I’m on board with the direction.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [January 25, 2021, 11:25pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/9 "2021-01-25T23:25:54Z")
> [@angus](#):
>
> I guess we’d also have a `group_sync_enabled?` method in the `Auth::Authenticator`
👍
> [@angus](#):
>
> This could also be used by the specific service authenticators
Yeah for sure. I was particularly thinking of Google, since people like to use that for their GSuite organisations, which I assume have some concept of groups.
> [@angus](#):
>
> we’ve switched to using “roles” instead of “groups” here. Not a huge issue, but want to be sure there’s not something I’m missing.
I guess I was trying to avoid any confusion between “Identity Provider Groups” and “Discourse Groups”… but it’s possible I just made things more confusing by changing the name. Quite happy to stick to “groups” here.
> [@angus](#):
>
> perhaps we should have a foreign key association with `user_associated_accounts` instead of `provider_name`
My thinking was that we still support non-managed-authenticator auth providers, so there might not be a user\_associated\_account record. We can still join into it via the user\_id as you said 👍
> [@angus](#):
>
> have this information on a provider basis by virtue of the site setting you proposed, which would also cover the case of when the role hasn’t been seen.
I’m not sure the site setting would help here. If we’re talking about an array of strings representing “roles”, the site setting wouldn’t specify what the roles actually are. It would just specify how to obtain the array. Does that make sense?
> [@angus](#):
>
> I think that’s a good compromise. Particularly if we also have the site setting mentioned below.
Sounds great 😃
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [January 26, 2021, 1:29am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/10 "2021-01-26T01:29:03Z")
> [@david](#):
>
> I’m not sure the site setting would help here. If we’re talking about an array of strings representing “roles”, the site setting wouldn’t specify what the roles actually are. It would just specify how to obtain the array. Does that make sense?
Yes, you’re right. I was still thinking about a previous implementation where I had included specific groups in the setting itself, but we’re not doing that here, which is fine.
I think there’s enough info here to start on work on the PR, which I’ll start later this week, unless you have any remaining concerns? I’ll update here if anything arises on the way.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [January 26, 2021, 10:54am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/11 "2021-01-26T10:54:58Z")
Sounds great - happy to help with any issues / questions along the way 🙂
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [March 19, 2021, 11:17am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/12 "2021-03-19T11:17:32Z")
Ok, I finally have a work in progress to share here
[https://github.com/discourse/discourse/pull/12446](https://github.com/discourse/discourse/pull/12446)
A few notes.
I’ve gone for the slightly tricker case of google apps hd groups for the initial implementation as I think it helps to think through the possible permutations of this, e.g. the need to account for domain-specific groups from a provider.
To implement that use case, I’ve also had to introduce a new concept of “secondary authorization” at the point of authentication, to allow for incremental authorization. I considered a few different ways to implement asking for specific user’s group permissions (i.e. if they were authenticating with a hd), and this seemed to be the most feasible. I appreciate this is perhaps a bigger change on that front than anticipated, but it’s perhaps worth discussing.
Note that to implement the google hd groups case you need to give the non-admin members of your google apps hd groups delegated admin authority in order to list their groups (via the admin directory API). There’s actually a “beta” pre-built administrator role called “Groups Reader” that works well for this. See [Prebuilt administrator roles | User management | Google Workspace Help](https://support.google.com/a/answer/2405986?hl=en)
The Google implementation works. If you set it up and then authenticate with a hd your google hd groups will be available in the automatic group membership setting, you’ll be added to that discourse group if that hd group is selected, removed if it’s removed (with both actions logged with some specificity), and subsequent users in that google hd group who authenticate will be added immediately.
The details should be evident from the code and the specs. You’ll also notice that I’ve ended up adding three new tables. I attempted a few more "lightweight’ solutions, but they each ended up being more convoluted and inefficient when it came to handling updates to a user’s associated groups and a group’s associated groups. It’s hard to avoid just creating new tables for each. Open to ideas on the data modelling front though, and more generally.
* * *
Some technical todos left (aside from the conceptual/product questions raised above). Suggestions also welcome on this front:
- Perhaps serialize the associated\_groups `label` (instead of modelling on the client).
- Add missing specs and qunits
- Perhaps move user\_associated\_group / group\_associated\_group creation/destruction to a background job, as with large numbers this could be slow.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [March 19, 2021, 3:31pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/13 "2021-03-19T15:31:16Z")
This is looking very cool!
> [@angus](#):
>
> I’ve also had to introduce a new concept of “secondary authorization” at the point of authentication
I am a **little** hesitant about the ‘secondary authorization’ thing, and also the `provider_domain` column. Can you elaborate some more on why they’re needed? Seems like they’re quite google-specific… is there a reason we can’t request the `admin.directory.group.readonly` scope during the first auth request? And maybe just prefix the group name with the domain? (or exclude the domain entirely, since I assume people will only be using this with a single google ‘hosted domain’?)
> [@angus](#):
>
> You’ll also notice that I’ve ended up adding three new tables. I attempted a few more "lightweight’ solutions, but they each ended up being more convoluted and inefficient when it came to handling updates to a user’s associated groups and a group’s associated groups
Yeah totally happy with 3 tables here - it keeps things cleaner
> [@angus](#):
>
> - Perhaps serialize the associated\_groups `label` (instead of modelling on the client).
Agreed 👍
> [@angus](#):
>
> - Perhaps move user\_associated\_group / group\_associated\_group creation/destruction to a background job, as with large numbers this could be slow.
We need to be careful here. Any group memberships need to be assigned **before** the user loads the site for the first time. Otherwise, they won’t see group-specific things the first time they log in (e.g. secure categories). So I think changes to `user_associated_group` records should be processed synchronously.
But for `group_associated_group` record changes, I think you’re right. Changes there could affect thousands of users, so they will need to be processed [in\_batches](https://www.rubydoc.info/github/rails/rails/ActiveRecord%2FBatches:in_batches). I think I’d start by doing it synchronously, with a loading spinner in the UI. That way, admins will clearly be able to see when it’s running/done.
If we see it getting close to 30 seconds (the unicorn request timeout), then we might need to think about a background job.
We might also need to think about adding some DistributedMutex locking here. For example, if a user logs in **while** a `group_associated_group` change is being processed, what happens? Happy to discuss this kind of stuff on GitHub once we get the overall architecture finalised.
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [March 19, 2021, 3:48pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/14 "2021-03-19T15:48:58Z")
Following this with interest — and throwing this in as related [Does `sso overrides groups` work with Oauth2?](https://meta.discourse.org/t/does-sso-overrides-groups-work-with-oauth2/175606)
For my use case, I’d be perfectly happy with the [Discourse Connect](https://meta.discourse.org/t/13045?silent=true) behavior to be implemented for other auth providers.
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [March 22, 2021, 12:24pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/15 "2021-03-22T12:24:05Z")
Great 🙂 we’re getting there.
> [@david](#):
>
> I am a **little** hesitant about the ‘secondary authorization’ thing, and also the `provider_domain` column. Can you elaborate some more on why they’re needed? Seems like they’re quite google-specific… is there a reason we can’t request the `admin.directory.group.readonly` scope during the first auth request? And maybe just prefix the group name with the domain? (or exclude the domain entirely, since I assume people will only be using this with a single google ‘hosted domain’?)
I’ll deal with each in turn.
### Incremental authorization
The reason you can’t request group permissions on the first request is that you don’t know who’s signing in, or what they’re comfortable with sharing. You could restrict google associated group mapping to people using the `google_oauth2_hd` setting, however this would limit the scope of the feature a fair bit. Having a team that uses google apps in addition to “public” users who also want to use google auth is relatively common.
\*edit I should clarify, that if you ask for a groups scope and the user cannot give it (e.g. their HD has not delegated authority to non admin users as described above) then the auth will fail. You can’t request optional scopes alongside required scopes.
Moreover, that approach, i.e. asking for a groups scope up front as a standard practice in implementing this in the various authentication methods, would be arguably more google-specific than the alternative, as you don’t always have the equivalent of the hosted domain system to restrict login to. For example, I could be wrong, but I don’t think there’s a way to restrict Github OAuth2 login to a specific github organisation.
In other words in a number of cases you’d be left with the choice of asking everyone who uses that auth method to grant the relevant `groups` scope, or not using the feature. That approach may work in some contexts, but not in many. This incremental authorization approach gives different auth methods more flexibility in implementing the feature.
It’s true Google has been championing incremental authorization in the OAuth2 space, for example the working papers on the subject have all been written by a googler
> **[OAuth 2.0 Incremental Authorization](https://datatracker.ietf.org/doc/html/draft-ietf-oauth-incremental-authz-04)**
>
> OAuth 2.0 authorization requests that include every scope the client might ever need can result in over-scoped authorization and a sub- optimal end-user consent experience. This specification enhances the OAuth 2.0 authorization protocol by adding...
However the concept is not Google-specific (other people don’t necessarily call it “incremental authorization”). It’s relatively common in different forms in mobile apps, and is being adopted in OAuth2 by other providers. For example, here’s [Facebook’s docs on the same subject](https://developers.facebook.com/docs/facebook-login/permissions/requesting-and-revoking/#optimizing).
You’re probably familiar with the field already, but it is considered “good practice” to
- tell people you’re about to ask for more permissions than the standard basic info;
- and maybe why.
If the user clicked “Sign up with Facebook” in a Discourse login form, and then, in addition to their email, they were also asked for access to their facebook groups, they might drop off. Facebook puts it like this
> As a general rule, the more permissions an app requests, the less likely it is that people will use Facebook to log into your app. In fact, our research shows that apps that ask for more than four permissions experience a significant drop off in the number of completed logins.
This raises the question of whether it’s a good idea to be asking for additional scopes at the point of authentication in the first place, and basically I concluded that we don’t really have any other good options. There’s the need to have immediate access to groups in some scenarios (as you alluded to), and there’s also just the reality that without it being asked up front many users would not take an additional step to authorize their groups on a service, in say their profile, or the groups page.
It has to be done at the point of authentication, which brings us back to the issue referenced above, and why I implemeted a “secondary authorizatiton” system. It is indeed intended to be a lightweight “system” insofar as it is relatively easy for another service, say Facebook or Github, to implement a secondary authorization request to obtain access to the user’s groups after they have authenticated and optionally passing a certain test relating to their basic info.
Each provider just needs to
- Return a result with `secondary_authorization_url`
- Use the `state` parameter to detect which authorization request the user is up to
- Provide a `omniauth_secondary_authorization_description` for `users/omniauth_callbacks/secondary_authorization.html.erb`. e.g. this is the one for google, which the user sees before they confirm the secondary authorization redirect
None of these parts are google-specific.
What I would like to do here is allow the user to say “no” to the secondary request, and still authenticate. In the Google Apps HD scenario this isn’t really an issue as if their account is part of a hosted domain, then they’re unlikely to want to, or be in a position to, say no. However it should be accomodated to allow for the full range of authentication secnarios here I think.
Finally, it should also be noted that secondary authorization is not _required_ for associated\_groups to work. An auth provider can just ask for the groups scope up front, and then add the groups to the auth result after receiving the first response. Indeed, we should probably build that in as an option to the basic oauth2 and openid connect plugins.
### Provider Domain
I do think there needs to be some form of secondary identifier in the `associated_groups` table that’s readable by the site admin. There are a number of scenarios in which a group name by itself may not be enough. There’s the possibiity of name conflicts across each services’ equivalent concept, e.g.
1. multi-domain google group management (you can have multiple domains in one workspace too)
2. mutli-org github group management
3. multi-server discord role management
etc
We could change `domain` to `namespace` perhaps. We could include it in the group’s `name`, but would that give as any advantages? It may be useful to query by the “domain” or “namespace” at some point. Yes, perhaps `namespace` would be better than `domain`.
The reason it needs to be “admin readable” is that it’s used in the label seen by the admin in the groups UI, partly for disambiguation purposes.
I’m mulling whether it makes sense to attempt to also store a `provider_id` here (if one exists). Could be useful down the track perhaps.
> [@david](#):
>
> We need to be careful here. Any group memberships need to be assigned **before** the user loads the site for the first time. Otherwise, they won’t see group-specific things the first time they log in (e.g. secure categories). So I think changes to `user_associated_group` records should be processed synchronously.
>
> But for `group_associated_group` record changes, I think you’re right. Changes there could affect thousands of users, so they will need to be processed [in\_batches](https://www.rubydoc.info/github/rails/rails/ActiveRecord%2FBatches:in_batches). I think I’d start by doing it synchronously, with a loading spinner in the UI. That way, admins will clearly be able to see when it’s running/done.
>
> If we see it getting close to 30 seconds (the unicorn request timeout), then we might need to think about a background job.
>
> We might also need to think about adding some DistributedMutex locking here. For example, if a user logs in **while** a `group_associated_group` change is being processed, what happens? Happy to discuss this kind of stuff on GitHub once we get the overall architecture finalised.
Yup, agree with all this, and thanks for the tips. I’ll give this part a go and we can discuss it further on the PR.
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [May 19, 2021, 9:02am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/16 "2021-05-19T09:02:55Z")
@david I’ve just pushed a few updates on this one including
- DistributedMutxes and `in_batches` in `group_associated_group`
- Acceptance tests (already had rspec)
Will need some further work no doubt, but it’s currently working according to spec and all tests are passing. Take it for a test drive, let me know what you think and what changes you’d like.
[https://github.com/discourse/discourse/pull/12446](https://github.com/discourse/discourse/pull/12446)
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [May 20, 2021, 6:08am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/18 "2021-05-20T06:08:29Z")
Perhaps worth marking it as a non draft for now?
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [November 4, 2021, 9:31pm UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/19 "2021-11-04T21:31:02Z")
Hi @angus! I’m curious if you’ve made any further progress on this? I’m very interested in the simple “strict” behavior, as I understand it, and since we control our Oauth2/OpenID connect provider, I’m not worried about the “secondary authorization” case. Any chance that something like that could land sooner?
If it’s any help, our environment is documented here: [Infrastructure/Authentication - Fedora Project Wiki](https://fedoraproject.org/wiki/Infrastructure/Authentication), and I’ve configured DIscourse to request oauth2 scope `openid profile email https://id.fedoraproject.org/scope/groups`
Basically, all I want is:
- Leave the trust level and staff groups alone
- Add the user to any existing Discourse groups in the list from SSO that are in the list if they’re not there already
- Remove the user from any groups they are in that aren’t in the list
I freely admit I don’t understand all the intricacies… is there a complication I don’t understand?
---
### Author:  [@angus](https://meta.discourse.org/u/angus)
#### Post date: [November 5, 2021, 2:24am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/20 "2021-11-05T02:24:27Z")
I’ve put aside time this weekend to work on this Matt. I’ll have an update next week, probably on the PR on github.
[https://github.com/discourse/discourse/pull/12446](https://github.com/discourse/discourse/pull/12446)
---
### Author:  [@mattdm](https://meta.discourse.org/u/mattdm)
#### Post date: [November 5, 2021, 11:23am UTC](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950/21 "2021-11-05T11:23:43Z")
Awesome — thank you so much. I don’t mean to nag, but Discourse Support suggested that posting in this topic would be the best way to see the current state of things. 🙂
I’m excited for your work on this, because there is so much we can do with the Fedora discourse sites once we have this that we just can’t right now!
[Next page](https://meta.discourse.org/t/managing-group-membership-via-authentication/175950.md?page=2)