Using the api for user management (the 100 user limit)

api

(eriko) #1

So I am trying to use the api for some management tasks and I can not find a way using the api to get all the users.

I am trying to do two things:

  • Using an external list + a white list suspend users that should no longer have active accounts.
  • Trigger a SSO log out to force everyone to have to re login (this is needed due to some upstream data issues that I do not control)

Two possible options:

  • The api call of list_users(type) only returns 100 users. The underlying call of users = ::AdminUserIndexQuery.new(params).find_users has a default limit of 100 so in theory this could return everyone. Though performance would probably be horrible.
  • The second way would be to use the export users functionality in the interface and while this would work it requires human intervention which is what I am trying to avoid.

Is there anything I am missing? Thanks.


User lists seem limited or buggy in Admin
(eriko) #2

So I guess there are no other options? I will redo the functionality as a custom plugin then.


(Dan Dascalescu) #3

You’re not the only one dumb-founded by the up-to-100-users limitation.


(Dean Taylor) #4

I recently highlighted way (work-a-round) of getting a list of all users via the API here (available to admins):


(Jared Reisinger) #5

I’ve just run into this problem as well. I have a user-account management scenario where I need to cross-check all Discourse users against a list of known valid (and invalid) employees.

It seems like the “right” thing is for the core admin users list API to support/expose the “limit” and/or “page” parameters. The limit value is there internally, and hard-coded to 100. It should be fairly easy to get this exposed. I have no idea how hard it might be to add pagination support. (I’m not really a Ruby dev, but I can probably get something cobbled together.)

I’m going to start working on exposing the “limit”, and just wanted to let folks know in case anyone had strong opinions about how it should work.


(Jeff Atwood) #6

Any thoughts here @blake ?


(Blake Erickson) #7

I personally am not a huge fan of increasing the limit pass 100. Usually I have seen limit in increments of 10, 25, 50, 100, but I don’t think having a limit of 1000, 2000, etc is a good idea

I would focus on doing something similar to the public users endpoint and add a page attribute.


(Jared Reisinger) #8

I do understand the general concern, that allowing a caller to pass “limit 999999” opens up a potential DoS-style attack. Is that one of your concerns? Does it matter that this is an admin-only API, and that check/requirement happens first, before the database is even queried?

Also, are you more concerned about the UI experience if someone passes ?limit=1000 in a browser than programmatic access of the API? (They both are things to worry about, but in my opinion, if someone has explicitly added ?limit=1000 to the url in their web browser and gets sluggish UI, they asked for that experience.) So it turns out that while the ?limit= parameter works on the API, the web UI somehow ends up doing a re-request that ignores it. I have no idea how or why… looking at the console, I can see the initial request with the custom LIMIT, but then there’s a second request from Admin::UsersController for the default LIMIT 100. So, at any rate, adding ?limit= to the API doesn’t actually seem to expose it in the web UI.

I do agree that adding pagination is probably better in the long run, but given that I don’t really have any/much experience with active record, the way that ruby does SQL calls, etc., making the change to adjust an already-existing “limit” value seemed a lot more approachable to me than adding pagination from scratch where none whatsoever exists.


(Blake Erickson) #9

I still think increasing the limit past 1000 or allowing an arbitrary amount is just bad API design and I want to protect admins from themselves. A lot of forums have well over 1000 users so at some point you have to cap it at something or they will create a query that will effect the performance of the rest of the app. So adding pagination is definitely the way to go.

This will have to be done in 2 phases. One for the rails api side and the other for the ember ui side.


(Jared Reisinger) #10

Gotcha…

I put up a PR in parallel with this discussion (Expose limit for admin/users/list API by JaredReisinger · Pull Request #4861 · discourse/discourse · GitHub), but I’ll go ahead and close it. I’d love to pitch in and add pagination, I’m just not sure how long that might take. (But I will take a look!)


(Blake Erickson) #11

would exporting users via csv temporarily solve your problem until pagination can be added?


(Jared Reisinger) #12

Probably not. I have a fully-automated job that runs every day and pulls account information from a bunch of 3rd-party services we use (like GitHub, Slack, etc.), and compares against a known list of “valid/not-valid” users. Waiting for a response to a request is one thing, using the export API in this context feels as wrong to me as ‘limit=1000’ does to you. The existing API is so close to being perfect for me; we only have about 250 users, and I would totally take the risk of asking for a large number of users at a time when I know that our Discourse site isn’t busy.

Yes, I could theoretically figure out how to programmatically kick off the export, then wait for it to complete, pull down the data, unzip, parse the CSV, etc… but I think it’ll actually be easier to re-ramp-up on Ruby and figure out how to add pagination. :smile: (I’ll keep the export idea in my back pocket just in case I get completely stumped.)


(Jeff Atwood) #13

This is the kind of thing we could cough prioritize if it was for a paying customer on a business or higher plan https://discourse.org/buy

Just throwing that out there :wink:


(Jared Reisinger) #14

Good news… pagination was as easy as "look for ‘page’ param, use it as an .offset() after the existing .limit()". I just completely overlooked this when I took a glance at the regular user page before. I should have a PR that adds pagination support (for the API, at least) shortly.


(Jeff Atwood) #15

Perhaps we should make this clearer in the docs somehow cc @blake


(Jared Reisinger) #16

Here’s the PR: Add pagination to /admin/users/list API by JaredReisinger · Pull Request #4866 · discourse/discourse · GitHub

And the docs, as well: Add `page` parameter to /admin/users/list API by JaredReisinger · Pull Request #5 · discourse/discourse_api_docs · GitHub