Loading times Admin User pages

AI-generated summary

Loading Times of Admin User Pages

DeanMarkTaylor reported that loading times for admin user pages are too long, taking around 32 seconds on a desktop browser. After analyzing the profiler entries, it appears that the issue is not with the queries, but with the “Executing action: show” process, which takes around 1750.50 ms.

Codinghorror suggested updating to the latest version of Discourse, as zogstrip had made some changes in this area. DeanMarkTaylor reviewed the code changes and found no relevant updates, but agreed to update soon.

After updating to the latest version, DeanMarkTaylor confirmed that the issue persists. Codinghorror suspected that trust level 3 requirements were being checked for every user on the page and asked neil to investigate.

Neil pushed a fix for the issue, which involved adding a missing index for the query and reducing the number of times the query was run.

Riking suggested splitting the trust level 3 requirements into a separate request.

Clicking though from users to a users /admin/users/{username} page takes too long…
… so long I consider this a bug.

Here is a screen capture of a click from the Admin Users list screen to a users admin page.
This is actually quite a fast example at ~32 seconds on a desktop browser (Chrome 39).

This makes clicking between profiles crazy painful.

I’m currently running:
Discourse 1.2.0.beta4 - https://github.com/discourse/discourse version 3cea85e09a5a86d7a965fb2ffd8b132659b79db0

##Profiler Entries
It doesn’t appear to be the queries that are actually slow - it appears to be the “Executing action: show”.

Here are a few examples

#1

Executing action: show
T+534.5 ms
Reader
0.5 ms

####Query

app/models/trust_level3_requirements.rb:218:in `flagged_post_ids'
app/models/trust_level3_requirements.rb:125:in `num_flagged_posts'
app/models/trust_level3_requirements.rb:36:in `requirements_met?'
app/serializers/trust_level3_requirements_serializer.rb:24:in `requirements_met'
lib/freedom_patches/ams_include_without_root.rb:55:in `include!'
app/controllers/application_controller.rb:207:in `serialize_data'
app/controllers/application_controller.rb:216:in `render_serialized'
app/controllers/admin/users_controller.rb:41:in `show'
lib/middleware/anonymous_cache.rb:119:in `call'
lib/middleware/unicorn_oobgc.rb:95:in `process_client'

SELECT "posts"."id" FROM "posts"  WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:36.359430' AND (spam_count > 0 OR inappropriate_count > 0))   

####Result
Executing action: show — 1750.50 ms

#2

Executing action: show
T+2296.5 ms
Reader
0.5 ms

####Query

app/models/trust_level3_requirements.rb:218:in `flagged_post_ids'
app/models/trust_level3_requirements.rb:138:in `num_flagged_by_users'
app/models/trust_level3_requirements.rb:37:in `requirements_met?'
app/serializers/trust_level3_requirements_serializer.rb:24:in `requirements_met'
lib/freedom_patches/ams_include_without_root.rb:55:in `include!'
app/controllers/application_controller.rb:207:in `serialize_data'
app/controllers/application_controller.rb:216:in `render_serialized'
app/controllers/admin/users_controller.rb:41:in `show'
lib/middleware/anonymous_cache.rb:119:in `call'
lib/middleware/unicorn_oobgc.rb:95:in `process_client'

SELECT "posts"."id" FROM "posts"  WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:38.134558' AND (spam_count > 0 OR inappropriate_count > 0))   

####Result
Executing action: show — 1804.70 ms

#3

executing action: show
T+209.2 ms
Reader
0.8 ms

####Query
app/models/trust_level3_requirements.rb:80:in num_topics_replied_to' app/models/trust_level3_requirements.rb:33:in requirements_met?’
app/serializers/trust_level3_requirements_serializer.rb:24:in requirements_met' lib/freedom_patches/ams_include_without_root.rb:55:in include!’
app/controllers/application_controller.rb:207:in serialize_data' app/controllers/application_controller.rb:216:in render_serialized’
app/controllers/admin/users_controller.rb:41:in show' lib/middleware/anonymous_cache.rb:119:in call’
lib/middleware/unicorn_oobgc.rb:95:in `process_client’

SELECT COUNT(distinct topic_id) FROM "posts"  WHERE ("posts"."deleted_at" IS NULL) AND "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:36.041733' AND post_number > 1)   
244.50 ms	

####Result
Executing action: show — 244.50 ms

##More
There are more too - but simply put they include:
1575.50 ms - SELECT "posts"."id" FROM "posts" WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:40.053569' AND (spam_count > 0 OR inappropriate_count > 0))
2175.90 ms - SELECT "posts"."id" FROM "posts" WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:41.625914' AND (spam_count > 0 OR inappropriate_count > 0))
1800.90 ms - SELECT "posts"."id" FROM "posts" WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:43.889573' AND (spam_count > 0 OR inappropriate_count > 0))
1385.30 ms - SELECT "posts"."id" FROM "posts" WHERE "posts"."user_id" = $1 AND (created_at > '2014-10-10 21:57:45.688083' AND (spam_count > 0 OR inappropriate_count > 0))

3 лайка

You should be on very latest (and we already have a beta5) as I know @zogstrip did some work in this area. Do that and try again.

I have reviewed every code change between my version (6 days ago) and the current version and I see no relevant changes.

I will update very soon but I’m currently tracking down an JavaScript issue right now and need to stay on the my current build whilst debugging.

I’ll come back real soon on this one.

I believe your site has millions of records? So the difference might be stark even compared to here, where the dataset is only of moderate size.

Here are some of the higher row counts for your reference.

        tablename         | rowcount
--------------------------+----------
 schema_migrations        |      502
 single_sign_on_records   |      202
 topic_users              |   268967
 posts                    |   428532
 uploads                  |    21589
 users                    |    10195
 topic_views              |    67712
 user_badges              |    10370
 post_timings             |   560405
 notifications            |    21713
 user_custom_fields       |    20281
 category_custom_fields   |       80
 topic_links              |    71052
 user_profiles            |    10176
 post_custom_fields       |   423228
 user_avatars             |     4054
 topics                   |    66130
 incoming_links           |      658
 post_replies             |     1994
 site_settings            |       62
 category_featured_topics |      241
 categories               |       84
 quoted_posts             |      177
 post_actions             |     3125
 topic_link_clicks        |     1072
 user_actions             |   543406
 incoming_referers        |       31
 incoming_domains         |       14
 category_featured_users  |      378
 email_logs               |    11988
 topic_search_data        |    65926
 topic_allowed_users      |    52137
 user_visits              |     4511
 user_search_data         |    10176
 category_search_data     |       81
 draft_sequences          |   244669
 email_tokens             |    10180
 post_search_data         |   428659
 drafts                   |     2261
 user_histories           |     1592
 groups                   |       21
 group_users              |    20559
 topic_allowed_groups     |        5
 category_groups          |      287
 optimized_images         |     1671
 post_uploads             |    24977
 user_stats               |    10195
 post_revisions           |    36757
 top_topics               |    39896
 category_users           |       12

I can confirm after an update to the latest this is still an issue:

Discourse 1.2.0.beta5 - https://github.com/discourse/discourse version 4fd1fdeb0102450a7a0418dff6e7ebe5a03105e9

It looks to me like trust level 3 requirements are possibly being checked for every user on that page? Can you have a look @neil?

Surely TL3 requirements should only be checked on click or when the scheduled task for promotion runs?

2 лайка

Yeah, those should probably be split out into a separate request.

I pushed a fix for this today. There were two issues: a missing index for that query, and the query was being run multiple times.

https://github.com/discourse/discourse/commit/4c0129ccddb9037467133d37e5a93520a5f86f77

2 лайка