Any progress on this issue? We have a topic with ~500 replies and it’s quite painful to scroll through.
Yes, this was addressed in Discourse 1.5.
Ah we are on 1.7 and the scroll behavior is still quite laggy. It appears to load replies in batches of 20, with a significant wait time between each batch. With 500 replies it can take minutes to scroll all the way through. Are we missing a configuration parameter that might eliminate these pauses?
Why are you scrolling without reading?
-
If you need to jump to the bottom, click the bottom date on the timeline at the right… or press end on your keyboard.
-
If you need to enter the topic at the bottom, click the last post date in the topic list.
3 Likes
I noticed a couple of months ago that our major off-topic/spam thread was becoming slower than other threads. Yesterday a user finally complained about it. At 188.7k replies it consistently takes 4 seconds or longer to perform any navigation in the thread, while other threads take less than a second.
We’re used to shutting these threads down and creating news ones, as this was a problem on vBulletin as well. I’m not a forum architect so I’m not here to say it can (or should) be fixed, and I don’t know how many other forums have topics with so many replies, but I figured I’d put the information here. This is obviously nonessential, but if you ever achieve all of your hopes for Discourse and want one last thing to work on, I guess there’s this. 
1 Like
By default we auto-close topics that reach 10k replies. So did you bypass that default?
Ah, we did. I guess we didn’t consider that there might be consequences. We have three other threads that are longer than 10k posts at the moment, so I guess it’d be good to keep an eye on them.
When auto-closing, is there any sort of thread re-creation that ties the new thread back to the old one? Just curious, as we might re-enable that limit if so. Otherwise we’ll just handle long threads ourselves since they don’t crop up too often.
Links between topics always connect them, this has been true of Discourse since beta. So just add a link to a post pointing to the new topic.
2 Likes
Whats the performance impact of long threads?
I have one that has escalated to 4.5K very rapidly and will keep escalating for at least 36 hours. Expecting thousands of messages, server is screaming.
Will it help anything to close the thread and start a new one, a “part 2” to continue the discussion? Ping @codinghorror, @sam
I would recommend keeping topics below the 10k count for now. Worth closing and doing a part2 when it reaches that point.
2 Likes
Yeah, I am guessing it will be below 10K and cools down tomorrow midnight (trade deadline). Just wondering if it gives me any benefit to cut it as we have our annual super peak.
We keep deferring this work, the bandage to stop the bleeding was a new default that auto-closes topics at 10k replies… but the underlying issue is still present.
Only ignorant brainstorming. If the bottleneck is a large number of posts belonging to a topic, would there be a way to insert a middle layer something like
topic has many blocks
block has many posts
similar to changing a large array into a multidimensional array but for database queries.
I stopped this bleeding in
https://github.com/discourse/discourse/commit/bad6a5142cd103b95a64250b3b6e89539187e279
While I was looking through the client side code, I noticed that the client only cares about the posts in the response payload and doesn’t do anything with the stream (all the post ids).
2 Likes
Maybe only the timeline does, and @sam already optimized that yesterday (by removing date from the timeline), which should also be covered in this topic I think.
From what I’m seeing the client do with the response I’m 99% sure that we don’t need to send down all the post ids when scrolling.
Maybe @eviltrout can review?
Yup we removed the date in
https://github.com/discourse/discourse/commit/9c925a66ff2b8bb19dd0d587606910e87b47e488
However, it didn’t actually affect the performance of the query much when we tried it out. The root cause is due to the fact that we’re over selecting the ids here.
Instead of having the client send the post ids that it wants to the server while scrolling, I plan to have the client send the current position and tell the server whether it is looking up/down the stream. I’ve got it working locally but I’ll need to sort out all the other edge cases.
5 Likes
I’m fine with a few days spent on this because we have deferred the work for two years now…
7 Likes
It’s tricky though, because what about filters? For example best of mode, or when restricting to one particular poster, or when moderators have “gaps” that they can expand to see deleted posts.
There are a lot of edge cases. I’m all for optimizing it, but be prepared for a LOT of regressions and testing if you take this path.
2 Likes
I’m not sure about the challenges with filters at the moment but my plan is to move most if not all of the calculations that we’re doing client side into the server side. The logic for scrolling up and down would simply be given the current position which is determined by the post_number ask the server for the next segment of posts/gaps. The server has knowledge of what filters are applied and can query for the segments accordingly. The client would just render what is given by the server. I’m still in the experimental stages but that is how I think the architecture should be.
8 Likes
Did you want to summarize here tomorrow?