I uploaded about 4000 tags to my forum and I am now making some edits. The nearly 4000 tags comprise the totality of tags. I am running three droplets with a load balancer, each containing 4 gigs of ram. My CPU usage is near 0, my memory is averaging 38%, and I have no traffic at all (I am still configuring and testing). At this rate, I will be configuring for a very long time because Discourse is EXTREMELY slow whilst I edit tags. Every other page within the admin area is quite fast. I only have this problem with tags. Any thoughts?
You have 4000 tags. Your browser has to download them and load them all into memory. This is pain that you asked for. Even if there were no performance issue it’ll be insanely hard for a user to choose the right one.
You need to rethink having 400 tags. 400 is probably too many.
2 Likes
Well, that is depressing. I read in a blog that thousands of tags was not an issue and I ran with that, haha. This changes my plans, and possibly the likelihood of me using Discourse.
1 Like
Probably better that you figured this out now, than once your site was live.
Can you not use tag groups and enable particular groups against categories?
2 Likes
Well, I guess I’m wrong. I’d defer to @HAWK, but I don’t see how you can load 4000 tags and be able to select among them easily.
But
1 Like
Yes. I am using tag groups. Every tag is part of a group. Each group has about five to 50 tags within it, depending. Every tag group includes a required tag parent, too. What is taking so long is the process of adding the appropriate tag parent to each tag group since the CSV upload did not provide a mechanism for including parent tags.
When exactly is the slowdown happening? Is it when you’re clicking into the parent tag input?
3 Likes
Yes, that is when. I would not complain, except for the fact that it takes SOO long for the page to respond when I click. There are multiple clicks per group in the whole process, so the combined time can be a bit…multiply that times about a hundred groups and that amounts to quite awhile. Mostly, I became concerned that there was a problem.
There is another issue that becomes a problem in this slowed process. I am not sure if anyone else has experienced this, but when I create a tag in the parent tag field and then click save, the tag eventually disappears. This does not happen right away, but after perhaps five minutes. I can navigate away, and even change pages and return within five minutes and it persists as the parent tag. However, after a bit longer I return to an empty field. The parent tag is not just missing from its field, but from the tag page entirely. I have only had success getting the parent tag to persist when I create it in the group field where the children are, click save, remove the tag from the children/group field, then call it in the parent tag field, saving. I don’t mind doing this, but this step makes for a very time-consuming process of adding over one hundred parent tags–especially when the page responds very slowly at each step.
We should support thousands of tags without issue, that is totally supported. So if we’re running into perf issues we should assign someone to deal with it @sam.
5 Likes
My main concerns were that there might have been something wrong with the way I was adding the parent tags or that there was some “trick” everyone else knew of that I had not yet discovered. Regarding the first issue, I do not have to do the work-around anymore that I described in my previous thread. I discovered that the system generates parent tags that match the group names when the CSV upload is performed, so as long as I do not try to create parent tags with names other than the group names, I can locate the existing parent tag in the parent tag field by pasting the name of the group there. The process is still slow, and the page is still significantly delayed in responding, but there are half as many clicks, copy/pastes as before. I can live with it. The second issue is whether I was just doing something wrong. It sounds like that is not the case.
In summary, we can probably close this ticket because although the tags page is very slow, I know there are many other more pressing problems/requests the discourse team is working on, and I do not want to distract from those now that I can at least press forward.
Thank you all for the responses and attempts to assist, though. As always, I am grateful.
@neil you know the tag system quite well, can you do a bit of testing on local with say 10k tags to see what cracks show up.
4 Likes
I’m following this description of the problem:
I tested 3 cases. In all cases, the /tags page is pretty slow due to rendering. At 4000 tags, it takes about 1 second to load. Navigating away from that page is also slow, probably because Ember is unloading all the components.
10k tags, no tag groups: no problem with tag inputs that I could see.
4k tags, one tag group per 25 tags: no problems
4k tags, one tag group per 25 tags, each tag group has a parent tag: also no problems.
Performance of tag inputs seems the same as if there were a low number of tags.
I tested inputs 1 and 2 in above screenshot, and they were fast. @Destry_Hunt Am I testing your situation or is there something else you’re doing?
5 Likes
You are definitely testing my situation. Did you go back to your tag groups (maybe at least 15 minutes later) to see if all of the parent tags were still present?
The parent tags are gone! I don’t see a job that would delete only the parent tags… mysterious.
But back on topic, I can’t reproduce the performance problem you’re seeing.
1 Like
Well, as you stated in an earlier post, the page is slow. Maybe it just feels extra slow and frustrating to me because I had to create parent tags in child groups, save, remove, then call them in the parent field. Those are a lot of steps to take on a slow page. I figured out that if I uploaded the CSV with group names matching my desired parent tags, then the parent tags would already exist–no need to create them in child groups. That seemed to help me a lot.
You mentioned thay it’s three droplets with a load balancer, can you elaborate any further on the network side of the setup?
I’m assuming you aren’t using anything like Cloudflare on top?
So “Yes, that is when” means “No, it’s the /tags page”?
Would need to be solved with a redesign of the /tags page when there are a large number of tags. Maybe show X most popular tags, and a search field to find more.
3 Likes
I just noticed that there may have been a couple questions from a few days ago that I overlooked. Let me know if any of those questions still require an answer.
Regarding the architecture, I am running three droplets with a load balancer. I am not running any “space” on Digital Ocean, and therefore I am not using their CDN service (nor am I using any other CDN). I plan on adding those later. I am, however, running DNS Made Easy, but I can’t imagine that affecting anything.
Regarding the droplets, they are Ubuntu web servers with 4 gigs of RAM and 80 gigs of hard drive (each). I spun all three up at the same time with a snapshot (image) that I had previously created. All three show good health.
Let me know if there was anything I didn’t cover.
You’re not using Cloudflare, correct?
Their implementation of brotli has done some very strange stuff in the past.