Unable to "Delete All Posts" when there are posts that belong to megatopic

[RobinTS]

I’m experiencing a problem on my community. I’m unable to “delete all posts” of certain users. From what I can tell it’s only happening to older accounts with a lot of posts. Newer accounts don’t face this problem.

I’ve checked the site settings and I’ve increased the limit on the delete all posts to 10 000.

I’m currently trying to delete a user with about 900 posts. The user does not want to be anonymized and is threatening legal action if the account and posts are not removed. I press the delete all button and then I get this message after about a minute:

I see this in my browser console, 502 on the post:

I’ve looked at the logs on my server, and I see this appear after I try to delete:

TypeError: undefined is not an object (evaluating ‘s.users’)
Url: https://myserver.com/assets/ember_jquery-d430881a3fb1403871256e5a02423c4b20a78793685e92088613ca9a701baf88.js
Line: 9
Column: 8994
Window Location: https://myserver/t/topic/64828/2502

Is there anywhere I can look to find the source of the problem? Logs?

[simon]

@RobinTS

Is there anywhere I can look to find the source of the problem? Logs?

Try going to Admin / Logs / Error Logs. Then try deleting the posts again. Hopefully there will be a log entry for the post that is causing the problem.

[RobinTS]

This is the error that appears in the log after I try and delete all posts:

TypeError: undefined is not an object (evaluating ‘s.users’)
Url: https://myserver.com/assets/ember_jquery-d430881a3fb1403871256e5a02423c4b20a78793685e92088613ca9a701baf88.js
Line: 9
Column: 8994
Window Location: https://myserver.com/t/topic/64828/2502

[RobinTS]

I’m noticing that it deletes a few posts (about 5), before the error dialog appears. Don’t know if this is relevant.

[simon]

@RobinTS

This is the error that appears in the log after I try and delete all posts:

Right, I should have read your post more carefully.

@RobinTS

I’m noticing that it deletes a few posts (about 5), before the error dialog appears.

It seems to be failing for the topic with the id 64828. Has the user created posts in this topic?

[RobinTS]

Apologies, upon closer inspection, that error seems to be related to something else.
I’m seeing this one now and I suspect it might be due to a timeout on the request:

I also tried doing the delete from an API request, but I guess that would be the same thing as the UI.

[RobinTS]

I wrote a small app that calls the discourse API on my installation. It individually deletes each post, instead of doing a batch. So for 700 posts it would do 700 API calls. Not the most efficient, but it was helpful.

I see that there are 20 problem posts. Looks like the “delete all” function stops after it reaches one of these and can’t delete them. The API was not able to delete these for some reason. I manually deleted most of them now by going into the post and tapping the delete button as a moderator would normally do.

There are still about 5 that I can’t delete, via API or normal. I don’t see any resemblance between them. Some are in massive topics, some are in small topics. Some are replies, have likes, have pictures, others not. When I try to delete them through the standard UI I get this error dialog:

Some of my users have reported getting the same dialog when they try to delete their own posts. I assumed it was due to a connection issue or something since almost all of them are on mobile, but it could be something different.

When trying to delete it manually, I get this log:

Any ideas/suggestions?

[RobinTS]

The five problem posts are all within two large topics. We recently closed both of them because they were just getting way too big. The one topic has 55k posts and the other has 17k posts. I’m guessing there’s a lot of processing that goes on when a post within such a large topic gets removed and that’s causing a timeout of some kind, but that’s only a guess.

[RobinTS]

I am now stuck. I am unable to delete the posts in those large topics and I am unable to delete the topics. Both of these end up with the 502 gateway dialog.

[dax (Daniela)]

@RobinTS

The one topic has 55k posts and the other has 17k posts

Have you tried deleting these 5 posts using the console?

./launcher enter app
rails c
Post.find(THE_POST_ID).destroy

[RobinTS]

@dax

./launcher enter app
rails c
Post.find(THE_POST_ID).destroy

Yes! That does the trick. I don’t know Ruby or Rails at all, but I’m able to chain together a few posts in one command like this:

Post.find(POST1_ID).destroy; Post.find(POST2_ID).destroy

I’ll probably use this at some point when I there are a few posts that get stuck. Is it possible (and safe) to also delete a topic this way?

[dax (Daniela)]

Generally the posts in Discourse are just soft-deleted and not hard-deleted . With the command destroy you have just deleted them from the database and I made you use it only because the posts you were trying to delete belonged to incredibly long topics.

I do not recommend using this method neither for posts nor for topics. They can never be restored and you will work on a database for a site in production from what I understand, with the risk of breaking the whole site if something goes wrong. However if you want using the destroy command you are free to do so. You should always make a backup if you intend to use this type of commands.

To permanently delete a topic run

Topic.find(THE_TOPIC_ID).destroy

[dax (Daniela)]

Just a note to say that to avoid this type of performance issues we have also introduced some settings to avoid the creation of megatopics. See The MEGATOPIC: public good, or public menace? for details

[RobinTS]

Thank you for the advice. It’s about what I expected and I will only use the destroy command as a last resort.

Since it’s normally soft deleting the posts, it sounds like it writes a bunch of other things instead of just deleting the record. Is this maybe why it doesn’t work well when deleting the posts from a mega topic? Just too many things to do and it hits a time out?

I’d love a way to resolve this without resorting to the risky console command.

[RobinTS]

Thank you. I changed our max topic limit to 2500. Users asked for the limit to be removed a while ago, which led to these mega topics. The mega topics are not good for the forum though. Server performance was severely impacted during peak times when a lot of people were replying to it. We ended up closing it to stop that from happening, but now we are still stuck with these massive topics that are causing other issues.

[dax (Daniela)]

The simplest solution is for some moderators to get to work and start splitting (and then close) megatopics into multiple topics, for example

• [Megatopic Title] part 1
• [Megatopic Title] part 2
• [Megatopic Title] part 3
• [Megatopic Title] part […]
• [Megatopic Title] part 10

Each part will be automatically linked to the previous and next part so users should not have navigation difficulties and you should definitely solve the performance problem.

Yes, it will be a boring job to do but this is also one of the reasons why having a topic with 55k or 17k posts is absolutely useless since nobody will read them.

[techAPJ (Arpit Jalan)]

@dax

The simplest solution is for some moderators to get to work and start splitting (and then close) megatopics into multiple topics

We now have a feature (site setting) for exactly this use case!

Linked topics (splitting and managing megatopics)

I’ve just merged a feature which creates a new linked topic automatically when a topic is auto-closed based on auto close topics post count site setting. This feature is especially useful for automatically splitting and managing megatopics. Here’s how it works Let’s say auto close topics post count setting is set to 10000 posts and a topic (with title “Introduce yourself!”) just reached that limit. Now we’ll close that topic and create a new linked topic to continue the discussion. The first…