Mass closing topics

[hendersj]

I found this post with instructions on how to use the rails console to mass close topics: Auto-close old topics from a migrated forum - #10 by zogstrip

A couple of questions about it, though:

1. Is there a newer way to do this?
2. If I wanted to do the close based on the last activity date (rather than the creation date), is there a variable to use in place of created_at?
3. Is there any way to exclude DMs from being closed? I ran the query in my test environment, and noticed that it hit every topic, whether it was public or private; we’d like to exclude private messages if possible.
4. On our forum, we have nearly 16 years of content that we imported from our previous solution. In terms of the time to run the query, (a) how can we determine how long it needs to run, and (b) would it be better to break it up (say, run for everything pre-2010, then for 2011, 2012, etc, until we hit 2023) or just run it as a single query?

Just trying to make sure (with #4) that we don’t impact system performance too much. I know a lot depends on what the hardware is that we’re running on (which I don’t actually know, because we have an infrastructure team that handles the installation itself and maintains all the hardware).

Appreciate any guidance!

[hendersj]

On #2, using the data explorer plugin (which I wasn’t aware of), it looks like updated_at is probably the value where the ‘last reply timestamp’ would be. Is that an accurate assessment?

[ondrej]

Administrative Bulk Operations

Below you will find a collection of bulk operations that can be initiated from the command line. You will need SSH access, so if you are a hosted customer, you will need to contact the Discourse team about running these commands. This topic is a wiki, so feel free to add or improve anything relevant. Before working with the console it is extremely important that you have a recent backup. Mistakes can always happen! First thing to do is enter your site’s container: cd /var/discou…

[hendersj]

Thanks for this pointer - that’s very useful for my first three questions, I think. Would appreciate some clarity on updated_at and planning to operate on our extended history of posts & topics.

[jericson (Jon Ericson)]

@hendersj

Just trying to make sure (with #4) that we don’t impact system performance too much.

I’d definitely test with a small batch first. I’ve, uh, taken down a community site with bulk actions and wished I tested a subset first.

Calvin and Hobbes by Bill Watterson for November 26, 1986 | GoComics.com

[hendersj]

Thanks - I suspected that might be the case. Good to have confirmation on the approach.

I think what I may do is see about pulling one of the recent backups and setting up a sandbox environment separate from my test environment. Not sure how the SSO piece will work in that setup, but it would be nice to see what the performance looks like before hitting the prod system.

[jericson (Jon Ericson)]

That’s a good idea:

Set up a staging server

There are several tricks that can help when you are setting up a staging server. What is a staging server? A staging server is essentially a clone of a production site. It also resides on a server, and functions identically. It runs inside a Docker container, just as a normal Discourse site does. It exists to give you a place to try risky things out, or to trial things that you cannot easily hide from your users. It is very useful for trialling adverts using the Discourse Ad Plugin, or if you…

I will note that if you have an especially active community, the staging site won’t completely simulate production since there’s some impact from people using the site organically in addition to the bulk actions.

[hendersj]

Definitely - appreciate the pointer to the staging server post as well - that will be quite helpful.

Yeah, user load is not likely to be a huge issue - looks like our page views per day is around 50K on average across all (crawlers, anonymous, and registered users). Understanding the potential increased load over the existing load will be useful for planning purposes.

[hendersj]

I did end up setting up a staging server (was pretty easy to do, actually - just restored a backup from prod and logged in using the admin recovery procedure, since it’s configured for OIDC only). Looks like we have about 160K topics, and a quick test on just one category with about 7500 took 6 minutes on my test system - so about 2 hours for all topics. The impact on system performance (monitored with htop) seemed pretty negligible here.

I’m sure we can find a low usage time period where the rake command can be run, and we can stage groups of categories if we want, so that’ll work really nicely for us.

Appreciate all the tips and help - learned a lot about the platform over the last couple of days as a result.