Administrative Bulk Operations

Discourse · May 22, 2019, 12:36am

Below you will find a collection of bulk operations that can be initiated from the command line. You will need SSH access, so if you are a hosted customer, you will need to contact the Discourse team about running these commands.

Before working with the console it is extremely important that you have a recent backup. Mistakes can always happen!

First thing to do is enter your site’s container:

cd /var/discourse
./launcher enter app

Additional Guides:

Change topic status

Before running the following commands, run rails c to enter the console.

Unlist all topics within a category (excludes post action)

You can replace visible with closed or archived and adjust the true/false values as needed
```
cat_id = Category.find_by_slug('admins').id
Topic.where(category_id: cat_id, visible: true).update_all(visible: false)
```

Unlist all topics within a category (includes post action)

cat_id = Category.find_by_slug("admins").id
Topic.where(category_id: cat_id, visible: true).find_each do |topic| 
  topic.update_status('visible', false, Discourse.system_user)
end

Close all topics created before a specified date (includes post action)

Topic.where(closed: false).where("created_at < '2015-01-01'").find_each do |topic| 
  topic.update_status('closed', true, Discourse.system_user)
end

Moving topics

Move a collection of topics from one category to another

rails c
topic_ids = [12,16,29]
cat_to = Category.find_by_slug('faq')
Topic.where(id: topic_ids).update_all(category_id: cat_to.id)
Category.update_stats

Users

Delete subset of users

Delete users that have never posted and have not visited since a specified date

rails c
User.joins(:user_stat).where("user_stats.post_count = 0 AND previous_visit_at <= '2016-05-20'::timestamp").destroy_all

Suspend a set of users based on criteria

Set who will be logged as suspending the users

rails c
logger = StaffActionLogger.new(User.find_by(username_lower: "tshenry"))

Create a suspension timeframe and reason

suspend_till = DateTime.new(2057,12,31)
reason = 'Completed Course'

In this example, our user criteria will be group membership.

target_group = Group.find_by_name("summer_students")
users = User.joins(:group_users).where(group_users: {group_id: target_group.id})

Suspend each user based on the values established above:

users.find_each do |u|
  u.suspended_till = suspend_till
  u.suspended_at = DateTime.now
  u.save!

  logger.log_user_suspend(u,reason)
  putc '.'
end

Update user suspension reasons

Perhaps you suspended users who completed a class (see example above), and now you want to add the year of the class as you’ve taught multiple years.

UserHistory.where(action: 10, details: "Completed Course").update_all(details: "Completed 2018 Course")

Unsuspend users

If you need to unsuspend users in bulk, say because they were part of a previous year’s cohort and are returning for this year, you can do so as shown below. In the example, we’re finding users by their user id.

user_list = [1, 3, 5, 7, 11]
users = User.where("id in (?)", user_list)

users.each do |user|
    user.suspended_till = nil
    user.suspended_at = nil
    user.save!
    StaffActionLogger.new(User.find(-1)).log_user_unsuspend(user)

    DiscourseEvent.trigger(:user_unsuspended, user: user)
end

Export/Import

Export/Import all site settings

To simply print out all of the settings that have been changed on your site, run:

rake site_settings:export

If you want to export the settings to a file:

rake site_settings:export > saved_settings.yml

If you want to import settings from a file:

rake site_settings:import < saved_settings.yml

Export/Import categories

There are two options for exporting and one method to handle importing.

Export a set of complete categories

First get a list of your category IDs:

rake categories:list

Then space-separate the category IDs in the export rake task. For example:

rake export:categories["12 6"]

Export your site’s category structure

This is essentially copying the “skeleton” of your Discourse site. It includes every category along with any groups associated with existing category permissions. It does not include topics:

rake export:category_structure

If you want the category structure along with any groups associated with the category permissions and any members of those groups:

rake export:category_structure[true]

Importing a category file

Use the exported file’s name like the example below:

rake import:file["category-export-2019-05-16-052430.json"]

Export/Import groups

Export all of the user groups

rake export:groups

Export all of the user groups including users

rake export:groups[true]

Importing a group file
Use the exported file’s name like the example below:

rake import:file["group-export-2019-05-16-052430.json"]

Set permissions for multiple categories

Note that this will remove any existing access restrictions you have set up for the categories involved. Make sure to include all of the relevant permissions.

Get a list of categories along with their IDs
```
rails c
Category.all.pluck("name", "id")
```
Create an array with the category IDs you wish to target.
```
category_ids = [6,7,8,10]
```

Change the permissions. The set_permissions function can utilize the following parameters: :full, :create_post, :readonly

A single permission. For example, making a set of categories staff only:

Category.where(id: category_ids).find_each do |category| 
 category.set_permissions(:staff => :full)
 category.save!
end

Multiple permissions. For example, making a set of categories read-only for normal users:

Category.where(id: category_ids).find_each do |category| 
  category.set_permissions(:everyone => :readonly, :staff => :full)
  category.save!
end

User group permissions. For example giving one group full permissions and another group read-only for a set of categories:

artists_group = Group.find_by_name("artists")
buyers_group = Group.find_by_name("buyers")
Category.where(id: category_ids).find_each do |category| 
  category.set_permissions(artists_group.id => :full, buyers_group.id => :readonly)
  category.save!
end

Bulk Tag All Topics Based on a Keyword

The following script will allow you to tag topics based on the presence of a keyword in the topic title or its posts. Start by creating an array of keywords:

rails c
keywords = ['apples','oranges']

Next we need to define a method:

def tag_by_keyword(word, tag_name)
  tag = Tag.find_by_name(tag_name) || Tag.create(name: tag_name)
  keyword_topics = Topic.joins(:posts).where("topics.title ~* :keyword or posts.raw ~* :keyword", keyword: "\\y#{word}\\y").distinct

  keyword_topics.each do |topic|
    if topic.tags.exclude?(tag)
      topic.tags << tag
    end
  end
end

And finally run each keyword through the method. The following with tag each relevant topic with a tag called “fruit”:

keywords.each { |word| tag_by_keyword(word, 'fruit') }

Bulk Tag All Topics Within a Category

Template: rake tags:bulk_tag_category["<tag>|<tag>",<category_id>]
This would be particularly useful when trying to convert a category to a tag.

First, use the following rake task to find the relevant category ID.

rake categories:list

Tag all topics of the category you specify. In this example, you would be tagging all topics in the category with an ID of 6 with the “support” tag. this will remove all other tags from each topic.

rake tags:bulk_tag_category["support",6]

Append all topics of the category you specify. In this example, you would be adding the “support” tag to all topics in the category with an ID of 6, while keeping existing tags.

rake tags:bulk_tag_category["support",6,true]

Move all topics with a specific tag to a single category

When trying to restructure your Discourse site, you may find that you want to move a collection of topics without triggering any notifications. One way to do this is to create a temporary tag, apply the tag to appropriate topics, move the topics to a specific category using the code below, then finally delete the temporary tag.

Get the tag.

rails c
tag = Tag.find_by_name("tutorial")

Get destination category.

For regular categories:

cat_to = Category.find_by_slug('guides')

For subcategories:

cat_to = Category.find_by_slug('child-slug','parent-slug')

Move the tagged topics to the destination category.

Topic.joins(:topic_tags).where("topic_tags.tag_id = ?", tag.id).update_all(category_id: cat_to.id)

Update the topic counts of the affected categories.

Category.update_stats

Move all topics from one category to another

Find the category IDs with the following rake task:

rake categories:list

The first value should be the starting category ID. The second value should be the destination category ID.

rake categories:move_topics[15,6]

Change owner of all topics in categories

Find the category IDs with the following rake task:

rake categories:list

Specify the new owner and categories to operate on. The categories should be an array of category ids, categories 1, 2 and 3 in the example:

rails c
user = User.find_by(username_lower: "lowercase-username")
categories = [1, 2, 3]

Get all topic ids for the given categories and change the owner of the first post in all matched topics.

topics = Topic.where(category_id: categories).pluck(:id)

topics.each do |topic|
  PostOwnerChanger.new(
    post_ids: Post.where(topic_id: topic).where(post_number: 1).pluck(:id),
    topic_id: topic,
    new_owner: user,
    acting_user: Discourse.system_user,
    skip_revision: true
  ).change_owner!
end

Grant a badge to all group members

Grant a badge to all users that belong to a specific group. The first value is the group ID and the second is the badge ID.

rails c
Group.find_by_name("event_participants").id
Badge.find_by_name("event_badge").id
exit
rake groups:grant_badge[42,102]

Note that the above rake task only grants a badge, it will not revoke a previously granted badge if a user is no longer part of the specified group. If you need to bulk revoke badges for all users that are no longer part of a group, you can run the following:

rails c

badge_id = Badge.find_by_name("Some Group Member").id

group = Group.find_by_name("Some_Group")

group_user_id = group.users.pluck("id")

userBadge = UserBadge.where.not(user_id: group_user_id).where(badge_id: badge_id)

userBadge.each do |ub|
  BadgeGranter.revoke(ub, revoked_by: Discourse.system_user)
end

exit

Ensure all users are at their automatic trust level

Say you set the default trust level for new or invited users to a value that isn’t working out quite the way you expected (such as TL4). Now you want to change it so your users are at the trust level they would be automatically, given their current stats. The following commands will ensure all users are at the trust level they should be according to Understanding Discourse Trust Levels. Note: users with locked trust-levels will not be affected.

Make sure all users are set to the correct trust level:

rails c
User.all.find_each do |user|
  Promotion.recalculate(user)
end

Refresh the group stats to reflect the changes:

Group.ensure_consistency!

Destructive rake tasks

Delete entire categories

The following will allow you to destroy multiple categories, along with any subcategories and topics that belong to those categories.

Print out a list of category IDs

rake categories:list

Destroy a set of categories based on their ID

rake destroy:categories[10,11,12,18,30]

Delete all topics in a category

Remove all private messages

rake destroy:private_messages

Destroy all groups

rake destroy:groups

Destroy all non-admin users

rake destroy:users

Destroy site stats

rake destroy:stats

Anonymize all users except staff

rake users:anonymize_all

I’ve tried to include the most useful rake tasks in this topic, but there are many others packaged with Discourse. If you would like to see a comprehensive list, you can use the following:

All tasks that have descriptions

rake --tasks

All tasks, including those that do not have descriptions

rake -AT

Last edited by @Moin 2024-09-28T17:49:06Z

Check document
Perform check on document:

Ventrilo · February 9, 2024, 12:52am

@Taylor what about bulk updating all topics with a different timestamp?

Say I wanted all topics on my discourse instance to instantly be changed to show as timestamp as if they was created Today how would I do that?

Firepup650 · February 9, 2024, 12:59am

Probably a modification of this query:

Taylor:

Change owner of all topics in categories

Find the category IDs with the following rake task:
rake categories:list
Specify the new owner and categories to operate on. The categories should be an array of category ids, categories 1, 2 and 3 in the example:
rails c
user = User.find_by(username_lower: "lowercase-username")
categories = [1, 2, 3]
Get all topic ids for the given categories and change the owner of the first post in all matched topics.
topics = Topic.where(category_id: categories).pluck(:id)

topics.each do |topic|
  PostOwnerChanger.new(
    post_ids: Post.where(topic_id: topic).where(post_number: 1).pluck(:id),
    topic_id: topic,
    new_owner: user,
    acting_user: Discourse.system_user,
    skip_revision: true
  ).change_owner!
end

(Assuming they’re in one category)

JammyDodger · June 7, 2024, 9:50pm

4 posts were split to a new topic: How to import category with category permissions?

wal · August 14, 2024, 3:55pm

Considering that this command needs to be run inside the Docker container, and the default pwd inside the container when running ./launcher enter app appears to be /var/www/discourse, the command described here seems a little odd for a few reasons

the site_settings includes the secret access keys used for S3 and other API’s, it feels like these should not be stored in a file under /var/www since that is traditionally the location for files that get served to the web
since we are inside the container at this default pwd, I would expect the file saved here would be lost the container stops?

From inside the container, I used this command mount | grep ^/dev/ | grep -v /etc/ to determine that the location /shared inside the container appears to map back to /var/discourse/shared/standalone on the host system. So it seems like maybe the command should be something like this?

cd /var/discourse
./launcher enter app
rake site_settings:export | grep -v key | grep -v secret > /shared/site_settings_$(date "+%Y-%m-%d-%H-%M-%S").yml

this would leave the file in a location such as /var/discourse/shared/standalone/site_settings_2024-08-14-15-53-11.yml on the host system

note that the extra grep commands piped here will remove any lines with the word “key” or “secret” in them in order to remove API keys but would also remove lines that included those words for non-sensitive reasons

does this sound about right?

pfaffman · August 14, 2024, 4:55pm

True, but I’m pretty sure that Discourse won’t serve the settings. You can run the command from any directory, and it’s assumed that if you’re doing this kind of stuff you understand what you’re doing.

So you could just

cd /shared/
mkdir -p my-settings
cd my-settings
rake ...

before you run the rake task.

Sure the dump has the keys in it, but lots of that is in plaintext in a bunch of places already (e.g., if you follow recommended procedures your S3 keys are in app.yml).

Yes.

wal · August 19, 2024, 1:12pm

oh I was not aware that the rake tasks would still work outside of the app’s pwd, that makes more sense then yea, thanks

pfaffman · August 19, 2024, 1:15pm

It still surprises me!

But also you can put the full path of the dump when you pipe out to a file.

Topic		Replies	Views
Discourse Version 2.0 Releases	5	18416	May 31, 2018
Re-purposing a Discourse installation for a yearly event Support	19	5803	June 8, 2024
Bulk delete all topics in a category Self-Hosting how-to	0	5673	September 23, 2018
Discourse Version 3.1 Releases	2	6731	August 1, 2023
Discourse 2.6.0.beta2 Release Notes Announcements release-notes	1	6339	August 20, 2020