Administrative Bulk Operations

tshenry · November 11, 2020, 5:53pm

Below you will find a collection of bulk operations that can be initiated from the command line. You will need SSH access, so if you are a hosted customer, you will need to contact the Discourse team about running these commands. This topic is a wiki, so feel free to add or improve anything relevant.

Before working with the console it is extremely important that you have a recent backup. Mistakes can always happen!

First thing to do is enter your site’s container:

cd /var/discourse
./launcher enter app

Change Topic Status (close, archive, unlist)
Move subset of topics
Delete subset of users
Suspend a set of users based on criteria
Update user suspension reasons
Unsuspend users
Export/Import all site settings
Export/Import categories
Export/Import groups
Set permissions for multiple categories
Tag all topics based on a keyword
Tag all topics within a category
Move all topics with a specific tag to a single category
Move all topics from one category to another
Grant a badge to all group members
Ensure all users are at their automatic trust level
Destructive rake tasks

Additional Guides:

Change topic status ↥

Before running the following commands, run rails c to enter the console.

Unlist all topics within a category (excludes post action)

You can replace visible with closed or archived and adjust the true/false values as needed
```
cat_id = Category.find_by_slug('admins').id
Topic.where(category_id: cat_id, visible: true).update_all(visible: false)
```

Unlist all topics within a category (includes post action)

cat_id = Category.find_by_slug("admins").id
Topic.where(category_id: cat_id, visible: true).find_each do |topic| 
  topic.update_status('visible', false, Discourse.system_user)
end

Close all topics created before a specified date (includes post action)

Topic.where(closed: false).where("created_at < '2015-01-01'").find_each do |topic| 
  topic.update_status('closed', true, Discourse.system_user)
end

Moving topics ↥

Move a collection of topics from one category to another

rails c
topic_ids = [12,16,29]
cat_to = Category.find_by_slug('faq')
Topic.where(id: topic_ids).update_all(category_id: cat_to.id)
Category.update_stats

Delete subset of users ↥

Delete users that have never posted and have not visited since a specified date

rails c
User.joins(:user_stat).where("user_stats.post_count = 0 AND previous_visit_at <= '2016-05-20'::timestamp").destroy_all

Suspend a set of users based on criteria ↥

Set who will be logged as suspending the users

rails c
logger = StaffActionLogger.new(User.find_by(username_lower: "tshenry"))

Create a suspension timeframe and reason

suspend_till = DateTime.new(2057,12,31)
reason = 'Completed Course'

In this example, our user criteria will be group membership.

target_group = Group.find_by_name("summer_students")
users = User.joins(:group_users).where(group_users: {group_id: target_group.id})

Suspend each user based on the values established above:

users.each do |u|
  u.suspended_till = suspend_till
  u.suspended_at = DateTime.now
  u.save!

  logger.log_user_suspend(u,reason)
end

Update user suspension reasons ↥

Perhaps you suspended users who completed a class (see example above), and now you want to add the year of the class as you’ve taught multiple years.

UserHistory.where(action: 10, details: "Completed Course").update_all(details: "Completed 2018 Course")

Unsuspend users ↥

If you need to unsuspend users in bulk, say because they were part of a previous year’s cohort and are returning for this year, you can do so as shown below. In the example, we’re finding users by their user id.

user_list = [1, 3, 5, 7, 11]
users = User.where("id in (?)", user_list)

users.each do |user|
    user.suspended_till = nil
    user.suspended_at = nil
    user.save!
    StaffActionLogger.new(User.find(-1)).log_user_unsuspend(user)

    DiscourseEvent.trigger(:user_unsuspended, user: user)
end

Export/Import all site settings ↥

To simply print out all of the settings that have been changed on your site, run:

rake site_settings:export

If you want to export the settings to a file:

rake site_settings:export > saved_settings.yml

If you want to import settings from a file:

rake site_settings:import < saved_settings.yml

Export/Import categories ↥

There are two options for exporting and one method to handle importing.

Export a set of complete categories

First get a list of your category IDs:

rake categories:list

Then space-separate the category IDs in the export rake task. For example:

rake export:categories["12 6"]

Export your site’s category structure

This is essentially copying the “skeleton” of your Discourse site. It includes every category along with any groups associated with existing category permissions. It does not include topics:

rake export:category_structure

If you want the category structure along with any groups associated with the category permissions and any members of those groups:

rake export:category_structure[true]

Importing a category file

Use the exported file’s name like the example below:

rake import:file["category-export-2019-05-16-052430.json"]

Export/Import groups ↥

Export all of the user groups

rake export:groups

Export all of the user groups including users

rake export:groups[true]

Importing a group file
Use the exported file’s name like the example below:

rake import:file["group-export-2019-05-16-052430.json"]

Set permissions for multiple categories ↥

Note that this will remove any existing access restrictions you have set up for the categories involved. Make sure to include all of the relevant permissions.

Get a list of categories along with their IDs
```
rails c
Category.all.pluck("name", "id")
```
Create an array with the category IDs you wish to target.
```
category_ids = [6,7,8,10]
```

Change the permissions. The set_permissions function can utilize the following parameters: :full, :create_post, :readonly

A single permission. For example, making a set of categories staff only:

Category.where(id: category_ids).find_each do |category| 
 category.set_permissions(:staff => :full)
 category.save!
end

Multiple permissions. For example, making a set of categories read-only for normal users:

Category.where(id: category_ids).find_each do |category| 
  category.set_permissions(:everyone => :readonly, :staff => :full)
  category.save!
end

User group permissions. For example giving one group full permissions and another group read-only for a set of categories:

artists_group = Group.find_by_name("artists")
buyers_group = Group.find_by_name("buyers")
Category.where(id: category_ids).find_each do |category| 
  category.set_permissions(artists_group.id => :full, buyers_group.id => :readonly)
  category.save!
end

Bulk Tag All Topics Based on a Keyword ↥

The following script will allow you to tag topics based on the presence of a keyword in the topic title or its posts. Start by creating an array of keywords:

rails c
keywords = ['apples','oranges']

Next we need to define a method:

def tag_by_keyword(word, tag_name)
  tag = Tag.find_by_name(tag_name) || Tag.create(name: tag_name)
  keyword_topics = Topic.joins(:posts).where("topics.title ~* :keyword or posts.raw ~* :keyword", keyword: "\\y#{word}\\y").distinct

  keyword_topics.each do |topic|
    if topic.tags.exclude?(tag)
      topic.tags << tag
    end
  end
end

And finally run each keyword through the method. The following with tag each relevant topic with a tag called “fruit”:

keywords.each { |word| tag_by_keyword(word, 'fruit') }

Bulk Tag All Topics Within a Category ↥

Template: rake tags:bulk_tag_category["<tag>|<tag>",<category_id>]
This would be particularly useful when trying to convert a category to a tag.

First, use the following rake task to find the relevant category ID.

rake categories:list

Tag all topics of the category you specify. In this example, you would be tagging all topics in the category with an ID of 6 with the “support” tag. this will remove all other tags from each topic.

rake tags:bulk_tag_category["support",6]

Append all topics of the category you specify. In this example, you would be adding the “support” tag to all topics in the category with an ID of 6, while keeping existing tags.

rake tags:bulk_tag_category["support",6,true]

Move all topics with a specific tag to a single category ↥

When trying to restructure your Discourse site, you may find that you want to move a collection of topics without triggering any notifications. One way to do this is to create a temporary tag, apply the tag to appropriate topics, move the topics to a specific category using the code below, then finally delete the temporary tag.

Get the tag and destination category.

rails c
tag = Tag.find_by_name("tutorial")
cat_to = Category.find_by_slug('guides')

Move the tagged topics to the destination category.

Topic.joins(:topic_tags).where("topic_tags.tag_id = ?", tag.id).update_all(category_id: cat_to.id)

Update the topic counts of the affected categories.

Category.update_stats

Move all topics from one category to another ↥

Find the category IDs with the following rake task:

rake categories:list

The first value should be the starting category ID. The second value should be the destination category ID.

rake categories:move_topics[15,6]

Grant a badge to all group members ↥

Grant a badge to all users that belong to a specific group. The first value is the group ID and the second is the badge ID.

rails c
Group.find_by_name("event_participants").id
Badge.find_by_name("event_badge").id
exit
rake groups:grant_badge[42,102]

Note that the above rake task only grants a badge, it will not revoke a previously granted badge if a user is no longer part of the specified group. If you need to bulk revoke badges for all users that are no longer part of a group, you can run the following:

rails c

badge_id = Badge.find_by_name("Some Group Member").id

group = Group.find_by_name("Some_Group")

group_user_id = group.users.pluck("id")

userBadge = UserBadge.where.not(user_id: group_user_id).where(badge_id: badge_id)

userBadge.each do |ub|
  BadgeGranter.revoke(ub, revoked_by: Discourse.system_user)
end

exit

Ensure all users are at their automatic trust level ↥

Say you set the default trust level for new or invited users to a value that isn’t working out quite the way you expected (such as TL4). Now you want to change it so your users are at the trust level they would be automatically, given their current stats. The following commands will ensure all users are at the trust level they should be according to Understanding Discourse Trust Levels. Note: users with locked trust-levels will not be affected.

Make sure all users are set to the correct trust level:

rails c
User.all.find_each do |user|
  Promotion.recalculate(user)
end

Refresh the group stats to reflect the changes:

Group.ensure_consistency!

Destructive rake tasks ↥

Delete entire categories

The following will allow you to destroy multiple categories, along with any subcategories and topics that belong to those categories.

Print out a list of category IDs

rake categories:list

Destroy a set of categories based on their ID

rake destroy:categories[10,11,12,18,30]

Delete all topics in a category

Remove all private messages

rake destroy:private_messages

Destroy all groups

rake destroy:groups

Destroy all non-admin users

rake destroy:users

Destroy site stats

rake destroy:stats

Anonymize all users except staff

rake users:anonymize_all

I’ve tried to include the most useful rake tasks in this topic, but there are many others packaged with Discourse. If you would like to see a comprehensive list, you can use the following:

All tasks that have descriptions

rake --tasks

All tasks, including those that do not have descriptions

rake -AT