CLI command to migrate a single category

feature
,
#1

We need to build a general purpose tool for moving a category from one Discourse instance to another.

This has come up on one of our customers and is also useful for a variety of use cases:

Use cases for category import / export

  1. (current need) During huge migrations we may need to avoid long term downtime, so we need to run a temporary site on one domain and then ship the content (eventually) to the new imported site

  2. In some cases category partitioning is not strong enough, we have seen this issue at the site @BCHK hosts. In cases like this it may be desirable to partition a site into 2 seperate sites.

  3. In some cases many tiny Discourse sites are created unnecessarily and an amalgamation is desirable

A CLI tool only

The scope of this spec is for a command line tool only. There will be no GUI at all for the first version.

Topic export with layered category export

At the heart of it we need a mechanism to export a single topic as a stand alone unit that can be imported into another instance. Format wise we need to export all users , user actions and posts as single tar-csv, xml or json blob.

We are only interested in exporting “involved” users, that is, users that have tracking state, user actions or direct involvement with the topic.

Once applied to a category the export will avoid double exporting users, the format should specify users first and then content in each of the exported entities.

For v1 we need to cover user case (1) and have an importer.

Example usage

discourse export some_category > my_export.tar
discourse import < my_export.tar
18 Likes
Moving topics between sites?
Becoming the new standard discussion tool for open source projects
How should category archiving work?
Can sub-forums of the same instance split up and move to new independent servers (or discourse instances)?
#2

I’ve been wanting to build a discourse_cli app for sometime now so that it’s possible to do admin tasks like this and have it use the discourse_api gem.

Is this something you would like to have as a separate discourse_cli project or do you think it needs to live in discourse so that it can make actual sql queries?

#3

I just created a CLI Tool that we can use for admin tasks like migrating categories.

https://github.com/blake/discourse_cli

This is just the basic structure for the CLI tool, it doesn’t haven any features yet other than displaying a list of categories.

4 Likes
#4

This is very deja vu. It’s basically the original spec for our backup/restore, which didn’t work because it broke whenever the db schema changed. We need to limit the scope of what it means to export a user, user action, topic, post, etc. otherwise this tool will become impossible to maintain.

2 Likes
#5

The tool is described here:

4 Likes
#6