Move topics from one Discourse instance to another

Need to move some stuff from one instance of Discourse to another?

There is a command line tool to export a topic, a set of topics, or an entire category, and another to import them in at the other end.

Topic Export/Import

  • one or more topics and their replies
  • the users who posted in the topic

Category Export/Import

  • the category and all its subcategories
  • its security settings
  • custom groups mentioned in the security settings
    • :warning: if any of the groups have ‘membership requests’ enabled, it will crash. Turn this off before the export
  • all topics and posts in the category and subcategories
  • all users who posted, and assigns them to the groups

Begin: Identify the IDs

First, identify the ID(s) of the =Export= that you wish to export. You’ll find this at the end of the URL used to access it.

  • https://meta.discourse.org/t/how-to-contribute-to-discourse/53797

  • https://meta.discourse.org/t/how-to-contribute-to-discourse/53797/12
    (in this case you are in the middle of a topic - grab the first number)

  • https://meta.discourse.org/c/howto/faq/4

Category or Topic ID(s)

Multiple topics

You can select more than one topic by adding IDs separated by a space; this works in the field above (I don’t think that this works for categories though):

  • 53797 200943 124096

Run the Export Script

The tool can be run as a root user inside the container:

cd /var/discourse
./launcher enter app

Run this command:

discourse export_=Export= =ID=

The output should look something like this:

Starting export of =Export=…

Where are the new users all coming from?
Export saved to =Export=-export-2021-08-27-015437.json

Done

Move the Exported File

The exported .json file needs to be moved from the inside of the application container where you did the export to the inside of the container of the receiving instance. This takes a few steps.

Copy it to the ‘outside’

Exit the container if you haven’t already

exit

You should now have a prompt like this:

root@yourserver:/var/discourse

Use docker to copy the file out of the container (you can’t move it I don’t think)

docker cp app:/var/www/discourse/=FileName= .

Move it to the other instance

This is beyond the scope of this guide, but this might help you get there:

Copy it into the container in the receiving instance

SSH into your receiving instance as root and get to the Discourse directory:

cd /var/discourse

Use docker once again to copy the file into the other container

docker cp =FileName= app:/var/www/discourse

Import the Content

Enter the container in the receiving instance:

./launcher enter app

Run the import script

discourse import_=Export= =FileName=

:cake: Done!

You might need to tidy up some loose ends, but hopefully, it should all be done.

Advanced Tips

Missing Ruby Gem

When doing a complex category import, my instance was missing the ‘Listen’ Ruby.
This was solved by doing this: (using the Discourse user inside the container)

./launcher enter app
su discourse
bundle config --delete without
bundle config --delete with
bundle install
exit
exit

This broke the rails console, necessitating a rebuild afterwards

./launcher rebuild app

Multisite

In a multisite environment, use the RAILS_DB env variable.

RAILS_DB=xxxmoves discourse export_=Export= =ID=
RAILS_DB=dancerehab discourse import_=Export= =FileName=

Please report bugs in the Contribute > Bug category, and request enhancements in the Contribute > Feature category.


Related Topics


Last Reviewed by @SaraDev on 2022-07-13T00:00:00Z

Last edited by @hugh 2024-05-29T06:25:11Z

Check documentPerform check on document:
45개의 좋아요
UI: Topic and Category Export/Import
CLI command to migrate a single category
Moving topics between sites?
Category/Topic export fails with error "database "discourse_development" does not exist"
Easiest way to export a single topic?
Saving a Conversation
How to export user data from a category on a hosted site
Import CSV User Activity from one Discourse to Another
How use API to convert topic in discourse1.com to discourse2.com
Can sub-forums of the same instance split up and move to new independent servers (or discourse instances)?
Help migrating category from one instance to another without admin on the source instance
I wonder how to add hundreds of users into a group? Is there any api?
How to download Comments and Topics
Adding default sets of topics, categories and users to a new Discourse
Can we combine 2 separate discourse sites?
Tracking and resolving a schema drift cause
Merge two Discourse forums into one
Moving topics from a forum on one website to a forum on another website
Transfer topics between forums?
Forum Aquisition
Transfering inbox group messages to another instance?
Transfering inbox group messages to another instance?
Imported categories are not in parent category
Copy categories / topics
Downloading Content
Selective backups skipping settings and config
Test driving admin/moderation features
Add backup to existing Discourse instance
Merging two Discourse forums
Migrating vBulletin onto existing and live Discourse instance
Migrate topics from one discourse instance to another?
A system where people have to start at lesson 1, and cannot see the other lessons?
How to divide my community into 2 parts
Migrate user custom fields to another instance
Error Importing Categories from a Discourse Instance to Another

I’m about to have a crack at this again.

Is there an easy way to migrate the relevant uploads along with the Topics?

1개의 좋아요

Not really. A thing that mostly works is if you get all of those uploads and paste the URL of the original in to any post on the site you’re moving to, then what should happen is that those uploads will get created with the same SHA and it Just Works.

So you’d do the thing that converts the stuff in the post to the URL to get those URLs. (I can never quite remember how to do that, so I have to figure it out every time).

1개의 좋아요

Hmmm - I remain a bit confused by that. Are you saying that this will work?

  1. Grab all of the upload files from the first site
    • how?
  2. Drop them into a single topic
  3. They will generate the same relative link as they had in the old site and magic!

Or did you mean this?

  1. Grab all of the absolute URLs in your categories of interest from the old site
  2. Paste these into a Topic
  3. They will all be downloaded to the new site with the same SHA and the existing links will all work like magic

Or something else entirely?

1개의 좋아요

I think either of those should work. However you can get those images to the new site, they should generate the same sha so that will magically make the existing links in the topics work.

Maybe you could do something like

cat_id=123
Topic.where(category_id: cat_id).each do| t|
  Post.where(topic_id: t.id) do |p|
      p.cooked.match(image url)....
         puts(the match) 

And I think if you do that and paste those links into a post anywhere on your site it should download all of those images to local and make the same sha, so then those links. You can try a couple by hand to see.

It’s the middle of the night. Your mileage may vary.

2개의 좋아요

Hello, I’m not sure if this tutorial is what I want to do, but raise the question here.

If I install discourse on a server, but due to growth I need to migrate to a more powerful one, are these the steps?

What happens, for example, with the backup option? Can’t I do it, install Discourse on another server and then restore the backup copy of the other instance from that option?

I await your clarification, since I want to know how to migrate when I need to and not lose the work I have been doing recently.

1개의 좋아요

No. To move to a bigger server a simple backup and restore is all that’s needed. You can also Move a Discourse site to another VPS with rsync

1개의 좋아요

오래전부터 업로드 마이그레이션 문제를 해결하고 싶었는데, 이제 LLM의 도움으로 마침내 해결했습니다. 관심 있는 분들을 위해 제 노트를 공유합니다! Jay의 위 게시물에는 시작하는 데 필요한 핵심 정보가 포함되어 있었으니, 그 점에 대해 감사드립니다.

첫 번째 시도는 Jay가 예측한 대로 정확히 작동했습니다. 이전 사이트의 이미지 URL 목록을 생성하여 새 사이트의 게시물에 붙여넣으면 이미지들이 다운로드됩니다. 이 기능이 작동하려면 Download remote images to local(원격 이미지를 로컬로 다운로드) 설정이 활성화되어 있어야 합니다.

LLM이 이를 위한 스크립트를 작성해 주었는데, 향후 같은 문제를 겪을 분들을 위해 여기에 공유합니다.

요약

이 코드를 Rails 콘솔에서 실행하세요:

topic = Topic.find(=TOPIC_ID=)
posts = Post.where(topic_id: topic.id)
urls = []

posts.each do |p|
  UploadReference.where(target: p).each do |ref|
    urls << "=SOURCE_URL=#{ref.upload.url}"
  end
end

puts urls.uniq.join("\n")
puts "\nTotal: #{urls.uniq.count} upload(s)"

:bulb: 전체 카테고리를 이동해야 하는 경우? 처음 두 줄을 다음과 같이 교체하세요:

topic_ids = Topic.where(category_id: =TOPIC_ID=).pluck(:id)
posts = Post.where(topic_id: topic_ids)

이 경우 =TOPIC_ID=는 Admin → Categories URL의 끝에 있는 카테고리 ID입니다.

하지만 제 경우엔 이것만으론 충분하지 않았습니다. 자동으로 로컬에 다운로드되지 않는 PDF 파일들도 있었기 때문입니다.

그래서 LLM에게 업로드를 마이그레이션하는 Python 스크립트를 작성해 달라고 했습니다. 꽤 잘 작동해서 이 또한 공유하고자 합니다. 직접 사용해 보고 싶거나 피드백이 있다면, 레포지토리는 여기 있습니다: tobias/discourse-upload-migrator: Migrates uploads (images, PDFs, and other attachments) from one Discourse instance to another via the Discourse API. - Digitally Sovereign Forgejo: Digitally Sovereign, for the win!

또한 LLM에게 주제나 카테고리를 이동하는 더 완전한 런북(runbook)도 작성해 달라고 했는데, 제 discourse에 올라가 있습니다: https://digitallysovereign.org/t/move-topics-or-categories-between-discourse-instances/742?u=tobias

2개의 좋아요