# Improving import and export support
**URL:** https://meta.discourse.org/t/improving-import-and-export-support/12110
**Category:** Feature
**Created:** [January 21, 2014, 11:36pm UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110 "2014-01-21T23:36:55Z")
**Posts on this page:** 10
**Page:** 2
### Author:  [@victorhooi](https://meta.discourse.org/u/victorhooi)
#### Post date: [March 4, 2014, 5:14am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/21 "2014-03-04T05:14:40Z")
Hmm - sorry, I didn’t quite follow the sentence - do you mean some kind of warning sentence on the /admin page, just like we do for ImageMagick, missing Facebook tokens etc.?
Or something else?
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [March 4, 2014, 5:30am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/22 "2014-03-04T05:30:39Z")
> [@victorhooi](#):
>
> So it wouldn’t even be possible to play around with it on any box with less than 2Gb?
No, any box with less than 2GB mem + swap combined.
I’m running one on a $10/mo DigitalOcean cloud server, which is 1GB of ram, but I also set up a 2GB swapfile.
---
### Author:  [@victorhooi](https://meta.discourse.org/u/victorhooi)
#### Post date: [March 4, 2014, 2:01pm UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/23 "2014-03-04T14:01:15Z")
Ok, so I have it setup now on a higher-specced box, and I was able to run the backup =).
I downloaded the backup tar.gz file, and extracted it. Inside, I have:
- dump.sql
- meta.json
Just to clarify - the meta.json file has a timestamp as the version number - does this mean I need to match the exact same git commit in order to do a successful import/export? Is this going to be the case all the way until 1.0?
And the `dump.sql` - that’s just a straight PostgreSQL dump, right? So there isn’t really a specific Discourse import/export or backup format per se, it’s just a dump of the database.
Hmm, my original purpose of testing this feature was to see how I might import another forum ([www.lefora.com](http://www.lefora.com)) into Discourse.
If I want to do that - do I need to generate a `dump.sql` file exactly like this, including all the DDL statements, in order for Discourse to import? There’s no easier way/format?
---
### Author:  [@probus](https://meta.discourse.org/u/probus)
#### Post date: [March 4, 2014, 2:12pm UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/24 "2014-03-04T14:12:36Z")
I set up a Digital Ocean box with low memory and enough swap to mock around and I run into the problem where running backups or restores from the admin panel would fail with ‘Waiting for sidekiq to finish running jobs..’. and multiple lines of ‘Waiting for 1 jobs…’. I don’t think it has anything to do with memory. I tried to resize the DO droplet and increase memory to 2GB with no help.
Turns out even though (default) queue was empty, sidekiq had one email worker from my registration email that I needed to remove for it to run. I didn’t setup email for this install since I just wanted to play around and not use the install for anything else than testing. After clicking ‘Clear workers list’ it runs fine even with 512MB memory (+swap of course).
---
### Author:  [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [March 4, 2014, 2:29pm UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/25 "2014-03-04T14:29:07Z")
> [@victorhooi](#):
>
> Just to clarify - the meta.json file has a timestamp as the version number - does this mean I need to match the exact same git commit in order to do a successful import/export? Is this going to be the case all the way until 1.0?
The `meta.json` file currently only holds the current database version (which is the timestamp when the last migration was created). This value is used [during the restore](https://github.com/discourse/discourse/blob/master/lib/import/importer.rb#L153-L160) to make sure you’re not importing a newer version of the database without migrating first.
> [@victorhooi](#):
>
> And the `dump.sql` - that’s just a straight PostgreSQL dump, right?
Not _ **exactly** _.
We do use [the standard pg\_dump command](https://github.com/discourse/discourse/blob/master/lib/export/exporter.rb#L154-L172) to generate the dump but then we add a slight modification.
In order to limit the amount of downtime during the restoration, we [make sure to restore](https://github.com/discourse/discourse/blob/master/lib/export/exporter.rb#L184-L209) the backup in the `restore` schema instead of the standard `public` schema.
This allow us to [limit the downtime](https://github.com/discourse/discourse/blob/master/lib/import/importer.rb#L219-L230) to the amount of time it takes to switch the `public` schema to the `backup` schema, and the `restore` schema to the `public` schema 😉
> [@victorhooi](#):
>
> Hmm, my original purpose of testing this feature was to see how I might import another forum ([www.lefora.com](http://www.lefora.com)) into Discourse.
You will need to write custom code for that. You may want to take a look at
- @Sam’s [import\_blog](https://github.com/SamSaffron/blog/blob/master/script/import_blog.rb) script
- @eviltrout’s [disqus.thor](https://github.com/discourse/discourse/blob/master/lib/tasks/disqus.thor) task
for inspiration.
> [@victorhooi](#):
>
> There’s no easier way/format?
Unfortunately no. Welcome to a world of pain :rage1:
---
### Author:  [@victorhooi](https://meta.discourse.org/u/victorhooi)
#### Post date: [June 16, 2014, 7:05am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/26 "2014-06-16T07:05:36Z")
Hi,
Have there been any changes/updates on the Discourse import/migration front?
Or is improving import/export support still on the roadmap somewhere?
There was talk about this [Discourse migration service](https://meta.discourse.org/t/communiteq-migration-service-for-your-existing-forum/12201) releasing some of their code as open-source, however, that doesn’t appear to have eventuated.
Is anybody aware of any up-to-date migration/import tools for Discourse?
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [June 16, 2014, 7:10am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/27 "2014-06-16T07:10:19Z")
There are some migration tools, @techapj can you provide a howto topic for finding it and getting started? Can be a very basic howto for now, just the location of the code basically. Longer term we will produce a detailed howto for migrating a sample PHPbb forum, but it may take a few months for us to get to that.
But in the meantime, the migration / conversion code is open source, you just have to know where in the tree to find it. Warning, it is evolving rapidly and meant only for developers at this time.
---
### Author:  [@techAPJ](https://meta.discourse.org/u/techAPJ)
#### Post date: [June 17, 2014, 2:37pm UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/28 "2014-06-17T14:37:04Z")
Here is the howto for migration:
> [@Migrate from another forum to Discourse](https://meta.discourse.org/t/migrating-to-discourse-from-another-forum-software/16616/):
>
> mega This guide serves as a high level overview of the migration process. For specifics on the wide range of forum softwares to import, including Vanilla, phpBB3, SMF2, bbPress, and vBulletin, review our [dedicated tutorials](https://meta.discourse.org/tags/c/documentation/10/migrations). Not all scripts have tutorials, but an exhaustive list of available imports is available under the [script/import\_scripts](https://github.com/discourse/discourse/tree/main/script/import_scripts) folder on GitHub. For larger migrations, there is also a faster [script/bulk\_import](https://github.com/discourse/discourse/tree/main/script/bulk_import) system that writes directly to the database. So you want to mi…
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 8, 2014, 11:20am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/29 "2014-10-08T11:20:05Z")
This is totally implemented and quite rocking.
Backup/Restore on Discourse is quite a lifesaver at times.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [October 8, 2014, 11:20am UTC](https://meta.discourse.org/t/improving-import-and-export-support/12110/30 "2014-10-08T11:20:08Z")
[Previous page](https://meta.discourse.org/t/improving-import-and-export-support/12110.md?page=1)