Migrate a phpBB3 forum to Discourse

After having the importer successfully used on my first testing-environment, I now wanted to import the old forum again / freshly into a new should-become-the-real-environment, but now it failed.

I get an error:
/var/www/discourse/plugins/discourse-migratepassword/plugin.rb:71:in `crypt’: incompatible character encodings: ASCII-8BIT and UTF-8 (Encoding::CompatibilityError).

Appearantly this happens oh the third user, and this one is with an umlaut.

Now, what did I change since it worked?

  • new VM with Ubuntu 18.04.3 LTS (old was a 16)
  • updated Discourse (latest)
  • set “unicode usernames” with [äöüßÄÖÜẞ] since loads of my users have Umlauts in their Usernames (first import was without this setting and Users got imported by losing the dots on the umlauts)
  • preinstalled migratepassword (last time installed it later).

Any Idea whereon it fails? I’d really like to have my users migrated without having changed usernames.

1 Like

Looks like it’s triggering a bug in the migratepassword plugin. It works in my tests, so I’m not sure what’s happening here. I suggest you disable the plugin during the migration. It’s not needed during the migration anyway.

6 Likes

Thank you - that worked :slight_smile:

2 Likes

hello, I’m trying to do a testrun on my local environment to see if I can get my forum migrated to discourse.

I used this install guide to install discourse

I have a phpbb3 database export locally.
I have installed a local discourse on my mac with docker for desktop.

I am now trying to follow the instructions in this topic but I get stuck because I use docker, but not ubuntu, and it is local environment. So I don’t know how to proceed :frowning:
there is no app.yml in my discourse directory.

could anyone help me with what steps to take ?

many thanks in advance!

1 Like

I’m sorry, I don’t have any experience with Mac, Docker and your setup. You might be able to roughly follow Importing using development environment… but I can’t really give you any support for this.

1 Like

I think you might be better off following the “official installation guide” (even on your local mac) and running the installer that way.

4 Likes

Hurray, I have successfully done an import!
In the end, I just decided to get a DO droplet and go from there. And it worked!

Question about sidekiq, @gerhard you say:

What does this do exactly? Do I have to wait until it’s finished before I can do anything in the forum?

2 Likes

Unless you need to backup to restore to a different server, you’re fine. You can go ahead and use it and sidekiq will process stuff.

4 Likes

Hi, currently I’m trying to import data of a phpBB 3.1 forum to Discourse. But importing polls doesn’t work for me:

If the thread owner (poll creator) was deleted, I’ll receive the following error message during import:

Error creating post 70978. Skipping. [“You are not allowed to create polls.”]

Regarding the following poll options
<12
12
>12
I receive the following error message:
Error creating post 345655. Skipping. [“Poll must have different options.”]

So, how do I solve this?

Thanks in advance

Best regards

1 Like

Do you have polls enabled?

1 Like

yes, some polls are imported but not all.

1 Like

You’ll need to create that user or modify the script to, say, use system if the owner doesn’t exist.

1 Like

Normal threads from deleted users are imported properly. How can i modify the import script so that polls from deleted users are imported as well?

1 Like

I’m currently trying a migration from phpBB the first time and I wonder if I transfer a Discourse backup later will it contain all meta information from the import (like the original post ID from phpBB). Reason is I’m thinking about doing the import on a bigger machine for quicker import and run the forum on a VPS later but may do another incremental import after moving everything to the VPS. I wonder if it’s enough to do a discourse backup / reimport or if I better dump the DB any other way (which?)

And one more question: Are the URL rewrites in the importer safe for a domain change afterwards? I was planning to use a different domain or sub domain for the initial migration and later switch to the real domain.

3 Likes

Yes, the backup contains the original post ID in a custom field. Running the final import on the cloud should be fine.

Changing hostname does require a bit of work. Change the domain name or rename my Discourse?

4 Likes

I just finished the first test migration and before anything else I have to say thank you to @gerhard and everyone else who contributed in this. It’s amazing how well this worked on a first try. It just took me roughly 24 hours to import shy of 900k posts with users, private messages and stuff and things really look good for a first shot. It’s just awesome to have such a great importer to get this done.

There’s a few things though and the most important one that I’m currently trying to hunt down is that some internal links seem to have gone completely wrong while others worked quite well. To be more precise way more seem to be right than wrong. I’m trying to find a pattern which ones didn’t work. Can anyone offer some insights on how this internal link changing works. Are there probably some gotchas where it might fail?

Sidenote: Right after the import I saw all avatars missing in quotes but then found out that they seem to be generated and looks like after 20 minutes everything’s there now. Fascinating :wink:

Regarding import time: This was on a Hetzner 4core VPS with exclusive cpu cores - I think I’m going to retry this on a bare metal server just for the import to improve the migration time. Need to see how moving the discourse backup works first.

5 Likes

The code for replacing internal links is here: https://github.com/discourse/discourse/blob/43ddf60cdf27a865b7b1aa0d54a144a3e46c74cf/script/import_scripts/phpbb3/support/text_processor.rb#L73-L114

Rewriting internal links will fail when a post contains a link to another post that hasn’t been imported yet. Posts are imported in the order of their original creation.

6 Likes

That’s an interesting hint. Can’t see why this would have been the case in some of the cases I checked but I will definitely go down that route. Weird thing is that it links the wrong posts in my case. In case of a fail I’d assume it would do nothing or leave the link alone. Unfortunately my ruby knowledge is close to zero but I’ll have a look at the code anyway.

Is there any way to work around this issue? I think it could happen a lot with tutorial style posts that get edited and ammended over and over and links are added. Any chance to get around this apart from manually changing those posts?

Thanks for helping! :+1:

1 Like

Hmm just noticed that user groups haven’t been imported and also saw that they aren’t listed in the initial post here but they are indeed one of the first things that get listed on the importer. I wonder if there’s a way to import them including attaching users to those groups or if that’s just not yet working? Iin the latter case we would likely need to find a workaround.

1 Like

Yes. That’s weird. That suggests that some of your local links are in some format other than what the import script expects and that makes the regex get the wrong post to look for. If someone has gone back and edited a post to point to a post that didn’t exist then the post was originally written (have you confirmed that is the case?) then they might have used some other means to make the link that confused the script.

It’s surprising the degree to which each import (especially for a mature forum) is a snowflake. It’s fairly rare that an import is just a matter of running the script (but, I suppose, people are more likely hire me if their import does have complications).

1 Like