Migrate a phpBB3 forum to Discourse

I can at least say that it not only happens for those cases where older posts link to newer ones.

I got one example where I first found the issue.

Original Post ID: 842948 links to Page 22 of a thread that already existed and at least the start post (&start=220) has an ID lower than the linking post (842880)

After import this original post (which was posted only days before the dump was created) links to a 7 year old thread where the first post has an original ID of 1353.

I can’t seem to find any hints on why this happens, any similarities in numbers in any way… The original link was just a plain posted URL which was auto linked by phpBB. Those links in general don’t cause any issues - they work in several other places.

Generally I don’t expect the import to go well immediately. I’m really happy to how it went so far but I assume this issue might be one of the hardest to fix, especially because I don’t have any clue of setting up a dev/debug environment with Discourse yet. Might need to dig into that soon.

Hi,
From what I remember, the phpbb importer was by default filling the Discourse user full name by guessing from the user email address. Am I right and is it still the case? I can’t find anything about this in the importer files…

This is something I wouldn’t want to happen in the next forum I may import.

Also, another question.

On the current phpbb forum, there are custom user fields (like users facebook or instagram urls). I would like to import this in Discourse custom fields. I guess I’ll do something like first installing and configuring Discourse by adding these custom user field, then importing the phpbb data from my custom import script?

That’s not happening anymore.

That sounds like a good plan.

Is there a reliable way to estimate the monthly volume of emails that Discourse will send to users after a migration from phpbb? New user registrations, mentions and replies, weekly digests and so on… Phpbb sends very few emails by default and I think we’ll have to change the current email provider.
I don’t currently have much stats from the actual phpbb. It exists since 2013, with 200000 messages and 5500 members. New members register everyday.

How many posts per day?

I dont know yet. I have currently nothing but the public stats. Maybe 20.

With only 20 posts per day I think you’d probably be looking at 3000 emails a month, at most, well within the free Mailgun plan.

However seeing as you’re migrating with lots of users I’d recommend you turn off the digest emails for everyone who hasn’t been to your forum within the last month or two (they can always turn it on themselves if they want). You can do that with a query in the Rails console, but it’s been a year since I did it for mine, so I don’t remember the exact code sorry.

That should actually be something the import script could do for you IMO, ideally with a setting for the time since last visit to enable digest emails.

I disagree. That’s not something an import script should do, but there’s a site setting for this: suppress digest email after days

Indeed After my first phpbb import on my current forum, I had to decrease the default value; my email provider automatically blocked the used address because of all the digest emails sent. It was flagged as spam.

It’s not quite the same thing. There’s a difference IMO between sending digest emails to someone for X months since their last visit when they signed up for it - just because they don’t visit the forum doesn’t mean they’re not reading the digest emails - and sending digest emails to a user who never signed up for them because they registered in the previous forum, which may not even be legal in some jurisdictions! You could simply turn off digests for every migrated user, and they can turn then on for themselves, or you could do as I did and turn them off for everyone except those who visited within the last month and who therefore would have seen the “we’re going to migrate the forum soon” announcement. I think that’s a fair middle ground.

It’s also practical - our phpBB forum had gained thousands of spammer accounts within the last year and sending the digest to them four times a month would have used all our free mailgun allowance and then started costing a lot. Reducing “clean up inactive users after days” eventually cleaned them up.

Not sure if this belongs here or in a separate topic. But it’s about migrating from phpbb 3 so here I am.

I have successfully migrated to discourse with 1 database. However I have 3 different, separate communities (3 different languages, 3 different phpbb installs), which ideally I would all have merged into one big happy discourse forum.

My question is: how would I go about doing this? What do I need to change in the import script / phpbb 3 database / anywhere else?
There are a few users who have accounts on both forums, (a few maybe even on all 3), but there is no single sign on so they have a different account on each forum with the same username. They are all different installs and different databases.

It would be awesome if this was feasible!

Thank you in advance for any help!

It won’t work out of the box, because the import script stores the original IDs in the Discourse database and it won’t import data from an additional forum if it finds a record with the same ID. However, I can think of two ways of solving this:

1. Hard: You need to edit the import script in various places and add a prefix for each forum to the import_id custom fields stored for posts, topics, categories and users.

2. Easy, but you will lose the ability for incremental imports: Delete the custom fields after each import.

   TopicCustomField.where(name: 'import_id').delete_all
   PostCustomField.where(name: 'import_id').delete_all
   CategoryCustomField.where(name: 'import_id').delete_all
   UserCustomField.where(name: 'import_id').delete_all
   

Solution 2 is untested, but in theory it should work. Create a backup before you give it a try.

After the first tests before new years we’re getting back to business with our migration project now. One of the biggest issues still is the failed link rewriting and i’m thinking about how to handle that best,

@gerhard my coding knowledge is very basic and I do have zero to no knowledge in rails in general and discourse structure in detail - hence please bear with me for the question: Would there be any chance to handle the case where the importer doesn’t find an imported post with the given ID from the link and maybe instead of replacing it with a wrong link append some Text that informs the user/admin and makes these problematic links findable after the import?

My main concern with how it currently works is that those Links just look like successfully replaced and you don’t have a way to find and manually fix them after the import.

Thanks for any help you could give me - it’s really appreciated.

One way to do it is to re-write the script so that it doesn’t replace those links until after all posts are imported. If you need help and have a budget you can contact me or post in marketplace.

Unfortunately this project is on a small budget and I guess the issue isn’t important enoug to spend the money on that one but definitely good to know you would be available for that.

One other question: I noticed bbcode seems to get handled differently when importing. [size] for example just is left as text while [color] seems to be removed completely. I wanted to reuse color and found there’s a discourse plugin for that but then I noticed it seems to be removed on import. The initial post says bbcodes can be imported but I couldn’t find any setting that controls which ones may get imported or not.

If there’s a chance I’d prefer to remove the size tags and instead leave color there. Is there something I am missing on how this is handled and how it could probably be changed?

I just discovered that topics that have been pruned on my phpbb forum, and which I thought were gone, have somehow been transferred to discourse.

How is that possible?

I’m trying to do this, but I’m lost at the instructions, which settings file do you mean here? and when do I do this? before importing or after?
thank you in advance!

edit: so I think you mean the emoji settings files in admin CP? Is there a way to do this with importing a text file or something? I have about 100 custom smilies, would prefer to somehow copy that mapping into discourse, is that possible? (If I could copy the smily.pak phpbb file somehow into discourse settings?)

You did see the importer’s settings file, right?

https://github.com/discourse/discourse/blob/1a31a403cefda7ead706076cc0a0decdb38a80c8/script/import_scripts/phpbb3/settings.yml#L62-L69

I guess phpBB handles pruned topics differently than regular deleted topics. In that case it’s not supported right now. You’ll have to find out how phpBB marks pruned topics as deleted and amend the import script.

Riiiiiiiiiight yes ofcourse! It has been too long since I did the first converting. I totally forgot that one, thank you!

About the pruned topics, yes, I thought it was something like that, it is not a big deal gor my forum but it was surprising and maybe a headsup for others that this may occur.