Importing / migrating from phpBB3

That’s all pretty typical.

I think that the changes you suggest likely won’t break any other imports. Often there are a set of changes like these and then a bunch of other changes that are specific to the import and figuring which are which and testing again with only those would be a bunch of work so a PR doesn’t get created.

Glad you got it done!

3 Likes

Thank you Constanza, very helpful link. :+1:


Thanks for the information Jay. I’ll do PR later after I’m done with the migration if it can help to have an even better phpBB3 migration tool.

The current script also ignores [size=XXX] tags which I encountered during other migrations.
On my current migration, I did a quick script in my importer and used the value of XXX to replace these tags with <small>content</small> and <big>content</big> since they are supported by Discourse.
But that was a personal wish and it might be a more proper solution (generally speaking) to simply get rid of these [size] tags, like the import script already does with the [color] tags.

4 Likes

Thanks @Canapin for all your feedback. A PR is definitely welcome!

Also, a small teaser: We are working on a solution which should make all imports – not just phpBB – a lot better, faster, easier to customize and get rid of those pesky problems with the BBCode conversion… :wink:

4 Likes

While importing my posts, I got a bunch of these errors (not for all posts):

   251491 / 251672 ( 99.9%)  [14140 items/min]  Exception while creating post 354629. Skipping.
undefined method `[]' for nil:NilClass
/var/www/discourse/script/import_scripts/phpbb3/importers/post_importer.rb:66:in `block in map_first_post'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/core_ext/object/try.rb:15:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.4.1/lib/active_support/core_ext/object/try.rb:15:in `try'
/var/www/discourse/script/import_scripts/base.rb:576:in `create_post'
/var/www/discourse/script/import_scripts/base.rb:523:in `block in create_posts'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/mysql2/alias_method.rb:8:in `each'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-2.3.3/lib/patches/db/mysql2/alias_method.rb:8:in `each'
/var/www/discourse/script/import_scripts/base.rb:510:in `create_posts'
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:192:in `block in import_posts'
/var/www/discourse/script/import_scripts/base.rb:874:in `block in batches'
/var/www/discourse/script/import_scripts/base.rb:873:in `loop'
/var/www/discourse/script/import_scripts/base.rb:873:in `batches'
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:254:in `batches'
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:188:in `import_posts'
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:38:in `execute'
/var/www/discourse/script/import_scripts/base.rb:47:in `perform'
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:22:in `perform'
script/import_scripts/phpbb3.rb:33:in `<module:PhpBB3>'
script/import_scripts/phpbb3.rb:14:in `<module:ImportScripts>'
script/import_scripts/phpbb3.rb:13:in `<main>'

But I couldn’t figure out what causes this.

Any clue?

How can I check if I have actually post missing? Couldn’t find how to output the data (post content for example) corresponding to “Exception while creating post 354629”

I can’t recall having this error in my other import tests the previous days, but maybe I just didn’t pay attention… Could it be related to this known issue?

251490 / 251672 ( 99.9%)  [14140 items/min]  Parent post 337703 doesn't exist. Skipping 354628: vw-camper est malade !?

I ran the import script twice as it should, regarding this particular issue.

1 Like

Make sure that you set tag_mappings: {} in your settings.yml file if you aren’t using this feature.

3 Likes

I feel dumb! I even thought doing about that, but I was so much sure that I didn’t encounter this error before that I didn’t try… :facepalm:

It resolved the issue, thank you very much.


I supposed you can’t say much at this time, but any clue about how it will work, and when some useable import scripts will be useable using this new thing?
I might migrate another phpBB this year, so that could be very interesting for me to know a bit more. :slight_smile:

1 Like

Hi, i’m testing a phpbb 3.2 migration to discourse for a decently sized forum (30k topics / 600k posts) and almost everything works fine, apart for a couple of topics not imported (“Parent post xx doesn’t exist”), not solved with multiple runs, but it’s not an issue.
My main problem is that subsequent imports with fresh data (verified with the sha256sum in import/mysql/imported file) do not import fresh posts in Discourse. I’m facing an exception early in the process, dunno if it’s linked :

Failed to map post with ID 6815

BIGINT UNSIGNED value is out of range in '(`phpbb_prod`.`o`.`poll_option_total` - (select count(distinct `phpbb_prod`.`v`.`vote_user_id`) from `phpbb_prod`.`phpbb3_poll_votes` `v` join `phpbb_prod`.`phpbb3_users` `u` join `phpbb_prod`.`phpbb3_topics` `t` where ((`phpbb_prod`.`u`.`user_id` = `phpbb_prod`.`v`.`vote_user_id`) and (`phpbb_prod`.`v`.`topic_id` = `phpbb_prod`.`t`.`topic_id`) and (`phpbb_prod`.`v`.`poll_option_id` = `phpbb_prod`.`o`.`poll_option_id`) and (`phpbb_prod`.`t`.`topic_id` = `phpbb_prod

Is this an issue with embedded polls in posts, from the SQL displayed ?

have a nice day !

2 Likes

I can’t say much yet as it’s still under development. Plans can change… But “this year” is a good bet. :wink:

2 Likes

BIGINT UNSIGNED value is out of range

Hmm, are there that many anonymous votes in this poll?

You will need to fiddle with this SQL:

Something like this maybe? mysql - BIGINT UNSIGNED VALUE IS out of range My SQL - Stack Overflow
Please let me know when you’ve found a solution or create a PR with a fix. I’d appreciate it.

3 Likes

Polls are from the very beginning of the forum, I wasn’t in charge at this time, but I think anonymous users were never allowed to post/answer polls

With

 SET sql_mode = 'NO_UNSIGNED_SUBTRACTION';

I’m able to query all polls votes, and for 2 of them, I have negative “anonymous_votes”. It seems this forum somehow allowed anonymous voting, actually :slight_smile:

Thanks @gerhard for the tip, I’m going to dig deeper…

4 Likes

I am currently looking into migrating a phpBB forum to Discourse, the problem is that our phpBB is using Postgres instead of MySQL, so I would like to adapt the script to work with that too.

Could anyone give me some pointers about the best way to go about this? As ideally I would like for everyone to benefit from this work, so would like it to use an upstream-accepted approach instead of just quick and dirty hacking it to work.

From what I can tell it seems to be mostly a matter for adding the proper SQL statements for the PostgreSQL schema using the right DB adapter, however I am not sure how the whole orchestration for the Docker container needs to be adapted to properly spin up a PostgreSQL instance, if needed by the DB type specified in the config, to import the DB dump into and execute the statements against.

1 Like

There’s a bulk import script for phpBB running on postgresql. Did you see it? discourse/phpbb_postgresql.rb at main · discourse/discourse · GitHub

However, I don’t think it has been used recently, so it’s probably broken and might not work with current phpBB versions because the Markdown storage format changed.

:+1:

Whatever path you choose, I wouldn’t spend too much time on gold plating your solution unless you really want to. The current import scripts will be deprecated sometime this year…

1 Like

Thanks, I wasn’t aware of that! And thanks for the heads-up that the scripts are going to be deprecated, so I guess it might not be worth upstreaming such a major feature for those if they will anyway be deprecated soon? Is there a replacement planned for them?

Yes, of course there will be a replacement.

1 Like

Another important thing I was wondering, do I need to set anything other than disable_edit_notifications to ensure none of the imported users are emailed by Discourse for anything that I import? As currently the instance is not public and we are just experimenting with the import and probably needing severals tries until it is good enough, I want to avoid any emails being sent.

You are looking for the disable_emails site setting…

2 Likes