Importing from phpBB3


(Todd Taylor) #234

thats what I was thinking as well, but is it a post? If so, I can just delete the post and try again, but not sure how to zero-in on the trouble maker

(Jay Pfaffman) #235

I believe the error is in clean_bbcodes. You can add some debugging puts with the message ID or something like that. Alternately, you could add rescue code to that function to make it do something more friendly.


Hey there,

I have successfull imported an old phpbb board to Discourse a couple of weeks ago. Everything went well and it’s still running in a good way. Now we would like to import another board which is running on phpbb3 aswell. Hence, a few questions:

  1. Is it possible to import an old phpbb3 to a running DC which is already full with content?

  2. There are existing usernames in DC which are the same like in the phpbb board. Will there be any issues with importing those to DC? Is it probably better to rename them first, or will the accounts beeing meged (somehow)?

Thanks for your helpfull tool and your answers.

(Gerhard Schlager) #237

It depends where the current content came from. If it is user-generated everything’s fine.
If (at least parts of it) were imported from phpBB3 (or some other forum software) you’ll need to delete a few things from the Discourse database, otherwise the importer will skip some content. The following SQL should do the trick:

DELETE FROM category_custom_fields WHERE name = 'import_id';
DELETE FROM post_custom_fields WHERE name = 'import_id';
DELETE FROM user_custom_fields WHERE name IN ('import_id', 'import_username');

This should work without any manual intervention. The importer tries to find an existing user which has the same email address or username before it creates a new user. It will reuse the existing user if it finds a match.

:warning: Oh, and you shouldn’t try this with your production database. Please create a backup and test the import in your development environment or at least a copy of your Discourse instance.

(Erlend Sogge Heggen) #238

Alternatively, I believe it should also be possible to:

  • Migrate the secondary phpBB forum to a brand new Discourse staging instance
  • Export the content as categories
  • Import those category exports into your existing forum (of course, test on a duplicate test instance first).

(matthias ) #239

thank you so much for this importer which seems so carefully done and well supported!
i have a problem though, with me it does not work. i get the error messages below.
i tried a few times and always get this. i have a 2 gb digital ocean droplet and installed discourse by the book and also prepared all the importer steps super carefully. my mysql file has about 2.5 gbyte / about 500.000 entries.
could you give me a hint what i could try in this case?
thank you :slight_smile: matthias

/var/www/discourse/script/import_scripts/phpbb3/importer.rb:52:in []': no implicit conversion of Symbol into Integer (TypeError) from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:52:inblock in disable_setting_validators’
from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:51:in each' from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:51:indisable_setting_validators’
from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:39:in change_site_settings' from /var/www/discourse/script/import_scripts/base.rb:44:inperform’
from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:20:in perform' from script/import_scripts/phpbb3.rb:31:inmodule:PhpBB3
from script/import_scripts/phpbb3.rb:12:in <module:ImportScripts>' from script/import_scripts/phpbb3.rb:11:in

(Gerhard Schlager) #240

This seems to be related to changes in Discourse that were made a few days ago. I’ll fix it.
In the mean time you could change to the beta channel in app.yml. It should work there.

After changing your app.yml you’ll need to stop the container, delete all data and start again.

./launcher stop app
rm -r /var/discourse/shared/standalone
./launcher rebuild app

(matthias ) #241

Hey Gerhard,
thank you so much for the quick response and a hint that seems to have solved all my problems. :slight_smile:
The beta didn’t work for me but as you referred to the changes having happened a few days ago I installed last month’s version of dicsourse and with this the import was perfectly fine so far.
Once sidekiq is finished and i updated to the current version i’ll report again

(Arpit Jalan) #242

We now have a phpBB PostgreSQL bulk import script: discourse/phpbb_postgresql.rb at master · discourse/discourse · GitHub

(matthias ) #243

Hi Gerhard,
just to let you know it all worked fine with this method, the forum is happily up and running.
Thank you :slight_smile:

(Arseniy Sheatakov) #244

Hello Gerhard,
I have similar problem with phpbb3 import:

loading existing groups...
loading existing users...
loading existing categories...
loading existing posts...
loading existing topics...
/var/www/discourse/script/import_scripts/phpbb3/importer.rb:52:in `[]': no implicit conversion of Symbol into Integer (TypeError)
        from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:52:in `block in disable_setting_validators'
        from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:51:in `each'
        from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:51:in `disable_setting_validators'
        from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:39:in `change_site_settings'
        from /var/www/discourse/script/import_scripts/base.rb:44:in `perform'
        from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:20:in `perform'
        from script/import_scripts/phpbb3.rb:31:in `<module:PhpBB3>'
        from script/import_scripts/phpbb3.rb:12:in `<module:ImportScripts>'
        from script/import_scripts/phpbb3.rb:11:in `<main>'

I have tried to change “version” to beta in both app.yml and after second failure for import.yml, clean data and rebuld, but no luck. It’s failing with the same error every time.

I’m trying to import it from phpbb 3.1.11 and Discourse version is v1.9.0.beta7 +341.

(Arseniy Sheatakov) #245

So I bypassed the problem by commenting-out body of change_site_settings function at importer.rb:39 and then manually set “max_image_size_kb” and “max_attachment_size_kb” to sufficient size to avoid problems with attachment import.

I still need to check how will import behave, but so far it’s seems to be working.

PS: Next problems I seen:

  • I suppose I’ll also need to setup more allowed attachment types in order to get them all imported.
  • Another problem is that it’s fail to import users with “invalid” temporary emails like from Mailinator, but I guess I can just disable this check before import and enable it afterwards.

(Gerhard Schlager) #246

Should be fixed soon:

(Gerhard Schlager) #247

You shouldn’t need to do that. During import all extensions are allowed. is the only domain that is blacklisted by default. You’ll need to change the email_domains_blacklist site setting in order to allow importing users with email addresses from that domain.

(Arseniy Sheatakov) #248

Great! Thanks for such fast fix.

How soon I can expect it to appear on “beta” channel or should I just apply patch from git?

I suppose it’s might have something to do with me commenting-out all change_site_settings content. :slight_smile:

(Gerhard Schlager) #249

Could be a few days before it’s merged into the beta branch… So, if you need it right now either apply the patch manually or use the tests-passed branch.

(Arseniy Sheatakov) #250

Thanks. I’ll still need to play with other import settings and see how well it’s work.
So at least for now I’ll use patch.

For some reason Discourse don’t want to parse some of quote bbcode like:

[quote="BigMgamer2309"]This is how it should look like.

So quotes appear as plain text. Though it does parse it well if [/quote] tags are on own lines. I’ll certainly test how it work with use_bbcode_to_md enabled.

PS: My work in progress is here, but not sure how long until I stop it and start to re-export.

UPD: Just in case forum I’m importing have database in utf8, but problem might have something to do with the fact it’s was just imported from phpbb2 to phpbb3.

(Jeff Atwood) #251

It was already merged

(Arseniy Sheatakov) #252

Does it mean it’s immediately appear if I rebuild?
Sorry, I never used discourse before so there are many questions.

UPD: Ok , got it, tests-passed is just branch in git just like beta. Good.

(Gerhard Schlager) #253

Yeah, the quotes need to be fixed in the importer because [quote] must be on a new line after the recent switch to CommonMark. I’ve put it on my todo list.