Thanks to ruby-bbcode and @haiku’s ruby-bbcode-to-md, some bbcode can optionally be converted to markdown before importing into Discourse. I made a fork that does some conversion that is specific for Discourse. The script will also handle some other phpbb-specific markup, like smileys and links that use <!-- m -->, etc.
Usage:
Edit the bbpress3.rb file to change the MySQL connection info (PHPBB_DB, and Mysql2::Client.new(...)) so it connects to a local copy of the phpBB database.
To convert phpBB’s bbcode to markdown before importing, install the ruby-bbcode-to-md gem. In a temp directory:
That’s how I’ve been doing it, but if you can connect to a remote mysql server, then that will work too. Just update the connection info to whatever works in the script.
If the PHPBB board (gah, to many 'B’s) is closed for maintenance though, that shouldn’t be a problem as very few people with specific privileges would be able to do anything on the board at the time.
Neil, as a creator of many import scripts, cleanup scripts and scheduled tasks for processes, I just want to say that there is a special rung in hell for import processes.
Seriously though, I give you much respect for doing ‘invisible-to-end-user’ work.
Probably a noobish question but I’ve edited the file and scp’d it to my server, where is the script/import_scripts folder if you used the docker setup?
Also, does this take care of not spamming users when new posts are created?
I’m trying out the phpbb3.rb conversion script and am having a little trouble. I suspect it’s a basic ruby thing… I’ve new to ruby.
What I’ve done:
install discourse locally with discourse_docker
Signed into discourse and created my admin account. Things look good, basically empty forum.
Entered the discourse conatiner
Edited phpbb3.rb to set the mysql setting
installed mysql-client, libmysql-dev,
gems install mysql2
I’m getting an error about missing roles in my DB. Does anyone have a suggestion? I know the stack trace suggest that I create a DB, but I already have one The stack trace is below
root@ubuntu-app:/var/www/discourse/script/import_scripts# ruby phpbb3.rb
2014-08-03T19:54:58Z 1990 TID-otwl8st54 INFO: Sidekiq client with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:898:in `rescue in connect': FATAL: role "root" does not exist (ActiveRecord::NoDatabaseError)
Run `$ bin/rake db:create db:migrate` to create your database
from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:888:in `connect'
from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-
...
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /var/www/discourse/script/import_scripts/base.rb:19:in `initialize'
from phpbb3.rb:14:in `initialize'
from phpbb3.rb:295:in `new'
from phpbb3.rb:295:in `<main>'
You need to set the Rails environment: RAILS_ENV=production sudo -E -u discourse ruby phpbb3.rb
Also, you can avoid the repetitive sudo cruft by becoming the discourse user instead of editing and running these scripts as root. Just run su - discourse after entering the container.
creating users
Skipping user id 1 because email is blank
641 / 1041 ( 61.6%) Skipping user id 68191 because email is blank
1043 / 1041 (100.2%)
creating categories
Admin
The Kiteboarding
The Gear
Everything Else
Snowboarding/Ski
Other sports - MTB, Wake, SUP, Surf and anything else
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/validations.rb:57:in `save!': Validation failed: Category Name is too long (maximum is 50 characters) (ActiveRecord::RecordInvalid)
I think this can be fixed by changing the mysql select to trim category names to 50 chars. I’ll put that in a pull request.
I’ve tried running the importer again - this time using bbcode-to-md as parameter.
I built and installed the gem using the instructions in the first post.
Now, when I’m executing ruby script/import_scripts/phpbb3.rb bbcode-to-md I get the following error:
/usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.6.5/lib/bundler/runtime.rb:34:in `block in setup': You have already activated i18n 0.6.11, but your Gemfile requires i18n 0.6.9. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
I know why it’s happening. When I install the ruby-bbcode-to-md-0.0.13.gem it also installs some dependencies which are the reason for this error. Am I doing something wrong during the installation or how can I fix this?
bundler: not executable: script/import_scripts/phpbb3.rb
I tried using bundle exec ruby script/import_scripts/phpbb3.rb bbcode-to-md too which results in the following error:
/var/www/discourse/script/import_scripts/base.rb:11:in `require': cannot load such file -- ruby-bbcode-to-md (LoadError)
from /var/www/discourse/script/import_scripts/base.rb:11:in `<top (required)>'
from script/import_scripts/phpbb3.rb:1:in `require'
from script/import_scripts/phpbb3.rb:1:in `<main>'
Here are the steps to reproduce this:
Bootstrap a new Docker image and log in with ./launcher ssh app
Install the mysql2 gem as root:
apt-get update && apt-get install libmysqlclient-dev
gem install mysql2
Change the user and execute the importer:
su - discourse
cd /var/www/discourse
RAILS_ENV=production ruby script/import_scripts/phpbb3.rb bbcode-to-md
It works without the bbcode-to-md, but fails when that parameter is used. Here’s the complete error message with stacktrace:
/usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.7.0/lib/bundler/runtime.rb:34:in `block in setup': You have already activated i18n 0.6.11, but your Gemfile requires i18n 0.6.9. Prepending `bundle exec` to your command may solve this. (Gem::LoadError)
from /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.7.0/lib/bundler/runtime.rb:19:in `setup'
from /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.7.0/lib/bundler.rb:121:in `setup'
from /usr/local/lib/ruby/gems/2.0.0/gems/bundler-1.7.0/lib/bundler/setup.rb:7:in `<top (required)>'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:135:in `require'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:144:in `require'
from /var/www/discourse/config/boot.rb:6:in `<top (required)>'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /var/www/discourse/config/application.rb:1:in `<top (required)>'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /var/www/discourse/config/environment.rb:2:in `<top (required)>'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /var/www/discourse/script/import_scripts/base.rb:19:in `initialize'
from script/import_scripts/phpbb3.rb:14:in `initialize'
from script/import_scripts/phpbb3.rb:295:in `new'
from script/import_scripts/phpbb3.rb:295:in `<main>'
As I said in my previous post I think it’s because of the ruby-bbcode-to-md gem, which installs a bunch of dependencies:
The stack trace is below