(Superseded) How to migrate / import from SMF2 to Discourse

:mega: This doc is now superseded by this other SMF2 guide: Migrate an SMF2 forum to Discourse

I really wanted to migrate our good ol’ SMF2 forum to Discourse. We have 1000+ members and ~20k posts in ~2k topics.
On SMF2, we did not use polls nor attachments – actually, not everyone even had an avatar. Some posts heavily relied on “advanced” BBCode, though.

Here’s how we pulled this off.

Pre-requisites

Just install Discourse the usual way.
I did add the Discourse BBCode plugin to handle lists, size, colors etc. out of the box.

If you can’t open a remote connection to your SMF2 MySQL database, you will have to import your data from a dump into a temporary Docker container.

If your database accepts remote connections – or runs on the same host, you can skip this step.

A temporary Docker MySQL container

Dump your database:

cd /tmp/
mysqldump –u[user name] –p[password] [database name] > sqldump.sql

Create the temporary Docker MySQL container:

cd /tmp/
docker run -d -e MYSQL_ROOT_PASSWORD=pass -e MYSQL_USER=user -e MYSQL_PASSWORD=pass -e MYSQL_DATABASE=db -v "$PWD":/backup --name=mysql mysql

Then enter it:

docker exec -it mysql bash

Import your database dump:

mysql -uuser -ppass db < /backup/sqldump.sql

If you get the following error:

ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)

Just wait a minute for the container to be fully booted.

For database compatibility, I had to change our temporary database authentification plugin, but I suppose your mileage may vary:

mysql -uuser -ppass db

Use this query:

ALTER USER user
IDENTIFIED WITH mysql_native_password
BY 'pass';

Then logout and check your MySQL instance IP address:

exit
docker inspect mysql | grep IPAddress

You now have a working copy of your SMF2 database.

A temporary Discourse import container

Let’s create a new container called “import”, based off the default app.yml container:

cd /var/discourse
cp containers/app.yml containers/import.yml

nano containers/import.yml

Add the mysql-dep template to your new import container:

templates:
  - "templates/postgres.template.yml"
  - "templates/redis.template.yml"
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  #- "templates/web.ssl.template.yml"
  #- "templates/web.letsencrypt.ssl.template.yml"
  - "templates/import/mysql-dep.template.yml"

Then just:

/var/discourse/launcher stop app
/var/discourse/launcher rebuild import

Importing into Discourse

Once the import container is rebuilt, we have to enter it for our latest tweak:

/var/discourse/launcher enter import
cd /var/www/discourse/ # the script has to be launched from that precise directory, but you should have landed there anyway
su discourse -c "bundle exec ruby script/import_scripts/smf2.rb -h 172.17.0.3 -u user -p pass -d db -f smf_ -t Europe/Paris"
  • -h: your SMF2 MySQL database hostname (here, a Docker container’s IP) ;
  • -u: SMF2 MySQL user;
  • -p: SMF2 MySQL password;
  • -d: SMF2 MySQL database;
  • -f: SMF2 tables prefix;
  • -t: SMF2 timezone.

Cleanup

If you created it, stop and remove the temporary MySQL Docker container:

docker stop mysql
docker rmi mysql

Stop the Discourse “import” container and use the launcher to cleanup:

/var/discourse/launcher destroy import
/var/discourse/launcher cleanup

I actually managed to destroy my main container :slight_smile: No biggies:

/var/discourse/launcher rebuild app

Discourse settings after import

Some default settings might cause problems with imported posts and permissions. Change these according to your needs from the Discourse settings panel:

  • min topic title length: I changed it to 3 – topics which title were too short proved very hard to work with (couldn’t be moved, etc.) ;
  • title prettify: I had to turn it off – topics which title didn’t start with an uppercase letter proved very hard to work with (couldn’t be moved, etc.).

Bibliography

10 Likes

Thanks for providing this. Maybe @marcozambi can use this for the guide he was planning to write after his successful import (including attachments)?

See here:

Also can anybody from the Discourse team let us know what happened to the old topic that was a guide for SMF2 imports? I can no longer find it via search.

1 Like

Still determined to contribute, as soon as all the details will be in place. I’m taking notes of everything and I’ll write someting asap

1 Like

poke @codinghorror @sam @HAWK

I think we can combine what @_vincent has compiled here, what was in the “missing” topic, and a few notes from @marcozambi’s migration to make a pretty comprehensive guide for migrating SMF2 to Discourse.

I have undeleted it here: https://meta.discourse.org/t/importer-for-simple-machines-2-forums/17656

I am assuming it was deleted because it contained info that is no longer up to date so grab what you need from it and we’ll remove it again to avoid future confusion.

1 Like

Thanks! For the most part it was up-to-date. I did a migration with it a couple months ago and I would have been lost as f*$# without it.

I think the guide isn’t the problem, so much as the importer itself could use a little attention in a few areas. Also, if possible, we should change some of the MySQL settings in the Discourse docker container when it gets created to prevent connection timeouts. That would probably take care of everything IMO.

1 Like

Ok, it’s out. SMF2 to Discourse - The Ultimate Guide
I hope the Guide is readable enough, as I am not a native english speaker. It goes without saying that any constructive feedback is more than welcome.

3 Likes

Hello guys,

Let me start that you did a wonderful job with [SMF2 to Discourse - The Ultimate Guide] and this migration tool.

I could migrate a huge SMF instante (61k members, 400k posts, 350k, topics).
I did a dry run and everything worked as it should.

My question is: will this script override Discourse specific settings like the ones in /admin/site_settings/ ? - I’m asking this as I want to make a final migration of the content that came up in the meantime on the SMF site, but I would not want to overwrite the settings already configured in /admin/site_settings/.

Thank you

1 Like

No it should not override any of the site settings.

1 Like

Thank you so much.

I don’t have the expertise to “read” the script and what exactly it is copying over. I was hoping the same

Thank you for the guide.

I’m stuck right after you say:

For database compatibility, I had to change our temporary database authentification plugin, but I suppose your mileage may vary:

I get the error:

Plugin caching_sha2_password could not be loaded: /usr/lib/x86_64-linux-gnu/mariadb19/plugin/caching_sha2_password.so: cannot open shared object file: No such file or directory (Mysql2::Error::ConnectionError)

I wonder if you could give me a clue as to how fix this please? Presumably it’s related to changing the ‘database authentication plugin’? If so, how do I do that please?

Thank you!

Next I tried importing the DB into a separate MySQL DB outside of Docker, etc. It’s running on localhost.

Now when I run the script I get this error at the same point as before:

from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.3/lib/mysql2/client.rb:90:in initialize' /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/mysql2-0.5.3/lib/mysql2/client.rb:90:in connect’: Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2) (Mysql2::Error::ConnectionError)

I have tried waiting for a period and re-running as per the guidance in the guide - but the same result always occurs unfortunately. Can anyone help me please?

Thank you

Here is a screenshot 2020-12-02_00-00-16.png - Google Drive