Re: Migrate a phpBB3 forum to Discourse

Support Migration

AI-generated summary

Re: Migrate a phpBB3 forum to Discourse

The user, JPD, is experiencing issues migrating a phpBB3 forum to Discourse. The initial problem was related to a low memory condition (1GB RAM) on a Lightsail server, which caused the import procedure to fail. After upgrading to 2GB RAM, the procedure was faster, but the import script still didn’t work properly for phpBB 3.3.3.

JPD reported that only user data was imported correctly, but no forums, categories, or posts were imported. The MySQL dump file was small (2.66MB), and all table prefixes were as required.

Arkshine and pfaffman suggested checking for errors and increasing RAM to at least 4GB. However, JPD believed that RAM was not the issue.

JPD tried to create a mapping section in the settings.yml file, but the import container created two main categories and one empty subcategory. The procedure imported all posts, but they were orphaned.

After making some fixes to the phpbb_mysql.sql file, JPD encountered another issue, where the script displayed a long list of “Parent post doesn’t exist. Skipping” errors and did not import any posts.

Conclusion

The default scripts may not work with phpBB 3.3.3 tables, as JPD experienced issues with the migration script. JPD eventually used a fresh installation and entered some text in the forum_desc field to solve the issue.

1

Hi,

Is phpBB3.3.3 supported or is it strictly 3.3.0? When performing the 1.5 step:

/var/discourse/launcher enter import

from Migrate a phpBB3 forum to Discourse - documentation / sysadmin - Discourse Meta
I’m receiving (already tried twice) the response:

x86_64 arch detected.
Error response from daemon: No such container: import

If I use
/var/discourse/launcher enter import.yml

the response is:

x86_64 arch detected.
ERROR: containers/import.yml.yml does not exist or is not readable.
Available configs ( app, import )

Any chances this can work?

Thanks

2

Hi, welcome! :wave:

I assume you followed the guide from the start; is /var/discourse/containers/import.yml file actually present?

It looks like the config file has been deleted. :thinking:

3

Yes, it’s like in the message above, both app.yml and import.yml are present in the “containers” directory.

4

If I understand well, you had no issue following the previous steps, creating/editing the import.yml next to app.yml, rebuilding the import container, and now you can’t even enter it. Is that right? That’s odd for sure!

What returns this command? ls -l /var/discourse/containers

5

It’s still the same: app.yml and import.yml (plus a “bak” file of app.yml).

There might be “issues” as I’ve been trying to install this on Lightsail 1GB (which is initially 940 and after updating Ubuntu becomes “921MB”). I used the official installation and what was installed was something like 3.3.* beta-something (while Wiki shows 3.2 as the most recent stable version). Installation takes 1h15min-1h20min with 4GB swap. the “launcher rebuild import” about 2h.
Could something be incorrectly terminated due to the low memory condition?

Btw, will backup files from the 2GB setup be readable in the 1GB setup?

6

Was it successful? After a rebuild, the container should be started.
You can’t enter the container if it’s not started (./launcher start import)

That would likely be the case. There are several topics related to low memory/swap and rebuilding failure here – there is usually no choice but to increase RAM/swap.

I’m not an expert here, but hopefully, someone can give you better insight.

7

Apparently, RAM is an issue. For the Lightsail 2GB/2GB, the clean installation is about 3.5-4x faster and the rebuilding of the import container was about (or even over) 10x faster.
Still, I’m afraid the import script might not work for phpBB 3.3.3. Anyway, I tried twice with the same result: only one word from one category name (out of ~10 categories) was imported as one empty “category”. The MySQL textual dump is pretty small 2.66MB. All table prefixes are as required. No images, no smilies etc.
All in all, no forums, no forum categories or posts, but all (a few hundred) users were imported correctly along with their statistics.

It seems phpBB 3.3.3 was the update that changed something about the used MySQL version, but this is a dump file here, so it shouldn’t matter after all.
Any ideas what else can be checked?

8

I would recommend at least 4gb of ram.

Are there any errors? It sounds like the script quit after it got one category?

9

You mean 4GB to open/read/parse that 2.6MB text file? Are you the Discourse developer? A binary executable to read that dump (extracting all the data used by Discourse) and save it to anything else should be imo around 200KB with a, say, 100KB buffer (without bothering with the inclusion of the whole MariaDB for this simple action).

I think it’s not about RAM. The procedure was fast and displayed importing progress for those imported tables. It didn’t “quit”. The users’ table is at the end of the dump, after the topics etc.

I wanted to do this again, experimenting with copying/pasting that users table to the top and perform the same steps from the above manual/page (also with a dump created by phpMyAdmin instead of phpBB), but now Discourse refuses to start the procedure trying to connect remotely although no connection data was added.

I rebuilt the import container, but perhaps something has to be deleted first?

10

I make my living supporting discourse and have done more migrations than I can count, but I think a hundred or more. 2gb will likely work, especially if you have only a few thousand topics and users, but since you need it for unit a small amount of time, more ram isn’t expensive and can’t hurt. I agree that it’s likely not the problem.

When the script ran, did it count through the categories, users, topics, and posts?

The script will skip over imported data, and creates custom field data to skip those in the future, so to run it again, you need to wipe the discourse database. Since you say it didn’t import anything it’s not clear whether that’s necessary. It would seem that either it did import it and you can’t find it for some reason, or it failed because it didn’t find the data, and you need to modify the script somehow; in neither of those cases should you need to wipe the database.

11

Since you say it didn’t import anything

Not exactly:
" all (a few hundred) users were imported correctly along with their statistics"

I tried to rebuild the container again. The same settings. I didn’t change anything and after the step:
import_phpbb3.sh # inside the Docker container
the script now displays:
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mysql2-0.5.6/lib/mysql2/client.rb:97:in `connect’: Unknown database ‘phpbb’ (Mysql2::Error)

Perhaps there are still some previously used temporary files to delete?

12

I eventually used the dump from phpMyAdmin. Is there any example (or some more detailed specs) how to create the mapping sections in the (import) settings.yml?
My php_forums table looks like the following:

forum_id, MEDIUMINT(8) ...	parent_id, MEDIUMINT(8) ...
3	0
8	3
7	3
9	3
10	3
11	0
12	11
13	11
14	11
15	11
17	3

It seems that whichever ids I use and no matter whether I also specify the “new_categories: ” section, the message generated by the import container is ok:
creating new categories
11 / 11 (100.0%) [2803 items/min] n]
creating categories
11 / 11 (100.0%) [2704 items/min]

but what is created are two main categories (that were “forums” in the phpBB with the parent_id 0) and one empty subcategory in each one of them. The procedure does import all posts but they are all "orphaned’.

13

Sounds like something has changed for your version of phpbb3.

14

Well, actually, I made one step forward fixing my own error: I just placed comments in the settings.yml file using the C/C++ style, at the line ends. This apparently prevented the subforums to be “processed” (though no errors were displayed).

I added the categories and subcategories as “new” and mapped the older ids.
Still there are all the posts not attached to any category. Are such topics stored in one Postgres table? How they can be deleted?

15

I added some fixes to the phpbb_mysql.sql at least to let the script create all the original categories, but it’s not enough.
When processing topics/posts there is just a long list of the type:

Parent post [some id] doesn't exist. Skipping [some id]

and, of course, there are no posts imported.
I wonder if anyone could confirm that the default scripts don’t work with phpBB 3.3.3 tables?

16

Eventually I was able to use the phpBB3.3.3 dump file without defining the categories as new and without all the mapping in the settings.yml file.

With the original phpBB_mysql.sql dump file the migration script quits the step where categories are loaded and displays an error about “an empty category body”.
It occurs if the “forum_desc” field in the “phpbb_forums” table for the main forum categories (that don’t have parent categories).
After importing only that 1st category partially, the rest apparently becomes messed up.
Using a fresh installation and entering some text in the above table fields seems to solve this issue.