Easier forum migration to Discourse

Feature
10

What goes though my head is:

A Discourse Import feature actually in Discourse that runs like this:

What is the URL of your existing forum?

Checking website…
I can see you are running phpBB 3.xxx

Just upload these two files to document root of your website:

  • discourse-import-bridge.php
  • discourse-import-bridge-key.php

When ready click I’ve Uploaded the Files

Obviously for PHP based forum site - which most are? (anyway another bridge could easily be implemented in any other language)

This would then run you though a few dialogs

What categories would you like to import?

  • General
  • Classifieds

Next

After confirming whatever details needed

This will import XXXX topics, XXXXX posts and XXX users.

The import will run in the background and continue to synchronise from your existing forum into your new Discourse install.

Start Import

After selecting to start

Import 32% complete

Importing at 30 posts per minute.
Estimated time until completion 4 hours.

Server health check: Good
Server last communicated with 30 seconds ago.

After all the data has been imported:

Import 100% complete - continuing to synchronise new topics, posts and users.

Server health check: Good
Server last communicated with 2 minutes ago.

Where:

  • discourse-import-bridge-key.php is contains some unique key specific to the Discourse instance allowed to access the server for extracting data for import. Probably downloadable from the Discourse instance.

  • the discourse-import-bridge.php would provide an API:

  • decrypt “Discourse Import Bridge API” HTTP API requests using the “key”.

  • executing a basic health check on the server config, PHP version, max request size etc, version of the “Discourse Import Bridge API”.

  • getting a list of files across the entire directory structure of the site

  • database proxy - receiving batched database queries, compressing responses and returning them in a chunked manor to avoid web server limitations.

  • downloading arbitrary files from file system, in compressed chunks.

  • the importer running on the Discourse server could use the “Discourse Import Bridge API” provided to:

  • request the URL of the site to import from the user

  • make encrypted “Discourse Import Bridge API” requests

  • run “health check” to ensure communication and server is “good”.

  • make a file system request for list of files

  • detect what type of forum installation (allowing the user to select if needed)

  • detect location of config file

  • read config file and identify DB location

  • read the DB as needed.

  • display prompts for any options specific to that importer (e.g. only import specific categories)

  • execute continuous background synchronisation of forum data so switch over is less of an issue.

  • notify and prompt user of synchronisation progress (sending emails if needed)

  • should use exponential back-off for all requests to discourse-import-bridge.php to avoid issues with hosting providers.

This Discourse importer would really just have an API layer for making file system requests, db requests, this layer would be switched between:

  • one version where “I have the DB and file system mounted locally”
  • another version where “I’m using the Discourse Import Bridge”.

The reason for the “encryption” of the requests is that most sites will not have HTTPS/SSL/TLS requests will contain information that should be kept secret, e.g. DB password.

It would / should be noted somewhere to the user that using the “Discourse Import Bridge” could take much longer to complete.

10 Likes