Script de migration Flarum vers Discourse

Lecter · Mai 24, 2021, 8:52

Bonjour à tous,

J’ai pensé faire un petit post pour vous expliquer brièvement comment j’ai migré notre forum de plus de 25 000 membres et 250 000 publications de Flarum vers Discourse.

Comme vous le savez probablement en lisant ceci, il n’existe pas de script d’importation officiel pour passer de Flarum à Discourse. Cependant, @koen360 a eu la gentillesse de partager son travail consistant à modifier l’importateur FluxBB pour importer les utilisateurs. Néanmoins, nous devions également importer les publications, car une grande partie du succès de notre forum repose sur la capacité des utilisateurs à rechercher des termes très spécifiques figurant dans les messages. C’est donc ce que nous avons fait… après plusieurs heures de migrations échouées, de maux de tête et quelques dégâts.

Je ne suis absolument pas un expert en écriture de scripts de migration, mais ce script fonctionne suffisamment bien pour la version actuelle de Flarum au moment de la rédaction (V0.1.0-beta.16). Le script (ci-joint) nécessitera l’accès à votre base de données MySQL/MariaDB, de préférence une copie, afin d’éviter d’endommager votre site de production en cas de problème. Il vous suffit de remplacer les valeurs fictives dans la connexion MySQL et vous devriez être prêt.

Assurez-vous simplement d’avoir une sauvegarde complète de votre base de données Flarum avant de commencer ce processus. En cas d’erreur, il est probable que la situation soit irréversible.

Si vous avez besoin d’aide pour votre migration de Flarum vers Discourse, n’hésitez pas à répondre à ce fil. Je ne suis pas un expert, mais je ferai de mon mieux pour vous aider.

Vous pouvez télécharger l’importateur ici ; apparemment, je ne peux pas uploader de fichier puisque c’est mon premier post

justin · Mai 25, 2021, 12:29

Merci d’avoir créé le script @Lecter ! Peux-tu le soumettre sous forme de PR à discourse/discourse ? Nous n’avons pas encore de script Flarum, donc cela nous serait utile comme base.

Lecter · Mai 25, 2021, 1:40

Absolument ! La RP arrive…

oshyan · Août 13, 2021, 8:50

C’est formidable, merci d’avoir fait faire un pas de plus à cet importateur ! J’espère qu’avec le temps, il pourra aussi être étendu pour inclure les balises, sinon les pièces jointes.

Combien de travail cela représenterait-il d’ajouter la prise en charge des balises ? Je vois que vBulletin 5 et peut-être quelques autres ont des modèles possibles pour comprendre comment cela fonctionne, mais je ne sais pas du tout dans quelle mesure cela devrait être adapté pour Flarum. Merci pour tout éclaircissement que quelqu’un pourrait apporter, et pas de souci si c’est quelque chose qui ne peut pas vraiment être répondu sans le faire réellement.

kovalr · Juin 13, 2022, 8:19

Bonjour @Lecter !
Votre script est-il toujours valide ?
Pouvez-vous le partager à nouveau ? Parce que votre lien est mort.

Merci de prendre en compte ma demande.

Falco · Juin 13, 2022, 8:46

Le script écrit par @Lecter se trouve maintenant à l’adresse

github.com/discourse/discourse

script/import_scripts/flarum_import.rb

main

# frozen_string_literal: true

require "mysql2"
require "time"
require "date"

require_relative "base"

class ImportScripts::FLARUM < ImportScripts::Base
  #SET THE APPROPRIATE VALUES FOR YOUR MYSQL CONNECTION
  FLARUM_HOST = ENV["FLARUM_HOST"] || "db_host"
  FLARUM_DB = ENV["FLARUM_DB"] || "db_name"
  BATCH_SIZE = 1000
  FLARUM_USER = ENV["FLARUM_USER"] || "db_user"
  FLARUM_PW = ENV["FLARUM_PW"] || "db_user_pass"

  def initialize
    super

    @client =

This file has been truncated. show original

kovalr · Juin 14, 2022, 3:42

Quelqu’un peut-il m’aider avec la migration, s’il vous plaît ?

J’ai Discourse avec des messages vides et 3 catégories par défaut.
Lorsque j’ai essayé d’importer le SQL de Flarum vers Discourse, j’ai obtenu cette erreur

su discourse -c 'bundle exec ruby script/import_scripts/flarum_import.rb'
Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...

creating users
       13 / 13 (100.0%)  [281 items/min]  n]  
importing top level categories...
       39 / 39 (100.0%)  [420 items/min]  n]  
importing children categories...
       39 / 39 (100.0%)  [290 items/min]  
creating topics and posts
Traceback (most recent call last):
	19: from script/import_scripts/flarum_import.rb:162:in `<main>'
	18: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
	17: from script/import_scripts/flarum_import.rb:32:in `execute'
	16: from script/import_scripts/flarum_import.rb:101:in `import_posts'
	15: from /var/www/discourse/script/import_scripts/base.rb:916:in `batches'
	14: from /var/www/discourse/script/import_scripts/base.rb:916:in `loop'
	13: from /var/www/discourse/script/import_scripts/base.rb:917:in `block in batches'
	12: from script/import_scripts/flarum_import.rb:122:in `block in import_posts'
	11: from /var/www/discourse/script/import_scripts/base.rb:224:in `all_records_exist?'
	10: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/transactions.rb:209:in `transaction'
	 9: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
	 8: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
	 7: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
	 6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
	 5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
	 4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
	 3: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
	 2: from /var/www/discourse/script/import_scripts/base.rb:231:in `block in all_records_exist?'
	 1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec': ERROR:  duplicate key value violates unique constraint "import_ids_pkey" (PG::UniqueViolation)
DETAIL:  Key (val)=(7) already exists.
	20: from script/import_scripts/flarum_import.rb:162:in `<main>'
	19: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
	18: from script/import_scripts/flarum_import.rb:32:in `execute'
	17: from script/import_scripts/flarum_import.rb:101:in `import_posts'
	16: from /var/www/discourse/script/import_scripts/base.rb:916:in `batches'
	15: from /var/www/discourse/script/import_scripts/base.rb:916:in `loop'
	14: from /var/www/discourse/script/import_scripts/base.rb:917:in `block in batches'
	13: from script/import_scripts/flarum_import.rb:122:in `block in import_posts'
	12: from /var/www/discourse/script/import_scripts/base.rb:224:in `all_records_exist?'
	11: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/transactions.rb:209:in `transaction'
	10: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
	 9: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
	 8: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
	 7: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
	 6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
	 5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-7.0.3/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
	 4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-7.0.3/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
	 3: from /var/www/discourse/script/import_scripts/base.rb:243:in `block in all_records_exist?'
	 2: from /var/www/discourse/script/import_scripts/base.rb:243:in `ensure in block in all_records_exist?'
	 1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec': ERROR:  current transaction is aborted, commands ignored until end of transaction block (PG::InFailedSqlTransaction)

Le problème vient de Key (val)=(7) already exists.

`exec': ERROR:  duplicate key value violates unique constraint "import_ids_pkey" (PG::UniqueViolation)
DETAIL:  Key (val)=(7) already exists.

root@raspberrypi-app:/var/www/discourse# rake db:validate_indexes
Starting postgres on port: 11000
Waiting for PG server to start...
PG server is ready and DB is loaded
Running migrations on blank database!

Testing indexes on the default database

No missing indexes

No extra indexes

Comment puis-je résoudre ce problème ?

kovalr · Juin 14, 2022, 10:18

Ce code SQL casse le script

 SELECT p.id id,
               d.id topic_id,
               d.title title,
               d.first_post_id first_post_id,
               p.user_id user_id,
               p.content raw,
               p.created_at created_at,
               t.tag_id category_id
        FROM posts p,
             discussions d,
             discussion_tag t
        WHERE p.discussion_id = d.id
          AND t.discussion_id = d.id
        ORDER BY p.created_at

kovalr · Juin 15, 2022, 4:56

J’ai trouvé le problème. Dans mon cas, un article a plusieurs étiquettes. Après avoir supprimé de tels articles, le script d’importation s’est traité sans aucune erreur.

Bishwas_Bhandari · Novembre 22, 2022, 6:19

J’ai eu cette erreur

ubuntu@ip-172-26-1-78:~/discourse$ bundle exec ruby script/import_scripts/flarum_import.rb
Loading existing groups...
Loading existing users...
Loading existing categories...
Loading existing posts...
Loading existing topics...

creating users
Skipping 1000 already imported users
Skipping 826 already imported users

importing top level categories...
       34 / 34 (100.0%)  [5728165 items/min]  
importing children categories...
       34 / 34 (100.0%)  [3194 items/min]  
creating topics and posts
Traceback (most recent call last):
	19: from script/import_scripts/flarum_import.rb:162:in `<main>'
	18: from /home/ubuntu/discourse/script/import_scripts/base.rb:47:in `perform'
	17: from script/import_scripts/flarum_import.rb:32:in `execute'
	16: from script/import_scripts/flarum_import.rb:101:in `import_posts'
	15: from /home/ubuntu/discourse/script/import_scripts/base.rb:916:in `batches'
	14: from /home/ubuntu/discourse/script/import_scripts/base.rb:916:in `loop'
	13: from /home/ubuntu/discourse/script/import_scripts/base.rb:917:in `block in batches'
	12: from script/import_scripts/flarum_import.rb:122:in `block in import_posts'
	11: from /home/ubuntu/discourse/script/import_scripts/base.rb:224:in `all_records_exist?'
	10: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/transactions.rb:209:in `transaction'
	 9: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
	 8: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
	 7: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
	 6: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
	 5: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
	 4: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
	 3: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
	 2: from /home/ubuntu/discourse/script/import_scripts/base.rb:243:in `block in all_records_exist?'
	 1: from /home/ubuntu/discourse/script/import_scripts/base.rb:243:in `ensure in block in all_records_exist?'
	 1: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec'
/home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec': ERROR:  duplicate key value violates unique constraint "import_ids_pkey" (PG::UniqueViolation)
DETAIL:  Key (val)=(53) already exists.
	20: from script/import_scripts/flarum_import.rb:162:in `<main>'
	19: from /home/ubuntu/discourse/script/import_scripts/base.rb:47:in `perform'
	18: from script/import_scripts/flarum_import.rb:32:in `execute'
	17: from script/import_scripts/flarum_import.rb:101:in `import_posts'
	16: from /home/ubuntu/discourse/script/import_scripts/base.rb:916:in `batches'
	15: from /home/ubuntu/discourse/script/import_scripts/base.rb:916:in `loop'
	14: from /home/ubuntu/discourse/script/import_scripts/base.rb:917:in `block in batches'
	13: from script/import_scripts/flarum_import.rb:122:in `block in import_posts'
	12: from /home/ubuntu/discourse/script/import_scripts/base.rb:224:in `all_records_exist?'
	11: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/transactions.rb:209:in `transaction'
	10: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/database_statements.rb:316:in `transaction'
	 9: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/transaction.rb:317:in `within_new_transaction'
	 8: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
	 7: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
	 6: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
	 5: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activesupport-7.0.3.1/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
	 4: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/activerecord-7.0.3.1/lib/active_record/connection_adapters/abstract/transaction.rb:319:in `block in within_new_transaction'
	 3: from /home/ubuntu/discourse/script/import_scripts/base.rb:243:in `block in all_records_exist?'
	 2: from /home/ubuntu/discourse/script/import_scripts/base.rb:243:in `ensure in block in all_records_exist?'
	 1: from /home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec'
/home/ubuntu/.rbenv/versions/2.7.6/lib/ruby/gems/2.7.0/gems/rack-mini-profiler-3.0.0/lib/patches/db/pg.rb:56:in `exec': ERROR:  current transaction is aborted, commands ignored until end of transaction block (PG::InFailedSqlTransaction)

mambonr5 · Février 21, 2024, 11:37

Salut,\n\nPeux-tu m’expliquer comment tu as fait ? Parce que je rencontre la même erreur que toi, mais au lieu de (val)=(7), j’obtiens :\n\n`exec’: ERROR: duplicate key value violates unique constraint "import_ids_pkey" (PG::UniqueViolation)\nDETAIL: Key (val)=(1) already exists.

Sujet		Réponses	Vues
Got error when try migrate from Flarum to Discourse Support	2	69	Avril 13, 2025
Error while importing topics from Flarum (users, groups import OK) Migration flarum	3	1077	Septembre 3, 2021
Flarum to Discourse Migration with users and discussion Migration flarum	22	398	Janvier 14, 2025
Makeshift but working flarum import script Development	0	642	Novembre 13, 2020
Migrate a XenForo forum to Discourse Sysadmins how-to	96	20670	Février 25, 2025

Script de migration Flarum vers Discourse

Sujets connexes