Mbox import: crash when Validation failed: Name is too long

dachary · November 2, 2021, 12:25am

Bonjour,

While importing a spam (not on purpose ) the 2.7.9 mbox import script breaks on a name longer than 255 characters instead of gracefully discarding it. Here is the stack trace.

Cheers

Invalid email '"from:mrs.karlijn"@home.sinet.sk' for '_from_mrs.karlijn'. Using 'ea189fafefc2ff24cd95ef7828f943dd@email.invalid'
     1485 / 2592 ( 57.3%)  [1496 items/min]  Error on record: {:id=>"leroysantosmaitre@yahoo.fr", :email=>"leroysantosmaitre@yahoo.fr", :name=>"EXCELLENT, Notre politique d'attribution de prime après sélection est dû à l'appel de nouvel investisseur au sein de notre Banque. C'est pour cette raison après une sélection des emails, le votre a été retenu. Et nous en sommes heureux de vous annoncer que vous êtes le gagnant de 60.000€. Et nous vous prions de bien vouloir contacter l’huissier Me SANTOS LEROY l’avocat qui a supervisé le jeu par son adresse mail pour recevoir le formulaire à remplir et pour procéder a l'encaissement de votre gain. Email :leroysantosmaitre@yahoo.fr", :trust_level=>1, :staged=>true, :active=>false, :created_at=>Fri, 02 Oct 2009 12:51:46.000000000 UTC +00:00}
Traceback (most recent call last):
        33: from script/import_scripts/mbox.rb:9:in `<main>'
        32: from script/import_scripts/mbox.rb:10:in `<module:ImportScripts>'
        31: from script/import_scripts/mbox.rb:12:in `<module:Mbox>'
        30: from /var/www/discourse/script/import_scripts/base.rb:47:in `perform'
        29: from /var/www/discourse/script/import_scripts/mbox/importer.rb:35:in `execute'
        28: from /var/www/discourse/script/import_scripts/mbox/importer.rb:63:in `import_users'
        27: from /var/www/discourse/script/import_scripts/mbox/importer.rb:84:in `batches'
        26: from /var/www/discourse/script/import_scripts/base.rb:873:in `batches'
        25: from /var/www/discourse/script/import_scripts/base.rb:873:in `loop'
        24: from /var/www/discourse/script/import_scripts/base.rb:874:in `block in batches'
        23: from /var/www/discourse/script/import_scripts/mbox/importer.rb:69:in `block in import_users'
        22: from /var/www/discourse/script/import_scripts/base.rb:264:in `create_users'
        21: from /var/www/discourse/script/import_scripts/base.rb:264:in `each'
        20: from /var/www/discourse/script/import_scripts/base.rb:276:in `block in create_users'
        19: from /var/www/discourse/script/import_scripts/base.rb:344:in `create_user'
        18: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:209:in `transaction'
        17: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `transaction'
        16: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:308:in `within_new_transaction'
        15: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `synchronize'
        14: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:21:in `handle_interrupt'
        13: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `block in synchronize'
        12: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:25:in `handle_interrupt'
        11: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activesupport-6.1.3.2/lib/active_support/concurrency/load_interlock_aware_monitor.rb:26:in `block (2 levels) in synchronize'
        10: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/transaction.rb:310:in `block in within_new_transaction'
         9: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:320:in `block in transaction'
         8: from /var/www/discourse/script/import_scripts/base.rb:345:in `block in create_user'
         7: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/suppressor.rb:48:in `save!'
         6: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:302:in `save!'
         5: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:350:in `with_transaction_returning_status'
         4: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/connection_adapters/abstract/database_statements.rb:318:in `transaction'
         3: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:354:in `block in with_transaction_returning_status'
         2: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/transactions.rb:302:in `block in save!'
         1: from /var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/validations.rb:53:in `save!'
/var/www/discourse/vendor/bundle/ruby/2.7.0/gems/activerecord-6.1.3.2/lib/active_record/validations.rb:80:in `raise_validation_error': Validation failed: Name is too long (maximum is 255 characters)

Maybe this hack is where it should be properly fixed?

diff --git a/script/import_scripts/base.rb b/script/import_scripts/base.rb
index 7cd7961cfc..652fcaf957 100644
--- a/script/import_scripts/base.rb
+++ b/script/import_scripts/base.rb
@@ -326,6 +326,7 @@ class ImportScripts::Base
     end
 
     opts[:name] = original_username if original_name.blank? && opts[:username] != original_username
+    opts[:name] = opts[:name][0..200] unless opts[:name].nil?
 
     opts[:trust_level] = TrustLevel[1] unless opts[:trust_level]
     opts[:active] = opts.fetch(:active, true)

pfaffman · November 2, 2021, 7:54am

That looks good to me. If that works you should submit a PR.

dachary · November 2, 2021, 7:56am

Don’t I know it but I don’t feel like agreeing to the GitHub term of services which, among other interesting things, explicitly exclude entire populations based on their geographical location.

pfaffman · November 2, 2021, 10:06am

Don’t submit a PR if you don’t want to.

But you’re using github already.

Topic		Replies	Views
Importing mbox files fails at creating topics with Chinese characters due to invalid byte sequence Bug	6	1432	October 31, 2018
"500 error" on username edit Bug	13	1250	June 8, 2017
Yahoo Groups Importation Errors Migration	7	1341	January 18, 2020
Import mbox maps charset=windows-1252 to � Support	14	1214	December 2, 2020
Help needed, I'm blocked in debugging email config (Infomaniak/Brevo) Installation email	14	1175	November 29, 2023

Mbox import: crash when Validation failed: Name is too long

Related topics