Mbox.rb replacing underscores with spaces

pfaffman · August 23, 2016, 6:28pm

Something in one of these files is replacing underscores with spaces in email addresses. I’m 99% sure that the one I forked did too, but I’m including my copy in case I did something boneheaded.

github.com/pfaffman/discourse

script/import_scripts/mbox.rb

master

# frozen_string_literal: true

if ARGV.length != 1 || !File.exist?(ARGV[0])
  STDERR.puts "", "Usage of mbox importer:", "bundle exec ruby mbox.rb <path/to/settings.yml>"
  STDERR.puts "",
              "Use the settings file from #{File.expand_path("mbox/settings.yml", File.dirname(__FILE__))} as an example."
  exit 1
end

module ImportScripts
  module Mbox
    require_relative "mbox/importer"
    Importer.new(ARGV[0]).perform
  end
end

github.com/pfaffman/discourse

script/import_scripts/base.rb

master

# frozen_string_literal: true

if ARGV.include?("bbcode-to-md")
  # Replace (most) bbcode with markdown before creating posts.
  # This will dramatically clean up the final posts in Discourse.
  #
  # In a temp dir:
  #
  # git clone https://github.com/nlalonde/ruby-bbcode-to-md.git
  # cd ruby-bbcode-to-md
  # gem build ruby-bbcode-to-md.gemspec
  # gem install ruby-bbcode-to-md-*.gem
  require "ruby-bbcode-to-md"
end

require_relative "../../config/environment"
require_relative "base/lookup_container"
require_relative "base/uploader"

module ImportScripts

This file has been truncated. show original

is replacing _ with a space in email addresses.

I can’t find it, and it’s making me crazy. I don’t see that any of the .gsubs could be doing it.

Any ideas where I should look, @eviltrout?

eviltrout · August 23, 2016, 7:36pm

Are you talking about underscores to spaces or spaces to underscores? The title of the topic is different than the body

I am pretty sure our system does not allow spaces in emails and will replace them, so I wouldn’t be surprised if it uses underscores.

pfaffman · August 23, 2016, 7:49pm

Fixed the title.

Right. That’s what’s bizarre. Emails that have underscores in them are ending up causing errors because the underscores are getting replaced with spaces.

m_smith@gmail.com becomes m smith@gmail.com.

Edit: but it only happens sometimes.

Anyway, I was hoping it was something silly and obvious that I was somehow overlooking. Thanks.

eviltrout · August 23, 2016, 8:13pm

I personally haven’t seen this, but it might depend on the mbox format. If you could provide an example of an email that is messed up and the original input I can take a look and try to figure it out.

pfaffman · August 23, 2016, 8:16pm

Thanks, @eviltrout. I’ve added some more debugging messages and will see what I can see.

Topic		Replies	Views
Importing mbox files fails at creating topics with Chinese characters due to invalid byte sequence Bug	6	1463	October 31, 2018
Import mbox maps charset=windows-1252 to � Support	14	1268	December 2, 2020
Import_mbox.sh not working with e-mails from Samsung phone sent via a listserv server Support	8	757	May 9, 2022
Missing space characters in activity summary email text Support activity-summary	13	242	October 9, 2024
Strip incoming email subject? Support	8	601	August 7, 2023

Mbox.rb replacing underscores with spaces

Related topics