[Paid] Need a Vanilla 2 Import tool

I’m also really in need of the importer. I do want to fix it myself if someone can explain to me how to debug it. Since it doesn’t really throw clear error messages.

@rjd22 So, yes, the importer is broken. I’m looking forward to this as well.

Just letting you know that I just pushed our new vanilla importer :slight_smile:

You will need to export your vanilla forum using Vanilla Porter and then run:

./launcher enter app
su discourse
cd /var/www/discourse
RAILS_ENV=production ruby script/import_scripts/vanilla.rb path/to/vanilla/exporter/file.txt
2 Likes

Thanks for the amazing work (it’s hard to even imagine how much work is to create an importer). I failed to run it anyway, this is the log I got… I’m not very good at… well nothing around here, mostly a Java developer myself, so I don’t quite get what I’m doing wrong:

2014-08-13T00:00:20Z 11640 TID-ovochl8d0 INFO: Sidekiq client with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:898:in `rescue in connect': FATAL:  role "root" does not exist (ActiveRecord::NoDatabaseError)
Run `$ bin/rake db:create db:migrate` to create your database
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:888:in `connect'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:568:in `initialize'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:41:in `new'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/postgresql_adapter.rb:41:in `postgresql_connection'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:435:in `new_connection'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:445:in `checkout_new_connection'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:416:in `acquire_connection'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_record/connection_adapters/abstract/connection_pool.rb:351:in `block in checkout'
        from /usr/local/lib/ruby/2.0.0/monitor.rb:211:in `mon_synchronize'
...

You’re running the import script as root. root is the equivalent of the SYSTEM account on Windows. You should not be running programs as root unless you intend to do maintenance on the operating system.

If you’re using the default recommended install and ./launcher enter app, you should switch to the proper unprivileged user account by running exec su - discourse.

1 Like

You are right, I’m such a dumbass, I realized that later. Thanks for the help!

Hey @zogstrip, k, i’m officially stuck on importing. Apologize about the question in advance. I was able to set the chmod of the development.log, but now I need to create “discourse_development” database. Any help would be appreciate to import vanilla data.

Do this instead:

RAILS_ENV=production ruby script/import_scripts/vanilla.rb path/to/vanilla/exporter/file.txt

@riking That worked! Though got some new errors as well.

 RAILS_ENV=production ruby script/impo                rt_scripts/vanilla.rb export_openbook_db_2014-08-13_190843.txt
2014-08-14T06:43:43Z 18358 TID-ox4vznp8w INFO: Sidekiq client with redis options                 {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
parsing file...
reading file...
parsing categories...
parsing comments...
parsing conversations...
parsing conversation_messages...
parsing discussions...
parsing media...
parsing permissions...
parsing roles...
parsing users...
parsing user_conversations...
parsing user_discussions...
parsing user_roles...
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4/lib/active_r                ecord/attribute_assignment.rb:50:in `rescue in _assign_attribute': unknown attri                bute: avatar_url (ActiveRecord::UnknownAttributeError)
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/attribute_assignment.rb:45:in `_assign_attribute'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/attribute_assignment.rb:32:in `block in assign_attributes'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/attribute_assignment.rb:26:in `each'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/attribute_assignment.rb:26:in `assign_attributes'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/core.rb:455:in `init_attributes'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/core.rb:198:in `initialize'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/inheritance.rb:30:in `new'
        from /var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.1.4                /lib/active_record/inheritance.rb:30:in `new'
        from /var/www/discourse/script/import_scripts/base.rb:244:in `create_use                r'
        from /var/www/discourse/script/import_scripts/base.rb:208:in `block in c                reate_users'
        from /var/www/discourse/script/import_scripts/base.rb:197:in `each'
        from /var/www/discourse/script/import_scripts/base.rb:197:in `create_use                rs'
        from script/import_scripts/vanilla.rb:79:in `import_users'
        from script/import_scripts/vanilla.rb:17:in `execute'
        from /var/www/discourse/script/import_scripts/base.rb:69:in `perform'
        from script/import_scripts/vanilla.rb:240:in `<main>'

Sorry, this should be fixed now :poodle:

https://github.com/discourse/discourse/commit/61842157bf7e96b2a23168cc8d06eaafb82c6f61

2 Likes

@zogstrip Awesome! Seems like everything is importing properly as I type this. Thanks!

@zogstrip I spoke too soon. Seems like it’s getting stuck at something called ‘Private Topics’. Also, seems like the posts that were imported, none of the embedded images are working.

RAILS_ENV=production ruby script/impo                                                                                                                               rt_scripts/vanilla.rb export_somedomain_db_2014-08-14_001737.txt
2014-08-14T08:19:46Z 413 TID-owzetv7vc INFO: Sidekiq client with redis options {:url=>"redis://localhost:6379/0", :namespace=>"sidekiq"}
parsing file...
reading file...
parsing categories...
parsing comments...
parsing conversations...
parsing conversation_messages...
parsing discussions...
parsing media...
parsing permissions...
parsing roles...
parsing users...
parsing user_conversations...
parsing user_discussions...
parsing user_roles...
       14 / 14 (100.0%)
importing first-level categories...
    General
    Videos
        
importing topics...
      111 / 111 (100.0%)
importing posts...
     1103 / 1103 (100.0%)
importing private topics...
script/import_scripts/vanilla.rb:199:in `block in import_private_topics': undefined method `[]' for nil:NilClass (NoMethodError)
        from /var/www/discourse/script/import_scripts/base.rb:325:in `block in create_posts'
        from /var/www/discourse/script/import_scripts/base.rb:324:in `each'
        from /var/www/discourse/script/import_scripts/base.rb:324:in `create_posts'
        from script/import_scripts/vanilla.rb:178:in `import_private_topics'
        from script/import_scripts/vanilla.rb:23:in `execute'
        from /var/www/discourse/script/import_scripts/base.rb:69:in `perform'
        from script/import_scripts/vanilla.rb:240:in `<main>'

Hey @zogstrip, any thoughts on the errors above?

Hello, I’m also curious to know the state of Vanilla 2 importer :slight_smile:

And how are the lists of emoticons (yahoo messenger style) converted to discourse emojis?

Do you have a list of these emoticons?

Yes!
This is exactly the same list as yahoo messenger

1 Like

We do have some of them, but most of them won’t be translated though. Is that something Vanilla has natively or is it a plugin?

I believe it’s a plugin that is included in the main package but isn’t enabled by default.

2 Likes

I’m fine with adding a pre-processing step in the vanilla importer to translate these emoticons to the ones we support if there is a match. PRs are welcome :wink:

2 Likes

Thank you, I’ll make a test this week and make the report :slight_smile: