Posts failed to import from phpBB3

danmaby · June 10, 2018, 5:35pm

Hi, we’ve completed an import of a phpBB3 forum using the import script provided, into a vanilla Discourse install. The import has completed but only imported the users. No posts or topics have been migrated in?

We’re also seeing 500 errors randomly, some pages receive a 500 error every time like /u/dan/preferences/account and some URLs are returning the “Ops The software powering this discussion forum encountered an unexpected problem”. I installed and activated the Data Explorer Plugin to see what’s going on in the database but each time I try to create a query it returns Internal Server Error modal.

gerhard · June 10, 2018, 9:19pm

Well, it sounds like the import failed completely. Did you get any errors during the import? Did the import script even try to import topics and posts? It should have printed a message like:

creating topics and posts
    89392 / 89392 (100%)   [983 items/min]

danmaby · June 10, 2018, 10:02pm

That’s the strange thing @gerhard, it’s imported the thousands of users but nothing else. The import didn’t seem to end right either, this is what displayed at the end:

669056 / 670485 ( 99.8%)  [1768 items/min]  Error on record: {:email=>"matias394@pinamail.com", :username=>"ZUVPamel", :password=>nil, :name=>"Matias394", :created_at=>Sat, 04 Nov 2017 00:27:41 UTC +00:00, :last_seen_at=>Sat, 04 Nov 2017 00:27:41 UTC +00:00, :registration_ip_address=>nil, :active=>false, :trust_level=>0, :approved=>false, :approved_by_id=>nil, :approved_at=>nil, :moderator=>false, :admin=>false, :date_of_birth=>nil, :import_mode=>true, :last_emailed_at=>2018-06-10 15:54:40 +0000}
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis/client.rb:119:in `call': ERR Error running script (call to f_b06356ba4628144e123b652c99605b873107c9be): @user_script:14: @user_script: 14: -MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.    (Redis::CommandError)
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis.rb:2448:in `block in _eval'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis.rb:45:in `block in synchronize'
from /usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis.rb:45:in `synchronize'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis.rb:2447:in `_eval'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/redis-4.0.1/lib/redis.rb:2499:in `evalsha'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.5/lib/message_bus/backends/redis.rb:381:in `cached_eval'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.5/lib/message_bus/backends/redis.rb:141:in `publish'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.1.5/lib/message_bus.rb:253:in `publish'
from /var/www/discourse/lib/site_setting_extension.rb:258:in `notify_changed!'
from /var/www/discourse/lib/site_settings/db_provider.rb:7:in `block in initialize'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:413:in `instance_exec'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:413:in `block in make_lambda'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:270:in `block in simple'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:511:in `block in invoke_after'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:511:in `each'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:511:in `invoke_after'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:132:in `run_callbacks'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:827:in `_run_commit_callbacks'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/transactions.rb:343:in `committed!'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/transaction.rb:86:in `commit_records'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/transaction.rb:178:in `block in commit_transaction'
from /usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/transaction.rb:168:in `commit_transaction'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/transaction.rb:207:in `block in within_new_transaction'
from /usr/local/lib/ruby/2.4.0/monitor.rb:214:in `mon_synchronize'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/transaction.rb:191:in `within_new_transaction'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/connection_adapters/abstract/database_statements.rb:235:in `transaction'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/transactions.rb:210:in `transaction'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/transactions.rb:381:in `with_transaction_returning_status'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/transactions.rb:313:in `save!'
from /var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/suppressor.rb:46:in `save!'
from /var/www/discourse/lib/site_settings/db_provider.rb:40:in `save'
from /var/www/discourse/lib/site_setting_extension.rb:251:in `add_override!'
from /var/www/discourse/lib/site_setting_extension.rb:358:in `block in setup_methods'
from /var/www/discourse/lib/site_setting_extension.rb:285:in `set'
from /var/www/discourse/script/import_scripts/base.rb:103:in `block in reset_site_settings'
from /var/www/discourse/script/import_scripts/base.rb:101:in `each'
from /var/www/discourse/script/import_scripts/base.rb:101:in `reset_site_settings'
from /var/www/discourse/script/import_scripts/base.rb:66:in `ensure in perform'
from /var/www/discourse/script/import_scripts/base.rb:66:in `perform'
from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:20:in `perform'
from script/import_scripts/phpbb3.rb:31:in `<module:PhpBB3>'
from script/import_scripts/phpbb3.rb:12:in `<module:ImportScripts>'
from script/import_scripts/phpbb3.rb:11:in `<main>'

danmaby · June 10, 2018, 10:14pm

Now if I try and rerun the import it won’t. This is what I see:

The phpBB3 import is starting...

Loading existing groups...

Loading existing users...

Loading existing categories...

Loading existing posts...

Loading existing topics...

/var/www/discourse/lib/site_settings/type_supervisor.rb:172:in `validate_value': undefined method `include?' for nil:NilClass (NoMethodError)

from /var/www/discourse/lib/site_settings/type_supervisor.rb:126:in `to_db_value'

from /var/www/discourse/lib/site_setting_extension.rb:250:in `add_override!'

from /var/www/discourse/lib/site_setting_extension.rb:358:in `block in setup_methods'

from /var/www/discourse/lib/site_setting_extension.rb:285:in `set'

from /var/www/discourse/script/import_scripts/base.rb:90:in `block in change_site_settings'

from /var/www/discourse/script/import_scripts/base.rb:88:in `each'

from /var/www/discourse/script/import_scripts/base.rb:88:in `change_site_settings'

from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:37:in `change_site_settings'

from /var/www/discourse/script/import_scripts/base.rb:45:in `perform'

from /var/www/discourse/script/import_scripts/phpbb3/importer.rb:20:in `perform'

from script/import_scripts/phpbb3.rb:31:in `&lt;module:PhpBB3&gt;'

from script/import_scripts/phpbb3.rb:12:in `&lt;module:ImportScripts&gt;'

from script/import_scripts/phpbb3.rb:11:in `&lt;main&gt;'

gerhard · June 10, 2018, 10:21pm

The import didn’t finish. It didn’t even import all users. It aborted at 669056 / 670485 ( 99.8%).

Did you run out of disk space?

danmaby · June 10, 2018, 10:52pm

The list of problems since running the import seems to be growing There’s around 40GB disk space available. The site keeps falling over and requiring a rebuild, possibly because of the number of queued jobs

But now I’m getting an error when trying to run the rebuild. I’ve tried rebooting and powering down the droplet

FAILED

--------------------

Pups::ExecError: socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 &amp;&amp; echo postgres already running stop container ; exit 1 failed with return #&lt;Process::Status: pid 44 exit 1&gt;

Location of failure: /pups/lib/pups/exec_command.rb:108:in `spawn'

exec failed with the params "socat /dev/null UNIX-CONNECT:/shared/postgres_run/.s.PGSQL.5432 || exit 0 &amp;&amp; echo postgres already running stop container ; exit 1"

def4e57bc78bb999e0b92227ab486b92787d0e058aaa813c1fd855f97cb61699

** FAILED TO BOOTSTRAP ** please scroll up and look for earlier error messages, there may be more than one

gerhard · June 10, 2018, 11:05pm

You really shouldn’t worry about those errors when it’s clear that the import didn’t finish at all. I suggest you create a new droplet (preferably with more disk space) and try again. Or maybe even run the import on your local machine. Also, if you have a budget, you might consider posting in marketplace to get some help with your import.

pfaffman · June 11, 2018, 12:10am

It could also be that the droplet for the import didn’t have enough RAM. I’d recommend at least 4GB for an import.

danmaby · June 11, 2018, 8:35am

@pfaffman yep it was the droplet maxing out it’s RAM. Upgraded to a 6GB, the import is now running and creating topics/posts. Thank you!

JammyDodger · June 8, 2024, 12:37pm

This topic was automatically closed after 2189 days. New replies are no longer allowed.

Topic		Replies	Views
phpBB 3 Importer (old) Feature	90	16886	October 8, 2014
Migrate a vBulletin 4 forum to Discourse Sysadmins how-to	160	28877	July 27, 2024
Re: Migrate a phpBB3 forum to Discourse Migration	15	459	March 23, 2024
Importers for large forums Announcements	50	9201	December 1, 2023
Migrate a Vanilla forum to Discourse Sysadmins how-to	44	15641	January 30, 2023

Posts failed to import from phpBB3

Related topics