Uploads:migrate_to_s3 crashes

So, I’m trying to migrate a Discourse installation to use S3 storage, in preparation for a server move. Sounds easy enough, except the rake task meant to do it crashes.

root@Jena-web:/var/www/discourse# rake uploads:migrate_to_s3
Migrating uploads to S3 ([BUCKET NAME]) for 'default'...
rake aborted!
TypeError: no implicit conversion of nil into String
/var/www/discourse/lib/tasks/uploads.rake:137:in `exists?'
/var/www/discourse/lib/tasks/uploads.rake:137:in `block in migrate_to_s3'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/relation/batches.rb:51:in `block (2 levels) in find_each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/relation/batches.rb:51:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/relation/batches.rb:51:in `block in find_each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/relation/batches.rb:124:in `find_in_batches'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activerecord-4.2.4/lib/active_record/relation/batches.rb:50:in `find_each'
/var/www/discourse/lib/tasks/uploads.rake:126:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:105:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rails_multisite-1.0.3/lib/rails_multisite/connection_management.rb:124:in `call'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rails_multisite-1.0.3/lib/rails_multisite/connection_management.rb:124:in `block in each_connection'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rails_multisite-1.0.3/lib/rails_multisite/connection_management.rb:122:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rails_multisite-1.0.3/lib/rails_multisite/connection_management.rb:122:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:105:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:101:in `block in <top (required)>'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)

It doesn’t reveal much extra information, but here’s a version with --trace.

The problem appears to be here - uploads where local.path_for(...) is nil crash, because File.exists?(nil) is not valid.

I’m assuming this happens because we were originally using S3 and migrated off it when you deprecated it, but some of our uploads got stuck on S3 for whatever reason (I let it be because it didn’t cause any actual problems). Now we’re moving back again, but some files do not exist locally, thus it freaks out.

Indeed, it seems to work if I change the offending line to:

if !path or !File.exists?(path)
1 Like

Awesome. Would you mind sending a pull request to fix the task for everyone? :wink:

Alright, sent!

I wanted to get a go-ahead first, in case this wasn’t the correct way to do it (it could have been a bug that the path was nil in the first place).

https://github.com/discourse/discourse/pull/3900

4 Likes