# 无法执行 rake uploads:migrate\_from\_s3

**URL:** https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541
**Category:** Bug
**Created:** [2018年一月20日 15:38 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541 "2018-01-20T15:38:56Z")
**Posts on this page:** 8
**Page:** 1

<div class="post-metadata">

### Author: ![Apecengo](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/apecengo/32/3945_2.png) [@Apecengo](https://meta.discourse.org/u/Apecengo)
#### Post date: [2018年一月20日 15:38 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/1 "2018-01-20T15:38:57Z")

</div>

I’m going to have to stop using S3 for file uploads, and today I tried the `rake uploads:migrate_from_s3`, but it failed:

```plaintext
$ rake uploads:migrate_from_s3

```

```plaintext
rake aborted!
NoMethodError: undefined method `Uploads' for #<Class:0x000056276a4f14f0>
Did you mean? Upload
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/dynamic_matchers.rb:22:in `method_missing'
/var/www/discourse/lib/tasks/uploads.rake:117:in `migrate_from_s3'
/var/www/discourse/lib/tasks/uploads.rake:110:in `block in migrate_all_from_s3'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rails_multisite-1.1.2/lib/rails_multisite/connection_management.rb:132:in `block in each_connection'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rails_multisite-1.1.2/lib/rails_multisite/connection_management.rb:130:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rails_multisite-1.1.2/lib/rails_multisite/connection_management.rb:130:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:110:in `migrate_all_from_s3'
/var/www/discourse/lib/tasks/uploads.rake:89:in `block in <top (required)>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_from_s3
(See full trace by running task with --trace)

```

I previously entered the container with the `./launcher enter app` command, and listed the available tasks, and the migrate\_from\_s3 is nowhere to be found:

> **Full output (long)**
>
> ```plaintext
> rake about # List versions of all Rails frameworks and the environment
> rake add_topic_to_quotes # Add the topic to quotes
> rake admin:create # Creates a forum administrator
> rake admin:invite[email] # invite an admin to this discourse instance
> rake api_key:get # generate api key if missing, return existing if already there
> rake app:template # Applies the template supplied by LOCATION=(/path/to/template) or URL
> rake app:update # Update configs and some other initially generated files (or use just update:configs or update:bin)
> rake assets:clean[keep] # Remove old compiled assets
> rake assets:clobber # Remove compiled assets
> rake assets:environment # Load asset compile environment
> rake assets:precompile # Compile all the assets named in config.assets.precompile
> rake assets:prestage # pre-stage assets on cdn
> rake autospec # Run all specs automatically as needed
> rake avatars:clean # Clean up all avatar thumbnails (use this when the thumbnail algorithm changes)
> rake avatars:refresh # Refresh all avatars (download missing gravatars, refresh system)
> rake build:stamp # stamp the current build with the git hash placed in version.rb
> rake build_test_topic # create pushstate/replacestate test topic
> rake cache_digests:dependencies # Lookup first-level dependencies for TEMPLATE (like messages/show or comments/_comment.html)
> rake cache_digests:nested_dependencies # Lookup nested dependencies for TEMPLATE (like messages/show or comments/_comment.html)
> rake db:create[multisite] # Creates the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:create:all to create all dat...
> rake db:drop[multisite] # Drops the database from DATABASE_URL or config/database.yml for the current RAILS_ENV (use db:drop:all to drop all databases...
> rake db:environment:set[multisite] # Set the environment value for the database
> rake db:fixtures:load # Loads fixtures into the current environment's database
> rake db:migrate[multisite] # Migrate the database (options: VERSION=x, VERBOSE=false, SCOPE=blog)
> rake db:migrate:status # Display status of migrations
> rake db:rebuild_indexes # Rebuild indexes
> rake db:rollback # Rolls the schema back to the previous version (specify steps w/ STEP=n)
> rake db:schema:cache:clear # Clears a db/schema_cache.yml file
> rake db:schema:cache:dump # Creates a db/schema_cache.yml file
> rake db:schema:dump # Creates a db/schema.rb file that is portable against any DB supported by Active Record
> rake db:schema:load # Loads a schema.rb file into the database
> rake db:seed # Loads the seed data from db/seeds.rb
> rake db:seed_fu # Loads seed data for the current environment
> rake db:setup # Creates the database, loads the schema, and initializes with the seed data (use db:reset to also drop the database first)
> rake db:stats # Statistics about database
> rake db:structure:load # Recreates the databases from the structure.sql file
> rake db:version # Retrieves the current schema version number
> rake dev:cache # Toggle development mode caching on/off
> rake docker:test # Run all tests (JS and code in a standalone environment)
> rake emails:import # use this task to import a mailbox into Disourse
> rake emoji:test # test the emoji generation script
> rake emoji:update # update emoji images
> rake enqueue_digest_emails # This task is called by the Heroku scheduler add-on
> rake export:categories[category_ids] # Export all the categories
> rake export:category_structure[include_group_users,file_name] # Export only the structure of all categories
> rake highlightjs:update # download latest version of highlight and prepare it
> rake i18n:stats # show the current translation status
> rake import:file[file_name] # Import existing exported file
> rake initializers # Print out all defined initializers in the order they are invoked by Rails
> rake integration:create_fixtures # Creates the integration fixtures
> rake log:clear # Truncates all/specified *.log files in log/ to zero bytes (specify which logs with LOGS=test,development)
> rake middleware # Prints out your Rack middleware stack
> rake multisite:generate:config # generate multisite config file (if missing)
> rake multisite:migrate # migrate all sites in tier
> rake multisite:rollback # rollback migrations for all sites in tier
> rake notes # Enumerate all annotations (use notes:optimize, :fixme, :todo for focus)
> rake notes:custom # Enumerate a custom annotation, specify with ANNOTATION=CUSTOM
> rake plugin:install[repo] # install plugin
> rake plugin:install_all_official # install all official plugins (use GIT_WRITE=1 to pull with write access)
> rake plugin:qunit[plugin,timeout] # run plugin qunit tests
> rake plugin:spec[plugin] # run plugin specs
> rake plugin:update[plugin] # update a plugin
> rake plugin:update_all # update all plugins
> rake poll:migrate_old_polls # Migrate old polls to new syntax
> rake posts:defer_all_flags # Defer all flags
> rake posts:delete_all_likes # Delete all likes
> rake posts:delete_word[find,type] # Delete occurrence of a word/string
> rake posts:fix_letter_avatars # Rebake all posts with a quote using a letter_avatar
> rake posts:normalize_code # normalize all markdown so <pre><code> is not used and instead backticks
> rake posts:rebake # Update each post with latest markdown
> rake posts:rebake_match[pattern,type,delay] # Rebake all posts matching string/regex and optionally delay the loop
> rake posts:refresh_emails[topic_id] # Refreshes each post that was received via email
> rake posts:refresh_oneboxes # Update each post with latest markdown and refresh oneboxes
> rake posts:remap[find,replace,type] # Remap all posts matching specific string
> rake qunit:test[timeout,qunit_path] # Runs the qunit test suite
> rake release_note:generate[from,to] # generate a release note from the important commits
> rake restart # Restart app by touching tmp/restart.txt
> rake routes # Print out all defined routes in match order, with names
> rake scheduler:run_all # run every task the scheduler knows about in that order, use only for debugging
> rake secret # Generate a cryptographically secure secret key (this is typically used to generate a secret for cookie sessions)
> rake smoke:test # run chrome headless smoke tests on current build
> rake stats # Report code statistics (KLOCs, etc) from the application or engine
> rake test # Runs all tests in test folder except system ones
> rake test:db # Run tests quickly, but also reset db
> rake test:system # Run system tests only
> rake time:zones[country_or_offset] # List all time zones, list by two-letter country code (`rails time:zones[US]`), or list by UTC offset (`rails time:zones[-8]`)
> rake tmp:clear # Clear cache and socket files from tmp/ (narrow w/ tmp:cache:clear, tmp:sockets:clear)
> rake tmp:create # Creates tmp directories for cache, sockets, and pids
> rake user_actions:rebuild # rebuild the user_actions table
> rake users:change_post_ownership[old_username,new_username,archetype] # Change topic/post ownership of all the topics/posts by a specific user (without creating new revision)
> rake yarn:install # Install all JavaScript dependencies as specified via Yarn
> 
> ```

Am I doing something wrong or is it a bug?

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2018年一月22日 00:42 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/2 "2018-01-22T00:42:51Z")

</div>

Good catch, should be fixed per:

[https://github.com/discourse/discourse/commit/83a346cd6879be75e6e9312e6b9d6edee1ae2c7b](https://github.com/discourse/discourse/commit/83a346cd6879be75e6e9312e6b9d6edee1ae2c7b)

---

<div class="post-metadata">

### Author: ![Apecengo](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/apecengo/32/3945_2.png) [@Apecengo](https://meta.discourse.org/u/Apecengo)
#### Post date: [2018年一月22日 13:05 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/3 "2018-01-22T13:05:56Z")

</div>

Another bug related to this rake task.

When executing the task, even tough I have the `s3_upload_bucket` set, the task fails with the following error: `The S3 upload bucket must be set before running that task.`

> <https://github.com/discourse/discourse/blob/83a346cd6879be75e6e9312e6b9d6edee1ae2c7b/lib/tasks/uploads.rake#L121>

---

<div class="post-metadata">

### Author: ![sam](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/sam/32/102149_2.png) [@sam](https://meta.discourse.org/u/sam)
#### Post date: [2018年一月22日 22:38 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/4 "2018-01-22T22:38:37Z")

</div>

I am not sure why this is required @zogstrip don’t we already store full urls in the upload table?

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [2018年一月23日 17:49 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/8 "2018-01-23T17:49:10Z")

</div>

That rake task’s logic hasn’t been updated in 4 years…

@Apecengo, can you try again with this newer version?

[https://github.com/discourse/discourse/commit/28d432263e58ae828470a87278f0c87a122be232](https://github.com/discourse/discourse/commit/28d432263e58ae828470a87278f0c87a122be232)

---

<div class="post-metadata">

### Author: ![Apecengo](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/apecengo/32/3945_2.png) [@Apecengo](https://meta.discourse.org/u/Apecengo)
#### Post date: [2018年一月23日 20:19 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/9 "2018-01-23T20:19:09Z")

</div>

Doesn’t seem to be working as intended:

```plaintext
Migrating uploads from S3 to local storage for 'default'...
..........................................................Done!

```

Each `.` should be a `#`, if i have understood correctly the code:

> <https://github.com/discourse/discourse/blob/28d432263e58ae828470a87278f0c87a122be232/lib/tasks/uploads.rake#L150>

My version: `Discourse 2.0.0.beta1 - https://github.com/discourse/discourse version 782d75069ecc7d710df70bf0366e260ddfc59a72`

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [2018年一月24日 12:23 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/10 "2018-01-24T12:23:35Z")

</div>

I’ve fixed the rake task to ensure we delete existing records before creating new ones (since we index based on the sha1 of the files, we weren’t able to create new records with the same sha1)

[https://github.com/discourse/discourse/commit/cbf4204eccff62b6eac00932a04b827bff6cf883](https://github.com/discourse/discourse/commit/cbf4204eccff62b6eac00932a04b827bff6cf883)

---

<div class="post-metadata">

### Author: ![zogstrip](https://sea3.discourse-cdn.com/meta/user_avatar/meta.discourse.org/zogstrip/32/512781_2.png) [@zogstrip](https://meta.discourse.org/u/zogstrip)
#### Post date: [2018年一月28日 09:57 UTC](https://meta.discourse.org/t/cannot-execute-the-rake-uploads-migrate-from-s3/78541/11 "2018-01-28T09:57:22Z")

</div>


