How to turn off S3 storage?

neounix · August 12, 2020, 7:38am

Here ya go, just for you…

That file is in the container.

You will find that file and other config files here:

# docker exec -it app bash
# cd /var/www/discourse/config

I absolutely recommend you do not exit any of those files unless you really know what are doing.

However, its definitely cool to read them and to understand them.

Pravi · August 12, 2020, 7:50am

But I can’t find discourse.config file in the container. I have the only app.yml in the container folder.

neounix · August 12, 2020, 7:52am

The container folder you are talking about is NOT the container.

The container is a Docker construct build and launched to run as a Docker container.

BTW, do not take this the wrong way, but do you understand what a Docker container is? It is perfectly OK if you don’t, BTW. Many people use Discourse daily and have not yet learned Docker container basics. It’s never too late to learn

Reference:

PS: @Pravi. I provided you the code to enter the container and change to the directory to where the config file are. Did you consider following that code I posted?

PS2: The rake task you are having trouble with (migrate_from_s3) can be found here:

github.com

discourse/discourse/blob/main/lib/tasks/uploads.rake

# frozen_string_literal: true

require "db_helper"
require "digest/sha1"
require "base62"

################################################################################
#                                    gather                                    #
################################################################################

require "rake_helpers"

task "uploads:gather" => :environment do
  ENV["RAILS_DB"] ? gather_uploads : gather_uploads_for_all_sites
end

def gather_uploads_for_all_sites
  RailsMultisite::ConnectionManagement.each_connection { gather_uploads }
end

This file has been truncated. show original

Pravi · August 12, 2020, 9:00am

I have not much idea about discourse.

Yes, I find those files.

How can I troubleshoot the problem from the rake code?

neounix · August 12, 2020, 9:05am

Hi @Pravi,

One approach would be to look at your log file entries immediately before your error message, and see if you can locate the line (or lines) in the rake task file; and try to isolate what is causing your issue, exactly.

Solutions are normally the easy part. The hard part is discovering what the problem is.

Pravi · August 12, 2020, 9:14am

Could you be more specific. You mean I have to look for log file in my website before executing the rake command. I didnt get it…

neounix · August 12, 2020, 9:20am

Hey @Pravi

This is becoming a lesson in basic software error troubleshooting… which is OK.

You already have your logging info in your post #4 in this topic:

There is a lot of info there.

You also a the full Discourse Ruby code for this rake task, above.

My apologies, but I have a lot of tasks to do as well, so I’m just trying to show you, at a high level, how to debug and the solve your own problems the same way most of us would go about it.

Hope this helps.

Pravi · August 12, 2020, 9:42am

I am sorry for taking too much time of yours. One last time, I tried to migrate to S3 instead of migrate from, which gives me following error even though I have provided all the details of s3 storage in my admin panel. Please take a look if you are familiar with this error.

root@ip-172-31-7-247:/var/discourse# ./launcher enter app
root@ip-172-31-7-247-app:/var/www/discourse# rake uploads:migrate_to_s3
rake aborted!
FileStore::ToS3MigrationError: Please provide the following environment variables:
  - DISCOURSE_S3_BUCKET
  - DISCOURSE_S3_REGION
  and either
  - DISCOURSE_S3_ACCESS_KEY_ID
  - DISCOURSE_S3_SECRET_ACCESS_KEY
  or
  - DISCOURSE_S3_USE_IAM_PROFILE
/var/www/discourse/lib/file_store/to_s3_migration.rb:38:in `s3_options_from_env'
/var/www/discourse/lib/tasks/uploads.rake:237:in `create_migration'
/var/www/discourse/lib/tasks/uploads.rake:245:in `migrate_to_s3'
/var/www/discourse/lib/tasks/uploads.rake:224:in `block in migrate_to_s3_all_sites'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.3.0/lib/rails_multisite/connection_management.rb:68:in `with_connection'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.3.0/lib/rails_multisite/connection_management.rb:78:in `each_connection'
/var/www/discourse/lib/tasks/uploads.rake:222:in `migrate_to_s3_all_sites'
/var/www/discourse/lib/tasks/uploads.rake:218:in `block in <main>'
/usr/local/bin/bundle:23:in `load'
/usr/local/bin/bundle:23:in `<main>'
Tasks: TOP => uploads:migrate_to_s3
(See full trace by running task with --trace)

Update:

I have added the all these DISCOURSE_S3_BUCKET, DISCOURSE_S3_REGION, DISCOURSE_S3_ACCESS_KEY_ID, DISCOURSE_S3_SECRET_ACCESS_KEY to app.yml and the above problem is solved. Now I am able to migrate to S3, but I cant migrate from S3. Its asking me to turn off S3 upload even though I have added DISCOURSE_USE_S3: false in my app.yml

How to confirm that S3 upload is turned off?

[5] pry(main)> SiteSetting.enable_s3_uploads
=> false
[6] pry(main)> GlobalSetting.use_s3
=> false
[7] pry(main)> GlobalSetting.use_s3?
=> true

neounix · August 12, 2020, 1:03pm

Hey @Pravi

I am passing the baton to the great discussion about this, in this meta team topic:

Lots of very details discussion there.

All the best… and hope this helps.

Topic		Replies	Views
Rake uploads:migrate_from_s3 fails Installation s3	31	2787	April 24, 2021
Help restoring - system hung at midnight Installation	26	2202	June 13, 2022
Migrate_from_s3: You must disable S3 uploads before running that task Support	21	1273	September 19, 2020
Migrate_to_s3 for Digital Ocean Spaces woes Installation s3	26	3098	April 27, 2019
Set up file and image uploads to S3 Self-Hosting configuring , how-to	23	116670	November 4, 2024

How to turn off S3 storage?

Related topics