I am also on latest. Did you get any errors on rebuild? If so, can you post those?
No, Not got any kind of error.
If the error is only after your rake command, then can you post exactly what you typed? When you first got S3 configured, didn’t you have to migrate files to S3?
Here is my full app.yml
## this is the all-in-one, standalone Discourse Docker container template ## ## After making changes to this file, you MUST rebuild ## /var/discourse/launcher rebuild app ## ## BE *VERY* CAREFUL WHEN EDITING! ## YAML FILES ARE SUPER SUPER SENSITIVE TO MISTAKES IN WHITESPACE OR ALIGNMENT! ## visit http://www.yamllint.com/ to validate this file as needed templates: - "templates/postgres.template.yml" - "templates/redis.template.yml" - "templates/web.template.yml" - "templates/web.ratelimited.template.yml" ## Uncomment these two lines if you wish to add Lets Encrypt (https) - "templates/web.ssl.template.yml" - "templates/web.letsencrypt.ssl.template.yml" ## - "templates/cloudflare.template.yml" ## which TCP/IP ports should this container expose? ## If you want Discourse to share a port with another webserver like Apache or nginx, ## see https://meta.discourse.org/t/17247 for details expose: - "80:80" # http - "443:443" # https params: db_default_text_search_config: "pg_catalog.english" ## Set db_shared_buffers to a max of 25% of the total memory. ## will be set automatically by bootstrap based on detected RAM, or you can override db_shared_buffers: "128MB" ## can improve sorting performance, but adds memory usage per-connection #db_work_mem: "40MB" ## Which Git revision should this container use? (default: tests-passed) #version: tests-passed env: LANG: en_US.UTF-8 # DISCOURSE_DEFAULT_LOCALE: en ## How many concurrent web requests are supported? Depends on memory and CPU cores. ## will be set automatically by bootstrap based on detected CPUs, or you can override UNICORN_WORKERS: 2 ## TODO: The domain name this Discourse instance will respond to DISCOURSE_HOSTNAME: engineersasylum.com ## Uncomment if you want the container to be started with the same ## hostname (-h option) as specified above (default "$hostname-$config") #DOCKER_USE_HOSTNAME: true ## TODO: List of comma delimited emails that will be made admin and developer ## on initial signup example 'email@example.com,firstname.lastname@example.org' DISCOURSE_DEVELOPER_EMAILS: 'email@example.com' ## TODO: The SMTP mail server used to validate new accounts and send notifications DISCOURSE_SMTP_ADDRESS: email-smtp.us-east-1.amazonaws.com DISCOURSE_SMTP_PORT: 587 DISCOURSE_SMTP_USER_NAME: AXXXXXXXXXXXXXXX DISCOURSE_SMTP_PASSWORD: "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" #DISCOURSE_SMTP_ENABLE_START_TLS: true # (optional, default true) ## If you added the Lets Encrypt template, uncomment below to get a free SSL certificate LETSENCRYPT_ACCOUNT_EMAIL: firstname.lastname@example.org ## The CDN address for this Discourse instance (configured to pull) ## see https://meta.discourse.org/t/14857 for details #DISCOURSE_CDN_URL: //discourse-cdn.example.com DISCOURSE_USE_S3: false ## The Docker container is stateless; all data is stored in /shared volumes: - volume: host: /var/discourse/shared/standalone guest: /shared - volume: host: /var/discourse/shared/standalone/log/var-log guest: /var/log ## Plugins go here ## see https://meta.discourse.org/t/19157 for details hooks: after_code: - exec: cd: $home/plugins cmd: - git clone https://github.com/discourse/docker_manager.git - git clone https://github.com/discourse/discourse-solved.git - git clone https://github.com/discourse/discourse-adplugin.git - git clone https://github.com/discoursehosting/discourse-sitemap.git - git clone https://github.com/discourse/discourse-whos-online.git - git clone https://github.com/discourse/discourse-push-notifications.git - git clone https://github.com/discourse/discourse-chat-integration.git - git clone https://github.com/davidtaylorhq/discourse-telegram-notifications.git - git clone https://github.com/discourse/discourse-knowledge-explorer.git - git clone https://github.com/discourse/discourse-math.git ## Any custom commands to run after building run: - exec: echo "Beginning of custom commands" ## If you want to set the 'From' email address for your first registration, uncomment and change: ## After getting the first signup email, re-comment the line. It only needs to run once. #- exec: rails r "SiteSetting.email@example.com'" - exec: echo "End of custom commands"
I don’t remember exactly, but I think I have migrated to S3 using
I have no clue but here’s a similar thread that has the exact same error you did:
You may want to check with a team member.
In your post of your
yml you “say”:
Is this correct? Is the password in double quotes in your
Yes, it is in double quotes. I have not changed it since I added it two years ago.
I have removed the double quotes, then rebuild the app again. Then tried
rake uploads:migrate_from_s3 again, but still no luck. Same error
To be honest, am an anti-fan of AWS and S3 so I do not use them.
Have you considered looking at the Rake task and reverse engineering that script and simply do the task manually, step-by-step?
I don’t have that much idea of discourse internal script and I am little scared to edit the script. I don’t want to ruin everything.
Can you tell me the location of discourse.config file? I can’t find it anywhere on my server. The find command returns nothing.
root@ip-172-31-7-247:/var/discourse# find . -name "*.config" root@ip-172-31-7-247:/var/discourse# find . -name "*.conf" ./shared/standalone/letsencrypt/engineersasylum.com/engineersasylum.com.conf ./shared/standalone/letsencrypt/engineersasylum.com/engineersasylum.com.csr.conf ./shared/standalone/letsencrypt/engineersasylum.com_ecc/engineersasylum.com.conf ./shared/standalone/letsencrypt/engineersasylum.com_ecc/engineersasylum.com.csr.conf ./shared/standalone/letsencrypt/ca/acme-v01.api.letsencrypt.org/ca.conf ./shared/standalone/letsencrypt/ca/acme-v02.api.letsencrypt.org/ca.conf ./shared/standalone/letsencrypt/account.conf ./shared/standalone/postgres_data/pg_hba.conf ./shared/standalone/postgres_data/postgresql.auto.conf ./shared/standalone/postgres_data/pg_ident.conf ./shared/standalone/postgres_data/postgresql.conf root@ip-172-31-7-247:/var/discourse#
Sorry that you misunderstood me.
I did not ask you to edit that file or any file.
To “Reverse engineer” a script, in my mind, means to study it and see how it works and has nothing to do with editing the file. It means to read it and understand it.
Normally if we read and understand any script or file giving us an error, we can normally get an understanding of how to fix the problem.
You are posting error messages from a file which you have not read, or so it seems.
Hope this helps.
I get your point. I will try to do it.
Can you suggest the location of the file ‘discourse.config’? Where are the configuration for the S3, stored?
Here ya go, just for you…
That file is in the container.
You will find that file and other config files here:
# docker exec -it app bash # cd /var/www/discourse/config
I absolutely recommend you do not exit any of those files unless you really know what are doing.
However, its definitely cool to read them and to understand them.
But I can’t find discourse.config file in the container. I have the only app.yml in the container folder.
The container folder you are talking about is NOT the container.
The container is a Docker construct build and launched to run as a Docker container.
BTW, do not take this the wrong way, but do you understand what a Docker container is? It is perfectly OK if you don’t, BTW. Many people use Discourse daily and have not yet learned Docker container basics. It’s never too late to learn
PS: @Pravi. I provided you the code to enter the container and change to the directory to where the config file are. Did you consider following that code I posted?
PS2: The rake task you are having trouble with (
migrate_from_s3) can be found here:
I have not much idea about discourse.
Yes, I find those files.
How can I troubleshoot the problem from the rake code?
One approach would be to look at your log file entries immediately before your error message, and see if you can locate the line (or lines) in the rake task file; and try to isolate what is causing your issue, exactly.
Solutions are normally the easy part. The hard part is discovering what the problem is.
Could you be more specific. You mean I have to look for log file in my website before executing the rake command. I didnt get it…
This is becoming a lesson in basic software error troubleshooting… which is OK.
You already have your logging info in your post #4 in this topic:
There is a lot of info there.
You also a the full Discourse Ruby code for this rake task, above.
My apologies, but I have a lot of tasks to do as well, so I’m just trying to show you, at a high level, how to debug and the solve your own problems the same way most of us would go about it.
Hope this helps.
I am sorry for taking too much time of yours. One last time, I tried to migrate to S3 instead of migrate from, which gives me following error even though I have provided all the details of s3 storage in my admin panel. Please take a look if you are familiar with this error.
root@ip-172-31-7-247:/var/discourse# ./launcher enter app root@ip-172-31-7-247-app:/var/www/discourse# rake uploads:migrate_to_s3 rake aborted! FileStore::ToS3MigrationError: Please provide the following environment variables: - DISCOURSE_S3_BUCKET - DISCOURSE_S3_REGION and either - DISCOURSE_S3_ACCESS_KEY_ID - DISCOURSE_S3_SECRET_ACCESS_KEY or - DISCOURSE_S3_USE_IAM_PROFILE /var/www/discourse/lib/file_store/to_s3_migration.rb:38:in `s3_options_from_env' /var/www/discourse/lib/tasks/uploads.rake:237:in `create_migration' /var/www/discourse/lib/tasks/uploads.rake:245:in `migrate_to_s3' /var/www/discourse/lib/tasks/uploads.rake:224:in `block in migrate_to_s3_all_sites' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.3.0/lib/rails_multisite/connection_management.rb:68:in `with_connection' /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rails_multisite-2.3.0/lib/rails_multisite/connection_management.rb:78:in `each_connection' /var/www/discourse/lib/tasks/uploads.rake:222:in `migrate_to_s3_all_sites' /var/www/discourse/lib/tasks/uploads.rake:218:in `block in <main>' /usr/local/bin/bundle:23:in `load' /usr/local/bin/bundle:23:in `<main>' Tasks: TOP => uploads:migrate_to_s3 (See full trace by running task with --trace)
I have added the all these DISCOURSE_S3_BUCKET, DISCOURSE_S3_REGION, DISCOURSE_S3_ACCESS_KEY_ID, DISCOURSE_S3_SECRET_ACCESS_KEY to app.yml and the above problem is solved. Now I am able to migrate to S3, but I cant migrate from S3. Its asking me to turn off S3 upload even though I have added
DISCOURSE_USE_S3: false in my app.yml
How to confirm that S3 upload is turned off?
 pry(main)> SiteSetting.enable_s3_uploads => false  pry(main)> GlobalSetting.use_s3 => false  pry(main)> GlobalSetting.use_s3? => true
I am passing the baton to the great discussion about this, in this meta team topic:
Lots of very details discussion there.
All the best… and hope this helps.