Continuing the discussion from `rake uploads:migrate_to_s3` needs some love : I still can’t get migrate_to_s3 to work with Digital Ocean Spaces. root@shadrach-rbx8888:/var/www/discourse# rails c [1] pry(main)> GlobalSetting.use_s3 => true [2] pry(main)> GlobalSetting.use_s3? => false [3] pry(main)> …

If it is urgent, I’ll recommend going with AWS S3 first since the rake task is used internally by us to migrate uploads to the S3 store. In theory, the rake task should work with Spaces as well but it wouldn’t be our top priority at the moment.

I ended up getting this working myself by adding those environment variables to app.yml and rebuilding. Feels like a workaround and probably not the best way forward long-term.

@rishabh and @zogstrip worked on this task recently so they will probably be able to advise better.

[image] pfaffman: The latest uploads.rb cranks up Aws::S3 with just a few environment variables (and not those required for Digital Ocean spaces). I’ve fixed this in: https://github.com/discourse/discourse/commit/e69634ec3aa594abd3c2d1057dcf1f65d09c48f5 For the GlobalSetting issue, can you …

I still don’t see how GlobalSetting.use_s3? is ever true. I have GlobalSetting.use_s3, but not with a ? and I don’t see how seeting those ENV vars sets GlobalSetting.use_s3?. Maybe that’s supposed to happen somewhere that I don’t know about or understand, but I removed the ? in order to be able to …

[image] pfaffman: I have GlobalSetting.use_s3 , but not with a ? It’s right here: app/models/global_setting.rb 6b006c383 def self.use_s3? (@use_s3 ||= begin s3_bucket && s3_region && ( …

Ah! I figured it must be in some place I didn’t understand. Thanks for that. Now that I understand, I think that it’s because I don’t have GlobalSetting.s3_region, even if I do pass the ENV variable. Any guess on what table is causing the value too long?

[image] rishabh: DISCOURSE_S3_ACCESS_KEY_ID="" \ DISCOURSE_S3_SECRET_ACCESS_KEY="" \ DISCOURSE_S3_REGION="us-east-1" \ DISCOURSE_S3_BUCKET="uploadbucket" \ DISCOURSE_S3_CDN_URL="https://xyz.com" \ DRY_RUN=1 rake uploads:migrate_to_s3 That works for me. You are setting a value for for the acce…

[image] gerhard: It must be the url column in the topic_links table. It’s the only column with a restriction of 500 chars. Thanks for that. I’ve got 30 on this site (different from the site discussed here ) with more than 490 characters. I’m not clear why the remap has this problem. Oh! Is it …

[image] gerhard: You are setting a value for for the access keys, right? I think the problem was that the S3_REGION wasn’t set. I just added it to the ENV in app.yml and tried again and now. DRY_RUN=1 rake uploads:migrate_to_s3 works as expected. Now I’d like to figure out what to do about…

Migrate_to_s3 for Digital Ocean Spaces woes

Support Self-hosting

hosna (hosna) February 21, 2019, 11:36am 7

Any news for minio?

Topic		Replies	Views
Rake uploads:migrate_from_s3 fails Self-hosting s3	31	3130	April 24, 2021
Uploads migration to S3 fails Support	26	3251	May 7, 2021
S3 uploads migrations to DigitalOcean S3 Spaces fails Support	0	570	April 15, 2020
1 of 16551 uploads are not migrated to S3. S3 migration failed for db 'default' Self-hosting cdn , s3	21	1835	July 11, 2023
Digital Ocean Spaces (S3) "unable to sign request without credentials set" Support uploads , backups , s3	19	358	January 7, 2026

Migrate_to_s3 for Digital Ocean Spaces woes

Related topics