Migration alter Uploads zu S3

Während meine neuen Uploads ordnungsgemäß auf Amazon S3 hochgeladen werden, schlägt dieser Befehl beim Verschieben der alten Dateien vom lokalen Speicher nach S3 fehl:

DISCOURSE_S3_BUCKET="bucketname" DISCOURSE_S3_REGION="ap-south-1" DISCOURSE_S3_ACCESS_KEY_ID="AKIchangedabitUNM2" DISCOURSE_S3_SECRET_ACCESS_KEY="rX6Pf3C_changedabit_YKKtdVrnbw6B" DISCOURSE_S3_CDN_URL="https://bucketname.s3.ap-south-1.amazonaws.com" rake uploads:migrate_to_s3

Vor 15 Tagen funktionierte dieser Befehl einwandfrei.
Heute erhalte ich diesen Fehler:

Prüfe, ob die Standardmigration bereits durchgeführt wurde...
7 von 14 Uploads wurden nicht nach S3 migriert. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
1 Beitrag wurde nicht auf die neue S3-Upload-URL umgemappt. Die S3-Migration für die Datenbank 'default' ist fehlgeschlagen.
Suche nach fehlenden Uploads auf: default

0 Beitrags-Uploads fehlen.

Keine Beiträge müssen neu verarbeitet werden.
Migriere Uploads zu S3 für 'default'...
Einige Uploads wurden nicht auf das neue Schema migriert. Bitte führen Sie diese Befehle in der Rails-Konsole aus:

SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)

Ich habe diese beiden Befehle auch in der Rails-Konsole ausgeführt, aber sie scheinen keinen Nutzen gebracht zu haben:

SiteSetting.migrate_to_new_scheme = true
Jobs::MigrateUploadScheme.new.execute(nil)

Hinweis: Meine Uploads sind so konfiguriert, dass sie sich in einem Unterordner des Buckets bucketname/uploads befinden.

3 „Gefällt mir“

Any idea here @vinothkannans?

Is it returning any errors? I’m unable to see any recent big changes in the code of the uploads:migrate_to_s3 rake task.

3 „Gefällt mir“

I had a similar problem. It had to do with the path of the uploads not matching what that test expects (like it’s missing ‘default’ in the path of something).

What I did was (in broad and likely unhelpful terms) to see what search is doing to find those uploads that it thinks are bad and fix them

3 „Gefällt mir“

I happen to solve it accidentally and in a way that I’m still not sure how it got solved.

I changed the command a couple of times mostly appending or removing the word ‘dualstack’ in the command and in CDN url field in Discourse Settings: https://bxyzbucket1.s3.dualstack.ap-south-1.amazonaws.com
But still it didn’t seem to work. Then randomly I gave the two same commands (told in the OP) in the rails console once again.
And 1 step I did randomly around this time was append the word /uploads to the bucketname in the command.

And ‘randomly’ (I need another random word here :slight_smile: ), I succeeded.
So, I can’t say who was the hero of the resolution here.
Thanks for listening anyway.

2 „Gefällt mir“

Whatever error was coming/showing, I already included in my original post. But I agree that that error/msg wasn’t very helpful in diagnosis.

Alas, today again, I thought since I couldn’t understand the problem in migration, I created 1 upload locally. Then tried to shift that again to S3 with this command:

DISCOURSE_S3_BUCKET="bxyzbucket1/uploads" DISCOURSE_S3_REGION="ap-south-1" DISCOURSE_S3_ACCESS_KEY_ID="AKIAchangedBRF" DISCOURSE_S3_SECRET_ACCESS_KEY="H0P6Oo8changed1AuLkRDCQK8" DISCOURSE_S3_CDN_URL="https://bxyzbucket1.s3.ap-south-1.amazonaws.com/uploads" rake uploads:migrate_to_s3

And though it shifted the 1 new upload to S3, but with this error included:

1 of 14 uploads are not migrated to S3. S3 migration failed for db 'default'.
1 posts are not remapped to new S3 upload URL. S3 migration failed for db 'default'.

What does this error signify? Can it pose some problem if there are thousands of images to be shifted from ‘Local >> S3’? @vinothkannans

Can you please run the following command in rails console and post the output here

Upload.by_users.where("url NOT LIKE '//%' AND url NOT LIKE '#{GlobalSetting.relative_url_root}/uploads/default/original/_X/%'")
3 „Gefällt mir“

Sorry that I was away from this operation for some time.
I’m running the rails command sent by you, but no output is coming:

I still get the error: " S3 migration failed for db ‘default’"
And although post uploads/images are showing ok in my bucket and on site, but site logos etc (under Admin Settings> Branding) are missing and I’m not even able to update them with fresh ones. They just stay blank.

I also ran (with no errors): rake posts:rebake_uncooked_posts and rake posts:rebake

@vinothkannans Selbst nach einem Jahr, wenn ich den Vorgang aus irgendeinem Grund wiederhole, tritt immer noch derselbe Fehler auf.