בעיות בהעלאות לאחר שינוי מרכז הנתונים של DO Spaces

תמיכה
7

Sorry, neet to reemerge this one.
During last week I spent some time reading the uploads rake tasks code trying to understand what goes on under the hood of the recover_from_tombstone and recover one.
It’s a difficult thing because of the encapsulation of the classes, so I’d say I mostly failed.

What I understood though (please @Falco correct me if I am wrong) is that the file name on disk of an upload is created combining its sha1 and its original extension. Then it’s stored on disk/aws in a directory whose path depends on the first and sometimes second letter in its name, within 1X or 2X or 3X… (how are these determined I do not understand).
Finally the sha1 and file name are stored, among other things, in records of the uploads table in PostgreSQL.

Going back to what happened during our change of Digital Ocean datacenter, this is what happened at the best of my understanding:

  1. we copied all the files from ams3 to fra1
  2. we failed to perform DbHelper.remap('oldbucketurl', 'newbucketurl') as suggested by @falco but it was not clear to us we had to in this case
  3. we launched a global rebake. At this stage thousands of images “broke” and many were moved to tombstone. It’s not completely clear to me why.
  4. I realised something was wrong, interrupted the on-going rebake and found about the remap command searching here in meta. We launched the DbHelper.remap('oldbucketurl', 'newbucketurl') task
  5. In order to recover the images that were moved to tombstone at step 3 we launched a rake uploads:recover_from_tombstone which recovered some, but left hundreds of others unrecovered, and showed errors about the sha1 of the files such as Warning /t/eclisse-parziale-di-sole-04-01-2011/14456/50 had an incorrect 3f5a1c136b97aebac4a188432c8e3ab7487f3bca should be ec88ee9eea18f3b8424bfef796345c68582911b5 storing in custom field 'rake uploads:fix_relative_upload_links' can fix this as if the file was somehow changed and hence the sha1 is now different. The recovery of such files fails.

We never changed the files while moving them between the two datacentres. Using s3cmd we were literally dumping them locally from the old bucket and immediately re-uploading them in the new one.

Why should the sha1 calculated by Discourse be different at all?

Would it be possible to just force the recover task to ignore the sha1 discrepancy and just adapt import in DB what is there or rename the existing files with the new sha1 while recovering them?

Am I missing something obvious? Thanks all for your help.