לא ניתן לשחזור בשל עדכון של gem ה-AWS SDK והגנות חדשות של שלמות נתוני AWS

התקנה
41

If you rebuilt the app then the changes made in the container are gone. Perhaps the backblaze folks fixed things while you were doing that.

לייק 1
42

I just commented out the S3 and rebuilt. I guess I’ll stick to local storage from now on.

לייק 1
43

Local storage works fine. It’s just deleting that does not work.

44

Thanks for posting and confirming there is no current workaround without downgrading the aws SDKs. I ran into issues with the b2 s3 apis from another project using the golang aws sdk, trying to back up a VictoriaMetrics database.

Do you have any sense from your engineering team how or when they may choose to resolve this issue, or have any way to track/get updates on a supported workaround? I’m trying to decide if I should fork and recompile a project with a dependency change or just wait a little while for an update!

Thanks!

45

הנה ההליך שעבד אצלי כדי להוריד את גרסאות ה-AWS Gems:

 # כדי להיכנס לקונטיינר:
./launcher enter app
# כנראה שצריך לשחרר את Gemfile.lock:
bundle config set frozen false
# להגדיר גרסה ישנה יותר של ה-gem sdk-s3:
sed -i 's/gem "aws-sdk-s3", require: false/gem "aws-sdk-s3", "1.177.0", require: false/' Gemfile
# להוריד את גרסת ה-S3 gem להתאמה עם התוכן הנוכחי ב-Gemfile:
bundle update aws-sdk-s3
# גם להוריד את הגרסה של ה-gem aws-sdk-core:
bundle add aws-sdk-core --version 3.215.

לאחר ביצוע שינויים אלה, הצלחתי לשמור גיבוי ל-B2. אני בטוח שזה רק פתרון זמני שיאבד עם העדכון הבא של Discourse, ולכן אני מקווה ש-@PatPatterson וצוות Backblaze יוכלו לספק תיקון תאימות קבוע בקרוב.

2 לייקים
46

Hopefully you won’t need to do this, but it’s possible to put that sed command in the app.yml so that it would happen automatically when you rebuild. I think just sticking it below where the plugins get cloned would do it. But it could be more complicated than I think.

לייק 1
47

Hi AntiMetaman,

I have spent the last week trying to get backups working with s3 storage and finally got the backup to work with this thread. Honestly, I am not sure which part worked.

I followed what you did with the gem uninstall and install but now I get an error trying to upload images. My error log shows this

not entitled /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/aws-sdk-core-3.219.0/lib/seahorse/client/plugins/raise_response_errors.rb:17:in `call’ /var/www/discourse/vendor/bundle/ruby/3.3.0/gems/aw

Before I have to trash my whole server and start again (honestly, at this point I’m not even sure I want to continue) do you know how I can reverse the gem uninstall and install so I can try and see if that fixes the issue?

48

ההתנהגות שראיתי במערכת שלנו (שמשתמשת ב-Backblaze לגיבויים) היא שהגיבויים התחילו לכשל, ללא התראה, בפברואר. פתאום שמתי לב לכך היום, כי אני בודק את הגיבויים שלנו מפעם לפעם. הייתי מחשיב את זה כבעיה מאוד חמורה?

האם אפשרי להוריד את ה-AWS-SDK עד שמסוגלים למצוא פתרון ל-Backblaze ולספקיות אחרות שאינן של AWS?

49

But I did for a site that’s using Backblaze. I created a template I put in /root/aws-revert-template.yml with this:

# This template reverts aws-sdk-s3 to a version that works with backblaze

params:
  home: /var/www/discourse

hooks:
  after_bundle_exec:
    - exec:
        cd: $home
        cmd:
          - bundle config set frozen false
          - "sed -i 's/gem \"aws-sdk-s3\", require: false/gem \"aws-sdk-s3\", \"1.177.0\", require: false/' Gemfile"
          - bundle update aws-sdk-s3
          - bundle add aws-sdk-core --version 3.215

And then added it to my app.yml like this:

# IMPORTANT: SET A SECRET PASSWORD in Postgres for the Discourse User
# TODO: change SOME_SECRET in this template

templates:
  - "templates/web.template.yml"
  - "templates/web.ratelimited.template.yml"
## Uncomment these two lines if you wish to add Lets Encrypt (https)
  - "templates/web.ssl.template.yml"
  - "templates/web.letsencrypt.ssl.template.yml"
  - "/root/aws-revert-template.yml"

And I ran an upgrade to stable and it seems to be working.

You could also just add what’s in the template to your app.yml

3 לייקים
50

Noticed that this page - S3-Compatible API - no longer lists the various checksum-* headers as being unsupported.

@PatPatterson not sure if you’re still in this thread or not - but was official support added for those?

2 לייקים
51

Can you confirm that your backblaze b2 is working without the fix I suggest?

52

Thanks for quick response, and sorry - I should have added more detail. In a different open source package (Mastodon) we wound up also locking the aws-sdk-core gem to be < 3.216.0 to handle this exact issue for people who are using backblaze (and I suspect others, but that’s where it came up).

But then in last few weeks I noticed:

  • That page on backblaze site no longers lists those headers as unsupported
  • A recent aws gem release mentions fixing the issue where when_required was not fully honored previously

In previous research I’d seen this thread with more or less the same issue and was trying to understand if we can safely gem update now if indeed backblaze has added that support, and/or the gem now fully works around the issue.

2 לייקים
53

Hi - yes, Backblaze B2 added official support for the checksum headers earlier this year.

2 לייקים
54

Excellent! Thanks for confirming.