백업용 Backblaze 문제 - S3에서 백업 목록을 가져오지 못함: 서명 검증 실패

제품 지원
1

안녕하세요.

백업 전용으로 Backblaze 버킷을 설정하려고 합니다.

다음 링크를 참고했습니다: REQ: Support S3 backup to a service like Backblaze - #4 by frold
그리고 링크된 게시물 "using-object-storage-for-uploads-s3-clones"도 참조했습니다.

app.yml 파일에 다음 내용을 추가했습니다:

  DISCOURSE_S3_REGION: “s3.us-west-002”
  DISCOURSE_S3_INSTALL_CORS_RULE: false
  DISCOURSE_S3_ENDPOINT: https://s3.us-west-002.backblazeb2.com
  DISCOURSE_S3_ACCESS_KEY_ID: xxxxxxxxxxxxxxxxxxxx
  DISCOURSE_S3_SECRET_ACCESS_KEY: xxxxxxxxxxxxxxxxxxxxxxx
  DISCOURSE_S3_BACKUP_BUCKET: bucket/backups
  DISCOURSE_BACKUP_LOCATION: s3

Set up file and image uploads to S3 글을 읽고 난 후, 공개(public)로 설정하고 소문자로 된 이름의 새 버킷을 다시 생성했습니다.

Backblaze에서는 discourse 버킷에만 접근 권한이 있는 새로운 앱 키(App key)를 생성했습니다. 이 키는 모든 버킷 이름을 나열할 권한은 있습니다.

Discourse의 백업 페이지에서는 슬픈 얼굴/깨진 페이지가 표시됩니다. 로그에는 다음과 같은 내용이 나옵니다:

Failed to list backups from S3: Signature validation failed

백트레이스(Backtrace):

/var/www/discourse/lib/backup_restore/s3_backup_store.rb:83:in `rescue in unsorted_files'

/var/www/discourse/lib/backup_restore/s3_backup_store.rb:72:in `unsorted_files'

/var/www/discourse/lib/backup_restore/backup_store.rb:23:in `files'

/var/www/discourse/app/controllers/admin/backups_controller.rb:22:in `block (2 levels) in index'

actionpack-6.1.3.1/lib/action_controller/metal/mime_responds.rb:214:in `respond_to'

/var/www/discourse/app/controllers/admin/backups_controller.rb:11:in `index'

actionpack-6.1.3.1/lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'

actionpack-6.1.3.1/lib/abstract_controller/base.rb:228:in `process_action'

actionpack-6.1.3.1/lib/action_controller/metal/rendering.rb:30:in `process_action'

actionpack-6.1.3.1/lib/abstract_controller/callbacks.rb:42:in `block in process_action'

activesupport-6.1.3.1/lib/active_support/callbacks.rb:117:in `block in run_callbacks'

/var/www/discourse/app/controllers/application_controller.rb:383:in `block in with_resolved_locale'

i18n-1.8.10/lib/i18n.rb:314:in `with_locale'

/var/www/discourse/app/controllers/application_controller.rb:383:in `with_resolved_locale'

activesupport-6.1.3.1/lib/active_support/callbacks.rb:126:in `block in run_callbacks'

activesupport-6.1.3.1/lib/active_support/callbacks.rb:137:in `run_callbacks'

actionpack-6.1.3.1/lib/abstract_controller/callbacks.rb:41:in `process_action'

actionpack-6.1.3.1/lib/action_controller/metal/rescue.rb:22:in `process_action'

actionpack-6.1.3.1/lib/action_controller/metal/instrumentation.rb:34:in `block in process_action'

activesupport-6.1.3.1/lib/active_support/notifications.rb:203:in `block in instrument'

activesupport-6.1.3.1/lib/active_support/notifications/instrumenter.rb:24:in `instrument'

activesupport-6.1.3.1/lib/active_support/notifications.rb:203:in `instrument'

actionpack-6.1.3.1/lib/action_controller/metal/instrumentation.rb:33:in `process_action'

actionpack-6.1.3.1/lib/action_controller/metal/params_wrapper.rb:249:in `process_action'

activerecord-6.1.3.1/lib/active_record/railties/controller_runtime.rb:27:in `process_action'

actionpack-6.1.3.1/lib/abstract_controller/base.rb:165:in `process'

actionview-6.1.3.1/lib/action_view/rendering.rb:39:in `process'

rack-mini-profiler-2.3.1/lib/mini_profiler/profiling_methods.rb:111:in `block in profile_method'

actionpack-6.1.3.1/lib/action_controller/metal.rb:190:in `dispatch'

actionpack-6.1.3.1/lib/action_controller/metal.rb:254:in `dispatch'

actionpack-6.1.3.1/lib/action_dispatch/routing/route_set.rb:50:in `dispatch'

actionpack-6.1.3.1/lib/action_dispatch/routing/route_set.rb:33:in `serve'

actionpack-6.1.3.1/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'

actionpack-6.1.3.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'

actionpack-6.1.3.1/lib/action_dispatch/journey/router.rb:50:in `block in serve'

actionpack-6.1.3.1/lib/action_dispatch/journey/router.rb:32:in `each'

actionpack-6.1.3.1/lib/action_dispatch/journey/router.rb:32:in `serve'

actionpack-6.1.3.1/lib/action_dispatch/routing/route_set.rb:842:in `call'

/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:71:in `call'

rack-2.2.3/lib/rack/tempfile_reaper.rb:15:in `call'

rack-2.2.3/lib/rack/conditional_get.rb:27:in `call'

rack-2.2.3/lib/rack/head.rb:12:in `call'

actionpack-6.1.3.1/lib/action_dispatch/http/permissions_policy.rb:22:in `call'

/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call'

/var/www/discourse/lib/middleware/anonymous_cache.rb:355:in `call'

rack-2.2.3/lib/rack/session/abstract/id.rb:266:in `context'

rack-2.2.3/lib/rack/session/abstract/id.rb:260:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/cookies.rb:689:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'

activesupport-6.1.3.1/lib/active_support/callbacks.rb:98:in `run_callbacks'

actionpack-6.1.3.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/actionable_exceptions.rb:18:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/debug_exceptions.rb:29:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'

logster-2.9.6/lib/logster/middleware/reporter.rb:43:in `call'

railties-6.1.3.1/lib/rails/rack/logger.rb:37:in `call_app'

railties-6.1.3.1/lib/rails/rack/logger.rb:28:in `call'

/var/www/discourse/config/initializers/100-quiet_logger.rb:23:in `call'

/var/www/discourse/config/initializers/100-silence_logger.rb:31:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/request_id.rb:26:in `call'

/var/www/discourse/lib/middleware/enforce_hostname.rb:23:in `call'

rack-2.2.3/lib/rack/method_override.rb:24:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/executor.rb:14:in `call'

rack-2.2.3/lib/rack/sendfile.rb:110:in `call'

actionpack-6.1.3.1/lib/action_dispatch/middleware/host_authorization.rb:92:in `call'

rack-mini-profiler-2.3.1/lib/mini_profiler/profiler.rb:373:in `call'

message_bus-3.3.4/lib/message_bus/rack/middleware.rb:61:in `call'

/var/www/discourse/lib/middleware/request_tracker.rb:177:in `call'

railties-6.1.3.1/lib/rails/engine.rb:539:in `call'

railties-6.1.3.1/lib/rails/railtie.rb:207:in `public_send'

railties-6.1.3.1/lib/rails/railtie.rb:207:in `method_missing'

rack-2.2.3/lib/rack/urlmap.rb:74:in `block in call'

rack-2.2.3/lib/rack/urlmap.rb:58:in `each'

rack-2.2.3/lib/rack/urlmap.rb:58:in `call'

unicorn-6.0.0/lib/unicorn/http_server.rb:634:in `process_client'

unicorn-6.0.0/lib/unicorn/http_server.rb:732:in `worker_loop'

unicorn-6.0.0/lib/unicorn/http_server.rb:547:in `spawn_missing_workers'

unicorn-6.0.0/lib/unicorn/http_server.rb:143:in `start'

unicorn-6.0.0/bin/unicorn:128:in `<top (required)>'

/var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `load'

/var/www/discourse/vendor/bundle/ruby/2.7.0/bin/unicorn:23:in `<main>'

각 구성 요소와 단계를 이해한다고 생각하지만, 이 문제를 디버깅하기 위한 다음 단계가 무엇인지 전혀 모르겠습니다.

키 ID와 키는 숫자와 알파벳으로만 구성되어 있으며 "+"나 기타 기호가 없습니다… 하지만 그렇더라도 따옴표로 묶어야 할까요?

제안해 주시면 감사하겠습니다!

2

Using Object Storage for Uploads (S3 & Clones)을 따라 해보실 수 있으신가요?

3

제안해 주셔서 감사합니다 @Falco !

현재 제 app.yml의 관련 섹션은 다음과 같이 변경되었습니다:

  ## DISCOURSE_USE_S3: true  -- CDN 사용을 위한 것인가요?!
  DISCOURSE_S3_REGION: “us-west-002”
  DISCOURSE_S3_INSTALL_CORS_RULE: false
  DISCOURSE_S3_CONFIGURE_TOMBSTONE_POLICY: false
  DISCOURSE_S3_ENDPOINT: https://s3.us-west-002.backblazeb2.com
  DISCOURSE_S3_ACCESS_KEY_ID: XXXXXXXXXXXXXXXXXX
  DISCOURSE_S3_SECRET_ACCESS_KEY: XXXXXXXXXXXXXXXXXX
  ## DISCOURSE_S3_CDN_URL: https://
  DISCOURSE_S3_BUCKET: discourse
  DISCOURSE_S3_BACKUP_BUCKET: discourse/backups
  DISCOURSE_BACKUP_LOCATION: s3

CORS에 대한 참고 사항도 확인하여, S3와 B2 프로토콜 모두에서 모든 도메인으로부터의 접근을 허용하도록 버킷의 CORS 정책을 설정해 보았습니다. (동작이 확인되면 보안 설정을 강화할 예정입니다.)

하지만 역시 성공하지 못했습니다.

모든 버킷에 대한 전체 접근 권한이 있는 새로운 앱 키를 생성하여 해당 키로 다시 빌드해 보았지만, 역시 성공하지 못했습니다.

추가적인 제안이 있을까요?

4

위 설정을 사용하되

DISCOURSE_USE_S3: true

으로 변경해서도 시도해 보았습니다.

다른 버킷들은 다른 서비스의 백업에 성공적으로 사용되고 있습니다(참고: 해당 백업 도구는 B2 API를 사용합니다).

소소한 업데이트: “all buckets” API 키를 제거했는데 이제 잘못된 키 오류가 발생합니다. 즉, 제 서버가 Backblaze에 연결되고 있다는 뜻입니다. 서명이 왜 틀어지는지 이유는 알 수 없습니다. 시간을 다시 확인해 보았는데 정확합니다(시각대는 UTC).

5

다음 날에는 “그냥” 작동했습니다. :roll_eyes: