업데이트 후 다이제스트 전송 실패 (NoMethodError)

Ed_S · 11월 2, 2020, 6:06오후

오늘 오후에 2.6.0.beta5 (7fa6aca9db)로 업데이트를 진행했는데, 대시보드에서 실패한 작업에 대한 알림이 뜨고 있습니다:

실패한 이메일 작업이 3건 있습니다. app.yml을 확인하고 메일 서버 설정이 올바른지 점검해 주세요. Sidekiq에서 실패한 작업 보기.

발송된 이메일(*)이 보이지 않고, sidekiq 작업이 재시도(retry) 상태에 있습니다:

Jobs::UserEmail	
{"type"=>"digest", "user_id"=>206, "current_site_id"=>"default"}
Jobs::HandledExceptionWrapper: Wrapped NoMethodError: undefined method `value' for nil:NilClass

Preview Summary를 사용하여 이메일 전송을 테스트해 보았지만, 다음과 같은 오류가 발생합니다.

Server Error
while trying to load /admin/email/preview-digest.json
Error code: 500 error

app.yml은 변경되지 않았습니다. Mailgun을 사용 중이며, 업데이트 전까지는 모든 것이 정상적으로 작동했습니다.

(*) 수정: 다이제스트가 약 12건 발송되었지만, "실패한 이메일 작업이 265건 있습니다."라는 메시지가 표시됩니다. evidently 실패 여부가 다이제스트의 콘텐츠에 따라 달라지는 것 같습니다.

justin · 11월 2, 2020, 6:07오후

제3자 플러그인이 설치되어 있나요?

Ed_S · 11월 2, 2020, 6:11오후

아니요: 플러그인은 discourse-data-explorer 하나뿐입니다.

Ed_S · 11월 2, 2020, 6:40오후

일부 데이터 값이 nil이거나 비어 있는 것 같은데, 그럴 수 없는 값인 것 같습니다 - 혹은 문제가 되지 않을 수도 있습니다. 로그에서 두 개의 traceback을 가져왔습니다. 하나는 preview용이고, 다른 하나는 sidekiq job용인 것 같습니다:

/var/www/discourse/lib/pretty_text.rb:440:in `block in strip_secure_media'
nokogiri-1.10.10/lib/nokogiri/xml/node_set.rb:238:in `block in each'
nokogiri-1.10.10/lib/nokogiri/xml/node_set.rb:237:in `upto'
nokogiri-1.10.10/lib/nokogiri/xml/node_set.rb:237:in `each'
/var/www/discourse/lib/pretty_text.rb:434:in `strip_secure_media'
/var/www/discourse/lib/email/styles.rb:347:in `replace_secure_media_urls'
/var/www/discourse/lib/email/styles.rb:281:in `to_html'
/var/www/discourse/lib/email/renderer.rb:31:in `html'
/var/www/discourse/lib/email/sender.rb:70:in `send'
/var/www/discourse/app/jobs/regular/user_email.rb:70:in `send_user_email'
/var/www/discourse/app/jobs/regular/user_email.rb:25:in `execute'
/var/www/discourse/app/jobs/base.rb:232:in `block (2 levels) in perform'
rails_multisite-2.5.0/lib/rails_multisite/connection_management.rb:76:in `with_connection'
/var/www/discourse/app/jobs/base.rb:221:in `block in perform'
/var/www/discourse/app/jobs/base.rb:217:in `each'
/var/www/discourse/app/jobs/base.rb:217:in `perform'
sidekiq-6.1.2/lib/sidekiq/processor.rb:196:in `execute_job'
sidekiq-6.1.2/lib/sidekiq/processor.rb:164:in `block (2 levels) in process'
sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:138:in `block in invoke'
/var/www/discourse/lib/sidekiq/pausable.rb:138:in `call'
sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:140:in `block in invoke'
sidekiq-6.1.2/lib/sidekiq/middleware/chain.rb:143:in `invoke'
sidekiq-6.1.2/lib/sidekiq/processor.rb:163:in `block in process'
sidekiq-6.1.2/lib/sidekiq/processor.rb:136:in `block (6 levels) in dispatch'
sidekiq-6.1.2/lib/sidekiq/job_retry.rb:111:in `local'
sidekiq-6.1.2/lib/sidekiq/processor.rb:135:in `block (5 levels) in dispatch'
sidekiq-6.1.2/lib/sidekiq.rb:38:in `block in <module:Sidekiq>'
sidekiq-6.1.2/lib/sidekiq/processor.rb:131:in `block (4 levels) in dispatch'
sidekiq-6.1.2/lib/sidekiq/processor.rb:257:in `stats'
sidekiq-6.1.2/lib/sidekiq/processor.rb:126:in `block (3 levels) in dispatch'
sidekiq-6.1.2/lib/sidekiq/job_logger.rb:13:in `call'
sidekiq-6.1.2/lib/sidekiq/processor.rb:125:in `block (2 levels) in dispatch'
sidekiq-6.1.2/lib/sidekiq/job_retry.rb:78:in `global'
sidekiq-6.1.2/lib/sidekiq/processor.rb:124:in `block in dispatch'
sidekiq-6.1.2/lib/sidekiq/logger.rb:10:in `with'
sidekiq-6.1.2/lib/sidekiq/job_logger.rb:33:in `prepare'
sidekiq-6.1.2/lib/sidekiq/processor.rb:123:in `dispatch'
sidekiq-6.1.2/lib/sidekiq/processor.rb:162:in `process'
sidekiq-6.1.2/lib/sidekiq/processor.rb:78:in `process_one'
sidekiq-6.1.2/lib/sidekiq/processor.rb:68:in `run'
sidekiq-6.1.2/lib/sidekiq/util.rb:15:in `watchdog'
sidekiq-6.1.2/lib/sidekiq/util.rb:24:in `block in safe_thread'

그리고

lib/pretty_text.rb:440:in `block in strip_secure_media'
nokogiri (1.10.10) lib/nokogiri/xml/node_set.rb:238:in `block in each'
nokogiri (1.10.10) lib/nokogiri/xml/node_set.rb:237:in `upto'
nokogiri (1.10.10) lib/nokogiri/xml/node_set.rb:237:in `each'
lib/pretty_text.rb:434:in `strip_secure_media'
lib/email/styles.rb:347:in `replace_secure_media_urls'
lib/email/styles.rb:281:in `to_html'
lib/email/renderer.rb:31:in `html'
app/controllers/admin/email_controller.rb:93:in `preview_digest'
actionpack (6.0.3.3) lib/action_controller/metal/basic_implicit_render.rb:6:in `send_action'
actionpack (6.0.3.3) lib/abstract_controller/base.rb:195:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/rendering.rb:30:in `process_action'
actionpack (6.0.3.3) lib/abstract_controller/callbacks.rb:42:in `block in process_action'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:112:in `block in run_callbacks'
app/controllers/application_controller.rb:357:in `block in with_resolved_locale'
i18n (1.8.5) lib/i18n.rb:313:in `with_locale'
app/controllers/application_controller.rb:357:in `with_resolved_locale'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:121:in `block in run_callbacks'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:139:in `run_callbacks'
actionpack (6.0.3.3) lib/abstract_controller/callbacks.rb:41:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/rescue.rb:22:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/instrumentation.rb:33:in `block in process_action'
activesupport (6.0.3.3) lib/active_support/notifications.rb:180:in `block in instrument'
activesupport (6.0.3.3) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (6.0.3.3) lib/active_support/notifications.rb:180:in `instrument'
actionpack (6.0.3.3) lib/action_controller/metal/instrumentation.rb:32:in `process_action'
actionpack (6.0.3.3) lib/action_controller/metal/params_wrapper.rb:245:in `process_action'
activerecord (6.0.3.3) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (6.0.3.3) lib/abstract_controller/base.rb:136:in `process'
actionview (6.0.3.3) lib/action_view/rendering.rb:39:in `process'
rack-mini-profiler (2.2.0) lib/mini_profiler/profiling_methods.rb:111:in `block in profile_method'
actionpack (6.0.3.3) lib/action_controller/metal.rb:190:in `dispatch'
actionpack (6.0.3.3) lib/action_controller/metal.rb:254:in `dispatch'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:50:in `dispatch'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:33:in `serve'
actionpack (6.0.3.3) lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
actionpack (6.0.3.3) lib/action_dispatch/routing/mapper.rb:48:in `serve'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:49:in `block in serve'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (6.0.3.3) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (6.0.3.3) lib/action_dispatch/routing/route_set.rb:834:in `call'
lib/middleware/omniauth_bypass_middleware.rb:68:in `call'
rack (2.2.3) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.3) lib/rack/conditional_get.rb:27:in `call'
rack (2.2.3) lib/rack/head.rb:12:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:354:in `call'
rack (2.2.3) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.3) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/cookies.rb:648:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (6.0.3.3) lib/active_support/callbacks.rb:101:in `run_callbacks'
actionpack (6.0.3.3) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
logster (2.9.4) lib/logster/middleware/reporter.rb:43:in `call'
railties (6.0.3.3) lib/rails/rack/logger.rb:37:in `call_app'
railties (6.0.3.3) lib/rails/rack/logger.rb:28:in `call'
config/initializers/100-quiet_logger.rb:23:in `call'
config/initializers/100-silence_logger.rb:31:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/request_id.rb:27:in `call'
lib/middleware/enforce_hostname.rb:22:in `call'
rack (2.2.3) lib/rack/method_override.rb:24:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/executor.rb:14:in `call'
rack (2.2.3) lib/rack/sendfile.rb:110:in `call'
actionpack (6.0.3.3) lib/action_dispatch/middleware/host_authorization.rb:76:in `call'
rack-mini-profiler (2.2.0) lib/mini_profiler/profiler.rb:368:in `call'
message_bus (3.3.4) lib/message_bus/rack/middleware.rb:61:in `call'
lib/middleware/request_tracker.rb:176:in `call'
railties (6.0.3.3) lib/rails/engine.rb:527:in `call'
railties (6.0.3.3) lib/rails/railtie.rb:190:in `public_send'
railties (6.0.3.3) lib/rails/railtie.rb:190:in `method_missing'
rack (2.2.3) lib/rack/urlmap.rb:74:in `block in call'
rack (2.2.3) lib/rack/urlmap.rb:58:in `each'
rack (2.2.3) lib/rack/urlmap.rb:58:in `call'
unicorn (5.7.0) lib/unicorn/http_server.rb:632:in `process_client'
unicorn (5.7.0) lib/unicorn/http_server.rb:728:in `worker_loop'
unicorn (5.7.0) lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
unicorn (5.7.0) lib/unicorn/http_server.rb:144:in `start'
unicorn (5.7.0) bin/unicorn:128:in `<top (required)>'
vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<main>'

Falco · 11월 2, 2020, 7:00오후

사이트 설정에서 보안 미디어가 활성화되어 있나요?

Ed_S · 11월 2, 2020, 7:02오후

아니요, secure media(보안 미디어)는 비활성화되어 있습니다(기본값).

참고로 secure media allow embed images in emails(이메일에 내장 이미지 허용)는 활성화되어 있지만, 이 또한 기본값입니다.

Falco · 11월 2, 2020, 7:06오후

github.com/discourse/discourse

lib/email/styles.rb

d824a5edd


      
                  <img src="#{url}" data-embedded-secure-image="true" style="#{style}" />
                HTML
                div.remove
              end
            end
          end
          
          def to_html
            # needs to be before class + id strip because we need to style redacted
            # media and also not double-redact already redacted from lower levels
            replace_secure_media_urls
            strip_classes_and_ids
            replace_relative_urls
          
            @fragment.to_html
          end
          
          def to_s
            @fragment.to_s
          end

@martin 왜 replace_secure_media_urls를 무조건적으로 호출하나요? secure media를 확인하는 가드(guard)를 추가할 수 있을까요?

Ed_S · 11월 2, 2020, 10:25오후

참고로: 브라우저에서 업데이트를 실행했을 때 실패하여, 명령줄에서 다시 빌드했습니다.

(이 설치 환경이 성공적으로 업데이트된 환경과 어떤 식으로든 다른 상태에 놓였을 수 있다는 점을 염두에 두고 있습니다. 다른 호스트에 있는 또 다른 포럼은 성공적으로 업데이트되었으며, 이메일 전송 실패나 미리보기 요약 준비 오류가 발생하지 않습니다.)

Ed_S · 11월 2, 2020, 10:41오후

19일에 업데이트한 대로, 그때 이후로 새로 도입된 무언가를 확인하고 있습니다.

스택 트레이스의 해당 줄을 보면, attributes["srcset"]는 아마도 새로운 것일까요? 아니면 어쨌든 value가 존재하지 않습니다.

github.com/discourse/discourse

lib/pretty_text.rb

3655062c6


      
            end
          end
          
          # images by themselves or inside a onebox
          doc.css('img[src]').each do |img|
            url = if img.parent.classes.include?("aspect-image")
          
              # we are using the first image from the srcset here so we get the
              # optimized image instead of the original, because an optimized
              # image may be used for the onebox thumbnail
              srcset = img.attributes["srcset"].value
              srcset.split(",").first
            else
              img['src']
            end
          
            width = img.classes.include?('site-icon') ? 16 : img['width']
            height = img.classes.include?('site-icon') ? 16 : img['height']
            oneboxed = (img.parent&.parent&.classes || []).include?('onebox-body')
          
            if Upload.secure_media_url?(url)

어제의 커밋이 왜 아직 많은 사람들이 이 문제를 보고 있지 않은지 설명해 줄 수 있을 것입니다.

github.com/discourse/discourse

FIX: Ensure oneboxed secure images which are optimized and also lightboxed optimized images are embedded in email (#11061)

committed 11:52PM - 01 Nov 20 UTC

martin-brennan

+152 -32

We had an issue where onebox thumbnail was too large and thus was optimized, and… we are using the image URLs in post to redact and re-embed, based on the sha1 in the URL. Optimized image URLs have extra stuff on the end like _99x99 so we were not parsing out the sha1 correctly. Another issue I found was for posts that have giant images, the original was being used to embed in the email and thus would basically never get included because it is huge. For example the URL http://localhost:3000/secure-media-uploads/optimized/2X/7/787b17ea6140f4f022eb7f1509a692f2873cfe35_2_690x335.jpeg was not parsed correctly; we would end up with 787b17ea6140f4f022eb7f1509a692f2873cfe35_2_690x335.jpeg as the sha1 which would not find the image to re-embed that was already attached to the email. This fix will use the first optimized image of the detected upload when we are redacting and then re-embedding to make sure we are not sending giant things in email. Also, I detect if it is a onebox thumbnail or the site icon and force appropriate sizes and styles.

Saiano · 11월 3, 2020, 8:11오전

같은 문제인데, 어떻게 해야 하나요?

Ed_S · 11월 3, 2020, 9:12오전

@martin (또는 다른 누군가)이 위에 언급된 변경 사항 중 하나를 되돌리거나, @Falco가 제안한 가드를 추가해 주길 바랍니다.

merefield · 11월 3, 2020, 10:58오전

저도 같은 문제를 겪고 있습니다.

AstonJ · 11월 3, 2020, 11:44오전

저도 같은 문제입니다..

Ed_S · 11월 3, 2020, 12:56오후

이것이 우회책이 될 수 있을지 궁금합니다. 문제가 다이제스트 이메일에서 (일부) 이미지의 처리와 관련되어 있고, 임시로 이미지를 포함하지 않은 다이제스트를 보내는 것에 동의한다면, short email length를 매우 큰 값으로 설정하여 strip images from short emails가 모든 이미지를 제거하도록 할 수 있습니다.

수정: 아니요, 시도해 보았지만 여전히 sidekiq 작업이 쌓이고 있습니다. 많은 작업이 반복적으로 실행되었음에도 불구하고 실험 기간 동안 이메일이 발송되지 않았습니다.

Falco · 11월 3, 2020, 2:51오후

수정 사항을 제안한 PR이 열려 있습니다

GuidoD · 11월 3, 2020, 3:17오후

여기서도 같은 문제가 발생했어요. PR이 빨리 구현되길 바랍니다.

ezworldwide · 11월 3, 2020, 5:16오후

확인했습니다… 저도 같은 문제를 겪고 있습니다.

Falco · 11월 3, 2020, 5:24오후

PR이 최신 브랜치에 병합되었습니다. 업데이트 후 결과를 보고해 주세요.

Ed_S · 11월 3, 2020, 5:32오후

지금은 정상 작동해요. 감사합니다!

merefield · 11월 3, 2020, 6:50오후

지금까지 나한테는 괜찮아 보여, 고마워.

글		댓글	조회수
Failing to send emails, undefined method with_secure_media Product Support	2	681	12월 6, 2019
NoMethodError: undefined method `should_secure_uploads? - latest updates Product Support	3	317	4월 15, 2024
Lots of retries mail in sidekiq after upgrade discourse to 3.5.0.beta8-dev( 4893bda14a ) Product Support	1	80	7월 9, 2025
Sidekiq email error: no implicit conversion of nil into String Bug	20	6850	4월 30, 2016
Severe Email Issue since last update a couple days ago - 3.4.0.beta4-dev Product Support	24	808	2월 16, 2025

업데이트 후 다이제스트 전송 실패 (NoMethodError)

Related topics