1.5 Upgrade, Failed Sidekiq Email jobs


(fearlessfrog) #1

Upgraded to 1.5 today, from the last stable 1.4. Upgrade via /admin didn’t work, so I did a manual ./launcher upgrade that seemed to go ok without errors.

EDIT:

Synopsis - sidekiq was failing to send out digest emails because of an update to allow the site color cusomization to be used on link styles. As we had never created a custom color scheme and the upgrade didn’t seed the ColorScheme model the color was returned as nil.

Workaround - Create a custom color scheme and save and enable it to create defaults.

Fix - Add a seed for ColorScheme.rb for tertiary in the -> 1.5 migration and/or nil protection on the helpers. Probably, not got as far as testing a fix.

END EDIT <

Getting errors on failing email jobs:

Has a site setting changed for 1.5 I need to alter?

Here’s the error from the log (405 times repeated):

/var/www/discourse/app/views/user_notifications/digest.html.erb:76:in `+'
/var/www/discourse/app/views/user_notifications/digest.html.erb:76:in `_app_views_user_notifications_digest_html_erb___341622358929525153_70223693149480'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/template.rb:145:in `block in render'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:166:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/template.rb:333:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/template.rb:143:in `render'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/rack-mini-profiler-0.9.9.2/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/template_renderer.rb:54:in `block (2 levels) in render_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/abstract_renderer.rb:39:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:164:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/abstract_renderer.rb:39:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/template_renderer.rb:53:in `block in render_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/template_renderer.rb:61:in `render_with_layout'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/template_renderer.rb:52:in `render_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/template_renderer.rb:14:in `render'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/renderer.rb:46:in `render_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/renderer/renderer.rb:27:in `render'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/rendering.rb:100:in `_render_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/rendering.rb:83:in `render_to_body'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.6/lib/abstract_controller/rendering.rb:25:in `render'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:904:in `block in collect_responses'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:918:in `each'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:918:in `each_template'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:900:in `collect_responses'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:829:in `mail'
/var/www/discourse/lib/email/message_builder.rb:9:in `build_email'
/var/www/discourse/app/mailers/user_notifications.rb:112:in `digest'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.6/lib/abstract_controller/base.rb:198:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.6/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.6/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionpack-4.2.6/lib/abstract_controller/base.rb:137:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionview-4.2.6/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:596:in `block in process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/activesupport-4.2.6/lib/active_support/notifications.rb:164:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:593:in `process'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/base.rb:584:in `initialize'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/message_delivery.rb:25:in `new'
/var/www/discourse/vendor/bundle/ruby/2.0.0/gems/actionmailer-4.2.6/lib/action_mailer/message_delivery.rb:25:in `__getobj__'
/usr/local/lib/ruby/2.0.0/delegate.rb:70:in `method_missing'
/var/www/discourse/lib/email/sender.rb:30:in `send'
/var/www/discourse/app/jobs/regular/user_email.rb:41:in `execute'
/var/www/discourse/app/jobs/base.rb:154:in `block (2 levels) in perform'

Looking at the stable source, it’s here:

So I guess ‘anchor_color’ is nil and it falls over?

I think this commit assumes a color scheme has been created - I’ll test to see if that helps (we’ve never created an admin/customize/color scheme as yet), as perhaps the lack of ‘tertiary’ defined seems to break the unsubscribe link in the ColorScheme.hex_for_name(‘tertiary’):

Ok, that worked. The temporary work around is to Save and Enable a custom color scheme, so that the ‘tertiary’ color definition is picked up and the email digest render now doesn’t fall over.

Longer term, I think ColorScheme.rb assumes some data, so need to tweak the 1.5 migration if this customization was added to stable.


(Jeff Atwood) #2

Hmm, is this something we should fix @eviltrout?


(Paul Apostolos) #3

I had the same issue…We upgraded (from an early 1.5 version) last night and this morning we noticed lots of log errors and failed Sidekiq jobs.

The log said missing method in /var/www/discourse/app/views/user_notifications/digest.html.erb

I’ve been working on it all morning thinking it was our custom email plugin (although our plugin is VERY vanilla and mostly just used to add extra information to the email template for user notifications).

Glad I searched here for an answer.

I’ve added a color scheme and I am rebuilding now.


(fearlessfrog) #4

We didn’t need to rebuild, as once the sidekiq jobs retried then they went through ok. Just by hitting in admin Customize, ‘New Color Scheme’ and hitting ‘Save’ and then ‘Enable’ fixed it for us.


(Paul Apostolos) #5

Yeah, I know…But, because I had messed around so much with our plugin’s code, I wanted to get that all sorted and then do a rebuild.

All better now.


(Robin Ward) #6

I unfortunately can’t reproduce this error locally. I even blew away my database and created one that didn’t have a color scheme.

Even without a color scheme on the stable branch ColorScheme.hex_for_name('tertiary') returns a value for me, which makes sense as it will read the values from the scss file if no color scheme is present.


(fearlessfrog) #7

Perhaps related to going from a 1.4 to 1.5 as an upgrade rather than a fresh 1.5 base install?

It might be safer for us off of stable, as it looks like more discourse users follow the beta releases. We didn’t want to have to update too frequently, but I’d trade being on the more ‘normal’ path than having bigger/riskier gaps between releases.


(Sebastian) #8

Sounds like something that should be fixed. I can confirm that on my installation also no digest emails, all others go out…