sau226
January 13, 2020, 6:12am
1
Description:
Sidekiq fails to load on upgrading to commit 9e399b42b96f115a8367c368862d62aeecfac680 of Discourse. This is running on a 2 container setup (web and DB, from ./discourse-setup 2container).
Buggy behavior:
When you go to <your-discourse-domain-name.tld>/sidekiq, you get the “The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.” error.
Behavior still occurs even after bootstrap, destroy and restart. Both the DB and web containers have been rebuilt but behavior still occurs too.
Expected behavior:
Sidekiq’s dashboard will open.
Bug self-diagnosis:
After visiting <your-discourse-domain-name.tld>/logs, I see only 1 error (I cleared all errors, viewed the sidekiq page and then checked). The error backtrace and info are in the relevant hide details boxes below.
Info
NoMethodError (undefined method transform_keys' for #<ActionDispatch::Request::Session:0x7fb46388f648 not yet loaded>) /var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:212:in stringify_keys’
Backtrace
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:212:in `stringify_keys'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:148:in `update'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:317:in `prepare_session'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:276:in `context'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:271:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:77:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/builder.rb:176:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/web.rb:104:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/sidekiq-6.0.4/lib/sidekiq/web.rb:109:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/routing/mapper.rb:19:in `block in <class:Constraints>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/routing/mapper.rb:48:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/journey/router.rb:49:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/journey/router.rb:32:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/journey/router.rb:32:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/routing/route_set.rb:837:in `call'
/var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:68:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/tempfile_reaper.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/conditional_get.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/head.rb:14:in `call'
/var/www/discourse/lib/content_security_policy/middleware.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:318:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:277:in `context'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/session/abstract/id.rb:271:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/cookies.rb:648:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/activesupport-6.0.1/lib/active_support/callbacks.rb:101:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/callbacks.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/actionable_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/debug_exceptions.rb:32:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/show_exceptions.rb:33:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/logster-2.5.1/lib/logster/middleware/reporter.rb:43:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/rack/logger.rb:28:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:18:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/remote_ip.rb:81:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/request_id.rb:27:in `call'
/var/www/discourse/lib/middleware/enforce_hostname.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/method_override.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/executor.rb:14:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/sendfile.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/actionpack-6.0.1/lib/action_dispatch/middleware/host_authorization.rb:77:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-mini-profiler-1.1.4/lib/mini_profiler/profiler.rb:184:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/message_bus-2.2.3/lib/message_bus/rack/middleware.rb:57:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:181:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/engine.rb:526:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/railties-6.0.1/lib/rails/railtie.rb:190:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:77:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `each'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/rack-2.1.1/lib/rack/urlmap.rb:61:in `call'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.2/lib/unicorn/http_server.rb:605:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.2/lib/unicorn/http_server.rb:700:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.2/lib/unicorn/http_server.rb:548:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.2/lib/unicorn/http_server.rb:144:in `start'
/var/www/discourse/vendor/bundle/ruby/2.6.0/gems/unicorn-5.5.2/bin/unicorn:128:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.6.0/bin/unicorn:23:in `<main>'
I have a feeling this was caused by the below PR upgrading the Rack gem based off my interpretation of the backtrace
master ← dependabot/bundler/rack-2.1.1
merged 01:09AM - 13 Jan 20 UTC
Bumps [rack](https://github.com/rack/rack) from 2.0.8 to 2.1.1.
<details>
<summa… ry>Changelog</summary>
<p><em>Sourced from <a href="https://github.com/rack/rack/blob/master/CHANGELOG.md">rack's changelog</a>.</em></p>
<blockquote>
<h2>[2.1.1] - 2020-01-12</h2>
<ul>
<li>Remove <code>Rack::Chunked</code> from <code>Rack::Server</code> default middleware. (<a href="https://github-redirect.dependabot.com/rack/rack/pull/1475">#1475</a>, <a href="https://github.com/ioquatix">@ioquatix</a>)</li>
</ul>
<h2>[2.1.0] - 2020-01-10</h2>
<h3>Added</h3>
<ul>
<li>Add support for <code>SameSite=None</code> cookie value. (<a href="https://github.com/hennikul">@hennikul</a>)</li>
<li>Add trailer headers. (<a href="https://github.com/eileencodes">@eileencodes</a>)</li>
<li>Add MIME Types for video streaming. (<a href="https://github.com/styd">@styd</a>)</li>
<li>Add MIME Type for WASM. (<a href="https://github.com/buildrtech">@buildrtech</a>)</li>
<li>Add <code>Early Hints(103)</code> to status codes. (<a href="https://github.com/egtra">@egtra</a>)</li>
<li>Add <code>Too Early(425)</code> to status codes. (<a href="(https://github.com/y-yagi)">@y-yagi</a>)</li>
<li>Add <code>Bandwidth Limit Exceeded(509)</code> to status codes. (<a href="https://github.com/CJKinni">@CJKinni</a>)</li>
<li>Add method for custom <code>ip_filter</code>. (<a href="https://github.com/svcastaneda">@svcastaneda</a>)</li>
<li>Add boot-time profiling capabilities to <code>rackup</code>. (<a href="https://github.com/tenderlove">@tenderlove</a>)</li>
<li>Add multi mapping support for <code>X-Accel-Mappings</code> header. (<a href="https://github.com/yoshuki">@yoshuki</a>)</li>
<li>Add <code>sync: false</code> option to <code>Rack::Deflater</code>. (Eric Wong)</li>
<li>Add <code>Builder#freeze_app</code> to freeze application and all middleware instances. (<a href="https://github.com/jeremyevans">@jeremyevans</a>)</li>
<li>Add API to extract cookies from <code>Rack::MockResponse</code>. (<a href="https://github.com/petercline">@petercline</a>)</li>
</ul>
<h3>Changed</h3>
<ul>
<li>Don't propagate nil values from middleware. (<a href="https://github.com/ioquatix">@ioquatix</a>)</li>
<li>Lazily initialize the response body and only buffer it if required. (<a href="https://github.com/ioquatix">@ioquatix</a>)</li>
<li>Fix deflater zlib buffer errors on empty body part. (<a href="https://github.com/felixbuenemann">@felixbuenemann</a>)</li>
<li>Set <code>X-Accel-Redirect</code> to percent-encoded path. (<a href="https://github.com/diskkid">@diskkid</a>)</li>
<li>Remove unnecessary buffer growing when parsing multipart. (<a href="https://github.com/tainoe">@tainoe</a>)</li>
<li>Expand the root path in <code>Rack::Static</code> upon initialization. (<a href="https://github.com/rosenfeld">@rosenfeld</a>)</li>
<li>Make <code>ShowExceptions</code> work with binary data. (<a href="https://github.com/axyjo">@axyjo</a>)</li>
<li>Use buffer string when parsing multipart requests. (<a href="https://github.com/janko-m">@janko-m</a>)</li>
<li>Support optional UTF-8 Byte Order Mark (BOM) in config.ru. (<a href="https://github.com/mikegee">@mikegee</a>)</li>
<li>Handle <code>X-Forwarded-For</code> with optional port. (<a href="https://github.com/dpritchett">@dpritchett</a>)</li>
<li>Use <code>Time#httpdate</code> format for Expires, as proposed by RFC 7231. (<a href="https://github.com/nanaya">@nanaya</a>)</li>
<li>Make <code>Utils.status_code</code> raise an error when the status symbol is invalid instead of <code>500</code>. (<a href="https://github.com/adambutler">@adambutler</a>)</li>
<li>Rename <code>Request::SCHEME_WHITELIST</code> to <code>Request::ALLOWED_SCHEMES</code>.</li>
<li>Make <code>Multipart::Parser.get_filename</code> accept files with <code>+</code> in their name. (<a href="https://github.com/lucaskanashiro">@lucaskanashiro</a>)</li>
<li>Add Falcon to the default handler fallbacks. (<a href="https://github.com/ioquatix">@ioquatix</a>)</li>
<li>Update codebase to avoid string mutations in preparation for <code>frozen_string_literals</code>. (<a href="https://github.com/pat">@pat</a>)</li>
<li>Change <code>MockRequest#env_for</code> to rely on the input optionally responding to <code>#size</code> instead of <code>#length</code>. (<a href="https://github.com/janko">@janko</a>)</li>
<li>Rename <code>Rack::File</code> -> <code>Rack::Files</code> and add deprecation notice. (<a href="https://github.com/postmodern">@postmodern</a>).</li>
<li>Prefer Base64 “strict encoding” for Base64 cookies. (<a href="https://github.com/ioquatix">@ioquatix</a>)</li>
</ul>
<h3>Removed</h3>
<ul>
<li>Remove <code>to_ary</code> from Response (<a href="https://github.com/tenderlove">@tenderlove</a>)</li>
<li>Deprecate <code>Rack::Session::Memcache</code> in favor of <code>Rack::Session::Dalli</code> from dalli gem (<a href="https://github.com/fatkodima">@fatkodima</a>)</li>
</ul>
<h3>Fixed</h3>
</tr></table> ... (truncated)
</blockquote>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a href="https://github.com/rack/rack/commit/799a520a015de5938bc01faa8e90b76589c6e7d3"><code>799a520</code></a> Updating changelog for 2-1-stable</li>
<li><a href="https://github.com/rack/rack/commit/4fabf4d70bb890d15a903ae4995c0349b0c69324"><code>4fabf4d</code></a> Bump version.</li>
<li><a href="https://github.com/rack/rack/commit/10512e68717e734f530d8f4d2f668a8c0ea6d5f8"><code>10512e6</code></a> Remove chunked middleware from default server stack.</li>
<li><a href="https://github.com/rack/rack/commit/db922514f49bbc8756a55da3cffe5ed5c0463bee"><code>db92251</code></a> Fix ActiveStorage use-case and add test case. Fixes <a href="https://github-redirect.dependabot.com/rack/rack/issues/1464">#1464</a>.</li>
<li><a href="https://github.com/rack/rack/commit/327a865a714942c4e3f257d846e01361f7eb9233"><code>327a865</code></a> Do not deprecate Rack::File</li>
<li><a href="https://github.com/rack/rack/commit/04f541ba9db04f3b13e5321c8d596c490955e1db"><code>04f541b</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rack/rack/issues/1462">#1462</a> from jeremyevans/sessionid-to_s</li>
<li><a href="https://github.com/rack/rack/commit/879ae7163a399a9ed36d876668f4ecae4ae8b9e4"><code>879ae71</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rack/rack/issues/1461">#1461</a> from p8/cleanup-changelog</li>
<li><a href="https://github.com/rack/rack/commit/acf7d44a078db48697f8d8bf92278c608a261c28"><code>acf7d44</code></a> Remove changes from CHANGELOG already present in older versions [ci skip]</li>
<li><a href="https://github.com/rack/rack/commit/7d14bac930a028501985441923fdca5d570fc778"><code>7d14bac</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rack/rack/issues/1162">#1162</a> from snood1205/master</li>
<li><a href="https://github.com/rack/rack/commit/2b61454982b47d066e5a0222dcd4a89d4808b32f"><code>2b61454</code></a> Merge pull request <a href="https://github-redirect.dependabot.com/rack/rack/issues/1459">#1459</a> from jimeh/add-breaking-change-warning-to-changelog</li>
<li>Additional commits viewable in <a href="https://github.com/rack/rack/compare/2.0.8...2.1.1">compare view</a></li>
</ul>
</details>
<br />
[](https://dependabot.com/compatibility-score.html?dependency-name=rack&package-manager=bundler&previous-version=2.0.8&new-version=2.1.1)
Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`.
[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)
---
<details>
<summary>Dependabot commands and options</summary>
<br />
You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it
- `@dependabot merge` will merge this PR after your CI passes on it
- `@dependabot squash and merge` will squash and merge this PR after your CI passes on it
- `@dependabot cancel merge` will cancel a previously requested merge and block automerging
- `@dependabot reopen` will reopen this PR if it is closed
- `@dependabot close` will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
- `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
- `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
- `@dependabot use these labels` will set the current labels as the default for future PRs for this repo and language
- `@dependabot use these reviewers` will set the current reviewers as the default for future PRs for this repo and language
- `@dependabot use these assignees` will set the current assignees as the default for future PRs for this repo and language
- `@dependabot use this milestone` will set the current milestone as the default for future PRs for this repo and language
- `@dependabot badge me` will comment on this PR with code to add a "Dependabot enabled" badge to your readme
Additionally, you can set the following in your Dependabot [dashboard](https://app.dependabot.com):
- Update frequency (including time of day and day of week)
- Pull request limits (per update run and/or open at any time)
- Out-of-range updates (receive only lockfile updates, if desired)
- Security updates (receive only security updates, if desired)
</details>
5 Likes
sam
(Sam Saffron)
January 13, 2020, 7:08am
3
This is fixed per:
committed 07:07AM - 13 Jan 20 UTC
We can not upgrade rack cause it breaks Sidekiq web.
I can not find a trivial f… ix short of disabling sessions in Sidekiq which
is a security concern.
We need to figure out how to reuse sessions with our Rails application in
Sidekiq.
This gets extra complex cause we use a special cookie store for sessions.
https://github.com/discourse/discourse/blob/9e399b42b96f115a8367c368862d62aeecfac680/lib/discourse_cookie_store.rb#L3-L21
We are tracking a separate TODO to upgrade Rack.
7 Likes