After rebuilding app, can't sign in with error 500

Bug
1

It says:

Oops

The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience.

Detailed information about the error was logged, and an automatic notification generated. We’ll take a look at it.

No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site’s feedback category.

In logs:

info

NoMethodError (undefined method `keys' for nil:NilClass)
(eval):76:in `_fast_attributes'
app/controllers/application_controller.rb:650:in `preload_current_user_data'
app/controllers/application_controller.rb:445:in `preload_json'
app/controllers/application_controller.rb:421:in `block in with_resolved_locale'
app/controllers/application_controller.rb:421:in `with_resolved_locale'
lib/middleware/omniauth_bypass_middleware.rb:74:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:389:in `call'
lib/middleware/gtm_script_nonce_injector.rb:10:in `call'
config/initializers/008-rack-cors.rb:24:in `call'
config/initializers/100-quiet_logger.rb:20:in `call'
config/initializers/100-silence_logger.rb:29:in `call'
lib/middleware/enforce_hostname.rb:24:in `call'
lib/middleware/request_tracker.rb:228:in `call'

backtrace

plugins/chat/lib/chat/channel_fetcher.rb:164:in `preload_custom_fields_for'
plugins/chat/lib/chat/channel_fetcher.rb:159:in `secured_public_channels'
plugins/chat/lib/chat/channel_fetcher.rb:9:in `structured'
plugins/chat/plugin.rb:239:in `block (2 levels) in activate!'
(eval):76:in `_fast_attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:468:in `rescue in attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:455:in `attributes'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:480:in `_serializable_hash'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:359:in `serializable_hash'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:347:in `as_json'
activesupport (7.0.5.1) lib/active_support/json/encoding.rb:22:in `encode'
activesupport (7.0.5.1) lib/active_support/json/encoding.rb:22:in `encode'
activesupport (7.0.5.1) lib/active_support/core_ext/object/json.rb:42:in `to_json'
active_model_serializers (0.8.4) lib/active_model/serializer.rb:331:in `to_json'
multi_json (1.15.0) lib/multi_json/adapters/oj.rb:56:in `dump'
multi_json (1.15.0) lib/multi_json/adapters/oj.rb:56:in `dump'
multi_json (1.15.0) lib/multi_json/adapter.rb:25:in `dump'
multi_json (1.15.0) lib/multi_json.rb:139:in `dump'
app/controllers/application_controller.rb:650:in `preload_current_user_data'
app/controllers/application_controller.rb:445:in `preload_json'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:400:in `block in make_lambda'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:199:in `block (2 levels) in halting'
actionpack (7.0.5.1) lib/abstract_controller/callbacks.rb:34:in `block (2 levels) in <module:Callbacks>'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:200:in `block in halting'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:595:in `block in invoke_before'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:595:in `each'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:595:in `invoke_before'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:116:in `block in run_callbacks'
app/controllers/application_controller.rb:421:in `block in with_resolved_locale'
i18n (1.14.1) lib/i18n.rb:322:in `with_locale'
app/controllers/application_controller.rb:421:in `with_resolved_locale'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:127:in `block in run_callbacks'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:138:in `run_callbacks'
actionpack (7.0.5.1) lib/abstract_controller/callbacks.rb:233:in `process_action'
actionpack (7.0.5.1) lib/action_controller/metal/rescue.rb:22:in `process_action'
actionpack (7.0.5.1) lib/action_controller/metal/instrumentation.rb:67:in `block in process_action'
activesupport (7.0.5.1) lib/active_support/notifications.rb:206:in `block in instrument'
activesupport (7.0.5.1) lib/active_support/notifications/instrumenter.rb:24:in `instrument'
activesupport (7.0.5.1) lib/active_support/notifications.rb:206:in `instrument'
actionpack (7.0.5.1) lib/action_controller/metal/instrumentation.rb:66:in `process_action'
actionpack (7.0.5.1) lib/action_controller/metal/params_wrapper.rb:259:in `process_action'
activerecord (7.0.5.1) lib/active_record/railties/controller_runtime.rb:27:in `process_action'
actionpack (7.0.5.1) lib/abstract_controller/base.rb:151:in `process'
actionview (7.0.5.1) lib/action_view/rendering.rb:39:in `process'
rack-mini-profiler (3.1.0) lib/mini_profiler/profiling_methods.rb:85:in `block in profile_method' 
actionpack (7.0.5.1) lib/action_controller/metal.rb:188:in `dispatch'
actionpack (7.0.5.1) lib/action_controller/metal.rb:251:in `dispatch'
actionpack (7.0.5.1) lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
actionpack (7.0.5.1) lib/action_dispatch/routing/route_set.rb:32:in `serve'
actionpack (7.0.5.1) lib/action_dispatch/routing/mapper.rb:18:in `block in <class:Constraints>'
actionpack (7.0.5.1) lib/action_dispatch/routing/mapper.rb:48:in `serve'
actionpack (7.0.5.1) lib/action_dispatch/journey/router.rb:50:in `block in serve'
actionpack (7.0.5.1) lib/action_dispatch/journey/router.rb:32:in `each'
actionpack (7.0.5.1) lib/action_dispatch/journey/router.rb:32:in `serve'
actionpack (7.0.5.1) lib/action_dispatch/routing/route_set.rb:852:in `call'
lib/middleware/omniauth_bypass_middleware.rb:74:in `call'
rack (2.2.8) lib/rack/tempfile_reaper.rb:15:in `call'
rack (2.2.8) lib/rack/conditional_get.rb:27:in `call'
rack (2.2.8) lib/rack/head.rb:12:in `call'
actionpack (7.0.5.1) lib/action_dispatch/http/permissions_policy.rb:38:in `call'
lib/content_security_policy/middleware.rb:12:in `call'
lib/middleware/anonymous_cache.rb:389:in `call'
lib/middleware/gtm_script_nonce_injector.rb:10:in `call'
config/initializers/008-rack-cors.rb:24:in `call'
rack (2.2.8) lib/rack/session/abstract/id.rb:266:in `context'
rack (2.2.8) lib/rack/session/abstract/id.rb:260:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/cookies.rb:704:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/callbacks.rb:27:in `block in call'
activesupport (7.0.5.1) lib/active_support/callbacks.rb:99:in `run_callbacks'
actionpack (7.0.5.1) lib/action_dispatch/middleware/callbacks.rb:26:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/debug_exceptions.rb:28:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/show_exceptions.rb:26:in `call'
logster (2.12.2) lib/logster/middleware/reporter.rb:43:in `call'
railties (7.0.5.1) lib/rails/rack/logger.rb:40:in `call_app'
railties (7.0.5.1) lib/rails/rack/logger.rb:27:in `call'
config/initializers/100-quiet_logger.rb:20:in `call'
config/initializers/100-silence_logger.rb:29:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/remote_ip.rb:93:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/request_id.rb:26:in `call'
lib/middleware/enforce_hostname.rb:24:in `call'
rack (2.2.8) lib/rack/method_override.rb:24:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/executor.rb:14:in `call'
rack (2.2.8) lib/rack/sendfile.rb:110:in `call'
actionpack (7.0.5.1) lib/action_dispatch/middleware/host_authorization.rb:131:in `call'
rack-mini-profiler (3.1.0) lib/mini_profiler.rb:260:in `call'
message_bus (4.3.7) lib/message_bus/rack/middleware.rb:60:in `call'
lib/middleware/request_tracker.rb:228:in `call'
railties (7.0.5.1) lib/rails/engine.rb:530:in `call'
railties (7.0.5.1) lib/rails/railtie.rb:226:in `public_send'
railties (7.0.5.1) lib/rails/railtie.rb:226:in `method_missing'
rack (2.2.8) lib/rack/urlmap.rb:74:in `block in call'
rack (2.2.8) lib/rack/urlmap.rb:58:in `each'
rack (2.2.8) lib/rack/urlmap.rb:58:in `call'
unicorn (6.1.0) lib/unicorn/http_server.rb:634:in `process_client'
unicorn (6.1.0) lib/unicorn/http_server.rb:739:in `worker_loop'
unicorn (6.1.0) lib/unicorn/http_server.rb:547:in `spawn_missing_workers'
unicorn (6.1.0) lib/unicorn/http_server.rb:143:in `start'
unicorn (6.1.0) bin/unicorn:128:in `<top (required)>'
vendor/bundle/ruby/3.2.0/bin/unicorn:25:in `load'
vendor/bundle/ruby/3.2.0/bin/unicorn:25:in `<main>'

env

hostname	iosre-app
process_id	226
application_version	497d5b4ca04794612509e43674aeba8aabb669e3
HTTP_HOST	iosre.com
REQUEST_URI	/
REQUEST_METHOD	GET
HTTP_USER_AGENT	Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
HTTP_ACCEPT	text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7
HTTP_REFERER	https://iosre.com/
HTTP_X_FORWARDED_FOR	115.171.177.163, 192.168.0.1
HTTP_X_REAL_IP	192.168.0.1
username	snakeninny
time	11:09 pm

Any ideas? Thank you!

2

The same even with safe mode

3

If you disabled discourse chat on your plugin page does it go away ?

What plugins do you have installed

4

I can’t even sign in… I can’t enter the plugin page
A list of installed plugins in app.yml:

- git clone https://github.com/discourse/docker_manager.git
- git clone https://github.com/discourse/discourse-solved.git
- git clone https://github.com/discourse/discourse-reactions.git
- git clone https://github.com/discourse/discourse-spoiler-alert.git
- git clone https://github.com/discourse/discourse-akismet.git
- git clone https://github.com/discourse/discourse-feature-voting
- git clone https://github.com/discourse/discourse-data-explorer.git
- git clone https://github.com/discourse/discourse-apple-auth.git
- git clone https://github.com/discourse/discourse-follow.git
- git clone https://github.com/discourse/discourse-cakeday.git
- git clone https://github.com/discourse/discourse-bcc.git
- git clone https://github.com/paviliondev/discourse-landing-pages.git
5

A list of plugins in /var/www/discourse/plugins:

root@iosre-app:/var/www/discourse/plugins# ls -al
total 96
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:02 .
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 ..
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 chat
drwxr-xr-x 12 discourse root      4096 Jan  7 03:07 discourse-akismet
drwxr-xr-x  7 discourse root      4096 Jan  7 03:07 discourse-apple-auth
drwxr-xr-x  9 discourse root      4096 Jan  7 03:07 discourse-bcc
drwxr-xr-x  9 discourse root      4096 Jan  7 03:07 discourse-cakeday
drwxr-xr-x 11 discourse root      4096 Jan  7 03:07 discourse-data-explorer
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 discourse-details
drwxr-xr-x 11 discourse root      4096 Jan  7 03:07 discourse-feature-voting
drwxr-xr-x 12 discourse root      4096 Jan  7 03:07 discourse-follow
drwxr-xr-x 13 discourse root      4096 Jan  7 03:02 discourse-landing-pages
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 discourse-lazy-videos
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 discourse-local-dates
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 discourse-narrative-bot
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 discourse-presence
drwxr-xr-x 11 discourse root      4096 Jan  7 03:07 discourse-reactions
drwxr-xr-x 10 discourse root      4096 Jan  7 03:07 discourse-solved
drwxr-xr-x  8 discourse root      4096 Jan  7 03:07 discourse-spoiler-alert
drwxr-xr-x 12 discourse root      4096 Jan  7 03:02 docker_manager
drwxr-xr-x  2 discourse root      4096 Jan  7 03:02 plugins
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 poll
drwxr-xr-x  1 discourse discourse 4096 Jan  7 03:01 styleguide
6

Try rebuilding without the pavilion plugin

7

Without paviliondev:

- git clone https://github.com/discourse/docker_manager.git
- git clone https://github.com/discourse/discourse-solved.git
- git clone https://github.com/discourse/discourse-reactions.git
- git clone https://github.com/discourse/discourse-spoiler-alert.git
- git clone https://github.com/discourse/discourse-akismet.git
- git clone https://github.com/discourse/discourse-feature-voting
- git clone https://github.com/discourse/discourse-data-explorer.git
- git clone https://github.com/discourse/discourse-apple-auth.git
- git clone https://github.com/discourse/discourse-follow.git
- git clone https://github.com/discourse/discourse-cakeday.git
- git clone https://github.com/discourse/discourse-bcc.git
# - git clone https://github.com/paviliondev/discourse-landing-pages.git

After rebuilding, error 500 persists.
It’s weird that this time safe mode didn’t save me. In backtrace I’ve noticed chat, can it be a chat bug? How can I disable chat plugin temporarily to test?

8

This time I rebuilt app without most plugins:

- git clone https://github.com/discourse/docker_manager.git
# - git clone https://github.com/discourse/discourse-solved.git
# - git clone https://github.com/discourse/discourse-reactions.git
# - git clone https://github.com/discourse/discourse-spoiler-alert.git
# - git clone https://github.com/discourse/discourse-akismet.git
# - git clone https://github.com/discourse/discourse-feature-voting
# - git clone https://github.com/discourse/discourse-data-explorer.git
# - git clone https://github.com/discourse/discourse-apple-auth.git
# - git clone https://github.com/discourse/discourse-follow.git
# - git clone https://github.com/discourse/discourse-cakeday.git
# - git clone https://github.com/discourse/discourse-bcc.git
# - git clone https://github.com/paviliondev/discourse-landing-pages.git

Still the same

9

Something about this feels like it is chat related … can you try

./mothership enter ...
rails c
SiteSetting.chat_enabled = false
1 Like
11

Thank you! It’s chat…

(base) root@iosre:/var/discourse# ./launcher enter app
x86_64 arch detected.
root@iosre-app:/var/www/discourse# rails c
Plugin name is 'discourse-topic-voting', but plugin directory is named 'discourse-feature-voting'
[1] pry(main)> SiteSetting.chat_enabled = false
=> false

After this, no error 500, I can enter the page

12

we need to figure out what happened in chat that caused this error, there is something about the channel configuration.

at a minimum we can put some protection in channel_fetcher to ensure this will never crash like this.

1 Like
13

Thank you! Anything I can help?

14

nothing yet, the chat team will have a look this week, will let you know!

4 Likes
16

Got the same thing, on stable.

I suspect this was fixed here FIX: Preload the right fields on categories (#24396) · discourse/discourse@0bc568f · GitHub

What version are you running @snakeninny ?

1 Like
17

Ah thanks, I had a suspicion it was related to our categories work but couldn’t prove it. The error is totally happening in category preloading so that would make sense.

However, from the OP:

So most likely it’s after the fix :confused:

1 Like
18

Ah, I missed that.

But that commit is before the fix! It is dated Nov 9 and the fix was made on Nov 15.

19

Oh yes indeed, I think I mixed the links, so yes it’s 99% the problem.

Try to update if you can @snakeninny

Thanks @RGJ :chefs_kiss:

4 Likes
21

:warning: This problem is surfacing when forums are updated from 3.1.3 to 3.1.4.
We’re investigating what is causing this and we will update you as soon as we have more information.

5 Likes
22

Yes, so the problem is that the fix mentioned above is not backported to stable, but stable now relies on it.

@joffreyjaffeux @daniel Can you please urgently backport this to stable? Currently stable is completely broken for any logged in user when chat is enabled.

2 Likes
23

Yes please, I have the same error, my site has crashed as well, I’m still awaiting for communiteq to answer.