Without me changing anything, my Discourse forum https://lilypond.community suddenly started returning an error page for any request with login cookies: Oops The software powering this discussion forum encountered an unexpected problem. We apologize for the inconvenience. Detailed information about the error was logged, and an automatic notification generated. We’ll take a look at it. No further action is necessary. However, if the error condition persists, you can provide additional detail, including steps to reproduce the error, by posting a discussion topic in the site’s feedback category. The site appears to work fine when you’re not logged in. I looked into /shared/log/rails/production.log (after ./launcher enter app) and it contains errors like this: Pitchfork worker is about to timeout, dumping backtrace for main thread /var/www/discourse/lib/git_utils.rb:33:in 'Kernel#`' /var/www/discourse/lib/git_utils.rb:33:in 'GitUtils.try_git' /var/www/discourse/lib/git_utils.rb:21:in 'GitUtils.has_commit?' /var/www/discourse/lib/discourse_updates.rb:167:in 'block in DiscourseUpdates.new_features' /var/www/discourse/lib/discourse_updates.rb:163:in 'Array#select!' /var/www/discourse/lib/discourse_updates.rb:163:in 'DiscourseUpdates.new_features' /var/www/discourse/lib/discourse_updates.rb:269:in 'DiscourseUpdates.latest_new_feature_created_at' /var/www/discourse/lib/discourse_updates.rb:254:in 'DiscourseUpdates.has_unseen_features?' /var/www/discourse/app/serializers/current_user_serializer.rb:193:in 'CurrentUserSerializer#has_unseen_features' (eval at /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:467):80:in 'CurrentUserSerializer#_fast_attributes' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:456:in 'ActiveModel::Serializer#attributes' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:480:in 'ActiveModel::Serializer#_serializable_hash' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:359:in 'ActiveModel::Serializer#serializable_hash' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:347:in 'ActiveModel::Serializer#as_json' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/json/encoding.rb:41:in 'Oj::Rails::Encoder#encode' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/json/encoding.rb:41:in 'ActiveSupport::JSON.encode' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/core_ext/object/json.rb:42:in 'ActiveSupport::ToJsonWithActiveSupportEncoder#to_json' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/active_model_serializers-0.8.4/lib/active_model/serializer.rb:331:in 'ActiveModel::Serializer#to_json' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/multi_json-1.20.1/lib/multi_json/adapters/oj.rb:67:in 'Oj.dump' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/multi_json-1.20.1/lib/multi_json/adapters/oj.rb:67:in 'MultiJson::Adapters::Oj#dump' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/multi_json-1.20.1/lib/multi_json/adapter.rb:92:in 'MultiJson::Adapter.dump' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/multi_json-1.20.1/lib/multi_json.rb:214:in 'MultiJson.dump' /var/www/discourse/lib/application_layout_preloader.rb:76:in 'ApplicationLayoutPreloader#preload_current_user_data' /var/www/discourse/lib/application_layout_preloader.rb:32:in 'ApplicationLayoutPreloader#preloaded_data' /var/www/discourse/app/helpers/application_helper.rb:1001:in 'ApplicationHelper#preloaded_json' /var/www/discourse/app/views/layouts/application.html.erb:101:in '_app_views_layouts_application_html_erb___243682310287886530_152408' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/base.rb:278:in 'Kernel#public_send' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/base.rb:278:in 'ActionView::Base#_run' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/template.rb:284:in 'block in ActionView::Template#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications.rb:212:in 'ActiveSupport::Notifications.instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/template.rb:589:in 'ActionView::Template#instrument_render_template' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/template.rb:272:in 'ActionView::Template#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-mini-profiler-4.0.1/lib/mini_profiler/profiling_methods.rb:116:in 'block in ActionView::Template#profile_method' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/template_renderer.rb:77:in 'block in ActionView::TemplateRenderer#render_with_layout' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/template_renderer.rb:75:in 'ActionView::TemplateRenderer#render_with_layout' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/template_renderer.rb:59:in 'ActionView::TemplateRenderer#render_template' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/template_renderer.rb:11:in 'ActionView::TemplateRenderer#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/renderer.rb:58:in 'ActionView::Renderer#render_template_to_object' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/renderer/renderer.rb:31:in 'ActionView::Renderer#render_to_object' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/rendering.rb:136:in 'block in ActionView::Rendering#_render_template' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/base.rb:305:in 'ActionView::Base#in_rendering_context' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/rendering.rb:135:in 'ActionView::Rendering#_render_template' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/streaming.rb:179:in 'ActionController::Streaming#_render_template' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/rendering.rb:122:in 'ActionView::Rendering#render_to_body' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/rendering.rb:186:in 'ActionController::Rendering#render_to_body' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/renderers.rb:140:in 'ActionController::Renderers#render_to_body' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/abstract_controller/rendering.rb:28:in 'AbstractController::Rendering#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/rendering.rb:167:in 'ActionController::Rendering#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:31:in 'block (2 levels) in ActionController::Instrumentation#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/benchmark.rb:17:in 'ActiveSupport::Benchmark.realtime' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:31:in 'block in ActionController::Instrumentation#render' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:100:in 'ActionController::Instrumentation#cleanup_view_runtime' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activerecord-8.0.5/lib/active_record/railties/controller_runtime.rb:46:in 'ActiveRecord::Railties::ControllerRuntime#cleanup_view_runtime' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:30:in 'ActionController::Instrumentation#render' /var/www/discourse/lib/topic_list_responder.rb:16:in 'block (2 levels) in TopicListResponder#respond_with_list' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/mime_responds.rb:224:in 'ActionController::MimeResponds#respond_to' /var/www/discourse/lib/topic_list_responder.rb:7:in 'TopicListResponder#respond_with_list' /var/www/discourse/app/controllers/list_controller.rb:117:in 'block (2 levels) in ' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/basic_implicit_render.rb:8:in 'ActionController::BasicImplicitRender#send_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/abstract_controller/base.rb:215:in 'AbstractController::Base#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/rendering.rb:193:in 'ActionController::Rendering#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/abstract_controller/callbacks.rb:261:in 'block in AbstractController::Callbacks#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/callbacks.rb:120:in 'block in ActiveSupport::Callbacks#run_callbacks' /var/www/discourse/app/controllers/application_controller.rb:443:in 'block in ApplicationController#with_resolved_locale' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/i18n-1.14.8/lib/i18n.rb:354:in 'I18n::Base#with_locale' /var/www/discourse/app/controllers/application_controller.rb:443:in 'ApplicationController#with_resolved_locale' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks' /var/www/discourse/app/controllers/application_controller.rb:1125:in 'ApplicationController#ensure_dont_cache_page' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/callbacks.rb:129:in 'block in ActiveSupport::Callbacks#run_callbacks' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/callbacks.rb:140:in 'ActiveSupport::Callbacks#run_callbacks' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/abstract_controller/callbacks.rb:260:in 'AbstractController::Callbacks#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/rescue.rb:27:in 'ActionController::Rescue#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:76:in 'block in ActionController::Instrumentation#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications.rb:210:in 'block in ActiveSupport::Notifications.instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications/instrumenter.rb:58:in 'ActiveSupport::Notifications::Instrumenter#instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/notifications.rb:210:in 'ActiveSupport::Notifications.instrument' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/instrumentation.rb:75:in 'ActionController::Instrumentation#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal/params_wrapper.rb:259:in 'ActionController::ParamsWrapper#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activerecord-8.0.5/lib/active_record/railties/controller_runtime.rb:39:in 'ActiveRecord::Railties::ControllerRuntime#process_action' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/abstract_controller/base.rb:152:in 'AbstractController::Base#process' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionview-8.0.5/lib/action_view/rendering.rb:40:in 'ActionView::Rendering#process' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-mini-profiler-4.0.1/lib/mini_profiler/profiling_methods.rb:116:in 'block in ActionController::Base#profile_method' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal.rb:252:in 'ActionController::Metal#dispatch' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_controller/metal.rb:335:in 'ActionController::Metal.dispatch' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/routing/route_set.rb:67:in 'ActionDispatch::Routing::RouteSet::Dispatcher#dispatch' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/routing/route_set.rb:50:in 'ActionDispatch::Routing::RouteSet::Dispatcher#serve' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/routing/mapper.rb:32:in 'block in ' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/routing/mapper.rb:62:in 'ActionDispatch::Routing::Mapper::Constraints#serve' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/journey/router.rb:53:in 'block in ActionDispatch::Journey::Router#serve' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/journey/router.rb:133:in 'block in ActionDispatch::Journey::Router#find_routes' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/journey/router.rb:126:in 'Array#each' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/journey/router.rb:126:in 'ActionDispatch::Journey::Router#find_routes' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/journey/router.rb:34:in 'ActionDispatch::Journey::Router#serve' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/routing/route_set.rb:908:in 'ActionDispatch::Routing::RouteSet#call' /var/www/discourse/lib/middleware/omniauth_bypass_middleware.rb:35:in 'Middleware::OmniauthBypassMiddleware#call' /var/www/discourse/lib/middleware/crawler_hooks.rb:13:in 'Middleware::CrawlerHooks#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/tempfile_reaper.rb:15:in 'Rack::TempfileReaper#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/conditional_get.rb:27:in 'Rack::ConditionalGet#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/head.rb:12:in 'Rack::Head#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/http/permissions_policy.rb:38:in 'ActionDispatch::PermissionsPolicy::Middleware#call' /var/www/discourse/lib/content_security_policy/middleware.rb:12:in 'ContentSecurityPolicy::Middleware#call' /var/www/discourse/lib/middleware/anonymous_cache.rb:425:in 'Middleware::AnonymousCache#call' /var/www/discourse/lib/middleware/csp_script_nonce_injector.rb:13:in 'Middleware::CspScriptNonceInjector#call' /var/www/discourse/lib/middleware/track_view_session_id_injector.rb:12:in 'Middleware::TrackViewSessionIdInjector#call' /var/www/discourse/config/initializers/008-rack-cors.rb:14:in 'Discourse::Cors#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/session/abstract/id.rb:266:in 'Rack::Session::Abstract::Persisted#context' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/session/abstract/id.rb:260:in 'Rack::Session::Abstract::Persisted#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/cookies.rb:706:in 'ActionDispatch::Cookies#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/callbacks.rb:31:in 'block in ActionDispatch::Callbacks#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/activesupport-8.0.5/lib/active_support/callbacks.rb:100:in 'ActiveSupport::Callbacks#run_callbacks' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/callbacks.rb:30:in 'ActionDispatch::Callbacks#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/debug_exceptions.rb:31:in 'ActionDispatch::DebugExceptions#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/show_exceptions.rb:32:in 'ActionDispatch::ShowExceptions#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/logster-2.21.0/lib/logster/middleware/reporter.rb:40:in 'Logster::Middleware::Reporter#call' /var/www/discourse/lib/middleware/default_headers.rb:13:in 'Middleware::DefaultHeaders#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/railties-8.0.5/lib/rails/rack/logger.rb:41:in 'Rails::Rack::Logger#call_app' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/railties-8.0.5/lib/rails/rack/logger.rb:29:in 'Rails::Rack::Logger#call' /var/www/discourse/config/initializers/100-quiet_logger.rb:20:in 'DiscourseRackQuietAssetsLogger#call' /var/www/discourse/config/initializers/100-silence_logger.rb:29:in 'SilenceLogger#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/request_id.rb:34:in 'ActionDispatch::RequestId#call' /var/www/discourse/lib/middleware/enforce_hostname.rb:23:in 'Middleware::EnforceHostname#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/method_override.rb:24:in 'Rack::MethodOverride#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/executor.rb:16:in 'ActionDispatch::Executor#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/sendfile.rb:127:in 'Rack::Sendfile#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-mini-profiler-4.0.1/lib/mini_profiler.rb:334:in 'Rack::MiniProfiler#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/message_bus-4.5.2/lib/message_bus/rack/middleware.rb:60:in 'MessageBus::Rack::Middleware#call' /var/www/discourse/lib/middleware/request_tracker.rb:380:in 'Middleware::RequestTracker#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/actionpack-8.0.5/lib/action_dispatch/middleware/remote_ip.rb:96:in 'ActionDispatch::RemoteIp#call' /var/www/discourse/lib/middleware/overload_protections.rb:18:in 'Middleware::OverloadProtections#call' /var/www/discourse/lib/middleware/processing_request.rb:14:in 'Middleware::ProcessingRequest#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/railties-8.0.5/lib/rails/engine.rb:535:in 'Rails::Engine#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/railties-8.0.5/lib/rails/railtie.rb:226:in 'Kernel#public_send' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/railties-8.0.5/lib/rails/railtie.rb:226:in 'Rails::Railtie.method_missing' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/urlmap.rb:74:in 'block in Rack::URLMap#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/urlmap.rb:58:in 'Array#each' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/urlmap.rb:58:in 'Rack::URLMap#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/tempfile_reaper.rb:15:in 'Rack::TempfileReaper#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/chunked.rb:105:in 'Pitchfork::Chunked#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/rack-2.2.23/lib/rack/content_length.rb:17:in 'Rack::ContentLength#call' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:868:in 'Pitchfork::HttpServer#process_client' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:1017:in 'Pitchfork::HttpServer#worker_loop' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:616:in 'block in Pitchfork::HttpServer#spawn_worker' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:1222:in 'block in Pitchfork::HttpServer#fork_sibling' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork.rb:170:in 'block in Pitchfork.clean_fork' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork.rb:168:in 'Kernel#catch' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork.rb:168:in 'Pitchfork.clean_fork' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:690:in 'Pitchfork::HttpServer#spawn_initial_mold' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/lib/pitchfork/http_server.rb:176:in 'Pitchfork::HttpServer#start' /var/www/discourse/vendor/bundle/ruby/3.4.0/gems/pitchfork-0.18.2/exe/pitchfork:110:in ' ' /var/www/discourse/vendor/bundle/ruby/3.4.0/bin/pitchfork:25:in 'Kernel#load' /var/www/discourse/vendor/bundle/ruby/3.4.0/bin/pitchfork:25:in ' ' Rendered layout layouts/application.html.erb (Duration: 29736.7ms | GC: 60.9ms) Completed 500 Internal Server Error in 29972ms (ActiveRecord: 0.0ms (0 queries, 0 cached) | GC: 98.5ms) Also, top -c sometimes shows git merge-base processes taking 100% CPU and substantial RAM. What might be going on?

[image] Ed_S: I would like my self-hosted forum to run unattended even if github (and discourse.org) fell into a very deep hole. :100: It is completely unacceptable to be any other way.

[image] Ed_S: I would like my self-hosted forum to run unattended even if And it will — unless an admin says .launcher rebuild app. Or what am I missing now?

@Ed_S I think I found the issue and I have a PR for it, can you check to see if it touches the area you saw with issues? FIX: Stop new-feature checks reaching GitHub during requests (#43143) main ← new-features-no-git-in-requests merged 05:13PM - 03 Sep 26 UTC (UTC) [image] xfalcox +261 -33 Previously, `CurrentUserSerializer#has_unseen_features` could derive the newest …new-feature timestamp inline, which filters the feed through `GitUtils.has_commit?` — and since we deploy from a partial clone (`--filter=tree:0`), `git merge-base --is-ancestor` on a commit missing from the local graph is lazily fetched from the promisor remote, i.e. a blocking, untimed network call to GitHub, once per feed entry, in a request. Compounding it, a fully-filtered feed cached no result at all, so every logged-in page render redid the whole loop until the worker timed out. This change runs that check with `GIT_NO_LAZY_FETCH=1` under a hard timeout so it stays local and bounded, memoizes it per process, and makes `DiscourseUpdates.latest_new_feature_created_at` a plain Redis read derived on write (`update_new_features`, plus a throttled `Jobs::RefreshLatestNewFeature` to self-heal after a deploy or an upcoming change changing status).

[image] Falco: FIX: Stop new-feature checks reaching GitHub during requests - Pull Request #43143 - discourse/discourse - GitHub oh, nice, that does seem to me to have the right shape! (Edit: I read the commentary, not the code!) For interest, when you say per process, is that per pitchfork?

On my forum for example, I became suddenly unable to log in without any admin action on my part. The problem isn’t that you can’t upgrade Discourse when GitHub is unreachable, it’s that GitHub being unreachable takes down your forum because Discourse tries to check for upgrades that you could perform.

[image] Ed_S: For interest, when you say per process, is that per pitchfork? Yes, exactly. I think this was caused by a way way too eager want of telling people about new features, in the current user serializer of all places, paired with a 3 years old change that moved us away from the normal shallow clones: DEV: Create partial clone of discourse repository (#691) committed 09:58AM - 13 Mar 23 UTC (UTC) [image] gschlager +3 -6 A treeless clone is nearly as small as a shallow clone, but it contains the full… history which makes it possible to easily checkout tags and commits and allows all features of docker_manager to work correctly. See https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/ for more information on treeless clones. Also this change makes it possible to correctly pull the main branch which didn't work before. @Ed_S and @jeanas you both admins on the affected instances right? This should only affect admins while GitHub is down.

[image] Falco: @Ed_S and @jeanas you both admins on the affected instances right? This should only affect admins while GitHub is down. Yes, exactly, I could log in with another account but not with my admin account.

Cool I have merged the PR. Update when it hits the release channel you run, then wait a couple of days for GitHub to go down again and let me know how it goes during it.

great, thanks. Just to confirm, I’m an admin, I have two mods, and one of those was affected. So, either it affects all staff, or it affects all logged-in users. I was OK with incognito access. And in my case, I could log in some of the time with much delay - I wasn’t entirely locked out.

Awesome, thanks a lot for the quick fix!

Error for logged in users and Git hang

Product Support

agemo (agemo) September 24, 2026, 9:11pm 34

Ok, this instance is US based DO droplet.

Topic		Replies	Views
The Git Hang and the Admin/Mod user Accs Oops Error Page Error Self-hosting login , bug	28	754	October 5, 2026
Fatal: could not read Username for 'https://github.com' Self-hosting	10	458	September 3, 2026
Twitter, Github and normal username and password logins not working after upgrade Self-hosting unsupported-install	34	7952	October 22, 2016
40 Fatals and 7 Errors in recent logs Product Support	12	1994	February 17, 2017
Discourse now stops loading at login dialog when viewed with any browser Self-hosting	21	1587	January 29, 2023

Error for logged in users and Git hang

Related topics