Hey all, I’m a volunteer admin with this forum and they have been seeing a lot of 429 errors, dropped emails, and other issues. I noticed sidekiq needs to have memory increased beyond 500mb and it also seems that the email server has run out of space in queue. Unsure of what other issues there are since I do not have access to the underlying server, but hoping others can offer suggestions. Thanks!
Discourse::InvalidParameters (string contains null byte) app/controllers/application_controller.rb:134:in `block in <class:ApplicationController>' lib/middleware/omniauth_bypass_middleware.rb:74:in `c
Message (919 copies reported)
Discourse::InvalidParameters (string contains null byte)
app/controllers/application_controller.rb:134:in block in <class:ApplicationController>' lib/middleware/omniauth_bypass_middleware.rb:74:in call’
lib/content_security_policy/middleware.rb:12:in call' lib/middleware/anonymous_cache.rb:387:in call’
lib/middleware/gtm_script_nonce_injector.rb:10:in call' config/initializers/100-quiet_logger.rb:20:in call’
config/initializers/100-silence_logger.rb:29:in call' lib/middleware/enforce_hostname.rb:24:in call’
lib/middleware/request_tracker.rb:233:in `call’
Backtrace
app/controllers/application_controller.rb:134:in block in <class:ApplicationController>' activesupport (7.0.7) lib/active_support/rescuable.rb:116:in instance_exec’
activesupport (7.0.7) lib/active_support/rescuable.rb:116:in block in handler_for_rescue' activesupport (7.0.7) lib/active_support/rescuable.rb:92:in rescue_with_handler’
activesupport (7.0.7) lib/active_support/rescuable.rb:165:in rescue_with_handler' actionpack (7.0.7) lib/action_controller/metal/rescue.rb:26:in rescue in process_action’
actionpack (7.0.7) lib/action_controller/metal/rescue.rb:22:in process_action' actionpack (7.0.7) lib/action_controller/metal/instrumentation.rb:67:in block in process_action’
activesupport (7.0.7) lib/active_support/notifications.rb:206:in block in instrument' activesupport (7.0.7) lib/active_support/notifications/instrumenter.rb:24:in instrument’
Job exception: execution expired
Message (1704 copies reported)
Job exception: execution expired
Backtrace
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:631:in initialize' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:631:in open’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:631:in tcp_socket' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:641:in block in do_start’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/timeout-0.4.0/lib/timeout.rb:186:in block in timeout' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/timeout-0.4.0/lib/timeout.rb:193:in timeout’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:640:in do_start' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-smtp-0.4.0/lib/net/smtp.rb:610:in start’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp.rb:109:in start_smtp_session' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/mail-2.8.1/lib/mail/network/delivery_methods/smtp.rb:100:in deliver!’
Job exception: Net::ReadTimeout
Message (75565 copies reported)
Job exception: Net::ReadTimeout
Backtrace
/usr/local/lib/ruby/3.2.0/net/protocol.rb:229:in rbuf_fill' /usr/local/lib/ruby/3.2.0/net/protocol.rb:199:in readuntil’
/usr/local/lib/ruby/3.2.0/net/protocol.rb:377:in each_message_chunk' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:958:in block in retr’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:1016:in critical' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:956:in retr’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:810:in pop' /var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:47:in block (2 levels) in poll_pop3’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:669:in each' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:669:in each_mail’
Job exception: X-MS-Exchange-CrossTenant-FromEntityHeader: Internet
Job exception: X-MS-Exchange-CrossTenant-FromEntityHeader: Internet
Backtrace
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:1004:in check_response' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:984:in quit’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:603:in do_finish' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in ensure in start’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in start' /var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:45:in poll_pop3’
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:14:in execute' /var/www/discourse/app/jobs/base.rb:292:in block (2 levels) in perform’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:82:in with_connection' /var/www/discourse/app/jobs/base.rb:279:in block in perform’
Job exception: Received-SPF: None (no SPF record) ...<UNKNOWN>
Message (4 copies reported)
Job exception: Received-SPF: None (no SPF record) identity=helo; client-ip=…; envelope-from=<>; receiver=
Backtrace
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:1004:in check_response' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:984:in quit’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:603:in do_finish' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in ensure in start’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in start' /var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:45:in poll_pop3’
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:14:in execute' /var/www/discourse/app/jobs/base.rb:292:in block (2 levels) in perform’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:82:in with_connection' /var/www/discourse/app/jobs/base.rb:279:in block in perform’
Job exception: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use
Message (92 copies reported)
Job exception: could not obtain a connection from the pool within 5.000 seconds (waited 5.000 seconds); all pooled connections were in use
Backtrace
Job exception: by mx with LMTP
Message
Job exception: by mx with LMTP
Backtrace
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:1004:in check_response' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:984:in quit’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:603:in do_finish' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in ensure in start’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/net-pop-0.1.2/lib/net/pop.rb:533:in start' /var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:45:in poll_pop3’
/var/www/discourse/app/jobs/scheduled/poll_mailbox.rb:14:in execute' /var/www/discourse/app/jobs/base.rb:292:in block (2 levels) in perform’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rails_multisite-5.0.0/lib/rails_multisite/connection_management.rb:82:in with_connection' /var/www/discourse/app/jobs/base.rb:279:in block in perform’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:126:in block in wait_poll' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:115:in loop’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:115:in wait_poll' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:76:in internal_poll’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:202:in internal_poll' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:71:in block in poll’
/usr/local/lib/ruby/3.2.0/monitor.rb:202:in synchronize' /usr/local/lib/ruby/3.2.0/monitor.rb:202:in mon_synchronize’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:80:in synchronize' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/activerecord-7.0.7/lib/active_record/connection_adapters/abstract/connection_pool/queue.rb:71:in poll’
Failed to send push notification : Timeout::Error : Timeout::Error
Message (52 copies reported)
Failed to send push notification : Timeout::Error : Timeout::Error
Backtrace
/var/www/discourse/lib/final_destination/resolver.rb:31:in block in lookup' /var/www/discourse/lib/final_destination/resolver.rb:8:in synchronize’
/var/www/discourse/lib/final_destination/resolver.rb:8:in lookup' /var/www/discourse/lib/final_destination/ssrf_detector.rb:127:in lookup_ips’
/var/www/discourse/lib/final_destination/ssrf_detector.rb:95:in lookup_and_filter_ips' /var/www/discourse/lib/final_destination/http.rb:13:in connect’
/usr/local/lib/ruby/3.2.0/net/http.rb:1248:in do_start' /usr/local/lib/ruby/3.2.0/net/http.rb:1237:in start’
/usr/local/lib/ruby/3.2.0/net/http.rb:1817:in request' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/rack-mini-profiler-3.1.1/lib/patches/net_patches.rb:19:in block in request_with_mini_profiler’
NoMethodError (undefined method `downcase' for #<ActionController::Parameters {"__proto__"=>{"expect"=>"100-continue"}, "constructor"=>{"prototype"=>{"expect"=>"100-continue"}}} permitted: false>) app
Message (65 copies reported)
Failed to handle exception in exception app middleware : ActionController::UnfilteredParameters : unable to convert unpermitted parameters to hash
Message (167 copies reported)
Failed to handle exception in exception app middleware : ActionController::UnfilteredParameters : unable to convert unpermitted parameters to hash
Backtrace
actionpack-7.0.7/lib/action_controller/metal/strong_parameters.rb:289:in to_h' actionpack-7.0.7/lib/action_controller/metal/strong_parameters.rb:336:in to_query’
actionpack-7.0.7/lib/action_dispatch/http/url.rb:85:in block in add_params' actionpack-7.0.7/lib/action_dispatch/http/url.rb:85:in reject!’
actionpack-7.0.7/lib/action_dispatch/http/url.rb:85:in add_params' actionpack-7.0.7/lib/action_dispatch/http/url.rb:76:in path_for’
actionpack-7.0.7/lib/action_dispatch/routing/route_set.rb:333:in block in <class:RouteSet>' actionpack-7.0.7/lib/action_dispatch/routing/route_set.rb:846:in url_for’
actionpack-7.0.7/lib/action_dispatch/routing/route_set.rb:271:in call' actionpack-7.0.7/lib/action_dispatch/routing/route_set.rb:327:in block in define_url_helper’
NoMethodError (undefined method downcase' for #<ActionController::Parameters {"__proto__"=>{"expect"=>"100-continue"}, "constructor"=>{"prototype"=>{"expect"=>"100-continue"}}} permitted: false>) app/models/tag.rb:26:in map’
app/models/tag.rb:26:in block in <class:Tag>' lib/topic_query.rb:1241:in filter_by_tags’
lib/topic_query.rb:768:in default_results' lib/topic_query.rb:538:in latest_results’
lib/topic_query.rb:281:in list_latest' app/controllers/tags_controller.rb:159:in public_send’
app/controllers/tags_controller.rb:159:in block (2 levels) in <class:TagsController>' app/controllers/tags_controller.rb:181:in show’
app/controllers/application_controller.rb:421:in block in with_resolved_locale' app/controllers/application_controller.rb:421:in with_resolved_locale’
lib/middleware/omniauth_bypass_middleware.rb:74:in call' lib/content_security_policy/middleware.rb:12:in call’
lib/middleware/anonymous_cache.rb:387:in call' lib/middleware/gtm_script_nonce_injector.rb:10:in call’
config/initializers/100-quiet_logger.rb:20:in call' config/initializers/100-silence_logger.rb:29:in call’
lib/middleware/enforce_hostname.rb:24:in call' lib/middleware/request_tracker.rb:233:in call’
Backtrace
app/models/tag.rb:26:in map' app/models/tag.rb:26:in block in class:Tag’
activerecord (7.0.7) lib/active_record/relation.rb:435:in instance_exec' activerecord (7.0.7) lib/active_record/relation.rb:435:in block in _exec_scope’
activerecord (7.0.7) lib/active_record/relation.rb:881:in _scoping' activerecord (7.0.7) lib/active_record/relation.rb:435:in _exec_scope’
activerecord (7.0.7) lib/active_record/scoping/named.rb:175:in block in scope' lib/topic_query.rb:1241:in filter_by_tags’
lib/topic_query.rb:768:in default_results' lib/topic_query.rb:538:in latest_results’
Sidekiq is consuming too much memory (using: 504.11M)
Message (4012 copies reported)
Sidekiq is consuming too much memory (using: 504.11M) for ‘discuss.domain.com’, restarting
Backtrace
config/unicorn.conf.rb:164:in check_sidekiq_heartbeat' config/unicorn.conf.rb:244:in master_sleep’
/var/www/discourse/vendor/bundle/ruby/3.2.0/gems/unicorn-6.1.0/lib/unicorn/http_server.rb:295:in join' /var/www/discourse/vendor/bundle/ruby/3.2.0/gems/unicorn-6.1.0/bin/unicorn:128:in <top (required)>’
/var/www/discourse/vendor/bundle/ruby/3.2.0/bin/unicorn:25:in load' /var/www/discourse/vendor/bundle/ruby/3.2.0/bin/unicorn:25:in ’