Sitemap plugin issue 502 Bad Gateway - in /sitemap.xml page


(Chaudhary) #1

I have a big site, based on discourse. pages more than 0.1 million. 16 gb ram and 4 core CPU in linode, But when i acess www.___________.com/sitemap.xml
gives 502 Bad Gateway nginx page error.


(Vinoth Kannan) #3

Hi, I am unable to reproduce the problem in my local development environment. Can you please post log entries related to this issue? You can find it at discourse.example.com/logs page.

Also did you enabled sitemap plugin from site settings?


(Chaudhary) #4
 /var/www/discourse/plugins/discourse-sitemap/plugin.rb:57:in `default'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/basic_implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:186:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rendering.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:131:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/rescue.rb:20:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications/instrumenter.rb:21:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/notifications.rb:166:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal/params_wrapper.rb:252:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activerecord-5.1.4/lib/active_record/railties/controller_runtime.rb:22:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/abstract_controller/base.rb:124:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionview-5.1.4/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:189:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_controller/metal.rb:253:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:49:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:31:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:17:in `block in <class:Constraints>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/mapper.rb:46:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:50:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/journey/router.rb:33:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/routing/route_set.rb:834:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-protection-2.0.0/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:189:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/strategy.rb:167:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/omniauth-1.6.1/lib/omniauth/builder.rb:63:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/conditional_get.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/head.rb:12:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:147:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:232:in `context'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/session/abstract/id.rb:226:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/cookies.rb:613:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:26:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/activesupport-5.1.4/lib/active_support/callbacks.rb:97:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/callbacks.rb:24:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/debug_exceptions.rb:59:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/show_exceptions.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/logster-1.2.8/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:36:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/rack/logger.rb:26:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:16:in `call'
/var/www/discourse/config/initializers/100-silence_logger.rb:29:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/remote_ip.rb:79:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/request_id.rb:25:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/method_override.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/runtime.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/actionpack-5.1.4/lib/action_dispatch/middleware/executor.rb:12:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/sendfile.rb:111:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-mini-profiler-0.10.7/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/message_bus-2.0.9/lib/message_bus/rack/middleware.rb:63:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:110:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/engine.rb:522:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/railties-5.1.4/lib/rails/railtie.rb:185:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:68:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `each'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/rack-2.0.3/lib/rack/urlmap.rb:53:in `call'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.3.1/lib/unicorn/http_server.rb:606:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.3.1/lib/unicorn/http_server.rb:702:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.3.1/lib/unicorn/http_server.rb:549:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.3.1/lib/unicorn/http_server.rb:142:in `start'
/var/www/discourse/vendor/bundle/ruby/2.4.0/gems/unicorn-5.3.1/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `load'
/var/www/discourse/vendor/bundle/ruby/2.4.0/bin/unicorn:23:in `<main>'
infobacktraceenvSolve  Delete  Protect Share  Debug   Info    Warning    Error    Fatal  
Search

(Vinoth Kannan) #5

Yes. I am sure you didn’t enabled sitemap plugin from settings page. Please navigate to the below URL in your site and enable it.

discourse.example.com/admin/site_settings/category/plugins?filter=sitemap

(Chaudhary) #6

Its already enabled. :frowning: no error in logs, but sitemap still giving error 502 bad gateway. Robots.txt have sitemap link


(Sam Saffron) #7

What is the error here, look at the other tab, my guess is that something timed out!


(Chaudhary) #8

When i set sitemap topics per page to 50000.
url www.-example.com/sitemap.xml works but again www._example.com/sitemap/1.xml gives 502 error


(Sam Saffron) #9

502 is very likely due to the backend timing out… 99% sure that is the case. We need to know where it is timing out.


(Chaudhary) #10

its a big forum. what should i look now ?


(Sam Saffron) #12
  • Try reducing the batch size
  • Try giving postgres more memory
  • Try giving your machine more CPU

I really want to know what query this is timing out on, in case there is an index missing. You can enable slow sql logs in posgtres via the container and paste a few queries that take longer than say 1 second here.


(Chaudhary) #13

Thanks, i will look into it and report where was the problem.