Straightforward direct-delivery incoming mail


(Jay Pfaffman) #29

I’d be happy to get the multisite documentation straightened out, & could probably figure out the issue, but I’ve been unable to figure out multsitie myself.


(Wes Osborn) #30

As noted in the title, this process is for handling inbound email instead of using a POP account.

You can send through a local SMTP server rather than SES, but be warned:

A thread to get you started:

You also run the risk of breaking your current working installation, so probably something worth trying to setup on a dev system first.


(Matt Palmer) #31

There’s more than just documentation that needs updating for multisite incoming e-mail; there’s some extensions that need to be made to the base mail-receiver image to have it lookup domains and delivery addresses.


(Jay Pfaffman) #32

I know, but I think that if I got a handle on the other multi site stuff that I might be able to figure out the email stuff.


#33

Hi all and thanks @mpalmer for this super helpful post! I just got this all set-up on my instance, and it’s working perfectly!

One question – about the language setting in the mail-receiver.yml file – what does this control? I am hosting a forum that is in Portuguese and I am wondering if I need to switch this from en-US to pt-BR?


(Matt Palmer) #34

The LANG env var is the default locale for the container itself. I would be very surprised if it would need to be changed for correct operation, since the mail container doesn’t generate anything user-visible anyway – everything gets generated by the app itself.


(Erik Mueller-Harder) #35

This is terrific functionality, @mpalmer — thank you so much!

One difficulty I had was related to redirecting archived .eml files (for a migration-to-Discourse project). Some of the .eml files were mal-formed, in that they began with a “From” header, à la an .mbox file.

This caused errors, of course, and a few hours of troubleshooting. (Why so long? My brain mis-parsed the error message “Error_parsing_from_header” as mail-receiver telling me that it was having a problem parsing the data from an (unspecified) header — not an official (or indeed an .mbox-style delimiter) “From:” header!)

Anyway, it occurred to me that this might be pretty easily worked around on the mail-receiver end so future migrations for others might go more smoothly…

Thanks again for this great plug-n-play container!


(Erik Mueller-Harder) #36

I’ve just found another similar issue.

I’ve been importing messages, and one of my .eml files has no date header. This is illegal, but again in the spirit of “accept liberally,” it would be great if mail-receiver could give a more helpful error message.

With no date header, mail-receiver rejects the message with the error:

 undefined method `>' for nil:NilClass

Took me a while to spot the trouble, but adding a fake but reasonably accurate Date header:

 Date: Wed, 14 Jan 2015 16:43:27 -0500

worked around the problem nicely.


(Matt Palmer) #37

Was it mail-receiver itself, or was it passing back an exception the application threw? We don’t do any e-mail parsing in mail-receiver, just shovel it along. If you can give the complete backtrace, I can see about getting a more informative error message put in, but given that we use the Date header to determine the effective posting date of the message, we can’t do without having one entirely.


(Erik Mueller-Harder) #38

Oh, I do know that it’s not processable, @mpalmer! (Well, I suppose you could choose a random “Receive:” header if you really had to, but really the onus should be on the admin to get the data right.) I was just suggesting a more helpful error message.

Again, I’m really grateful that this exists and that it works so very well; I really had no other way to import my particular set of data. (The group started as a Google Group, migrated to Posterous, migrated to Minigroup, and has been hobbling along with blogging software for the past year!)

Thanks!



First, I get this:

“Email can not be processed: undefined method `>’ for nil:NilClass”

Here’s the (slightly redacted) envelope:

hostname	scw-c85ec4-app
process_id	142
application_version	495a511862b859a98516fae90179b15598fff110
HTTP_HOST	sub.example.com
REQUEST_URI	/admin/email/handle_mail?api_key=blahblahblah&api_username=system
REQUEST_METHOD	POST
HTTP_USER_AGENT	Ruby
HTTP_ACCEPT	*/*
HTTP_X_FORWARDED_FOR	1.23.45.6
HTTP_X_REAL_IP	1.23.45.6
username	system
params	
api_key	bff79b0269b4cb63f9d1a80aa8378291ca9016a791a52eb950e779507b449859
api_username	system
email	Received: from out2-smtp.example2.com (out2-smtp.example2.com [66.111.4.26]) by scw-c8

and the backtrace:

/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/logger.rb:76:in `add_with_opts'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/logger.rb:37:in `add'
/usr/local/lib/ruby/2.3.0/logger.rb:498:in `warn'
/var/www/discourse/lib/email/processor.rb:107:in `log_email_process_failure'
/var/www/discourse/lib/email/processor.rb:25:in `rescue in process!'
/var/www/discourse/lib/email/processor.rb:15:in `process!'
/var/www/discourse/lib/email/processor.rb:11:in `process!'
/var/www/discourse/app/controllers/admin/email_controller.rb:74:in `handle_mail'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/base.rb:198:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:117:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:505:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rescue.rb:29:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications.rb:164:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/base.rb:137:in `process'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionview-4.2.7.1/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal.rb:196:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal.rb:237:in `block in action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:43:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:43:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:30:in `each'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:30:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:817:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/builder.rb:63:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/conditionalget.rb:38:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/head.rb:13:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/flash.rb:260:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/query_cache.rb:36:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/rack/logger.rb:22:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:13:in `call_with_quiet_assets'
/var/www/discourse/config/initializers/100-silence_logger.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/methodoverride.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/runtime.rb:18:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/sendfile.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/message_bus-2.0.2/lib/message_bus/rack/middleware.rb:62:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:73:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/engine.rb:518:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/application.rb:165:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/railtie.rb:194:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/railtie.rb:194:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:66:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:562:in `process_client'
/var/www/discourse/lib/scheduler/defer.rb:85:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:658:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:132:in `start'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `load'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `<main>'

Then I get this:

“Email can not be processed: undefined method `>’ for nil:NilClass”

Here’s the (slightly redacted) envelope:

hostname	scw-c85ec4-app
process_id	142
application_version	495a511862b859a98516fae90179b15598fff110
HTTP_HOST	sub.example.com
REQUEST_URI	/admin/email/handle_mail?api_key=blahblahblah&api_username=system
REQUEST_METHOD	POST
HTTP_USER_AGENT	Ruby
HTTP_ACCEPT	*/*
HTTP_X_FORWARDED_FOR	1.23.45.67
HTTP_X_REAL_IP	1.23.45.67
username	system
params	
api_key	bff79b0269b4cb63f9d1a80aa8378291ca9016a791a52eb950e779507b449859
api_username	system
email	Received: from out2-smtp.example2.com (out2-smtp.example2.com [66.111.4.26]) by scw-c8
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/logger.rb:76:in `add_with_opts'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/logger.rb:37:in `add'
/usr/local/lib/ruby/2.3.0/logger.rb:507:in `error'
/var/www/discourse/lib/email/processor.rb:84:in `handle_failure'
/var/www/discourse/lib/email/processor.rb:27:in `rescue in process!'
/var/www/discourse/lib/email/processor.rb:15:in `process!'
/var/www/discourse/lib/email/processor.rb:11:in `process!'
/var/www/discourse/app/controllers/admin/email_controller.rb:74:in `handle_mail'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/base.rb:198:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rendering.rb:10:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:117:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:555:in `block (2 levels) in compile'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:505:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:92:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:778:in `_run_process_action_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/callbacks.rb:19:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rescue.rb:29:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/instrumentation.rb:32:in `block in process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications.rb:164:in `block in instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/notifications.rb:164:in `instrument'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/abstract_controller/base.rb:137:in `process'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionview-4.2.7.1/lib/action_view/rendering.rb:30:in `process'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiling_methods.rb:76:in `block in profile_method'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal.rb:196:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_controller/metal.rb:237:in `block in action'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:74:in `dispatch'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:43:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/mapper.rb:49:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:43:in `block in serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:30:in `each'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/journey/router.rb:30:in `serve'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/routing/route_set.rb:817:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-protection-1.5.3/lib/rack/protection/frame_options.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:186:in `call!'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/strategy.rb:164:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/omniauth-1.3.1/lib/omniauth/builder.rb:63:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/conditionalget.rb:38:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/head.rb:13:in `call'
/var/www/discourse/lib/middleware/anonymous_cache.rb:138:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/flash.rb:260:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:225:in `context'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/session/abstract/id.rb:220:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/query_cache.rb:36:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/abstract/connection_pool.rb:653:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:88:in `__run_callbacks__'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:778:in `_run_call_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/activesupport-4.2.7.1/lib/active_support/callbacks.rb:81:in `run_callbacks'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/logster-1.2.5/lib/logster/middleware/reporter.rb:31:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/rack/logger.rb:38:in `call_app'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/rack/logger.rb:22:in `call'
/var/www/discourse/config/initializers/100-quiet_logger.rb:13:in `call_with_quiet_assets'
/var/www/discourse/config/initializers/100-silence_logger.rb:26:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/actionpack-4.2.7.1/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/methodoverride.rb:22:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/runtime.rb:18:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/sendfile.rb:113:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-mini-profiler-0.10.1/lib/mini_profiler/profiler.rb:171:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/message_bus-2.0.2/lib/message_bus/rack/middleware.rb:62:in `call'
/var/www/discourse/lib/middleware/request_tracker.rb:73:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/engine.rb:518:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/application.rb:165:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/railtie.rb:194:in `public_send'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/railties-4.2.7.1/lib/rails/railtie.rb:194:in `method_missing'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:66:in `block in call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `each'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/rack-1.6.5/lib/rack/urlmap.rb:50:in `call'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:562:in `process_client'
/var/www/discourse/lib/scheduler/defer.rb:85:in `process_client'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:658:in `worker_loop'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:508:in `spawn_missing_workers'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/lib/unicorn/http_server.rb:132:in `start'
/var/www/discourse/vendor/bundle/ruby/2.3.0/gems/unicorn-5.2.0/bin/unicorn:126:in `<top (required)>'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `load'
/var/www/discourse/vendor/bundle/ruby/2.3.0/bin/unicorn:22:in `<main>'


(Matt Palmer) #39

I’ve just merged this PR, which should provide a more helpful error message in the future:


(Erik Mueller-Harder) #40

Wow! Quick work!

Just checking, as I’m pretty new to this — I should just pretty much follow some of the installation instructions again to pick this up? Like this?

  1. ssh in and sudo to root
  2. cd to /var/discourse
  3. git pull
  4. bootstrap and start mail-receiver via ./launcher

Is that about right, or am I missing anything?

Thanks!


(Matt Palmer) #41

This is a change to Discourse itself, not mail-receiver, so you’ll just need to do a rebuild of the app to get the changes.


(Christoph) #42

I am very tempted to switch from pop3 polling to this but I’m wondering: are there any downsides of this method? Is this the recommended method now?

Also, has anyone transitioned from pop3 polling to this on a live instance? What would be the steps to ensure a smooth transition?


(Joshua Rosenfeld) #43

I did during on a live site. However, this is an internal site, so the “steps to ensure a smooth transition” were sending a full company email saying that in 5 days at this time email-in to the forums will be unreliable while we complete an update. We sent another email when we were starting the conversion, and one upon completion. It was awhile ago, and if memory serves me correctly it took less than an hour.


(Jay Pfaffman) #44

How hard would it be to make this work on a host that received email for a number of Discourse sites?


(Matt Palmer) #45

It should be smooth as silk. You can have both POP3 polling and direct delivery active at once, so the sequence would be something like:

  1. Setup direct delivery.
  2. Change the forwarding rules (or MX config, or whatever) to deliver to the direct delivery MTA rather than the POP3 mailbox.
  3. PROFIT!
  4. After a while (once you’re pretty sure no more mail’s going to land in the POP3 mailbox, due to cached MX records or whatever), nuke the POP3 polling config.

You’ve just got to do something to map incoming envelope-to addresses to site hostnames, and then modify mail-receiver to set the Host: header on connection (if it’s a multisite container), or connect to a different host/port (if separate containers). Since that is exactly how we do it here at Discourse (we lookup the envelope-to address in our master hosting database, and then connect to the appropriate site), mail-receiver is structured to make that pretty easy – just build a new container with an alternate entrypoint script that includes the default script then overrides post_email. If you need assistance, I’m happy to give specific pointers and share bits and pieces of how we do things privately.


(Wes Osborn) #46

My Discourse site requires TLS. We’re using a publicly signed cert from Digicert, but when I attempt to receive mail via mail-receiver, I’m seeing the following in the logs:

<19>May 23 02:50:01 receive-mail[86]: Failed to POST the e-mail to https://discourse.clcohio.org/admin/email/handle_mail: SSL_connect returned=1 errno=0 state=error: certificate verify failed (OpenSSL::SSL::SSLError)
<19>May 23 02:50:01 receive-mail[86]:   /usr/local/lib/ruby/2.3.0/net/http.rb:933:in `connect_nonblock'
  /usr/local/lib/ruby/2.3.0/net/http.rb:933:in `connect'
  /usr/local/lib/ruby/2.3.0/net/http.rb:863:in `do_start'
  /usr/local/lib/ruby/2.3.0/net/http.rb:852:in `start'
  /usr/local/lib/ruby/2.3.0/net/http.rb:1398:in `request'
  /usr/local/bin/receive-mail:66:in `post_email'
  /usr/local/bin/receive-mail:39:in `main'
  /usr/local/bin/receive-mail:81:in `<main>'
<20>May 23 02:50:01 postfix/pipe[85]: warning: write/read private/defer socket: Permission denied

(Matt Palmer) #47

It doesn’t look like you’re sending a chain to a trust anchor:

$ openssl s_client -connect discourse.clcohio.org:443 -servername discourse.clcohio.org
CONNECTED(00000003)
depth=0 C = US, ST = Ohio, L = Columbus, O = Central Library Consortium, CN = *.clcohio.org
verify error:num=20:unable to get local issuer certificate
verify return:1
depth=0 C = US, ST = Ohio, L = Columbus, O = Central Library Consortium, CN = *.clcohio.org
verify error:num=21:unable to verify the first certificate
verify return:1
---
Certificate chain
 0 s:/C=US/ST=Ohio/L=Columbus/O=Central Library Consortium/CN=*.clcohio.org
   i:/C=US/O=DigiCert Inc/OU=www.digicert.com/CN=DigiCert SHA2 High Assurance Server CA

While some browsers will do chain construction and AIA fetching, it’s far from universal, and it certainly isn’t done by OpenSSL (which is what underlies the HTTP library used by mail-receiver). Configure your web server to present a complete chain to a trust anchor, and I suspect things will go much better for you.


(Wes Osborn) #48

Is that the way things would be setup if we used these instructions?

We followed those, but it has been a long time, so may be we need to revisit the configuration that we used.