他のウェブサイトとのディスコース、SMTPの問題:End of file reached

(to staff: this is unrelated to the hosted subscription that my account is also associated with)

I installed Discourse on a VPS using instructions from Run other websites on the same machine as Discourse but with this difference: I adapted the app.yml before the first install:

sudo mkdir /var/discourse
sudo git clone https://github.com/discourse/discourse_docker.git /var/discourse
cd /var/discourse
sudo cp samples/standalone.yml containers/app.yml
sudo nano containers/app.yml

In the YAML I commented out ports section, added - "templates/web.socketed.template.yml" and set hostname + SMTP:

  ## TODO: The domain name this Discourse instance will respond to
  ## Required. Discourse will not work with a bare IP number.
  DISCOURSE_HOSTNAME: 'discuss.mydomain.community'

  ## TODO: List of comma delimited emails that will be made admin and developer
  ## on initial signup example 'user1@example.com,user2@example.com'
  DISCOURSE_DEVELOPER_EMAILS: 'someuser@protonmail.com,anotheruser@otherdomain.io'

  ## TODO: The SMTP mail server used to validate new accounts and send notifications
  # SMTP ADDRESS, username, and password are required
  # WARNING the char '#' in SMTP password can cause problems!
  DISCOURSE_SMTP_ADDRESS: smtp.myprovider.email
  DISCOURSE_SMTP_PORT: 465
  DISCOURSE_SMTP_USER_NAME: mydomain-no-reply@otherdomain.io
  DISCOURSE_SMTP_PASSWORD: pa$$word
  #DISCOURSE_SMTP_ENABLE_START_TLS: true           # (optional, default true)

Then I bootstrapped and launched Discourse app:

sudo ./launcher bootstrap app
sudo ./launcher start app

That worked. I stopped discourse and configured the Outer nginx (at /etc/nginx/conf.d/discourse.conf) with the suggested http-only config (just for testing). I could access http://discuss.mydomain.community.

I stopped discourse once again and with certbot I upgraded the discourse.conf and now have:

server {
	server_name discuss.mydomain.community;  # <-- change this

	location / {
		proxy_pass http://unix:/var/discourse/shared/standalone/nginx.http.sock:;
		proxy_set_header Host $http_host;
		proxy_http_version 1.1;
		proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
		proxy_set_header X-Forwarded-Proto $scheme;
		proxy_set_header X-Real-IP $remote_addr;
	}

    listen [::]:443 ssl ipv6only=on; # managed by Certbot
    listen 443 ssl; # managed by Certbot
    ssl_certificate /etc/letsencrypt/live/discuss.mydomain.community/fullchain.pem; # managed by Certbot
    ssl_certificate_key /etc/letsencrypt/live/discuss.mydomain.community/privkey.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot

}
server {
    if ($host = discuss.mydomain.community) {
        return 301 https://$host$request_uri;
    } # managed by Certbot


	listen 80; listen [::]:80;
	server_name discuss.mydomain.community;
    return 404; # managed by Certbot
}

So far, so good. I can access Discourse over https (I later did a ./launcher rebuild app when things didn’t work, but that made no difference).

In the ‘Finish installation’ screen after submitting the registration form I get " We sent an activation mail to someuser@protonmail.com" .

Problem: no mail arrives.

No email arrives in the admin’s email inbox. Checking on mydomain-no-reply@otherdomain.io mailbox, and nothing to see either.

Note: The whole problem might be caused because the mail user is on otherdomain.io and must reside on mydomain.community. But I as far as I know this is no requirement.

Started troubleshooting and subsequently tested with following app.yml changes (with a rebuild app in between):

  DISCOURSE_SMTP_ENABLE_START_TLS: false

No success. Continuing:

## If you want to set the 'From' email address for your first registration, uncomment and change:
  - exec: rails r "SiteSetting.notification_email='mydomain-no-reply@otherdomain.io"

So I proceeded with Troubleshoot email on a new Discourse install :

  • otherdomain.io has DKIM and SPF properly set in DNS
  • Using telnet from the VPS to the SMTP provider works
  • Using telnet from the Docker container (with docker exec and telnet install) works too

I ran ./discourse-doctor and there were 2 parts that jumped out (rest of the results were as expected):

========================================
Discourse version at discuss.mydomain.community: Discourse 2.6.0.beta2 
Discourse version at localhost: NOT FOUND
==================== DNS PROBLEM ====================
This server reports NOT FOUND, but discuss.mydomain.community reports Discourse 2.6.0.beta2 .
This suggests that you have a DNS problem or that an intermediate proxy is to blame.
If you are using Cloudflare, or a CDN, it may be improperly configured.
==================== MAIL TEST ====================
For a robust test, get an address from http://www.mail-tester.com/
Sending mail to REDACTED  . . 
Testing sending to test-k86jiyqb9@srv1.mail-tester.com using smtp.myprovider.email:465.
======================================== ERROR ========================================
                                    UNEXPECTED ERROR

end of file reached

====================================== SOLUTION =======================================
This is not a common error. No recommended solution exists!

Please report the exact error message above to https://meta.discourse.org/
(And a solution, if you find one!)

I tried twice. Once with the configured email, once with a mail-tester.com provided one. Same results. So that does not look good.

Note: Apparently the docker install procedure also serves beta versions. Good to know.

Looking at the production log it has:

Started GET "/" for REDACTED-IP at 2020-09-03 06:21:57 +0000
Processing by FinishInstallationController#index as HTML
  Rendering finish_installation/index.html.erb within layouts/finish_installation
  Rendered finish_installation/index.html.erb within layouts/finish_installation (Duration: 3.2ms | Allocations: 356)
  Rendered layouts/_head.html.erb (Duration: 15.7ms | Allocations: 2969)
Completed 200 OK in 323ms (Views: 140.3ms | ActiveRecord: 0.0ms | Allocations: 32137)
Started GET "/finish-installation/register" for REDACTED-IP at 2020-09-03 06:22:01 +0000
Processing by FinishInstallationController#register as HTML
  Rendering finish_installation/register.html.erb within layouts/finish_installation
  Rendered finish_installation/register.html.erb within layouts/finish_installation (Duration: 7.1ms | Allocations: 1607)
  Rendered layouts/_head.html.erb (Duration: 22.3ms | Allocations: 3139)
Completed 200 OK in 107ms (Views: 41.3ms | ActiveRecord: 0.0ms | Allocations: 11760)
Started POST "/finish-installation/register" for REDACTED-IP at 2020-09-03 06:22:22 +0000
Processing by FinishInstallationController#register as HTML
  Parameters: {"utf8"=>"✓", "authenticity_token"=>"l1hEEsK/gur5yplJdMIHttZAgYcuzLLkESaAI87IMb88nAFqNwi7l3yJ+3EJBw7leFypVGbH4C5hJl7VnJVYBQ==", "email"=>"someuser@protonmail.com", "username"=>"someusername", "password"=>"[FILTERED]", "commit"=>"Register"}
Redirected to https://discuss.mydomain.community/finish-installation/confirm-email
Completed 302 Found in 48ms (ActiveRecord: 0.0ms | Allocations: 4246)
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
Started GET "/finish-installation/confirm-email" for REDACTED-IP at 2020-09-03 06:22:22 +0000
Processing by FinishInstallationController#confirm_email as HTML
  Rendering finish_installation/confirm_email.html.erb within layouts/finish_installation
  Rendered finish_installation/confirm_email.html.erb within layouts/finish_installation (Duration: 2.3ms | Allocations: 418)
  Rendered layouts/_head.html.erb (Duration: 7.9ms | Allocations: 1612)
Completed 200 OK in 25ms (Views: 22.1ms | ActiveRecord: 0.0ms | Allocations: 4611)
Delivered mail ea8af868-4a2c-4312-85dd-f57061a3cd90@discuss.mydomain.community (60015.9ms)
Job exception: Net::ReadTimeout

Waiting for a bit and then via the browser requested ‘Resend notification email’. The following was now added to the production log:

Delivered mail 47ca6f15-cd9e-4c96-a670-6646e2bda585@discuss.mydomain.community (60007.9ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
Delivered mail 02367872-af3a-4df4-9a68-4e5e7c5eda60@discuss.mydomain.community (60007.4ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
Delivered mail ee9ee1fc-6fd1-4970-89fa-260efe2dd04c@discuss.mydomain.community (60014.1ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.1ms | Allocations: 31)
Delivered mail 9030f87b-99df-4de2-9a60-2c57f7c752de@discuss.mydomain.community (60007.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
Started PUT "/finish-installation/resend-email" for REDACTED-IP at 2020-09-03 06:31:22 +0000
Processing by FinishInstallationController#resend_email as HTML
  Parameters: {"authenticity_token"=>"NE21scxxyZz3/DxDkoF8kwi9GXoNKvnstNJdKZjQs7afigDKWcbw4XK/XnvvRHTApqExqUUghybE1oPfyo3aDA=="}
  Rendering finish_installation/resend_email.html.erb within layouts/finish_installation
  Rendered finish_installation/resend_email.html.erb within layouts/finish_installation (Duration: 0.9ms | Allocations: 163)
  Rendered layouts/_head.html.erb (Duration: 2.5ms | Allocations: 269)
Completed 200 OK in 63ms (Views: 6.5ms | ActiveRecord: 0.0ms | Allocations: 6408)
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
Delivered mail 127cc350-3fb0-4ef1-8759-391fb407b1cb@discuss.mydomain.community (60008.0ms)
Job exception: Net::ReadTimeout

Delivered mail d9c70dfd-5a8b-4f4c-bafa-5a540fc8ed4f@discuss.mydomain.community (60010.0ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.1ms | Allocations: 31)
Delivered mail e9e67ed1-d7b1-4e8a-ba11-30501c6fae89@discuss.mydomain.community (60012.4ms)
Job exception: end of file reached

  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.2ms | Allocations: 31)
  Rendering layouts/email_template.html.erb
  Rendered layouts/email_template.html.erb (Duration: 0.1ms | Allocations: 31)
Delivered mail 82d0361e-0349-4a1a-928e-dcb16dcffdbc@discuss.mydomain.community (60008.6ms)
Job exception: end of file reached

I contacted my hosting provider to see if the problem was on their end. They said that according to them this configuration should work fine, but that the VPS had caused no activity at all on the SMTP server.

So that’s my current situation.. posting to Meta.

メールサーバーへの送信トラフィックがブロックされている可能性がありますか?そこからtelnetできますか?コンテナ内部からですか?

はい、Docker コンテナ内から telnet を使用して SMTP サーバーに正常に接続できます。ただし、プロバイダーがそれ以降のコマンドを送信すると接続を切断するため、telnet でテストメールを送信することはできません。

解決策はシンプルでした。プロバイダーが上記のポートを示していましたが、587 に telnet 接続すると動作していることがわかりました。そこで app.yml を変更し、再構築を実行したところ、問題なく動作しました :smiley:

これを見る前、私はこの変更を推奨しようとしていました。

ポート 465 は SMTP-over-SSL で、非推奨です。
ポート 587 は MSA ポート(メール送信エージェント)であり、通常、STARTTLS(平文から TLS 接続へのアップグレード)がサポートされています。

設定に 465 を指定していた場合、Discourse は平文で SSL ポートと通信しようとしていました。

@supermathie さん、ありがとうございます。あなたの回答を解決策として採用しました。明確な説明でした。

こんにちは @supermathie

RFC 8314 に従い、ポート465は(廃止から)復活した(または復活の過程にあります)。私の現在のメールプロバイダー(mailfence.com)もこの推奨事項に従っているようです。

ポート465を使用した接続でTLSを強制する方法はありますか?これで問題が解決するはずです。容器内から swaks を使用して確認したところ、STARTTLS (–tls) は失敗しますが、–tls-on-connect (–tlsc) は正常に動作しました:

swaks --to user@example.com --from myuser@mailfence.com --server smtp.mailfence.com:465 --auth LOGIN --tlsc --auth-user myuser@mailfence.com

よろしくお願いいたします。

ああ、TLS 経由の送信用に再利用されているのですね。それは理にかなっています。

現時点ではありません。smtp_settings:ssl = true を設定すると ActionMailer がそれをサポートしている ようですが、現在それを有効にするコードパスはありません。

そのサポートを追加する PR があれば、ぜひ歓迎したいと考えています。