Verification Email will not send due to errors caused by Docker filesystem

jd2066 · September 3, 2015, 9:07pm

I have a Linode with Gentoo Linux that I decided to install Discourse on.
I installed Docker and then using the 30-min install options installed discourse.
This part worked as I can go to the site and register my admin user but after registering, I never get a Verification Email.
I configured Discourse via nano -w containers/app.yml with ./launcher bootstrap app after editing it.
I signed up with Mailjet to do email and got an API Key and Secret key to use.

To show examples without revealing those, I’ll use these:

API Key: hotdog
Secret Key: ketchup
Dev Email: justin@mydomain.tld
Hostname: forum.mydomain.tld

Here is my app.yml configuration:

DISCOURSE_DEVELOPER_EMAILS: 'justin@mydomain.tld'
DISCOURSE_HOSTNAME: 'forum.mydomain.tld'
DISCOURSE_SMTP_ADDRESS: in-v3.mailjet.com
DISCOURSE_SMTP_DOMAIN: mydomain.tld
DISCOURSE_SMTP_USER_NAME: hotdog
DISCOURSE_SMTP_PASSWORD: ketchup

I have tried using using these options (separately and together) also without any effect:

DISCOURSE_SMTP_PORT: 587`
DISCOURSE_SMTP_ENABLE_START_TLS: false

After looking at this topic, I copied an example python script and edited it’s configuration to be:

SMTPserver = 'in-v3.mailjet.com'
sender =     'forum@mydomain.tld'
destination = ['justin@mydomain.tld']
USERNAME = "hotdog"
PASSWORD = "ketchup"

The python script worked just fine so I know the Mailjet account works.
No idea why Discourse can’t send email.

Are there any logs I can look at to figure out why Discourse won’t send email?
I’ve looked in shared/standalone/log/rails/production.log but all I find about email is various entries of this (with different ms values):

Sent mail to justin@mydomain.tld (30058.0ms)
Job exception: execution expired

codinghorror · September 3, 2015, 9:16pm

See the email troubleshooting howto, and list your response to every section of that topic.

jd2066 · September 3, 2015, 9:23pm

Did you fill in the SMTP address, SMTP username, and SMTP password fields correctly, as specified by Mandrill?
Yes (for Mailjet).

Make sure you copy and paste the Mandrill password – do not type it in manually.
I did do this (with MailJet).

Does the value for DISCOURSE_DEVELOPER_EMAILS correctly have one set of single quotes around it ‘like this’?
Yes

Are the values aligned with the correct indentation for the email section? YML files are alignment sensitive. Even one extra space at the beginning of a line will cause problems.
I believe so, I didn’t change the alignment that was there.

Did you remove the YML comment marker # in front of the lines?
Yes

Is Digital Ocean blocking your email?
I don’t use Digital Ocean, so no.
And Linode is fine with email that isn’t spam.

Is Mandrill blocking your email?
I use MailJet so no, also because I could use the expample python script to send email, I know they aren’t blocking email.

Did you properly set up DKIM and SPF records for your domain?
Yes I did.

codinghorror · September 3, 2015, 9:48pm

You didn’t quite answer all the questions. Do MailJet logs show an attempt to connect and send email from the IP of the linode? Digital Ocean will show a connection attempt from the Digital Ocean IP in its web UI for email logs.

Looks like something in your network, somewhere, is blocking connections based on

Sent mail to justin@mydomain.tld (30058.0ms)
Job exception: execution expired

You can look at /logs URL in Discourse but you need an admin account to do that.

jd2066 · September 3, 2015, 10:06pm

I don’t see anywhere in MailJet logs of connect attempts.
Just a “Real-time Email Feed” showing 1 email sent via the python test script (Running on the Linode).

In theory it seems like there shouldn’t be a difference between a connection from a python script or a ruby on rails script (which is what the discourse mail code is if I understand correctly) in a docker container.

Why can’t I look at those logs via the command line like most other logs?

codinghorror · September 3, 2015, 10:21pm

Because they are served via the web browser. Sorry, if you want, buy a $99 install and I will set up email for you that’s guaranteed to work with a Digital Ocean Discourse droplet, using a secret Mandrill reseller code that provides a large number of free emails per month.

http://discourse.org/buy

Email is the hardest problem in computer science.

jd2066 · September 3, 2015, 10:29pm

Is there a way to manually set the user as verified just to check the logs (I wouldn’t leave it that way as it’s a good idea to verify people’s email addresses)?

At the moment, I don’t want to spend any more money then I’m spending on the Linode hosting for my WordPress site/blog along with other small traffic sites (which run using apache port 80 so for now I have discourse set to use port 8080)

I can’t tell if you are being serious or not.

Mittineague · September 3, 2015, 10:49pm

Jeff is serious. A subjective opinion but not far off, if at all.

The alternative is to spend more time. Up to you.

codinghorror · September 3, 2015, 11:05pm

There are howto topics here on creating an admin account manually at the command line, then you can get to /logs in the web browser.

But without functional email you are 100% screwed since nobody will be able to sign up and use your Discourse.

jd2066 · September 3, 2015, 11:45pm

If I kept it that way, yes.
The idea was to login, check /logs, find and fix the problem so there would be functional email for all future accounts Verifications.

jd2066 · September 3, 2015, 11:57pm

Ok, I created the admin account manually and looked at /logs
First Error:
PG::UndefinedTable: ERROR: relation "users" does not exist LINE 5: WHERE a.attrelid = '"users"'::regclass ^ : SELECT a.attname, format_type(a.atttypid, a.atttypmod), pg_get_expr(d.adbin, d.adrelid), a.attnotnull, a.atttypid, a.atttypmod FROM pg_attribute a LEFT JOIN pg_attrdef d ON a.attrelid = d.adrelid AND a.attnum = d.adnum WHERE a.attrelid = '"users"'::regclass AND a.attnum > 0 AND NOT a.attisdropped ORDER BY a.attnum
I’m not sure what this means.

Second Error:
Job exception: execution expired
This is related to the email but doesn’t seem to have much more detail than production.log did.

codinghorror · September 4, 2015, 12:05am

Are you on the latest version of Discourse? Any plugins or unusual site settings?

jd2066 · September 4, 2015, 12:34am

I followed the instructions at discourse/docs/INSTALL-cloud.md at main · discourse/discourse · GitHub for the most part.
Here are the exact steps I did:
* Set up Swap (if needed)
Because of ‘If you’re using 2 GB+ memory, you can probably get by without a swap file.’, I figured I didn’t need one so I didn’t set one up.
Edit:
Checking with free -m it returns a Memory total of 1999 so this could be part of the problem as that says 2GB+ not 2GB (which would be 2048 MB so it seems I’m 49MB short of that).
* Install Docker / Git
I skipped running the command wget -qO- https://get.docker.com/ | sh as I figured for Gentoo the packages app-emulation/docker and dev-vcs/git covered it.
* Install Discourse
I ran the listed commands:
mkdir /var/discourse git clone https://github.com/discourse/discourse_docker.git /var/discourse cd /var/discourse cp samples/standalone.yml containers/app.yml
Which appeared to complete successfully.
* Edit Discourse Configuration
I edited containers/app.yml with nano as listed in my first post.
* Bootstrap Discourse
Ran ./launcher bootstrap app
Than ran ./launcher start app
* Register New Account and Become Admin
I was able to do this except of course for the email validation
So I haven’t really been able to add plugins or change site settings.
I can now with the manual admin user creation but without working email, not much point in changing other things yet.

Edit:
I did end up doing these steps twice because the first time docker used the vfs storage driver and ran out of space so I wiped out /var/lib/docker, told docker to use overlayfs and restarted it.
I also ended up wiping out /var/discourse incase something was wrong there.

codinghorror · September 4, 2015, 12:46am

What does docker --info return and docker --version ?

jd2066 · September 4, 2015, 2:40am

It appears the – prefix is not used in newer docker versions so I ran the commands without it.

The command docker info returns:

Containers: 1
Images: 11
Storage Driver: overlay
 Backing Filesystem: extfs
Execution Driver: native-0.2
Logging Driver: json-file
Kernel Version: 4.1.5-x86_64-linode61
Operating System: Gentoo/Linux
CPUs: 2
Total Memory: 1.952 GiB
Name: linodevps
ID: 2C7H:MOAS:GM2N:JMG7:GYY5:CNNF:R33M:QNKK:3U6K:2N27:OOAU:PZRG
WARNING: No swap limit support

The command docker version returns:

Client:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:
 OS/Arch:      linux/amd64

Server:
 Version:      1.8.1
 API version:  1.20
 Go version:   go1.4.2
 Git commit:   d12ea79
 Built:
 OS/Arch:      linux/amd64

Also, in case it matters, when merging the docker package the following notice appeared:

 * Checking for suitable kernel configuration options...
 *   CONFIG_MEMCG_KMEM: is optional
 *   CONFIG_MEMCG_SWAP: is required if you wish to limit swap usage of containers
 *   CONFIG_MEMCG_SWAP_ENABLED:  is not set when it should be.
 *   CONFIG_CGROUP_HUGETLB:      is not set when it should be.
 *   CONFIG_RT_GROUP_SCHED:      is not set when it should be.

codinghorror · September 4, 2015, 4:18am

Is overlay safe to use now @sam ? If you are on latest 1.4 beta Discourse code, I cannot explain the errors you are seeing. But last I recall overlay was new and experimental and could potentially cause unknown issues.

sam · September 4, 2015, 4:39am

I have been using overlay locally for about 3 months with no issues

jd2066 · September 4, 2015, 4:58am

Logged in to Discourse with the manual admin user I created and it shows the installed version being “v1.4.0.beta10 +176”.

codinghorror · September 4, 2015, 5:02am

I think you have a busted install, somehow. If you don’t have any plugins or any other unusual settings, I can’t think of any reason you would have strange errors related to email.

I just set up two instances today following the guide literally step by step on Digital Ocean with Mandrill, and everything worked perfectly.

riking · September 4, 2015, 5:08am

It’s probably that overlayfs is still broken, maybe it’s more broken on Linode than it is on @sam’s laptop?

Actually sam probably has a newer kernel version. @jd2066 does Linode let you pick a newer kernel?

Topic		Replies	Views
No connection accepted on http / https after fresh installation on Ubuntu 22.04 LTS Installation	5	2097	December 16, 2022
Can't get discourse working again Installation	8	1588	February 16, 2020
Generals Subfolder recommendations and tips Installation	9	1982	January 12, 2020
Error executing command "./launcher rebuild app" Installation	6	610	August 3, 2021
Can't send mail by smtp server(mailjet) Installation email	6	1160	June 6, 2023

Verification Email will not send due to errors caused by Docker filesystem

Related topics