# Replacing Mailing lists: Email-In
**URL:** https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099
**Category:** Feature
**Created:** [February 26, 2014, 6:24pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099 "2014-02-26T18:24:44Z")
**Posts on this page:** 20
**Page:** 1
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [February 26, 2014, 6:24pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/1 "2014-02-26T18:24:45Z")
Hello everyone,
**Update** : I’ve finished developing this. You can find the [entire patchset at the pull-request](https://github.com/discourse/discourse/pull/2012) with the three features as three commits in the same order.
I am on the quest on supporting discourse [to replace mailing lists](http://www.discourse.org/faq/#what) (our internal google group system to be precise). One feature discourse is clearly missing to be able to replace mailing lists is to allow the creation of new topics via email. I call this \* Email-In\*. These are the three development steps I currently see:
### 1. General Email in
1. let admins enable and define an incoming-email-address
2. parse every email to the email-address in the POP3-account
3. if it comes from a user of the forum and that user has a certain configurable trust level, post the message as a new topic in the forum
### 2. Email-In per Category
1. to reduce the noise, we are using categories people can mute and such
2. therefore it makes sense to allow to configure email-addresses per category if the general incoming-email-feature is enabled
3. an email to that address then will be posted in that category instead of globally
4. the privacy options for that category apply
### 3. Allowing “unknown” Email-In
1. mailing lists are also sometimes configured as “a shared inbox”, allowing third parties to post to it
2. add another configuration option in the email-in-configuration (globally and per category) to allow emails from third-parties (off per default)
3. when an email arrives in the inbox, that is not by any user known to the system, the header and its content will be posted as a quote in a new topic by the configured “system”-user
4. also emails by users without the appropriate rights would be handled this way
Any feedback, ideas, features or other uses cases I might should consider, too?
Thanks
Ben
---
### Author:  [@mcwumbly](https://meta.discourse.org/u/mcwumbly)
#### Post date: [February 26, 2014, 7:05pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/4 "2014-02-26T19:05:57Z")
> [@lightyear](#):
>
> if it comes from a user of the forum and that user has a certain configurable trust level, post the message as a new topic in the forum
If the user does not have the required trust level, what happens?
Oh, I see:
> [@lightyear](#):
>
> 1. when an email arrives in the inbox, that is not by any user known to the system, the header and its content will be posted as a quote in a new topic by the configured “system”-user
> 2. also emails by users without the appropriate rights would be handled this way
---
### Author:  [@erlend\_sh](https://meta.discourse.org/u/erlend_sh)
#### Post date: [February 26, 2014, 7:47pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/5 "2014-02-26T19:47:20Z")
@watchmanmonitor has also done a take on ‘new posts by e-mail’:
[https://meta.discourse.org/t/mailing-list-and-nntp-bridge/3453/63?u=erlend\_sh](https://meta.discourse.org/t/mailing-list-and-nntp-bridge/3453/63)
Also a lot of good discussion in that thread as a whole regarding Discourse and mailing lists. Well worth reading.
---
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [February 26, 2014, 7:54pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/6 "2014-02-26T19:54:30Z")
@mcwumbly, yeah, actually this is one I am still scratching my head about. We were discussing an [“inbox”-like-permission for category-system before](https://meta.discourse.org/t/new-category-permission-create-and-see/12053), which I think makes more sense at least in the case if a user can’t post because they aren’t part of that group. But then again, this should be part of the usual discourse permission system and not act any different for mails imho. (and as my implementation totally relies on the guardian, this would probably work in the future, too).
@erlend_sh, interesting indeed. Didn’t see that one yet, but it is very similar to my code actually. @watchmanmonitor , what do you think about collaborating on this feature?
---
### Author:  [@uckelman](https://meta.discourse.org/u/uckelman)
#### Post date: [February 26, 2014, 9:13pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/7 "2014-02-26T21:13:34Z")
> [@lightyear](#):
>
> @watchmanmonitor , what do you think about collaborating on this feature?
I think all three of us should try working together on this. It’s a lot of wasted effort otherwise.
---
### Author:  [@watchmanmonitor](https://meta.discourse.org/u/watchmanmonitor)
#### Post date: [February 26, 2014, 9:30pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/8 "2014-02-26T21:30:31Z")
Bring it on- I’m happy to add you to my repo for the work
[https://github.com/cidercooler/discourse/](https://github.com/cidercooler/discourse/)
with the idea that we’ll submit it as a proper pull request once there’s some admin interface ready to include with the feature (which will need to be off by default for acceptance)
---
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [February 27, 2014, 12:57pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/9 "2014-02-27T12:57:40Z")
I included your work @watchmanmonitor and continued with step 2 this morning. The [updated pull-request now contains both features](https://github.com/discourse/discourse/pull/2012) sets listed in 1 and 2. I’ll continue with feature 3 now.
---
### Author:  [@ipcamit](https://meta.discourse.org/u/ipcamit)
#### Post date: [February 27, 2014, 4:20pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/10 "2014-02-27T16:20:42Z")
Hi I was about to ask very similar things. Though would like to know if one more feature can be added by the admin- the ability to filter incoming mail on the basis of keywords or attachments etc. As its really tiresome on a mailing list if first 25 mails are automated out of office replies
---
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [February 27, 2014, 5:13pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/11 "2014-02-27T17:13:05Z")
Update: done. The [same pull-request](https://github.com/discourse/discourse/pull/2012) now contains all three part-features. Enjoy.
@ipcamit, that sounds like a misuse of ML in general. Not sure how to add this atm.
---
### Author:  [@ipcamit](https://meta.discourse.org/u/ipcamit)
#### Post date: [February 27, 2014, 5:32pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/12 "2014-02-27T17:32:56Z")
Well I want to pitch discourse as a viable new community tool to The
Microscopy Society of America ([http://microscopy.com/](http://microscopy.com/)). But It being 30 years old community chances of their
accepting it are slim. Every time you send a mail there you get around 20
out of office reply. So I thought, more features, more stronger will be my
case! Though their current filters don’t allow any attachment to mails,
even the basic HTML. So having capabilities to control incoming mails would
be better. In current scenario, how will you protect it from the spam?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 3, 2014, 12:01am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/13 "2014-03-03T00:01:47Z")
I am concerned about the massive spam/impersonation vector we are introducting. allow\_strangers really does nothing to protect against spam, once people discover an email address in the forum they can just keep emailing in forever.
What should happen is that on the user page users should be given a “secret” incoming email address. Eg mine would be something like
`info+A12312BF5@discourse.org`
That way we can validate that the incoming mail is coming from who they say it is and regenerate a private key if the email is compromised.
This feature may work in low volume forums but as soon as spammers are on to it there is nothing you can do except for disable it.
---
### Author:  [@ipcamit](https://meta.discourse.org/u/ipcamit)
#### Post date: [March 3, 2014, 4:43am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/14 "2014-03-03T04:43:36Z")
Going slightly off topic, on many lists it is done by asking strangers to fill up form to post. So only registered users can post through email, but once form is submitted, admin can review the content and allow outgoing notification to that particular user, no incoming still. Something on the lines of running [contactify](http://www.contactify.com/)
Well actually that is how the Microscopy listserver is managed.
---
### Author:  [@riking](https://meta.discourse.org/u/riking)
#### Post date: [March 3, 2014, 4:54am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/15 "2014-03-03T04:54:32Z")
“fill out a form”
So that’s basically the same thing as signing up for the forum and setting the threads to Watched. No need to do any more work here.
---
### Author:  [@ipcamit](https://meta.discourse.org/u/ipcamit)
#### Post date: [March 3, 2014, 5:34am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/16 "2014-03-03T05:34:05Z")
No. Its just a subscription to interact with a single post. No need to signup. I think its more akin to Q and A site like stackexchange but with focus on discussing rather than having an “expert” answer. It is useful at least in academic research where you might need a consultation on something but not keen on having membership of a forum that you will never use again.
So by filling the form your message will be posted on behalf of Admin,where it can be reviewed with you getting notifications through mail on that particular post. Giving a direct post address will be vulnerable to spam and giving a secret post email address will be ideal for such one time users.
---
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [March 3, 2014, 9:27am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/17 "2014-03-03T09:27:58Z")
> [@sam](#):
>
> I am concerned about the massive spam/impersonation vector we are introducting. allow\_strangers really does nothing to protect against spam, once people discover an email address in the forum they can just keep emailing in forever.
I agree, which is why this option isn’t existing for the global ML-address but only on a category level. I don’t think many people will actually activate it to be honest, as for us it serves a specific use case that is currently handled with a normal ML: using the ML as the sign-up and notification email for third-parties. A great example would be the ML currently handling external communication, that receives a twitter-notification-email whenever we get mentioned. The same goes for bills send from AWS or technical messages from [meetup.com](http://meetup.com) . I do see in the long-run it would be nicer to replace this with “native” implementations of the services posting stuff directly but I bet there will always be system that simply send emails instead.
Ergo, we (and the few others, who might be using it at all) will be using it private categories and not share that email-address outside of that limited group in the first place. But I do see your concern and am thinking about how to fix that.
> [@sam](#):
>
> What should happen is that on the user page users should be given a “secret” incoming email address. Eg mine would be something like
>
> [info+A12312BF5@discourse.org](mailto:info+A12312BF5@discourse.org)
>
> That way we can validate that the incoming mail is coming from who they say it is and regenerate a private key if the email is compromised.
Not sure how that would solve that issue. The “allow\_strangers” feature explicitly allows posting of those, who’ve no knowledge about that there is such a thing as discourse at all behind it. If you are sending from a user-validated email-address you don’t need the “allow strangers” to be set, the message will be assigned to the user and all other messages will be discarded. I don’t see how creating a custom incoming-email per user makes much more of a difference than using their authenticated from-email for matching.
> [@sam](#):
>
> This feature may work in low volume forums but as soon as spammers are on to it there is nothing you can do except for disable it.
As said, the “allow\_strangers” option is mainly for third-party-usage and doesn’t exist for the global email at all. I don’t think there is much point in publishing any email-address that does have the “allow\_strangers”-field set.
* * *
But on that regard and thinking of it, what I’d rather have for this case are “shadow”-users like the system user to cover this use-case. So you could create a “twitter”-system-user and a "meetup.com|-system-user that can’t log it and you could assign a bunch of email-addresses to the user – maybe even with a simple option of where it should be posted – that you could then safely use as notification email on a third-party service and if compromised would just be disabled. But that sounds like a harder case to make …
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 3, 2014, 10:05am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/18 "2014-03-03T10:05:52Z")
> [@lightyear](#):
>
> If you are sending from a user-validated email-address you don’t need the “allow strangers” to be set,
no no no, that is not the case, SMTP sucks big time, just cause the record says it came from [sam.saffron@gmail.com](mailto:sam.saffron@gmail.com) does not mean it came from me.
> **[Email spoofing](https://en.wikipedia.org/wiki/Email_spoofing)**
>
> Email spoofing is the creation of email messages with a forged sender address. The term applies to email purporting to be from an address which is not actually the sender's; mail sent in reply to that address may bounce or be delivered to an unrelated party whose identity has been faked. Disposable email address or "masked" email is a different topic, providing a masked email address that is not the user's normal address, which is not disclosed (for example, so that it cannot be harvested), bu Th...
“user-validated email address” is meaningless unless there is some extra amount of security around it.
* * *
Allow strangers could reply to the first email sent with a magic key email. That in effect could enforce a secure channel.
---
### Author:  [@lightyear](https://meta.discourse.org/u/lightyear)
#### Post date: [March 3, 2014, 10:29am UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/19 "2014-03-03T10:29:39Z")
What I meant was that the “allow strangers” option doesn’t have any influence there, that’s all. As it explicitly is meant for non-users. Even sending an email to verify would simply not work for the use cases mentioned above.
* * *
Which doesn’t mean your point is legit. I didn’t think about that as we also just send emails with the reply-key, which - if intercepted - would create a secure channel of spam to that topic at least. What I said is that spoofing would work even without the allow\_stranger-feature to be set. I just wanted to make that separation, as allow\_strangers does never care about the sender at all at the moment.
So, let’s get back to your proposal of “customized incoming emails” then. I like that approach. What I am wondering about it, is how many providers do support the ±email-suffix-matching feature outside of google though. I do know we area already enforcing this via the reply-key but I was wondering how much we are expecting from the servers here and why we don’t simply set the reply-key as the corresponding email-header in the first place in which case the incoming-email-address-configuration wouldn’t matter that much.
And secondly, if we do make the posting to certain email-private-address, how we want to handle categorisation. Do we set another, second suffix for each category? Or a totally customized email-address per category. Or maybe just a simple text-matching of a “[CATEGORY]”-prefix in the subject?
The problem with subject-lines would be that this won’t work with third-parties again. But a some-how customized email could indeed. We could potentially support both if we want to.
That in combination with a simplified-system-user-system-for-third-parties would probably be a much better implementation for the previously mentioned use cases.
---
### Author:  [@watchmanmonitor](https://meta.discourse.org/u/watchmanmonitor)
#### Post date: [March 3, 2014, 9:02pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/20 "2014-03-03T21:02:03Z")
> [@sam](#):
>
> [sam.saffron@gmail.com](mailto:sam.saffron@gmail.com) does not mean it came from me.
I don’t remember the last email [that I received] that didn’t come from the true sender. That’s what the inbound spam filter in front of our mailhost is there for, right?
I can expect to get spam from users’ hacked aol & yahoo accounts from time to time, however it’s a risk much smaller than the utility we get from allowing our users to shoot off a quick email to our Mailing List.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [March 3, 2014, 9:05pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/21 "2014-03-03T21:05:32Z")
This is all about creating topics, reply via email is pretty damn safe. The issue is that as soon as an email address here leaks out you can flood the forum with new topics.
---
### Author:  [@watchmanmonitor](https://meta.discourse.org/u/watchmanmonitor)
#### Post date: [March 3, 2014, 9:11pm UTC](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099/22 "2014-03-03T21:11:19Z")
Understood, and I’m OK with that.
I have three large Discourse installs in the works, and none of them would gain traction if our users couldn’t leverage the community with a quick “How do I do this” email to the list.
I’ve experienced mailing lists die when moved to a web-only forum. It made me sad when it happened, and I’m hoping to revitalize them with Discourse.
I have no concept that enabling this option is good for a forum where people can self-enroll, but for private lists, I express my gratitude that the feature is a part of your product.
-Allen Hancock
[Next page](https://meta.discourse.org/t/replacing-mailing-lists-email-in/13099.md?page=2)