Update: I’ve finished developing this. You can find the entire patchset at the pull-request with the three features as three commits in the same order.
I am on the quest on supporting discourse to replace mailing lists (our internal google group system to be precise). One feature discourse is clearly missing to be able to replace mailing lists is to allow the creation of new topics via email. I call this * Email-In*. These are the three development steps I currently see:
1. General Email in
let admins enable and define an incoming-email-address
parse every email to the email-address in the POP3-account
if it comes from a user of the forum and that user has a certain configurable trust level, post the message as a new topic in the forum
2. Email-In per Category
to reduce the noise, we are using categories people can mute and such
therefore it makes sense to allow to configure email-addresses per category if the general incoming-email-feature is enabled
an email to that address then will be posted in that category instead of globally
the privacy options for that category apply
3. Allowing “unknown” Email-In
mailing lists are also sometimes configured as “a shared inbox”, allowing third parties to post to it
add another configuration option in the email-in-configuration (globally and per category) to allow emails from third-parties (off per default)
when an email arrives in the inbox, that is not by any user known to the system, the header and its content will be posted as a quote in a new topic by the configured “system”-user
also emails by users without the appropriate rights would be handled this way
Any feedback, ideas, features or other uses cases I might should consider, too?
@mcwumbly, yeah, actually this is one I am still scratching my head about. We were discussing an “inbox”-like-permission for category-system before, which I think makes more sense at least in the case if a user can’t post because they aren’t part of that group. But then again, this should be part of the usual discourse permission system and not act any different for mails imho. (and as my implementation totally relies on the guardian, this would probably work in the future, too).
@erlend_sh, interesting indeed. Didn’t see that one yet, but it is very similar to my code actually. @watchmanmonitor , what do you think about collaborating on this feature?
with the idea that we’ll submit it as a proper pull request once there’s some admin interface ready to include with the feature (which will need to be off by default for acceptance)
Hi I was about to ask very similar things. Though would like to know if one more feature can be added by the admin- the ability to filter incoming mail on the basis of keywords or attachments etc. As its really tiresome on a mailing list if first 25 mails are automated out of office replies
Well I want to pitch discourse as a viable new community tool to The
Microscopy Society of America (http://microscopy.com/). But It being 30 years old community chances of their
accepting it are slim. Every time you send a mail there you get around 20
out of office reply. So I thought, more features, more stronger will be my
case! Though their current filters don’t allow any attachment to mails,
even the basic HTML. So having capabilities to control incoming mails would
be better. In current scenario, how will you protect it from the spam?
I am concerned about the massive spam/impersonation vector we are introducting. allow_strangers really does nothing to protect against spam, once people discover an email address in the forum they can just keep emailing in forever.
What should happen is that on the user page users should be given a “secret” incoming email address. Eg mine would be something like
info+A12312BF5@discourse.org
That way we can validate that the incoming mail is coming from who they say it is and regenerate a private key if the email is compromised.
This feature may work in low volume forums but as soon as spammers are on to it there is nothing you can do except for disable it.
Going slightly off topic, on many lists it is done by asking strangers to fill up form to post. So only registered users can post through email, but once form is submitted, admin can review the content and allow outgoing notification to that particular user, no incoming still. Something on the lines of running contactify
Well actually that is how the Microscopy listserver is managed.
No. Its just a subscription to interact with a single post. No need to signup. I think its more akin to Q and A site like stackexchange but with focus on discussing rather than having an “expert” answer. It is useful at least in academic research where you might need a consultation on something but not keen on having membership of a forum that you will never use again.
So by filling the form your message will be posted on behalf of Admin,where it can be reviewed with you getting notifications through mail on that particular post. Giving a direct post address will be vulnerable to spam and giving a secret post email address will be ideal for such one time users.
I agree, which is why this option isn’t existing for the global ML-address but only on a category level. I don’t think many people will actually activate it to be honest, as for us it serves a specific use case that is currently handled with a normal ML: using the ML as the sign-up and notification email for third-parties. A great example would be the ML currently handling external communication, that receives a twitter-notification-email whenever we get mentioned. The same goes for bills send from AWS or technical messages from meetup.com . I do see in the long-run it would be nicer to replace this with “native” implementations of the services posting stuff directly but I bet there will always be system that simply send emails instead.
Ergo, we (and the few others, who might be using it at all) will be using it private categories and not share that email-address outside of that limited group in the first place. But I do see your concern and am thinking about how to fix that.
Not sure how that would solve that issue. The “allow_strangers” feature explicitly allows posting of those, who’ve no knowledge about that there is such a thing as discourse at all behind it. If you are sending from a user-validated email-address you don’t need the “allow strangers” to be set, the message will be assigned to the user and all other messages will be discarded. I don’t see how creating a custom incoming-email per user makes much more of a difference than using their authenticated from-email for matching.
As said, the “allow_strangers” option is mainly for third-party-usage and doesn’t exist for the global email at all. I don’t think there is much point in publishing any email-address that does have the “allow_strangers”-field set.
But on that regard and thinking of it, what I’d rather have for this case are “shadow”-users like the system user to cover this use-case. So you could create a “twitter”-system-user and a "meetup.com|-system-user that can’t log it and you could assign a bunch of email-addresses to the user – maybe even with a simple option of where it should be posted – that you could then safely use as notification email on a third-party service and if compromised would just be disabled. But that sounds like a harder case to make …
What I meant was that the “allow strangers” option doesn’t have any influence there, that’s all. As it explicitly is meant for non-users. Even sending an email to verify would simply not work for the use cases mentioned above.
Which doesn’t mean your point is legit. I didn’t think about that as we also just send emails with the reply-key, which - if intercepted - would create a secure channel of spam to that topic at least. What I said is that spoofing would work even without the allow_stranger-feature to be set. I just wanted to make that separation, as allow_strangers does never care about the sender at all at the moment.
So, let’s get back to your proposal of “customized incoming emails” then. I like that approach. What I am wondering about it, is how many providers do support the ±email-suffix-matching feature outside of google though. I do know we area already enforcing this via the reply-key but I was wondering how much we are expecting from the servers here and why we don’t simply set the reply-key as the corresponding email-header in the first place in which case the incoming-email-address-configuration wouldn’t matter that much.
And secondly, if we do make the posting to certain email-private-address, how we want to handle categorisation. Do we set another, second suffix for each category? Or a totally customized email-address per category. Or maybe just a simple text-matching of a “[CATEGORY]”-prefix in the subject?
The problem with subject-lines would be that this won’t work with third-parties again. But a some-how customized email could indeed. We could potentially support both if we want to.
That in combination with a simplified-system-user-system-for-third-parties would probably be a much better implementation for the previously mentioned use cases.
I don’t remember the last email [that I received] that didn’t come from the true sender. That’s what the inbound spam filter in front of our mailhost is there for, right?
I can expect to get spam from users’ hacked aol & yahoo accounts from time to time, however it’s a risk much smaller than the utility we get from allowing our users to shoot off a quick email to our Mailing List.
This is all about creating topics, reply via email is pretty damn safe. The issue is that as soon as an email address here leaks out you can flood the forum with new topics.
I have three large Discourse installs in the works, and none of them would gain traction if our users couldn’t leverage the community with a quick “How do I do this” email to the list.
I’ve experienced mailing lists die when moved to a web-only forum. It made me sad when it happened, and I’m hoping to revitalize them with Discourse.
I have no concept that enabling this option is good for a forum where people can self-enroll, but for private lists, I express my gratitude that the feature is a part of your product.