Replacing Mailing lists: Email-In

Honestly, that could happen with any mailing list now… and at least with Discourse we can clean up the detritus easily!

For any user of the forum who isn’t subscribed to instant notification, they may never know it happened if one of our moderators is around to clean up the mess quickly :slight_smile:

No, I agree @sam has a really good point here. @watchmanmonitor, the reason that works for your inbox is because Gmail and a lot of others do a lot of counter-proofing email-origination with each other. And one problem we are starting to face is that some crawlers founds the even non-public google-groups of ours and now I get a daily spam-digest-report from google for these lists, which is every annoying.

Revisiting the unique-incoming-email makes sense then. And though not all providers support the smth+else@example.com format, I think we can stick with it as most would at least offer a catch-all option through which this can be supported.

If we change the phrasing of the site-setting to a more general discourse+%{KEY}@example.com, we could also easily support personalised incoming-emails with it. In which case I’d propose to have [CATEGORY]-Prefix in the subject line indicate the desired category it is supposed to go to. Alternatively we can also think of allowing the key to contain a category, too (to allow users to put them into their address book), like discourse+3bc038ea_feature@example.com.

This email would be exposed to the user through their account preference and can be regenerated by them or by the admin (in which case the user would receive a notification about it).

This would allow for an easy email-in-system that I do consider more secure than what we have currently in place.

Note: If implemented this way, this could already be used for the desired other use-case of third-party-apps. An admin could just create a new user, generate the incoming-email-address and copy-paste-it including the category-handle to the third party notification system. Which if you allow the admin to generate multiple random-addresses per user, could be totally sufficient for my described use-case. Unless there is another great use case for the allow_strangers-feature ( @watchmanmonitor, @riking do you have any), I’d consider this covered.

@sam, we’ve not released the feature as it is currently in, right? If we can agree on this implementation instead, I’d get right on it and ask you to disable the current site-settings until then. But please don’t revert it, I’d build this feature on top of the existing code.

Gentlemen, agreed?

4 Likes

Sounds good to me - and hey, the reply_by_email_address is already in that format - you just need to generate the keys with _category on the end.

Yay for code reuse! :wink:

1 Like

The email address which our forum POPs is a spam filtered inbox like any other.

I would like it to be optional.

If you want spam + simple email for incoming topics that is fine. If you want spam resilience that should be in too. I would just have a site setting with the “incoming template” and have it flexible to have key or no key.

3 Likes

I’m unclear on something:

Will there be an option to allow topics created by non-existing users (aka strangers)? I would very much like that.

Our basic way to prevent spam would be:

  • Maintain a whitelist of allowed e-mails (based on our e-mail subscribers)
  • Make all first-time posts hidden by default. If that email has had 2 posts made visible, future posts will be automatically accepted.

@erlend_sh, there currently is, but only on a category-bases (so users could mute the category if it comes to worst). You can activate it by giving a category an incoming email-address and check the checkbox under it. Emails from strangers will then be posted by the system user referencing the original email-address.

Thanks, that’s good news. Is this live on Discourse.org as we speak? I’d love a basic manual on how to use it so I can make a few tests on Try.Discourse.

This part I don’t like, though I realize it’s tricky. So if I start out interacting with a forum by e-mail only, then decide to register later, my initial posts will belong to the system user?

Couldn’t we just create a new user based on the e-mail? I realise this would not be desirable for many, but if you have a whitelist the chance of spammer accounts would be minimal.

Actually, thinking of the Support-Forum and CRM-System approach (which is a little how we are using it), I was thinking of building a “ghost”-accounts system: accounts handled by third-parties like apps (a twitter account for e.g.) but also the mailing system, so when the same users emails twice, they are understood as one user. Like in this case, I’d love for a user, who joins later to get their emails attached to them, but as a support person I might also simply be able to look up our conversations with them and refer to one profile account (with potential meta-data). And if I answer the post, I’d also want the system to send that person behind the ghost account this answer as an email-reply.

But all of that is way out of scope for a discussion here, needs way more discussion and is by far not needed to fix the use case at hand (described above), so I went for the simpler system first. I’m open to discussing this kind of system though, too.

2 Likes

Hi all. I’m the creator of a different group discussion platform called Lumen. It was designed with email-in from the beginning. You can check it out at https://github.com/wordsandwriting/lumen if you’re looking for ideas of how to go about things.

5 Likes

Is the current feature set documented anywhere? I found the “email_in” option in settings, but how do I target a specific category for instance? And if I’m set up for replies by e-mail am I good and ready for new posts too?

Also, is it safe to rename a category after you’ve set up a mail for it?

What about moving the category? E.g. from child-category to parent category.

The ‘custom incoming email address’ is the only thing that really matters there.

@lightyear, I am quite interested in what your actual experience is with replacing Google Groups with Discourse?

I am consulting with a large, multi-billion dollar Silicon Valley chip company that is using Lithium for forums and Google Groups for mailing lists.

They are creating a Linux User Group for hardware hacking. Part of this may deal with kernel patches. They want to have a forum for discussion on “cool hardware hacks” (like 3d vision processing for autonomous planes) and a separate mailing list for driver patches to the Linux kernel or embedded Linux distribution updates. I would like to recommend that they just use Discourse. However, I have not been able to compare the feature set of Discourse as a mailing list replacement for GoogleGroups. I would be interested in your experience.

There is some connection to their GitHub repository and I imagine that the mailing list will have code snippets. If I look at the Linux Kernel Mailing List, they appear to actually use the mailing list to exchange some patches. Is that correct?

The Linux kernel mailing list is primarily used for sending around patches to the kernel, and discussion of said patches.

I also noticed that the incoming emails from unregistered users (the ones that get posted by @system) show the e-mail address of the poster. I find this very worrisome. I could be sending a ticket to support, completely oblivious to the fact that my e-mail address is being made available on a public forum.

Then the support of your company sucks, obviously. After the first post they’d be very aware that these email addresses are included (how else should you know how to contact them otherwise) and I hope they either disable the feature or move it into a private category (which is what we are doing).

@codetricity
We have only little emails on the mailing lists and they are primary for discussions, support and other non-code-related stuff. The kernel ML is indeed mostly about sending around patches and if you want to have something like this I’d not recommend discourse ATM. The support for real-email-patches in GIT and other tools is high, while discourse poorly manages patches-like files, merge-requests and alike at the moment. That said, I’d personally recommend Github and Pull-Requests (or your custom hosted Gitlab) alongside a discourse instance – like done here on meta. With this combination (and an IRC-Channel for direct communication and support), I don’t see the need for any noisy Mailing list (or crappy google groups).

They should absolutely be in a private category. I went at it from the wrong angle. All I really meant to say was that there ought to be a prominent warning in the Discourse settings panel (been wanting tooltips for a while) for things like these, because there is a big difference between e-mails sent from knowns and unknowns.

So, this is also kind of a hack to allow people to post to a private category that they do not have access to…

The only thing that makes it less ‘email like’ is that you can’t reply directly to them, and you can’t add new people to the thread.

When you get a message in this private category, what do you do? Open your email and reply to the person? Or create a new private message to them?

It’d be interesting to eventually have a separate permissions for “Topic Creator” and “Mentioned Users” so that a conversation can be ‘opened up’ to certain users within a private category.

Wow, great recommendation. This is what I’ll recommend. I know they already have a GitHub system open to the public to manage stuff like driver patches. I actually think that most of the patches are coming from in-house staff at the moment. However, they hope to change that mix with training, developer events, a new forum, and project highlights. They are also coming out with new dev tools. Great point about IRC. I already suggested that they set something up on freenode.net after we clear the first hurdle of the forum.

I think I can just show the guy that hired us the interaction between GitHub and the dev community here on this forum. With the blowout number of stars that Discourse gets on GitHub, there’s clear proof that this method works. Follow the leaders. Much thanks for the info.

1 Like