"Email all Posts" option to allow email client interaction

(Deferred Procrastination) #1

I was looking at Discourse as a possible replacement to a Google Group for a makerspace. While the web interface of Discourse is much better for viewing posts (especially for new users) and mixed threading is great, some of our power users prefer the non-web email/newsgroup interface; so items come to them, instead of having to visit each site they follow.

For a more recent example of that behaviour, it’s like having a twitter client on your phone/computer, rather than going to twitter’s site to check what’s happening every day.

I know there are email digests and reply via email support currently. Is there any opportunity to add “email all posts” as an option that sends reply via email messages?

Can Discourse replace mailing lists?
Support for migration of a Yahoo! group
(Brentley Jones) #2

I believe it’s in the works, please see @codinghorror’s post over at BoingBoing which covers some of the plans:

(Jeff Atwood) #3

Yes this is all planned. As you can see from What is Discourse? | Discourse - Civilized Discussion displacing mailing lists is a primary goal of Discourse.

The main thing I am wanting at the moment is a way to get emailed when there is a new post in certain categories on some Discourse forums. That’ll probably be next, but I can’t say exactly when.

(Deferred Procrastination) #4

Yes, the FAQ reference was one of the reasons I thought it might be part of Discourse’s remit.

I suppose that if categories are fairly stable (new ones aren’t normally created) then the subscribing to new posts in all current categories amounts to almost the same thing.

(Jeff Atwood) #5

Categories are only createable by staff. There is some discussion of trust level 4 users eventually being able to do this, though.

(Benjamin Kampmann) #6

Is there any way one could help with this?

We are testing discourse now as our main communication tool at OpenTechSchool and that feature (or a “email every new post” at least) is what keeps engagement low and makes it impossible for us to move away from mailing lists. And as I am (and many others in the community are, too) a coder, I was wondering how we could help with this.

Is there already a plan on what this is supposed to look like and where and how it should be implemented? Maybe even some pointers on where to get started digging in the code to make it happen?

I’d love to help here.

(Sam Saffron) #7

We would love the help.

My suggestion would be … allow users to specify that they “watch all topics” by default, the change runs fairly deep.

  1. Add: automatically watch all topics, checkbox on the user page (site setting to enable)
  2. Add: disable context from notification emails on user page (site setting to enable)
  3. Ensure all watched topics result in emails for every new post
  4. Add: Send me emails while I am active on the site (site setting to enable)

(Benjamin Kampmann) #8

Just started implementing the feature as you were saying, @sam, but I feel it kinda is weird or I didn’t understand it entirely.

I added that checkbox and tried to find where to add the other things when I realised that feels kinda odd from a user perspective. Just thinking of wanting to replace a mailing list and what kind of features would make sense for that and tracking everything doesn’t sound like the approach I’d take instead I would suggest to make the following two changes in the preferences:

  • in the section for emails add a checkbox “Inform me about all new posts/topics”
  • with a select box similar to the one of the digest with the options:
    • “Include new topics in digest email” (default)
    • “Send all new topics immediately”
    • “Send all new topics and posts immediately”

This allows the user to have a full mailing list like behavior (last option), get informed about any new topic started (already much smarter than any ML out there and less noisy) or just make sure they are in your daily/weekly digest, too (Google Groups has this kind of support).

That kinda feels a bit more straight forward from a user perspective. What do you think?

(Jeff Atwood) #9

It is definitely more related to the digest email. Note that the digest email does (as of about a week ago) include all the new TOPICS (up to a threshold, we can’t send an email with ten thousand links in it) on the site.

However you will only get the digest email if you haven’t been on the site at all in a specified interval.

Probably the first thing to do is enable a setting that forces you to get the digest email even if you have been on the site. Then you’ll have a list of all new TOPICS in the time interval.

(Sam Saffron) #10

I am fine with the box that ties this to digests. Eg:

“Include new topics in digest email” (default)
“Send all new topics immediately”

However the “Send all new topics and posts immediately” detached from watching/tracking worries me.

It means we are introducing a second way of watching topics that is orthogonal to the box at the bottom of posts. I don’t like that.

(Jeff Atwood) #11

I was just referring to topics, sorry – this will give you a list of all new topics in the time frame. I really think some form of that should be sufficient for now. And as previously mentioned, this already exists… I can forward you a digest email if you haven’t seen it yet, but all new TOPICS are present in that email at the bottom, and have been for over a week.

You could also enable a per-user setting to force you to watch every new topic on the site, or a category. That’d be… pretty god damn rough, and probably a bad idea for 99.99% of the public though. It’s like implementing a “blow up the whole world” button and putting it in a public park where children play. So… yeah.

(Sam Saffron) #12

Sorry, my post was not complete at the time of posting (it actually was but somehow the markdown engine ate half my reply or something)

(Sam Saffron) #13

Completing my previous post I suggest the following settings:

[x] Include previous posts as context in notification emails
[ ] Send me digest emails even if I am present on the site
[ ] Send me notification emails even if I am present on the site
[ ] Send me an email as soon as a new topic is created

Default notification level for new topics

  • Always watch new topics
  • Always track new topics
  • Regular tracking level

Copy needs to be improved, but I think these settings cover what we need.

Regarding the “include all new topics in digest email”, I don’t think we want that setting, we should simply always include new topics up to a threshold

(Benjamin Kampmann) #14

Hey @sam,

I’ve made a pull-request implementing the features as discussed here. Just a small difference: I made the autotracking only available if you also ask for emails for any new topic, because it feels otherwise it doesn’t make much sense.

Looking forward to hearing about the patchset from you.

(Dave McClure) #15

This is pretty exciting stuff.

Any thoughts on a feature to ‘Create a new topic via email’. That would seem to be the one missing piece after this to completely replace mailing lists. The part needing some design / thought would be how these topics get categorized.

(Jeff Atwood) #16

This is great! Such a big PR is kind of a concern though, I think we failed to communicate the smaller steps we’d like to see towards the goal first:

  1. Normally we only send digest emails to users after a period of inactivity. Enable a setting that lets people get digest emails even if they are active on the site. The digest emails already contain a list of all new topics (within reason, we can’t put ten thousand links in an email).

  2. Let people opt in to getting notified of new topics via email on a per-category basis. So for example if I decide I like the “politics” category, but I don’t come to the site very often, I will get an email letting me know about the new politics topic someone created. This also means you’d have to set those topics set to “watching” by default so you will get emailed all new posts in those new politics topics as they arrive, also.

  3. We absolutely need to add a general “how often can we email you” threshold as a user setting to group emails by. I would object to getting 10 emails a day on new topics in a specific category, but I would not object to one email per day (or per week, or whatever) that contained a summary of the 10 new topics that occurred in that day. If you do want an email every time a new topic arrives, you can set it to “immediately” versus “daily”.

This is also complex because it means you need a way to stop watching these topics via email, otherwise the minute you say “email me every new politics topic” you’re signing up to get emailed non-stop by every new politics topic, forever, and every new post in those politics topics, too!

My apologies, when we talk about difficult, large stuff like this the implied assumption is that it is so daunting that nothing will be contributed so there’s no risk – but in this case you did the work which is awesome :), but it also means the spec we provided was kind of… incomplete and maybe even wrong. That’s our bad.

(Benjamin Kampmann) #17


not sure I can follow entirely, correct me, when I’m wrong.

Not sure what you are trying to tell me here. I found that bit, too and it was a little weird because “present on site” isn’t really defined properly. Specifically I stumbled about this when trying to adapt the digest email at Implement Email all posts option by gnunicorn · Pull Request #1426 · discourse/discourse · GitHub - doesn’t that define the “minimum-date” to look for new topics just since the last email was send? So, like if I am watching a topic and get a notification email about this at 5:50am, my digest would only contain new topics from after that?

I wasn’t sure about this, because it is still fuzzy to me under which circumstances that attribute is updated. But still, is there any clear definition of what is supposed to be expected in the digest? Are you referring to that? Aside from that, I didn’t change the digest email and I know it got changed meanwhile and I like that new one much more.

I am also following that conversation and I wanted to implement this next. The main problem for me with that was only that within that thread it wasn’t clear yet, where you were supposed to do this setting - UI-wise. Though I would have a proposal, I didn’t have time to mock it up and discuss it with the group yet, so I settled for having the global switch first and implement this as soon as those details are figure out.

Oh, this feels awry. If there is an “urgent” conversation going on, you would stop sending emails though the person is watching because you reached a threshold of x-emails-per-day giving the person the false believe of no further communication happening? That sounds like a very bad idea. Instead I was thinking of something you are referring to here:

Aside from the above mentioned next step another one is adding the change-watch-level-feature as part of the “unsubscribe-footer” in every email. So if you receive two emails about a conversation you don’t want to follow any longer, it is just a link-click away to opt-out (preferably without having to login in case one isn’t at the moment). This feature alone would make this already much better than any existing mailing list solution as you can opt-out on a per-topic-basis, reducing the noise very gracefully.

Well, on one side you are saying this is already a big PR, which is also what I felt and why I submitted it without having added those two features yet but then you are saying it is still missing stuff. Now I am confused how to continue from here. I gladly can implement those two features, too if you feel this is incomplete otherwise, which would make the PR bigger though.

But I’d gladly do that, as long as someone already looks over what I have done so far to give me hints and tips if I overlooked stuff or you guys want to have stuff done differently. Just so that I know I’m on the right track. I don’t like the idea of blowing this up even more just to have to rewrite it at the end, it’s my first contribution to this project after all :wink: .

(Sam Saffron) #18

I think the key here is to break this PR down into smaller bits. The big kaboodle is a bit hard to digest and we need to work on one bit at a time.

I think we need to figure this one out this is critical:


Additionally, we can look a single PR for a light weight “Email me all new topics” option per user with a site setting to enable it.

Changes to default tracking levels are a separate, more controversial PR, I am not sure @codinghorror agrees with giving users this fidelity globally. Even though there is little contention that we need that on the local category level.

Can you try breaking the PR into multiple PRs so we can work on each piece in the puzzle individually?

(note our user table is getting way too wide, we probably need a user_settings table)

(Benjamin Kampmann) #19

Sure, could do that. As you can see I have separated commits for all the features, so that would be easy to split up. The only thing I didn’t split up is adding the whole settings, as this would mean editing the same line in a bunch of files, so merging this back together in the end would be a pain.

Let me split out the “email everything” option and the “don’t include the context” as they seem the most logical to me and do separated pull request for them. The other two features are more intertwined and are referring to the watch-thing you are talking about.

(David García-Navas) #20

Sorry for the necromancy :skull:

Is there any progress in this functionality until now?

For me, it would be fine to have an option where the user could indicate that he wants to receive an email for all that occurs on a Discourse instance.