Mirroring a read-only mailing list in Discourse

[Discourse]

This guide explains how to create a read-only mirror of a mailing list using Discourse. It covers the necessary site settings, category configuration, and subscription process.

Required user level: Administrator

Discourse can function as an archive for a mailing list while allowing registered users to utilize most of its features. This guide will walk you through the process of setting up your own mailing list mirror.

See this in our action on the Ruby-Talk mailing list mirror.

Summary

• Configure site settings for email handling
• Create or modify a category for the mailing list
• Subscribe to the mailing list
• Understand how the mirroring process works

The following guide uses two email addresses as placeholders. You need to replace them with your actual addresses.

discourse@example.com is address for incoming emails on your Discourse
mailing-list@foo.test is the submission address of the mailing list^[1]

Configuring site settings

1. If you’re not on a hosting plan that pre-configures incoming email, set up POP3 polling or manual polling via API for the account associated with your Discourse email address (e.g., discourse@example.com).

2. Enable the following site settings:

   • email_in: Allow incoming emails
   • enable_staged_users: Enable staged users (crucial for the mirroring process)

3. Optional settings:

   • Set authorized_extensions to * or add specific file extensions to allow the attachment types you need
   • Enable always_show_trimmed_content to allow all users to see trimmed content (hidden behind a button)
   • Enable display_name_on_posts and disable prioritize_username_in_ux for name display preferences

Creating or modifying a category

1. Create a new category or edit an existing one for the mailing list posts.

2. Configure category permissions:

   • Click the “Security” tab, then “Edit Permissions”
   • Remove all existing permissions
   • Add “everyone” with “See” permission
     
     

When you’re done, it should look like this:

3. Configure category settings:

   • Click the “Settings” tab
   • Enable “Accept emails from anonymous users with no accounts”
   • Set “Custom incoming email address” to the mailing list’s email address (e.g., mailing-list@foo.test)
   • Enable “Category mirrors mailing list”
     
     

4. Save the category settings.

Subscribing to the mailing list

1. Subscribe to the mailing list using your Discourse email address (e.g., discourse@example.com).

2. If the mailing list has a web interface, ensure the following settings:

   • Mail delivery is enabled
   • Digest mode is off
   • Password reminder emails are disabled

For Mailman 2, make sure:

• “Mail delivery” is “Enabled”
• “Set Digest Mode” is “Off”
• “Get password reminder email for this list?” is set to “No”

How it works

Enabling “Category mirrors a mailing list” changes how incoming emails are processed for that specific category:

• Auto-generated emails are allowed, even when block_auto_generated_emails is enabled
• All users can post via email, ignoring the email_in_min_trust setting
• Incoming emails use the Message-ID to find related posts, regardless of the find_related_post_with_key setting
• Markdown interpretation is limited to code blocks enclosed in triple backticks
• Rejection emails are not sent for mailing list posts

FAQs

Q: Can I mirror multiple mailing lists?
A: Yes, create a separate category for each mailing list and configure them as described above.

Q: Can users reply to notifications and post to the mailing list?
A: No, replies to notifications are sent to Discourse, not the original mailing list. Users cannot post directly to the mailing list through Discourse.

Q: How can I prevent users from posting to the mirrored category via email?
A: While the category allows email posting, you can keep the incoming email address private to prevent unauthorized posting.

---

1. i.e. people send to this address to post to the list, and it also appears on the To:/CC: lines of emails sent by the list to members ↩︎

Last edited by @supermathie 2024-10-18T20:01:40Z

Check document

Perform check on document:

[mattdm (Matthew Miller)]

@gerhard

• Every user can post within that category via email. The value of email_in_min_trust is ignored.

Does this mean “if any user knows the incoming email address, they can post to this category”? This can be prevented by not telling anyone that address, right?

But, presumably if someone gets a notification by email, they can reply. Is that reply sent to Discourse as a post, or to the mailing list?

[gerhard (Gerhard Schlager)]

Those are good questions.

@mattdm

Does this mean “if any user knows the incoming email address, they can post to this category”? This can be prevented by not telling anyone that address , right?

Yes, I think it’s possible if they know the incoming email address and spoof the From address so that it matches the mailing lists address.

@mattdm

But, presumably if someone gets a notification by email, they can reply.

I want to say no, but I’m not 100% sure on this one. So, maybe. Please give it a try and let me know. It’s possible that it also requires spoofing the sender address.

@mattdm

Is that reply sent to Discourse as a post, or to the mailing list?

It’s definitely sent to Discourse.

[pfaffman (Jay Pfaffman)]

I have a site that’s getting a Google group forwarded to a category with accept emails from anonymous and mirrors a mailing list are set but it’s still rejecting mail with Email::Receiver::UserNotFoundError

Is there something else that could trigger this?

[RGJ (Richard - Communiteq)]

Did you enable staged users?

[pfaffman (Jay Pfaffman)]

No. Staged users was off. So I guess it must be on for it to work? (Makes sense that it would…) So maybe there should be a check for that when you try to enable it?

[RGJ (Richard - Communiteq)]

@pfaffman

So I guess it must be on for it to work?

I’m not sure, but it’s my best guess.

EDIT there is only one place where this error is raised and it looks like this:

raise UserNotFoundError unless SiteSetting.enable_staged_users

[mattdm (Matthew Miller)]

@gerhard

@mattdm

Is that reply sent to Discourse as a post, or to the mailing list?

It’s definitely sent to Discourse.

I’ll have to try it out, but my assumption is that quite a few people will try to reply. It might be nice to have one or more of these options:

1. Don’t send mail even if someone is watching this category. Possibly give them a notification (or a one-time message) saying that if they want email delivery of this list, they should {do whatever steps it takes to sign up for the list directly}.

2. Let the site admin define the reply-to address when mail is sent from a mailing-list-mirror category. (On a per category basis, of course.) Then, users could subscribe to that list without delivery enabled, get notifications from Discourse, and if they reply, that reply would actually go to the original list.

3. A “divert replies to category” option, where when someone replies, a linked topic is created in a different category. For example, category Announce List is a mailing list mirror, and then Discussion About The Announcements for replies. The title needs some workshopping.

[codinghorror (Jeff Atwood)]

@RGJ

Did you enable staged users?

Does the guide make this clear @gerhard?

[gerhard (Gerhard Schlager)]

No, it didn’t, but now it does.

@gerhard

Make sure that enable_staged_users is enabled.

[mattdm (Matthew Miller)]

Thanks! This previously confused me, because I assumed that the special handling here also special-handled staged users somehow (possibly only creating them for posts to this category?).

[JonLewit]

Can this be made bi-directional by creating an account on Discourse that is a member of the Google Group, Then posts to forums would be sent to GG as if it was just another user on the email list (we would need to reject all incoming mail from that user to avoid a loop). I ask this as a way to run Discourse and GG in parallel as users transition over.