Create a read-only mailing list mirror

documentation admins
1

This guide describes how you can use Discourse for creating a read-only mirror of a mailing list.
Discourse acts as an archive for the mailing list and allows registered users to use most of its features.

Do you want to see it in action? Take a look at our mirror of the Ruby-Talk mailing list.

Are you ready to setup your own mailing list mirror? Here we go!

The following guide uses two email addresses as placeholders. You need to replace them with your actual addresses.

:incoming_envelope: discourse@example.com is address for incoming emails on your Discourse
:postbox: mailing-list@foo.test is the address where emails get sent by the mailing list

Site Settings

You can skip the first step if you are on a hosting plan that pre-configures your incoming email.

  1. Make sure you configured POP3 polling or manual polling via API, so that it polls the account associated with :incoming_envelope: discourse@example.com

  2. Enable email_in to allow incoming emails.

  3. Make sure that enable_staged_users is enabled.

  4. Optional: Allow all incoming attachments by setting authorized_extensions to * or add all the file extensions you want to allow.

  5. Optional: Enable always_show_trimmed_content in order to allow every user to see trimmed content. It will be hidden behind a image button.

  6. Optional: Enable display_name_on_posts and disable prioritize_username_in_ux.

Category

You need to create a new category or edit an existing one which will be used for posting all emails that get sent to the mailing list.

  1. First, click on the Security tab and then click on the Edit Permissions button.

    image

  2. Remove all existing permissions and add “everyone” with the permission “See”. When you are done, it should look like this:

    image

  3. Next, click on the Settings tab. There should be the following three fields. If there aren’t, make sure you have enabled the email_in site setting.

    • “Accept emails from anonymous users with no accounts” – Enable this to allow the creation of staged users.

    • “Custom incoming email address” – Enter the :postbox: email address of the mailing list. Look at the To: header of one of the emails sent by the mailing list, if you are unsure what to use here.

    • “Category mirrors mailing list” – You must enable this option. It activates all the special handling for mailing lists.

    image
    image548Ă—608 39.8 KB

  4. Click on New Category or Save Category to create the new category or save your changes.

Subscribe to mailing list

Now it’s time to subscribe to the mailing list using :incoming_envelope: discourse@example.com.

If the mailing list provides a web interface, take a look at the settings and make sure that you will receive only messages that should appear in the mirror category. For Mailman 2 it’s important that

  • “Mail delivery” is “Enabled”
  • “Set Digest Mode” is “Off”
  • “Get password reminder email for this list?” is set to “No”

:tada: That’s it! Now it’s time to wait for the first email to arrive in your new mailing list mirror.

FAQ

I want to mirror multiple mailing lists. Is that possible?
Sure! All you need to do is create a category for each mailing list and configure it like described above.

How does all this work?

Enabling “Category mirrors a mailing list” changes how incoming emails are processed for that specific category:

  • Emails sent by a mailing list are usually marked as auto-generated. This setting allows incoming emails to be auto-generated even when block_auto_generated_emails is enabled in site settings.

  • Every user can post within that category via email. The value of email_in_min_trust is ignored.

  • It’s a good idea to make the category read-only to prevent users from posting and expecting that their post will appear on the mailing list. However, everyone can create new posts by sending an email to the mailing list.

  • Usually find_related_post_with_key is enabled in the site settings. Disabling it for the whole site is not recommended, since it allows user impersonation based on email address. Incoming emails that were sent to the mailing list always use the email’s Message-ID to find related posts and disregard the value of that site setting.

  • Normally Discourse expects incoming emails to contain text formatted as Markdown. Mailing list users are usually unaware of that requirement, so Discourse doesn’t interpret any Markdown (except for code blocks enclosed in three backticks) or HTML within plain text emails and posts them with the original formatting intact.

  • Discourse usually notifies the sender when it rejects an email. That doesn’t make sense for mailing lists and probably would annoy the mailing list user, so sending of rejection emails is disabled.

28 Likes
23

Does this mean “if any user knows the incoming email address, they can post to this category”? This can be prevented by not telling anyone that address, right?

But, presumably if someone gets a notification by email, they can reply. Is that reply sent to Discourse as a post, or to the mailing list?

2 Likes
24

Those are good questions.

Yes, I think it’s possible if they know the incoming email address and spoof the From address so that it matches the mailing lists address.

I want to say no, but I’m not 100% sure on this one. So, maybe. Please give it a try and let me know. It’s possible that it also requires spoofing the sender address.

It’s definitely sent to Discourse.

3 Likes
25

I have a site that’s getting a Google group forwarded to a category with accept emails from anonymous and mirrors a mailing list are set but it’s still rejecting mail with Email::Receiver::UserNotFoundError

Is there something else that could trigger this?

2 Likes
26

Did you enable staged users?

4 Likes
27

No. Staged users was off. So I guess it must be on for it to work? (Makes sense that it would…) So maybe there should be a check for that when you try to enable it?

2 Likes
28

I’m not sure, but it’s my best guess.

EDIT there is only one place where this error is raised and it looks like this:

raise UserNotFoundError unless SiteSetting.enable_staged_users

4 Likes
29

I’ll have to try it out, but my assumption is that quite a few people will try to reply. It might be nice to have one or more of these options:

  1. Don’t send mail even if someone is watching this category. Possibly give them a notification (or a one-time message) saying that if they want email delivery of this list, they should {do whatever steps it takes to sign up for the list directly}.

  2. Let the site admin define the reply-to address when mail is sent from a mailing-list-mirror category. (On a per category basis, of course.) Then, users could subscribe to that list without delivery enabled, get notifications from Discourse, and if they reply, that reply would actually go to the original list.

  3. A “divert replies to category” option, where when someone replies, a linked topic is created in a different category. For example, category Announce List is a mailing list mirror, and then Discussion About The Announcements for replies. The title needs some workshopping. :slight_smile:

2 Likes
30

Does the guide make this clear @gerhard?

3 Likes
31

No, it didn’t, but now it does.

5 Likes
32

Thanks! This previously confused me, because I assumed that the special handling here also special-handled staged users somehow (possibly only creating them for posts to this category?).

4 Likes