We are happy to introduce an alpha version of IMAP support for group inboxes. This feature adds the ability to enter IMAP credentials and settings on a per-group basis to sync group inboxes with an email inbox.
This is an alpha-stage feature, and as such it is not 100% complete and polished. Use at your own risk!
- Your group can hit the ground running with using Discourse inboxes, because all your existing mail will be synced!
- Your group can have their messages synced between Discourse and your email provider so there is no single point of failure. People can reply from the email account or from within Discourse and everything will sync up
- You will no longer need to set up crazy forwarding rules from your email provider to your Discourse incoming mailbox.
- Email labels will sync with Discourse tags so things can be nice and organized
- You will be replying to people who email your group from the email address you set up, even if you reply from Discourse. No more confusion!
- All incoming and outgoing will be synced between the IMAP server and Discourse, with the appropriate topics and post replies created based on parsed emails. Replies can be created from Discourse or from the IMAP server and everything will sync up!
- Emails can be replied to from the Discourse group inbox OR from the email account.
- Tags applied to the Discourse topic will be created as labels and applied to the email on the IMAP server (this is provider-specific).
- Emails archived on the IMAP server will be archived in the group inbox.
- Group private message topics that are archived on Discourse will be archived on the IMAP server.
- Emails deleted on the IMAP server will delete the topic in the group inbox.
- Group private message topics that are deleted on Discourse will be deleted on the IMAP server.
- Emails sent by groups with this feature enabled will have their reply-to address set to the same email username set up in the IMAP settings.
First of all, there are some site settings you must configure before you can set up this functionality on a group.
enable smtpboth need to be enabled. IMAP is used for syncing with your mail server, and SMTP is used to send emails from your mail server.
allow staff to tag pms- Must both be enabled, as label sync applies tags to PMs.
enable imap write- Must be enabled if you want changes made in Discourse to reflect in your mail server (for example tags, archiving topics, and deleting topics).
enable imap idle- Lets us receive live updates from your mail server IF your mail server supports IDLE. This greatly speeds up the reflection of changes in your email provider to Discourse. You should turn this on (Gmail supports it, for example). If you want all the gory details you can read about them in the RFC for IDLE.
imap polling period mins- If
IDLEis not supported, this is the number of minutes we wait before checking the IMAP server for changes. It is also the time we wait between sending changes to the IMAP server from Discourse if
enable imap writeis enabled. The minimum is 1 minute.
These settings can be left at their defaults, and tweaked if you are finding issues with the syncing:
imap polling old emails- The maximum number of old emails (processed) to be updated every time an IMAP box is polled (0 for all).
imap polling new emails- The maximum number of new emails (unprocessed) to be updated every time an IMAP box is polled.
imap batch import email- The minimum number of new emails that trigger import mode (disables post alerts).
If you want labels/tags to sync with the IMAP server you need to enable the
tagging enabled and
allow staff to tag pms site settings enabled.
Next, you need to go into the group that you want to sync with your IMAP server and fill in the settings.
These settings will be provider-specific, see below for more details. Once you have entered the settings and credentials click “Save Changes” and we will validate the credentials against the IMAP provider’s servers. If successful, the list of mailboxes will be populated, and you need to choose which one you want to sync (guidance on this is also provider-specific):
If the credential validation was not successful, an error message will be shown on the page. To be clear, the SMTP server entered will be used to send emails on behalf of the account entered under username & password, not the main Discourse SMTP configured for bulk emails.
Finally, inside your
app.yml file you will need to add
DISCOURSE_ENABLE_EMAIL_SYNC_DEMON: true inside the
env section and run
./launcher rebuild. This will enable the background worker to start up and start syncing emails!
- You must generate an app password, otherwise you need to turn on “Less secure app access” which Google is getting rid of at some point anyway. Use this password instead of your Gmail account password in the IMAP settings. See Sign in using app passwords - Gmail Help for more details.
- Make sure you use these settings for SMTP and IMAP:
- It is strongly advised (soon to be a UI feature) that you only select the “[Gmail]/All Mail” mailbox to sync with.
- We do not immediately delete emails in Gmail, only move them to the trash inbox, when the topic is deleted in Discourse. The “30 day” delete functionality inside Gmail will take over from there.
- Tags applied to Discourse topics will create Labels in Gmail, and apply them to email threads. Labels in Gmail are also IMAP mailboxes!
This is an alpha-stage feature, and as such it is not 100% complete and polished. Therefore, the following limitations currently apply:
- Only Gmail is currently supported as an IMAP provider. We have some generic IMAP functionality but it is not guaranteed to work. Outlook online is our next big target for support.
- Resurrecting deleted emails from Gmail’s Trash and reflecting the state in Discourse is currently flaky.
- Changing the sync mailbox after emails have already synced is not advised and can lead to a lot of weird problems.
- Only one mailbox can be synced per group.
- Group-to-group communication is uncharted territory and will not work well (e.g. emailing email@example.com from firstname.lastname@example.org if each of those emails is set up for a different group).
There may also be other subtle gotchas and rough edges as this is an alpha feature.
Feedback & Roadmap
We would love feedback from anyone using this feature with Gmail. I will be the one working on hammering out any bugs/issues. Debugging is a little tricky right now but I am working on that!
The next things I will work on:
- Improve debugging to make it easier to view IMAP logs and figure out things that go wrong.
- Improve the group email interface to limit the mailboxes that can be selected from Gmail and to also discourage/disallow changing mailboxes.
- Validation to prevent using the same IMAP details for multiple groups.
- Possibly improvements to how the group IMAP settings are stored and a better UX for validating credentials.
- Outlook support.
Currently while we work out the kinks this feature is not available to our hosted customers.
Both @udan11 and @joffreyjaffeux are the primary contributors of this huge feature which has been worked on for a long time. Without their amazing work I would not have been able to bring the feature to this point and make this announcement .