Encrypted personal messages

[sam (Sam Saffron)]

We would like to commission a plugin that enables private, encrypted messaging between end-users.

This plugin now exists and is officially supported, see Discourse Encrypt (for Private Messages)

As it stands there are very few venues online for secure, private, long form communication. The existing tooling is usually quite arcane and requires extreme technical skills to use. As a result majority of “private” communication is done “short form” in Telegram, Skype or WhatsApp. Additionally, this mode of mass communication is not properly auditable as it builds on closed source tooling and often closed source protocols.

We would like to build a highly usable and auditable solution for encrypted messaging in a dedicated Discourse plugin.

To achieve this we would like to build on a few principle

1. Build the solution on top of Web Crypto API

2. Have the server store an encrypted private key and public key per user

3. Have the server store an encrypted conversation key per participant in private messages (encrypted using end users private key)

4. Have all markdown and titles stored in the raw encrypted form per message (encrypted using conversation key)

5. Accept the limitation that the server knows who talked to who and when (we are only protecting the “what”). Accept the limitation that search will not work for this content.

Proposed end user experience

Jane heads to her user page and clicks the “enable encrypted messaging” button. Once clicked she is prompted to enter a secret, the UI clearly explains that if she forgets this secret she will NEVER have access to her encrypted messages anymore.

Once encrypted messaging is enabled a new [ ] will appear in the create message UI with the text [ ] encrypt message. This will only be clickable if all the recipients have “enable encrypted messaging” turned on, if some do not have public keys, when attempting to click it we will say: “Sorry but some of the participants do not have encrypted messaging enabled”.

Jane decides she wants to talk privately to Pete, she adds him to the participant list and clicks encrypt message and sends a message to Pete.

Pete receives a notification saying that Jane sent him an encrypted message. There is no title, just information that an encrypted message exists and links to the message (both in email and web notification)

If Pete is on the original device where he enabled encrypted messaging his secret passphrase was already entered and the credential already stored in index db. If Pete is on a new device he will be prompted to enter his secret passphrase to turn on encrypted messages.

The UI clearly displays an overlay or visual hint to show that the message is encrypted.

Once communication starts between Jane and Pete, either of them can invite new people to the message provided they have public keys.

Pete or Jane may change the secret passphrase whenever they wish, if they do so the private key will be re-encrypted using the new passphrase and shipped to the server. (For V1 we will not allow changes to the actual private key, just passphrase)

Technical details

At no point in time will any non encrypted private conversation or private credentials be sent to the server. The only trust that will be placed on the server is that nobody messed with the JavaScript payload sent to the client.

Private and public key pairs will be generated 100% on the client side and then the private key will be encrypted using AES symmetric encryption prior to handing it to the server for safekeeping, we will stretch the passphrase using randomly generated salt stored on the server. Key stretching will be done on the client using PBKDF2 (available in web crypto api). The private/public key pair will be stored on the client in IndexDB using a non exportable CryptoKey object. Exportable CryptoKey data will be removed from memory as soon as possible.

Encrypted private key and public key will be stored in a user custom fields. Only current_user will be allowed to read/write the encrypted private key, all logged on users will be allowed to read public keys for all users.

Conversation keys will be generated client side on conversation initiation or invite and encrypted using all the public keys involved in the conversation. This data will be stored in a dedicated table (user_id, topic_id, encrypted_conversation_key), this row will be creatable by any user in the conversation but only readable by current_user == user_id.

Special care will be taken never to leak out unencrypted data to the server, this includes ensuring drafts do not get stored unencrypted.

To avoid a large class of errors new DOM elements (with new ids) will be used for the composer edit and preview. Current .d-editor-input will only ever contain encrypted blobs and traditional preview will be disabled. (meaning 2 textareas will be at play, encrypted and unencrypted) Encryption will be heavily debounced to avoid a class of performance issues.

For V1 (and the scope of this work item) we are happy to eliminate certain features, including upload support, oneboxing of encrypted content and other complex edge cases.

We are fine to decrypt / bake and pass HTML through whitelister on-the-fly when rendering this private content.

Documentation requirements

Prior to implementation a very detailed spec will be created including web mockups and most importantly detailed security overview.

Testing requirements

This plugin must include a comprehensive set of client and server based unit/integration tests. If you take this project on, write tests early, do not wait until the project is finished to retrofit tests.

Concerns

Many of the concerns in: https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2011/august/javascript-cryptography-considered-harmful/ are no longer relevant given the Web Crypto API (true random byte generation, private key store and the list goes on). The only major vector that exists is the server sending the client malicious JavaScript. Short term users will simply be expected to trust the server same as they trust 1password, lastpass and other web security UIs. Longer term (v3/v4 - next year or after) we can consider shipping a browser extension that places a padlock on sites where all the JS payloads stability hashes are well known.

WebCrypto API is still relatively young but quite widely supported, the plugin should work across all our supported browsers.

Simple rough mockups

* when starting a message user can opt to encrypt the conversation, a warning will be displayed if any of the participants are missing public keys.

* users can enable encrypted messages by clicking that button and working through a UI that generates the key pair.

If encrypted messages are enabled it should simply display “You have encrypted messaging enabled”.

* Some sort of icon, or overlay will denote that a message is encrypted.

Budget

This is a very complex piece to build that required very deep integration and careful review. We expect it is extremely well tested (using both client side and server side tests). Our current budget for an MVP is $10K USD.

[DNSTARS]

If the implementation is open source with the rest of discourse I could probably get our lot to throw in £1000 or thereabouts to your budget.

[sam (Sam Saffron)]

Cool, yes, this will be open source under the MIT license, with an additional full license assignment to Discourse.

Regarding browser support, we want to make sure it works on every platform: Chrome + Safari is a minimum (so it works on iPhone / Android / Desktop), but my preference is that we hit Firefox as well. Edge is in the nice-to-have and IE11 is in the I don’t really care department, especially if it is missing crypto stuff.

[DNSTARS]

I’ll get the ball rolling now then, couple of questions though.

• Is it plausible for this project to accommodate group encrypted chat, even if up to a point?
• Would “Force every PM encrypted” option be on the cards?

[sam (Sam Saffron)]

Is it plausible for this project to accommodate group encrypted chat, even if up to a point?

Not for the initial version. Maybe for version 2 or 3, a PM between 20 users should absolutely work for V1.

Would “Force every PM encrypted” option be on the cards?

Yes this is fine as an option. Basically would disallow PM to users that don’t have encryption enabled (but still allow admins to PM them unconditionally). We would need to think this through, but I don’t want to add this to the scope of the spec quite yet.

[DNSTARS]

Thanks i’m obligated to sample an arbitrarily acceptable number of our PMs and reading people’s intimate conversations if only skimming is not only something I don’t want to do it takes time I haven’t got.

a PM between 20 users should absolutely work for V1.

That’s group chat in my context (Forgot where I am ) and will be fine. I’d expect strong support from our lot.

[sam (Sam Saffron)]

Thanks i’m obligated to sample an arbitrarily acceptable number of our PMs and reading people’s intimate conversations if only skimming is not only something I don’t want to do it takes time I haven’t got.

Keep in mind with this plugin it is technically impossible for you to read members private conversations provided they have a reasonably strong passphrase.

They are encrypted in the database and only decrypted client side. You would have to add a code exploit to your server for you to be able to swing reading encrypted PMs. Longer term (in v3 / v4) this code exploit would result in a giant red flag on the screen for people who install the “confirm my encrypted discourse conversations have not been exploited” browser plugin.

[DNSTARS]

Yes that’s ideal, i’m obligated to look at them because I can look at them.

[angus (Angus McLeod)]

This is fascinating. It would be refreshing to do something really technically challenging. But, at least for me, the cost / benefit of taking the time to master (assuming, optimistically, that I could) all the relevant pieces doesn’t make sense right now.

That said, I just did a little reading on cryptography and have a few questions / thoughts:

• Am I correct in understanding that you would need to generate a new conversation key every time you added or removed a member from the PM?

• You would need some protection or warning against using other plugins as well. I can imagine a scenario in which you install this plugin, then someone writes another plugin that exposes or stores the entered text in some way before it is encrypted.

• Wouldn’t you need to also only allow this feature on sites that force https? From the article you linked.

  You can’t simply send a single Javascript file over SSL/TLS. You have to send all the page content over SSL/TLS. Otherwise, attackers will hijack the crypto code using the least-secure connection that builds the page.

  Otherwise, in addition to the issue of the server sending malicious javascript you could also have the issue of ‘hijacking’.

• Searchable semantic encryption is possible (by no means do I mean to imply that I understand the description in that paper). I assume you’re excluding it as the cost / benefit isn’t worth it, at least for the MVP.

• Why not use the Signal Protocol? (javascript library).

ps. for whoever takes this on

• there are some useful Web Crypto API examples here, including a cool live table of what parts of the Web Crypto API are supported by the browser you’re using. You may have issues with safari.

• This article has some good advice on implementing the Web Crypto API.

[sam (Sam Saffron)]

Great questions:

@angus

Am I correct in understanding that you would need to generate a new conversation key every time you added or removed a member from the PM?

The encrypted conversation key does not need to change on add. All members of the conversation have a copy of the decrypted conversation key. So to add a member you would encrypt this conversation key using the “invitee” public key and teach the server about that.

Technically it would be correct to amend the conversation key if you are removing members from a conversation, but we can wait on v2 for that. Cause it would be a very expensive operation from the client side, entire conversation would have to be re-encrypted, or complex multi conversation key systems need to be built.

@angus

You would need some protection or warning against using other plugins as well. I can imagine a scenario in which you install this plugin, then someone writes another plugin that exposes or stores the entered text in some way before it is encrypted

Yes, the long term V3 / V4 goal here is to have a whitelist of JS integrity hashes stored in a browser extension. That is really the only way a client can know 100% that the “server is not messing with stuff”. However for V1 we trust the server and server operator not to be malicious.

@angus

Wouldn’t you need to also only allow this feature on sites that force https?

Yes, if SSL is off all bets are off. We do not expect this feature to even remotely work if SSL is off.

@angus

I assume you’re excluding it as the cost / benefit isn’t worth it, at least for the MVP.

Yes search is off the cards for now cause it is just going to be too hard to get right for the MVP.

@angus

Why not use the Signal Protocol? (javascript library).

Maybe, not against leaning on existing standards, but I suspect the Discourse integration work here is going to be enormous regardless.

[sam (Sam Saffron)]

Congrats! @dan will be taking this project! We will update here as we make progress!

[BeardMadden (Beard McMadden)]

Signed up to comment on this because I’m curious on the implementation as a developer myself.

I’m personally no cryptography pro, so forgive me if these are silly questions.

Technically it would be correct to amend the conversation key if you are removing members from a conversation, but we can wait on v2 for that. Cause it would be a very expensive operation from the client side, entire conversation would have to be re-encrypted, or complex multi conversation key systems need to be built.

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation, in this case?
Essentially I guess this would lead to a double layer of encryption - the data is encrypted client-side for true E2E encryption as described with the conversation key, and then encrypted on the way out by the server to only the public keys active in the conversation. At that point, removing a member from the conversation would mean any future messages wouldn’t be decryptable by them.

I’m not too sure on the security implications of nested encryption like that, however.

My second question is how well this would work cross-device? Does it require a private key on the device, or is that handled by the passphrase encrypted private key being stored?

[sam (Sam Saffron)]

@BeardMadden

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation , in this case?

Interesting idea the “double” encrypting. It does though add more responsibility to the server and a lot of this is about removing responsibility from the server so I am not sure it is a great idea protocol wise.

@BeardMadden

My second question is how well this would work cross-device? Does it require a private key on the device, or is that handled by the passphrase encrypted private key being stored?

Cross device is handled by retrieving the encrypted private key from the server and decrypting using the key phrase.

Longer term we can introduce more “security” modes with a super strict one when say the server never even stores the private key encrypted (like say 1 password), but short term this is out of scope.

[dan (Dan Ungureanu)]

@BeardMadden

Signed up to comment on this because I’m curious on the implementation as a developer myself.

Welcome!

@BeardMadden

I’m personally no cryptography pro, so forgive me if these are silly questions.

Those are not silly questions at all! If you want to dig deeper, I highly recommend Dan Boneh’s courses on Coursera.org.

@BeardMadden

I’m not too sure on the security implications of nested encryption like that, however.

This depends a lot on the particularities of the system you use. Check out double DES (i.e. vulnerable to meet-in-the-middle) and triple DES.

@BeardMadden

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation , in this case?
Essentially I guess this would lead to a double layer of encryption - the data is encrypted client-side for true E2E encryption as described with the conversation key, and then encrypted on the way out by the server to only the public keys active in the conversation. At that point, removing a member from the conversation would mean any future messages wouldn’t be decryptable by them.

Let’s say Eve used to be a part of a group discussion with Alice and Bob. Even if Alice or Bob kick Eve out she will still have the conversation key and theoretically, she could intercept the messages before they get to the server and decrypt them.

If the conversation key gets compromised, the safest approach is to revoke it and use a new one for new messages. Ideally, you’d probably want to re-encrypt the old messages.

[BeardMadden (Beard McMadden)]

@dan

Even if Alice or Bob kick Eve out she will still have the conversation key and theoretically, she could intercept the messages before they get to the server and decrypt them.

That makes sense to me, thanks for the explanation! (@sam too)

So adding or removing users from the conversation would involve decrypting everything client-side, revoking the conversation key and generating a new one, then re-encrypting messages before sending to the server?
Plus encrypting the new conversation key with the user passphrase and storing it.

[Alexander_Wright]

I’ve been talking to my wife about this feature. While this may raise an eyebrow at mealtime conversation in our family, she’s a security engineer for Thales eSecurity.

She’s of the opinion that the Specification needs some use cases with bad actors and how the system should respond. For example: Malicious Admin, Malicious SysOps.

This would then give rise to a security statement that could be presented to users of the system.

[sam (Sam Saffron)]

I would love it if your wife or anyone interested in this problem at her work would pop by and give feedback.

@dan has made tremendous progress here, I am testing v1 with him this week.

To answer your specific question of bad sys admins. Overall if you can commandeer the web site and deliver malicious payloads our current v1 can not offer privacy. This is by design.

Our plans for v3/v4 is to have a browser plugin that validates every single JavaScript file on the page matches a particular hash in the whitelist. That way if you would be running this plugin you would be able to disallow unrecognized JS payloads.

Overall the underlying design means that passwords can not be fished out after the fact even if the server is malicious, but it can trick the client if it wishes to decrypt arbitrary payloads. Native browser plugins or a more refined web standard longer term will help eliminate this