加密的个人消息

[sam (Sam Saffron)]

我们希望委托开发一款插件，实现终端用户之间的私密、加密消息通信。

该插件现已存在并得到官方支持，详见 Discourse Encrypt (deprecated)

目前，在线安全、私密且适合长文交流的渠道非常有限。现有的工具通常晦涩难懂，需要极高的技术技能才能使用。因此，大多数“私密”交流仅限于在 Telegram、Skype 或 WhatsApp 等平台上进行“短文本”沟通。此外，这种大规模通信模式由于基于闭源工具和通常也是闭源的协议，无法进行适当的审计。

我们希望构建一个高度易用且可审计的解决方案，通过专门的 Discourse 插件实现加密消息功能。

为实现这一目标，我们将遵循以下原则：

1. 基于 Web Crypto API 构建解决方案。
2. 服务器为每个用户存储一个加密的私钥和一个公钥。
3. 服务器为私密消息中的每位参与者存储一个加密的会话密钥（使用终端用户的私钥进行加密）。
4. 所有 Markdown 内容和标题均以原始加密形式存储（使用会话密钥加密）。
5. 接受以下限制：服务器知道谁与谁交谈以及何时交谈（我们仅保护“内容”本身）；接受此类内容无法被搜索的限制。

拟定的终端用户体验

Jane 进入她的用户页面并点击“启用加密消息”按钮。点击后，系统会提示她输入一个秘密短语，界面会明确说明：如果她忘记了该秘密，将永远无法再访问她的加密消息。

启用加密消息后，创建消息的界面中将出现一个新的 [ ] 复选框，文字为 [ ] 加密消息。仅当所有收件人都已启用“加密消息”且拥有公钥时，该选项才可点击。如果部分用户未启用，点击时将提示：“抱歉，部分参与者未启用加密消息”。

Jane 决定与 Pete 进行私密交谈，她将 Pete 添加到参与者列表，点击 加密消息 并向 Pete 发送消息。

Pete 会收到通知，告知 Jane 发送了一条加密消息。通知中没有标题，仅显示存在加密消息的信息，并包含指向该消息的链接（在邮件通知和网页通知中均如此）。

如果 Pete 使用的是当初启用加密消息的原始设备，他的秘密短语此前已输入，凭据已存储在 IndexedDB 中。如果 Pete 使用的是新设备，系统会提示他输入秘密短语以启用加密消息。

界面会清晰显示覆盖层或视觉提示，表明消息已加密。

一旦 Jane 和 Pete 开始通信，其中任何一方都可以邀请新用户加入对话，前提是该用户拥有公钥。

Pete 或 Jane 可随时更改秘密短语。更改后，私钥将使用新短语重新加密并发送至服务器。（V1 版本不允许更改实际的私钥，仅允许更改短语。）

技术细节

在任何情况下，非加密的私密对话内容或私密凭据都不会发送至服务器。对服务器的唯一信任在于：发送给客户端的 JavaScript 负载未被篡改。

公私钥对将 100% 在客户端生成，随后私钥将使用 AES 对称加密进行加密，再交由服务器安全存储。我们将使用随机生成的盐（存储于服务器）对短语进行扩展。密钥扩展将在客户端使用 PBKDF2（Web Crypto API 中可用）完成。公私钥对将以不可导出的 CryptoKey 对象形式存储在客户端的 IndexedDB 中。可导出的 CryptoKey 数据将尽快从内存中清除。

加密后的私钥和公钥将存储在用户的自定义字段中。仅 current_user 被允许读写加密后的私钥，所有登录用户均可读取所有用户的公钥。

会话密钥将在对话开始或邀请时于客户端生成，并使用对话中涉及的所有公钥进行加密。该数据将存储在专用表 (user_id, topic_id, encrypted_conversation_key) 中。该记录可由对话中的任何用户创建，但仅当 current_user == user_id 时方可读取。

需特别小心，确保绝不向服务器泄露未加密的数据，包括确保草稿不会以未加密形式存储。

为避免一大类错误，我们将为编辑器编辑和预览使用新的 DOM 元素（带新 ID）。当前的 .d-editor-input 将仅包含加密的数据块，传统预览功能将被禁用（即同时存在两个文本区域：加密的和未加密的）。加密操作将进行大量防抖处理，以避免性能问题。

对于 V1（以及本工作项的范围），我们可以接受移除某些功能，包括文件上传支持、加密内容的链接预览（oneboxing）以及其他复杂的边缘情况。

在渲染此类私密内容时，我们接受在运行时解密、生成并经由白名单过滤器处理 HTML。

文档要求

在实施之前，将创建一份非常详细的规范文档，包括网页线框图，最重要的是，详细的网络安全概述。

测试要求

该插件必须包含一套全面的基于客户端和服务器的单元测试/集成测试。如果您承接此项目，请尽早编写测试，切勿等到项目结束才补写测试。

关注点

https://www.nccgroup.trust/us/about-us/newsroom-and-events/blog/2011/august/javascript-cryptography-considered-harmful/ 中提到的许多担忧，鉴于 Web Crypto API（真正的随机字节生成、私钥存储等），已不再相关。目前存在的主要攻击向量是服务器向客户端发送恶意的 JavaScript。短期内，用户将被期望信任服务器，就像他们信任 1Password、LastPass 和其他 Web 安全 UI 一样。长期来看（V3/V4 - 明年或之后），我们可以考虑发布一个浏览器扩展，在所有 JS 负载的稳定性哈希值已知的网站上显示挂锁图标。

WebCrypto API 虽然相对较新，但已得到广泛支持，该插件应能在我们支持的所有浏览器上运行。

简单的粗略线框图

* 开始消息时，用户可以选择加密对话，如果任何参与者缺少公钥，将显示警告。

* 用户可以通过点击该按钮并经过生成密钥对的界面来启用加密消息。

如果启用了加密消息，界面应简单显示“您已启用加密消息”。

* 某种图标或覆盖层将标明消息已加密。

预算

这是一项构建非常复杂、需要深度集成和仔细审查的工作。我们期望该插件经过极其完善的测试（包括客户端和服务器端测试）。我们目前对 MVP（最小可行性产品）的预算为 10,000 美元。

[DNSTARS]

If the implementation is open source with the rest of discourse I could probably get our lot to throw in £1000 or thereabouts to your budget.

[sam (Sam Saffron)]

Cool, yes, this will be open source under the MIT license, with an additional full license assignment to Discourse.

Regarding browser support, we want to make sure it works on every platform: Chrome + Safari is a minimum (so it works on iPhone / Android / Desktop), but my preference is that we hit Firefox as well. Edge is in the nice-to-have and IE11 is in the I don’t really care department, especially if it is missing crypto stuff.

[DNSTARS]

I’ll get the ball rolling now then, couple of questions though.

• Is it plausible for this project to accommodate group encrypted chat, even if up to a point?
• Would “Force every PM encrypted” option be on the cards?

[sam (Sam Saffron)]

@DNSTARS

Is it plausible for this project to accommodate group encrypted chat, even if up to a point?

Not for the initial version. Maybe for version 2 or 3, a PM between 20 users should absolutely work for V1.

@DNSTARS

Would “Force every PM encrypted” option be on the cards?

Yes this is fine as an option. Basically would disallow PM to users that don’t have encryption enabled (but still allow admins to PM them unconditionally). We would need to think this through, but I don’t want to add this to the scope of the spec quite yet.

[DNSTARS]

Thanks i’m obligated to sample an arbitrarily acceptable number of our PMs and reading people’s intimate conversations if only skimming is not only something I don’t want to do it takes time I haven’t got.

a PM between 20 users should absolutely work for V1.

That’s group chat in my context (Forgot where I am ) and will be fine. I’d expect strong support from our lot.

[sam (Sam Saffron)]

@DNSTARS

Thanks i’m obligated to sample an arbitrarily acceptable number of our PMs and reading people’s intimate conversations if only skimming is not only something I don’t want to do it takes time I haven’t got.

Keep in mind with this plugin it is technically impossible for you to read members private conversations provided they have a reasonably strong passphrase.

They are encrypted in the database and only decrypted client side. You would have to add a code exploit to your server for you to be able to swing reading encrypted PMs. Longer term (in v3 / v4) this code exploit would result in a giant red flag on the screen for people who install the “confirm my encrypted discourse conversations have not been exploited” browser plugin.

[DNSTARS]

Yes that’s ideal, i’m obligated to look at them because I can look at them.

[angus (Angus McLeod)]

This is fascinating. It would be refreshing to do something really technically challenging. But, at least for me, the cost / benefit of taking the time to master (assuming, optimistically, that I could) all the relevant pieces doesn’t make sense right now.

That said, I just did a little reading on cryptography and have a few questions / thoughts:

• Am I correct in understanding that you would need to generate a new conversation key every time you added or removed a member from the PM?

• You would need some protection or warning against using other plugins as well. I can imagine a scenario in which you install this plugin, then someone writes another plugin that exposes or stores the entered text in some way before it is encrypted.

• Wouldn’t you need to also only allow this feature on sites that force https? From the article you linked.

  You can’t simply send a single Javascript file over SSL/TLS. You have to send all the page content over SSL/TLS. Otherwise, attackers will hijack the crypto code using the least-secure connection that builds the page.

  Otherwise, in addition to the issue of the server sending malicious javascript you could also have the issue of ‘hijacking’.

• Searchable semantic encryption is possible (by no means do I mean to imply that I understand the description in that paper). I assume you’re excluding it as the cost / benefit isn’t worth it, at least for the MVP.

• Why not use the Signal Protocol? (javascript library).

ps. for whoever takes this on

• there are some useful Web Crypto API examples here, including a cool live table of what parts of the Web Crypto API are supported by the browser you’re using. You may have issues with safari.

• This article has some good advice on implementing the Web Crypto API.

[sam (Sam Saffron)]

Great questions:

@angus

Am I correct in understanding that you would need to generate a new conversation key every time you added or removed a member from the PM?

The encrypted conversation key does not need to change on add. All members of the conversation have a copy of the decrypted conversation key. So to add a member you would encrypt this conversation key using the “invitee” public key and teach the server about that.

Technically it would be correct to amend the conversation key if you are removing members from a conversation, but we can wait on v2 for that. Cause it would be a very expensive operation from the client side, entire conversation would have to be re-encrypted, or complex multi conversation key systems need to be built.

@angus

You would need some protection or warning against using other plugins as well. I can imagine a scenario in which you install this plugin, then someone writes another plugin that exposes or stores the entered text in some way before it is encrypted

Yes, the long term V3 / V4 goal here is to have a whitelist of JS integrity hashes stored in a browser extension. That is really the only way a client can know 100% that the “server is not messing with stuff”. However for V1 we trust the server and server operator not to be malicious.

@angus

Wouldn’t you need to also only allow this feature on sites that force https?

Yes, if SSL is off all bets are off. We do not expect this feature to even remotely work if SSL is off.

@angus

I assume you’re excluding it as the cost / benefit isn’t worth it, at least for the MVP.

Yes search is off the cards for now cause it is just going to be too hard to get right for the MVP.

@angus

Why not use the Signal Protocol? (javascript library).

Maybe, not against leaning on existing standards, but I suspect the Discourse integration work here is going to be enormous regardless.

[sam (Sam Saffron)]

Congrats! @dan will be taking this project! We will update here as we make progress!

[BeardMadden (Beard McMadden)]

Signed up to comment on this because I’m curious on the implementation as a developer myself.

I’m personally no cryptography pro, so forgive me if these are silly questions.

@sam

Technically it would be correct to amend the conversation key if you are removing members from a conversation, but we can wait on v2 for that. Cause it would be a very expensive operation from the client side, entire conversation would have to be re-encrypted, or complex multi conversation key systems need to be built.

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation, in this case?
Essentially I guess this would lead to a double layer of encryption - the data is encrypted client-side for true E2E encryption as described with the conversation key, and then encrypted on the way out by the server to only the public keys active in the conversation. At that point, removing a member from the conversation would mean any future messages wouldn’t be decryptable by them.

I’m not too sure on the security implications of nested encryption like that, however.

My second question is how well this would work cross-device? Does it require a private key on the device, or is that handled by the passphrase encrypted private key being stored?

[sam (Sam Saffron)]

@BeardMadden

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation , in this case?

Interesting idea the “double” encrypting. It does though add more responsibility to the server and a lot of this is about removing responsibility from the server so I am not sure it is a great idea protocol wise.

@BeardMadden

My second question is how well this would work cross-device? Does it require a private key on the device, or is that handled by the passphrase encrypted private key being stored?

Cross device is handled by retrieving the encrypted private key from the server and decrypting using the key phrase.

Longer term we can introduce more “security” modes with a super strict one when say the server never even stores the private key encrypted (like say 1 password), but short term this is out of scope.

[dan (Dan Ungureanu)]

@BeardMadden

Signed up to comment on this because I’m curious on the implementation as a developer myself.

Welcome!

@BeardMadden

I’m personally no cryptography pro, so forgive me if these are silly questions.

Those are not silly questions at all! If you want to dig deeper, I highly recommend Dan Boneh’s courses on Coursera.org.

@BeardMadden

I’m not too sure on the security implications of nested encryption like that, however.

This depends a lot on the particularities of the system you use. Check out double DES (i.e. vulnerable to meet-in-the-middle) and triple DES.

@BeardMadden

Would it not be viable to encrypt the message on the server-side using the public keys of people currently in the conversation , in this case?
Essentially I guess this would lead to a double layer of encryption - the data is encrypted client-side for true E2E encryption as described with the conversation key, and then encrypted on the way out by the server to only the public keys active in the conversation. At that point, removing a member from the conversation would mean any future messages wouldn’t be decryptable by them.

Let’s say Eve used to be a part of a group discussion with Alice and Bob. Even if Alice or Bob kick Eve out she will still have the conversation key and theoretically, she could intercept the messages before they get to the server and decrypt them.

If the conversation key gets compromised, the safest approach is to revoke it and use a new one for new messages. Ideally, you’d probably want to re-encrypt the old messages.

[BeardMadden (Beard McMadden)]

@dan

Even if Alice or Bob kick Eve out she will still have the conversation key and theoretically, she could intercept the messages before they get to the server and decrypt them.

That makes sense to me, thanks for the explanation! (@sam too)

So adding or removing users from the conversation would involve decrypting everything client-side, revoking the conversation key and generating a new one, then re-encrypting messages before sending to the server?
Plus encrypting the new conversation key with the user passphrase and storing it.

[Alexander_Wright]

I’ve been talking to my wife about this feature. While this may raise an eyebrow at mealtime conversation in our family, she’s a security engineer for Thales eSecurity.

She’s of the opinion that the Specification needs some use cases with bad actors and how the system should respond. For example: Malicious Admin, Malicious SysOps.

This would then give rise to a security statement that could be presented to users of the system.

[sam (Sam Saffron)]

I would love it if your wife or anyone interested in this problem at her work would pop by and give feedback.

@dan has made tremendous progress here, I am testing v1 with him this week.

To answer your specific question of bad sys admins. Overall if you can commandeer the web site and deliver malicious payloads our current v1 can not offer privacy. This is by design.

Our plans for v3/v4 is to have a browser plugin that validates every single JavaScript file on the page matches a particular hash in the whitelist. That way if you would be running this plugin you would be able to disallow unrecognized JS payloads.

Overall the underlying design means that passwords can not be fished out after the fact even if the server is malicious, but it can trick the client if it wishes to decrypt arbitrary payloads. Native browser plugins or a more refined web standard longer term will help eliminate this