GDPR countdown and compliance

[GBrowning]

As we’re quickly approaching May 25th, here’s my (perhaps naive) understanding of certain key Discourse GDPR issues:

Right to Be Forgotten

If the user requests, the admin can use the user info panel to delete all of a user’s posts. Note you may need to up the “delete user max post age” and “delete all posts max” settings but then it’s possible. Once there are no more posts you can then delete the user.

Alternatively, a less disruptive solution is the “Anonymize User” button which leaves the posts but changes the username and everything else about the user so they are no longer identifiable, although recorded IP addresses still linger (?) so that may be an issue (see below).

Be sure to uncheck Settings > Legal > “log anonymizer details” (“Whether to keep a user’s details in the log after being anonymized. When complying to GDPR you’ll need to turn this off.”)!

The user has a “Delete My Account” button on their own info screen, but I believe only if they have a small number of posts.

So I think Discourse is okay on GDPR compliance here, as long as it’s documented in the privacy policy that they must contact the admin to do some of these operations.

Data Portability

Via the user’s own info panel they can click the “Download All” button to download all of their posts. I believe this is only the text, not any uploads, although I’m not sure about that. It’s not clear if this also includes private messages sent via Discourse, although I assume it would need to. Again, if documented in the PP then I’m hoping we’re GDPR-compliant here.

Explicit Consent to Emails

By default a user will receive email if someone messages, quotes, replies, mentions, or invites them. They will also receive, by default, a digest email of popular topics and replies.

On sign-up they consented to the terms of service but not to these emails. Perhaps something like this:

The forums service can send me emails when my forums user is messaged, quoted by, replied to, mentioned, or invited by another forums user.
The forums service can send me a weekly digest email of popular topics and replies.
I agree to the terms of service and privacy policy.

If they consent then Discourse would need to record the consent date, not just a boolean, so we can match it to a ToS/PP revision.

If not checked, then all emails like those should be off by default. Perhaps after activating their account Discourse could show an email opt-in screen. Again, consent records the opt-in timestamp, unchecking should record the opt-out timestamp.

In lieu of this, is there a way to turn all of this off (a) by default for new users, and (b) for all existing users? They could then turn it on manually via their info screen. This tip is related but doesn’t change the default:

GDPR and the Digest email

This hasn’t been tested (so do a database-only backup first), but this should do it: cd /var/discourse ./launcher enter app rails c User.real.where("not admin").update_all(email_digests: false)

This doesn’t tackle the issue of recording opt-in / opt-out dates but it’s better than automatic opt-in.

I realize I can disable digest emails for all users and even disable emails entirely, but I simply want to turn it off by default until they opt-in, not disable it entirely.

IP Addresses

Discourse stores each user’s last IP address and registration IP address. These are visible to admins in the user info screen. It is my understanding that GDPR considers IP addresses as personally identifiable information so, while there are legitimate uses for rate limiting, etc, if Discourse is keeping this data indefinitely then explicit consent may be needed. Or ideally Discourse can decay/hash this data automatically after some period of time.

Many more details in this thread:

Problematic IP address fields

Continuing the discussion from Providing data for GDPR: I did a first pass over Discourse’s tables, and I found several places where IP addresses are being accidentally correlated with user IDs. This is toxic data generating liability for Discourse forums. List of problematic IP address fields: incoming_links: stores timestamped IP address correlated with user ID and an exact post ID, topic ID, and Referer: header Fixed Storage: PR#5826 …

Re-consent to Changed ToS / Privacy Policy

If we have to update our ToS and Privacy Policy new users would be okay but we’d need existing users to re-consent to these changes before continuing use on the forum. Again, we’d need to record the timestamp of the consent so we can pair it to a ToS/PP revision.

More details here:

How to make users to explicitly agree to ToS

It’s one of the GDPR requirements in Europe - the ToS acceptance must be an explicit act (like marking a checkbox), not an effect of a separate action (like pressing the sign up). Is it possible with the SSO?

To be honest, I’m not sure why a blanket pinned topic “Our ToS and PP have changed. By continuing to use this service you agree to these changes.” (with hyperlinks to the ToS and PP) wouldn’t suffice. Perhaps some more knowledgeable person can chime in here. Does every site now need to record an explicit opt-in?

What Others are Doing

It looks like some other discussion forums vendors are rolling out GDPR changes and statements. Something similar from Discourse would be most welcome as we get closer to GDPR Day.

[pfaffman (Jay Pfaffman)]

@GBrowning

This tip is related but doesn’t change the default:

Site setting: default email digest frequency – set to never.

@GBrowning

It looks like some other discussion forums vendors are rolling out GDPR changes and statements.

I have a client whose solution is to use IP location data to exclude people from outside of the US (his site is US specific).

[codinghorror (Jeff Atwood)]

I believe @hawk might have a general update, but we’re focused on our paying customers at the moment. There are definitely lots of good trickle-down changes that will go out to everyone in the open source changes, but as far as direct hand holding and advice… uh, you kinda gotta pay us for that

[HAWK]

I’ll put together a general summary/update first thing next week.

In the meantime, we’ve published our updated privacy policy here.

[GBrowning]

@codinghorror

I believe @hawk might have a general update, but we’re focused on our paying customers at the moment. There are definitely lots of good trickle-down changes that will go out to everyone in the open source changes, but as far as direct hand holding and advice… uh, you kinda gotta pay us for that

My apologies. I didn’t realize this. I’ll dig around and see if I can find the GDPR features in your paid-for product offerings and learn about migrating. Thank you.

@HAWK

I’ll put together a general summary/update first thing next week.
In the meantime, we’ve published our updated privacy policy here

Thanks @HAWK!

[ljpp]

Disclaimer: I am no lawyer.

I have been studying the legal basis for the issue of user consent to our software collecting the data and so far I am betting that no consent would be needed.

The GDPR provides six different legal bases for processing personal data (excluding special categories), which include:

1. The data subject had given consent to the processing;
2. Performance of a contract to which the data subject is party or in order to take steps at the request of the data subject prior to entering into a contract;
3. Compliance with a legal obligation to which the controller is subject;
4. To protect the vital interests of the data subject or of another natural person;
5. For the performance of a task carried out in the public interest or in the official authority of the controller;
6. For the ‘legitimate interests’ of the controller or a third party.

Source: https://www.boldonjames.com/gdpr-dumb-ways-to-fail-part-2/

Bullet 2: We cannot comply to users request (of joining the community) without collecting the data (email)
Bullets 4 & 6: It is in the community’s interest as a whole as well as the individual’s for us to record the IP. Even though IP is nowadays mostly inefficient in forum moderation, it can be useful in protecting the community from misbehaving individuals as well as protecting individual users in cases like account theft.

So it is for the best of the community and all the members to record the IP. I don’t however have any idea will the indefinite time of record keeping will be an issue?

So my primary worries are:

• Opt-in consent to email. Private message notifications can be argued not require consent though?
• Opt-in consent, with re-consent after modifications, to privacy policy and ToS

[RGJ (Richard - Communiteq)]

@ljpp

Opt-in consent to email.

In my opinion collecting the e-mail address can be seen as required for performance of a contract.

@ljpp

I don’t however have any idea will the indefinite time of record keeping will be an issue?

Recording the IP address can indeed be a legitimate interest as far as the usage is meant for network security. Indefinite time is certainly an issue. But Discourse also stores IP addresses in other places (for instance, when someone initiates a search query) where they are not needed.

@GBrowning

the “Anonymize User” button which leaves the posts but changes the username and everything else about the user so they are no longer identifiable, although recorded IP addresses still linger (?) so that may be an issue

This leaves IP addresses and also @mentions.

@GBrowning

Via the user’s own info panel they can click the “Download All” button to download all of their posts

For the data portability this is sufficient. But for the right of Access by the Data subject it is not since there is a lot more information stored about a user.

@GBrowning

To be honest, I’m not sure why a blanket pinned topic “Our ToS and PP have changed. By continuing to use this service you agree to these changes.” (with hyperlinks to the ToS and PP) wouldn’t suffice. Perhaps some more knowledgeable person can chime in here. Does every site now need to record an explicit opt-in?

For the cases where processing requires consent the opt-in must be active, separate and explicit. But I agree with you: I haven’t been able to find any sign that the ToS must be opted in actively. We will give anyone who can point this out in the official GDPR text a full year of free hosting on our Basic plan, and our eternal respect

[codinghorror (Jeff Atwood)]

@RGJ

But Discourse also stores IP addresses in other places (for instance, when someone initiates a search query) where they are not needed.

Yes, this is something we were hoping @riking could assist us with, as he created a whole topic about it. Wherever we’re storing IP and we don’t need to be, we should stop doing that.

[GBrowning]

@RGJ

In my opinion collecting the e-mail address can be seen as required for performance of a contract.

I agree somewhat. I think collecting an email address is fine to become a member of the forums (to authenticate the user). But I don’t think that should automatically sign you up for mailings, especially the “popular posts” digest mailing. I think a more granular separation of consent would be good here.

So, if I were to set the digest default to ‘never’ and use that script in my first post to turn off everyone’s digest, that’s a start. Something similar for the other emails would be handy. Then I could pin a topic describing how members can turn on these settings, which allows them to manually opt-in to the emailings. Not perfect but gets us there…

@RGJ

This leaves IP addresses and also @mentions.

Oh! But simply renaming a username changes all @ mentions, so I’m surprised anonymize doesn’t do the same since it also renames the username.

@codinghorror

Yes, this is something we were hoping @riking could assist us with, as he created a whole topic about it. Wherever we’re storing IP and we don’t need to be, we should stop doing that.

That would be fantastic! I was surprised to have such easy access to member’s registration and last-login IP addresses.

[dacap (David Capello)]

@codinghorror

Wherever we’re storing IP and we don’t need to be, we should stop doing that.

Could be possible to store just a hash of the IP address instead? That would remove the personal information and will be useful for moderation purposes too.

[ChrisBeach (Chris)]

@GBrowning

Then I could pin a topic describing how members can turn on these settings, which allows them to manually opt-in to the emailings. Not perfect but gets us there…

I’d prefer a checkbox on the signup form like this (mockup):

[Stephen]

@dacap

Could be possible to store just a hash of the IP address instead?

Hashes aren’t a magic bullet to privacy issues. The systems which need to retain IP data can do so lawfully. The remainder just don’t need to do so.

One of the benefits of having access to IPs is being able to see if a user is still coming from the same subnet, source network or region. A hash would provide a simple Boolean response as to whether the user was on the exact same IP. It would be true in 1 out of potentially millions of IPs controlled by their ISP, and false for another ~3.7 billion address, while giving us no more information about the user and their behavior.

[RGJ (Richard - Communiteq)]

@dacap

Could be possible to store just a hash of the IP address instead?

No - the amount of IPv4 addresses is sufficiently small (4 bytes) that it would only take a very short time to generate a lookup table containing all possible values.

[riking (Kane York)]

@GBrowning

That would be fantastic! I was surprised to have such easy access to member’s registration and last-login IP addresses.

Those are actually the IP storage we’ll be keeping – I’m going to work on removing any IP storage not shown in the UI.

[HAWK]

Ok, general update as promised. It’s important to remember that Discourse can be used in GDPR compliant ways, but software itself isn’t compliant or non-compliant. If you host Discourse for users covered by GDPR, you’ll need to do so in a GDPR-compliant way. What that looks like for you will depend on your interpretation of the guidelines and how you specifically use your member’s data.

A user’s right to be informed
We have updated the privacy statement that ships with Discourse. You can see our version here. You can edit your own version to suit.

A user’s right to be forgotten
Users and their posts can be deleted or anonymised by an admin. We have added support in
v2.0.0beta8 to rename users in mentions and quotes when anonymising, as well as support for anonymising a user’s IP addresses

A user’s right to a copy of their data
A user can download their activity as a .csv file by going to their activity summary. An admin can do this for other members by impersonating them.

A user’s right to modify their data
Depending on how you have configured your Discourse instance, a user can modify their data via their personal preferences and/or by contacting an Admin.

Gaining consent
You can customise your own instance to include a mandatory custom field on registration or you could use the Custom Wizard Plugin as a means to gain consent.

[michaeld (Michael - Communiteq)]

@HAWK

but software itself isn’t compliant or non-compliant.

That is put a bit too simple. GDPR (especially article 25) but also other standards and regulations (think ISO27000, PCI) set very specific requirements for software. So technically you could argue that “software only helps to achieve compliance” but if your software doesn’t meet the requirements, it can break compliance for your entire organisation.

[HAWK]

Technically I could argue lots of things but that’s not how I’m choosing to spend my time right now. I’d rather wait and see how this pans out.

[richp10 (Richard Phillips)]

Discourse is amazing, the Discourse team are amazing, the Discourse Community is amazing - but I am not convinced that the GDPR issue has seen any of these at their best. One week from the deadline and it is very hard to comply with GDPR if you use Discourse.

As far as key stakeholders in my projects are concerned, Discourse makes it hard for them to meet their legal responsibilities under GDPR. Some people here seem to think there is not a problem I am making too much fuss - but the lawyers of people I am trying to engage with a product using Discourse are saying it is not good enough and they cannot allow their staff or service users to use Discourse until this is resolved. As such I am doing back-end coding to work around their concerns.

I shared my original concerns: Providing data for GDPR

Of these, I was wrong / reassured about the Discourse approach to Right to be Forgotten. I have set things up so users can choose whether to completely remove or anonymize and I explain the merits of each. I think Right to be forgotten has a big tick - thanks team.

Also mentioned there and alluded to but not explicit in @GBrowning post that started the current thread, we still have a big red cross beside the ‘Right of Access’ box. This is a separate issue to Data Portability.

Whatever people here may think, my stakeholders (NHS and major UK charities) demand that I am able to respond to data access requests in a way that is compliant with GDPR. At the moment I cannot comply with this basic requirement under GDPR using native Discourse tools.

The ‘download all posts’ simply does not cut the mustard because it does not provide all the personal data held on the database. My work around: on my backend I am coding an extraction directly from the database to try and pull together all the ‘personal information’ that must as a matter of law, be provided in response to a request. This includes IP addresses, private messages and other data that is not currently included in the download. I am finding it hard going because my knowledge of the database is limited.

As of next Friday, I could email @codinghorror making a Data Access Request and Discourse would as a matter of law have to carry out exactly the same exercise - building queries to extract all my personal data to send to me within a month. If they do not do this, the penalty could (theoretically) be 4% of your turnover.

Of course I am going to make no such request, but across the EU companies, Charities and Public Bodies are working hard to establish whether they are in a position to comply with Data Access Requests.

If they use Discourse, they cannot, unless they do some strenuous deep diving with SQL.

There might be a few other issues still worthy of more discussion (such as whether some old data no longer has legitimate use and should be deleted) but Right of Access requests are the big headache for me.

[mpj (Mattias Petter Johansson)]

@HAWK

A user’s right to be informed
We have updated the privacy statement that ships with Discourse. You can see our version here . You can edit your own version to suit.

This is awesome, is there any way to get it in markdown in a handy way? I can re-craft by hand of course, but just checking.

[pfaffman (Jay Pfaffman)]

Actually, just copy and paste will likely be pretty good. (looks like there is no route to get raw for it).