When helping a client with a data explorer query, they noticed a difference between the topic views returned by the query, and the number on the category topic list. Where does the difference between SELECT COUNT(*) FROM topic_views WHERE topic_id = X and Topic.find(X).views come from?

[image]Topic view count is inaccurate in topic index Note that topic view counts are reset once per 8 hours per IP (for anons). So if you had a lot of anon visitors coming from the same IP that won’t show up in view counts. Topic.views is integer count which includes both user and anonymous traffic. SELECT COUNT(*) FROM topic_views WHERE topic_id = X is used to track topic views by registered users. It will have below details. app/models/topic_view_item.rb cb824a6b3 # topic_id :integer not null # viewed_at :date not null # user_id :integer # ip_address :inet

[image] vinothkannans: used to track topic views by registered users No, that is not entirely true. topic_views registers anonymous views as well, in those cases user_id is null. So what’s I think that’s left is that Topic.views counts for anonymous users are counted once per topic_view_duration_hours (default: 8) hours per IP, and topic_views for anonymous users are counted once per IP?

Yes I can see now. As per the index uniq_ip_or_user_id_topic_views either user id or ip address is required.

So what is the intended use of topic_views ? For registering statistics about topic views it would make sense if it used the same rules as Topic.views. For registering which user saw what topic it wouldn’t make sense to register IP addresses for anonymous visitors (which is a left over GDPR toxic item anyway) EDIT: it seems like TopicViewItem is only being used for registering which user/IP saw what topic, and that is only used in TL3 requirement calculations. So that would mean that we could (optionally) get rid of registering IP addresses for anonymous visitors and be a bit more GDPR compliant? Or am I going too quickly now?

In a 20 users private community a topic can have 100 views count. Since new topic visits after topic_view_duration_hours are calculated as new view count. But select count(*) from topic_views where topic_id = X will never exceed the count 20. That’s why Topic.views count is different. This is applicable for open communities too. [image] RGJ: which is a left over GDPR toxic item anyway I’m not sure here. I will check it on Monday.

[image] RGJ: So what’s I think that’s left is that Topic.views counts for anonymous users are counted once per topic_view_duration_hours (default: 8) hours per IP, and topic_views for anonymous users are counted once per IP? Sounds about right. Long term I don’t even see much value in carrying topic views in the raw form in the topic_views table for longer than a few months.

[image] vinothkannans: I’m not sure here. I will check it on Monday. Did you check already?

[image] GDPR countdown and compliance Community Building As we’re quickly approaching May 25th, here’s my (perhaps naive) understanding of certain key Discourse GDPR issues: Right to Be Forgotten white_check_mark If the user requests, the admin can use the user info panel to delete all of a user’s posts. Note you may need to up the “delete user max post age” and “delete all posts max” settings but then it’s possible. Once there are no more posts you can then delete the user. Alternatively, a less disruptive solution is the “Anonymize User” button … [image] Problematic IP address fields Feature Continuing the discussion from Providing data for GDPR: I did a first pass over Discourse’s tables, and I found several places where IP addresses are being accidentally correlated with user IDs. This is toxic data generating liability for Discourse forums. List of problematic IP address fields: white_check_mark x incoming_links: stores timestamped IP address correlated with user ID and an exact post ID, topic ID, and Referer: header Fixed Storage: PR#5826 white_check_mark … It’s already discussed in above topics.

It’s discussed there, but there is no solution. IP addresses for anonymous visitors are still being stored. [image] Problematic IP address fields Feature Although I do absolutely welcome these PR’s I do want to emphasize that storing the IP addresses of visitors without an account (for a longer time than needed for deduplication) is a much more problematic issue since those people cannot easily be asked to give their consent. So I guess we’re back at my former statement: [image] RGJ: For registering which user saw what topic it wouldn’t make sense to register IP addresses for anonymous visitors (which is a left over GDPR toxic item anyway)

I don’t think we have quite gotten around to it, but it is definitely on our list. If you would like it to be done faster, have you considered contributing a PR to get it done?

Topic_views vs Topic.views

Development

sam (Sam Saffron) September 28, 2018, 10:47am 13

No we need to roll this up into a different table, I want to retain 1 day fidelity on page views

Topic		Replies	Views
How many views are counted per IP? Feature	38	10945	March 30, 2017
View count difference in category vs data-explorer Data & reporting	3	316	November 9, 2024
Topic view count is inaccurate in topic index Bug	7	2161	August 5, 2020
Views counter specification UX	6	1869	May 23, 2023
Total Views Error Product Support	1	637	May 30, 2019

Topic_views vs Topic.views

Related topics