Tracking of Pageviews and Unique Users at Category and Tag Level

Do you have any advice here @hawk? It seems like a reasonable question and maybe new dashboard / reports covers this a bit?

2 Likes

It does seem like a reasonable request, yes. At the moment the reports on the dashboard aren’t segment-able but if we can work out a way to do that I think it would be an excellent addition to v2. We’ve also talked about exposing approved Data Explorer queries on the dashboard.

cc @joffreyjaffeux – do you have plans to revisit the dashboard any time soon?

6 Likes

Yes, it will be step by step but might come to something close to this. I don‘t want it to become too complex though and it will never be a “combine any segment you want”, there’s sql (and data-explorer) for this. But will probably try to provide this on a per report basis where we think it makes sense.

4 Likes

This query is specifically about pageviews per category, which I don’t think can be handled with Data Explorer can they?

2 Likes

Yes, I would also be interested in a report about the unique users on my site.

Yes this specific report is using ApplicationRequest table, which doesn’t have any notion of category. So not possible through reports, data-explorer or raw sql.

2 Likes

Can we pull a report on the number of anonymous users instead of the number of pages viewed anonymously?

1 Like

Do you mean the number of people that have used the anonymous posting feature? They all get unique anonymous usernames so it will be very easy to pull a report from Data Explorer

2 Likes

I’m trying to find out how many people are visiting the site who have not logged in at all and do not have an account.

2 Likes

Would love to see this as well! Any advice on how to construct a query? Thx!

We also want this number (PVs per category, excluding crawler activity). Is there a way to get this from the Data Explorer?

I don’t believe so. We don’t granularly track page views like that. You would need to hook up your GA and use that.

1 Like

@HAWK: Do you reckon this won’t be possible then? Counting Pageviews for users (non-staff, non-crawler)

Has anyone found a solution to count page views for all posts within a specific category? Possible with GA?

1 Like

It’s possible to get total category views from the Data Explorer by summing up the views of all topics under a category:

SELECT 
       c.id as category_id, 
       SUM(views) as "total views"
FROM categories c 
JOIN topics t ON t.category_id = c.id
WHERE read_restricted is false
GROUP BY c.id
order by sum(views) desc

Getting total views by tag is similar:

SELECT tags.name,
       sum(views)
from topics t
     join topic_tags tt on t.id = topic_id
     join tags on tags.id = tt.tag_id
group by tags.name
order by sum(views) desc

There’s also a topic_views table which could, in theory, be used to divide views by user and date. I haven’t found that table particularly useful, however, because queries time out when topics have a lot of views. It also shows a lot fewer views per topic because, if I understand it correctly, it’s not counting anonymous views. I also think maybe topic.views includes not only anonymous views but also bot traffic? It’s a lot more than what I see on GA.

Speaking of GA, it has all the data you need, but it’s not easy to group it by category or tag. The best I can do try to parse the pageTitle to find categories. I’m doing this in R using googleAnalyticsR. Follow the instructions in the manual to authorize your Google account and get the metrics you want. Be sure to include pageTitle as a dimension. My API calls look a bit like this:

ga_this_year <- ga_data(ga_id, 
                        date_range = c("2023-01-01", "2023-05-30" ),
                        metrics=c("screenPageViews","averageSessionDuration", "sessions"), 
                        dimensions = c("pageTitle", "deviceCategory")
                       )

The key to understanding the next bit is to see that pageTitles fit this general format:

Topic title - Category - Site title

If the topic is not categorized, the Category is missing. There are also a bunch of utility pages (“Latest topics”, “New topics”, etc.) that don’t have categories. (I’m not counting “Latest [category] topics” as being part of the category, though it might be better to include them.) Finally the homepage uses Site title - short site description for the page title. We don’t care about any of those, however. So the regex I’m using is:

str_extract(pageTitle,
            str_glue(".*? - (.*) - {sitename}"),
            group=1)

Putting it together into a function that groups by category, I get something like this:

category_views <- function(data, sitename = "Meta Jon") {
  data %>% 
  mutate(category = str_extract(
                                pageTitle,
                                str_glue(".*? - (.*) - {sitename}"),
                                group=1
                                )) %>% 
  group_by(category) %>% 
  summarise(views = sum(screenPageViews)) %>% 
  arrange(desc(views)) #%>% head(20)
}

category_views(ga_this_year)

(Obviously change “Meta Jon” for your own site’s name.)

I don’t currently know how to extract GA data based on tag, however.

2 Likes

Are those total views including bots, aka. anonymouses, too?

GA excludes bot traffic, according to their documentation. I can’t find documentation for topic_views, but a comment in the code says:

# Only store a view once per day per thing per (user || ip)

I don’t know for sure, but topic.views seems to show bot page views because it shows many more views than GA does for the same page.

Yes but counts from Discourse when using data explorer?

Earlier GA told same and it can be true when there is legit bots. But majority of calls happends by rogue ones, bad behaving SEO-bots, knckers etc. and then one visit rule will break. Or so I reckon if I’m honest.

Anyway, I’m using Matomo. I’m not that big fish I would need GA.

Just an FYI, This is actually because the topic_views table limits the counting to one new view per day, whereas the topics.views field counts allows one new view every 8 hours (by default, but changeable using topic view duration hours). It does include users and anon. :+1:

UPDATE

Turns out the topic_views table isn’t even counting one new view a day… It actually counts the first time someone views a topic and no more views after that. So there’ll only ever be one record per user or IP per topic.

2 Likes

It is great that there are ways to do this with Data Explorer, but it would be lovely to just have a Category filter on the main /admin dashboard. I came across this: https://meta.discourse.org/t/visualised-traffic-statistics/ and it looks like there may have been a Category filter at one point, unless that is something different.

The “Community Health” dashboard is an awesome way to see at a glance the simple overview statistics on whether you are building a healthy community. I look after one category on a bigger discourse and for me, my “Community” that I want to build and grow is that category. I have different team members looking after sub-categories and again, I would love to see how they are building and growing their communities by having simple overview statistics of their sub-category.