Admin Panel Usage Export Missing Dates

Hope this one is fairly self explanatory. Today I attempted to do some data analysis on the Discourse usage. Was trying to compare New Users with Unique Visits with Posts. I went into the admin panel, opened the usage view for each stat, switched the start date, and clicked export.

When attempting to combine the 3 stats in my spreadsheet, I realized that there was different amounts of data for each stat, and found that on days where the stat was 0 (mostly had this issue with new users), no data was listed, and it just “skipped” the date. This required me to manually go through 3 years of dates and add the missing date (with 0 for stat) to allow the data comparison to work.

Expected result: Data should export with all dates within range and 0 where applicable. Much easier to filter out 0s from data than to find missing dates and add them.

6 Likes

To add some more clarity…this was the end goal/product:

In order to make that work I needed all the stats to line up with each other, and while Visits and Posts were always non-zero during the past 3ish years, New Users, especially towards the beginning of the timeline, had some dates where no one joined, thus causing ~50 less data points than the other stats.

P.S. For those curious about the large spike, that was the date the game was first released…

1 Like

Not sure if this is kosher, but I am going to “bump” this thread. I noticed that codinghorror and zogstrip liked the post (not pinging them as again I am not sure if that is allowed), so does that mean the bug is acknowledged? Should I expect a fix for this?

Again, sorry if this is not the right way to do things…

I have it bookmarked and it’s on my TODO list. I have other priorities, but will see if I can move it up the list :wink:

1 Like

Thanks @zogstrip! My “bumping” was not an effort to get it corrected quicker (I have already modified the data by hand), but simply to confirm if a discourse team member liking a bug indicated that it was acknowledged and queued for an (eventual) fix.

While I can’t speak for all the team members as we have different workflows, when I like a post describing a bug it means that I agree it is a bug. It doesn’t necessarily mean that I will fix it. I bookmark all the posts I want to fix, but since they aren’t public, I guess there’s no way for other users to see them. When I’m about to fix the bug though, I usually write a short post mostly because I want to check whether someone else is already working on it :wink:

3 Likes

Looking to pull this data again, can you confirm if this has been fixed?

It has not. I’m sorry I haven’t had the time to look at it yet :frowning:

1 Like

No worries! :slightly_smiling: Thanks for the quick response!

I wouldn’t mind a PR :wink: I’m pretty sure the fix is only updating the SQL query to return rows with 0 when there are no values.

I’ll see what I can do. Never dug into the Discourse code, nor do I have much experience with Github…but I’ll give it a try!

1 Like

OK, so I signed the CLA, and started digging through the code, but I have no experience with Ruby and have no idea where to look. I found https://github.com/discourse/discourse/blob/master/app/assets/javascripts/admin/templates/reports.hbs, as well as https://github.com/discourse/discourse/blob/master/app/assets/javascripts/admin/models/report.js.es6, but they only seem to control what appears when viewing the report on the web. https://github.com/discourse/discourse/blob/master/app/controllers/export_csv_controller.rb and https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/lib/export-csv.js.es6, seem to control the csv creation, but not what is stored in the csv.

Can you point me in the right direction?

Everything is in the report.rb model class and particularly the req_report method.

Well, I took a look int that code, but I don’t understand much of it :wink: Any chance some Ruby Master could take a look at this issue maybe?

This issue has been fixed when viewing graph/table, but export is still missing dates with zero value. Added on my list.

4 Likes

Fixed via:

https://github.com/discourse/discourse/commit/33df2d6a029cb9dd97124cd7866e1a090caf8b4f

6 Likes