Admin Panel Usage Export Missing Dates

admin

(Joshua Rosenfeld) #1

Hope this one is fairly self explanatory. Today I attempted to do some data analysis on the Discourse usage. Was trying to compare New Users with Unique Visits with Posts. I went into the admin panel, opened the usage view for each stat, switched the start date, and clicked export.

When attempting to combine the 3 stats in my spreadsheet, I realized that there was different amounts of data for each stat, and found that on days where the stat was 0 (mostly had this issue with new users), no data was listed, and it just “skipped” the date. This required me to manually go through 3 years of dates and add the missing date (with 0 for stat) to allow the data comparison to work.

Expected result: Data should export with all dates within range and 0 where applicable. Much easier to filter out 0s from data than to find missing dates and add them.


In admin graphs, days with 0 activity are excluded from the x-axis
What would you like to see on your Discourse 2.0 Admin Dashboard?
(Joshua Rosenfeld) #2

To add some more clarity…this was the end goal/product:

In order to make that work I needed all the stats to line up with each other, and while Visits and Posts were always non-zero during the past 3ish years, New Users, especially towards the beginning of the timeline, had some dates where no one joined, thus causing ~50 less data points than the other stats.

P.S. For those curious about the large spike, that was the date the game was first released…


(Joshua Rosenfeld) #3

Not sure if this is kosher, but I am going to “bump” this thread. I noticed that codinghorror and zogstrip liked the post (not pinging them as again I am not sure if that is allowed), so does that mean the bug is acknowledged? Should I expect a fix for this?

Again, sorry if this is not the right way to do things…


(RĂ©gis Hanol) #4

I have it bookmarked and it’s on my TODO list. I have other priorities, but will see if I can move it up the list :wink:


(Joshua Rosenfeld) #5

Thanks @zogstrip! My “bumping” was not an effort to get it corrected quicker (I have already modified the data by hand), but simply to confirm if a discourse team member liking a bug indicated that it was acknowledged and queued for an (eventual) fix.


(RĂ©gis Hanol) #6

While I can’t speak for all the team members as we have different workflows, when I like a post describing a bug it means that I agree it is a bug. It doesn’t necessarily mean that I will fix it. I bookmark all the posts I want to fix, but since they aren’t public, I guess there’s no way for other users to see them. When I’m about to fix the bug though, I usually write a short post mostly because I want to check whether someone else is already working on it :wink:


(Joshua Rosenfeld) #7

Looking to pull this data again, can you confirm if this has been fixed?


(RĂ©gis Hanol) #8

It has not. I’m sorry I haven’t had the time to look at it yet :frowning:


(Joshua Rosenfeld) #9

No worries! :slightly_smiling: Thanks for the quick response!


(RĂ©gis Hanol) #10

I wouldn’t mind a PR :wink: I’m pretty sure the fix is only updating the SQL query to return rows with 0 when there are no values.


(Joshua Rosenfeld) #11

I’ll see what I can do. Never dug into the Discourse code, nor do I have much experience with Github…but I’ll give it a try!


(Joshua Rosenfeld) #12

OK, so I signed the CLA, and started digging through the code, but I have no experience with Ruby and have no idea where to look. I found https://github.com/discourse/discourse/blob/master/app/assets/javascripts/admin/templates/reports.hbs, as well as https://github.com/discourse/discourse/blob/master/app/assets/javascripts/admin/models/report.js.es6, but they only seem to control what appears when viewing the report on the web. https://github.com/discourse/discourse/blob/master/app/controllers/export_csv_controller.rb and https://github.com/discourse/discourse/blob/master/app/assets/javascripts/discourse/lib/export-csv.js.es6, seem to control the csv creation, but not what is stored in the csv.

Can you point me in the right direction?


(RĂ©gis Hanol) #13

Everything is in the report.rb model class and particularly the req_report method.


#14

Well, I took a look int that code, but I don’t understand much of it :wink: Any chance some Ruby Master could take a look at this issue maybe?


(Arpit Jalan) #15

This issue has been fixed when viewing graph/table, but export is still missing dates with zero value. Added on my list.


(Arpit Jalan) #16

Fixed via:


(Arpit Jalan) #17