Search Logs Page

Announcements
1

Now that we have search logging in place it’s time to put that data into action.

I propose creating a new page under “Admin” → “Logs” → “Search Logs” tab, like so:

39
Screen Shot 2017-11-02 at 17.46.39.png2408×980 214 KB

The initial version of search logs page will contain following data points:

  • term: the search term.

  • searches: number of times the term has been searched for.

  • continued from search: number of times user has clicked on topic from search results.

  • most viewed topic: the topic that has been visited most for a particular term.

  • unique: number of unique searches (from different ip addresses) performed for a term.

Following filters will be available in initial version:

  • start date

  • end date

Improvements to consider in future:

  • term filter to fetch search log data for a particular term
  • topic id filter to fetch search log data for a particular topic
  • how to handle multi-word search term like “wordpress sso”
16 Likes
3

Nice work. For site admins very useful to know what users want. :+1:

In future if we had an option to add tags automatically based on continued from search threshold then it will become great.

4

“Click through” is the term that SEO wizards use.

7 Likes
5

Could we also get a simple Top style timespan-dropdown there?

58
58972×442 36.2 KB

Would like a simple way to check for current trends.

8 Likes
6

Looks good to me, many people have asked for it, will be awesome to have it in the product.

2 Likes
9

Here is a PR for this feature:

https://github.com/discourse/discourse/pull/5313

14 Likes
10

@sam merged the PR. The new search logs/trends page is now live at /admin/logs/search_logs. :mag_right:

3 Likes
11

Looks good but I think we need some refinements:

  1. [:white_check_mark:] We need to be tracking click through to user, tag and category. If you click through to any of those the search is still a success, we now only track click through to topic AFAIK.

  2. [:white_check_mark:] We should log fullpage vs mini search in the table (then we can add a “all” / “header” / “fullpage” filter to this page)

    • This allows you to isolate terms that seem to get click throughs on fullpage and dont on the mini search like sso on meta

  3. [:white_check_mark:] Left align search terms, left align most viewed topic

  4. It would be nice to be able to click through to the actual search, but its a tricky UI change cause you don’t really want to make all the terms blue and you don’t want to add an icon that repeats 100 time

  5. [:white_check_mark:] We should sort by unique descending, not by searches

  6. [:white_check_mark:] Add tooltip on unique: “unique users performing the search”

  7. We should think about ways of allowing people to track “progress” here, for example:

    • “sso”: Click through 13% (30% last week) - oh no - we changed something that made click through worse

    • This is probably most relevant in month/week time frames on stuff that is searched more than say 20 unique.

8 Likes
13

@techAPJ how much feature creep would it be to allow user sorting like /u?

14

Not much, but I feel like it’s unnecessary at this point. I can add it on my low-priority list.

4 Likes
15

@techAPJ

“Click through to a topic” doesn’t look like getting tracked properly.

This is what I did

  1. Searched for a term. Clicked on a topic and scrolled till the bottom of the page.
  2. Accessed /admin/logs/search_logs?period=daily
  3. I can see the term that I searched for. But it shows “0” on “click through count”, “most viewed topic” column is also blank.

I thought something is wrong with “click through tracking to topics” because, when I checked “today’s search logs”, I can see like 100+ searches, but “click through count” and “most viewed topic” column is filled for just 2 search terms. This is when I did a search myself to confirm if something is really wrong. And after my tests “like I mentioned above”, it looks like its not tracking “click through to topics” correctly.

3 Likes
16

PR for review here:

5 Likes
17

I noticed that currently we are tracking clicks only from header search, and not from full page search. Is this intentional? (cc @eviltrout)

2 Likes
18

It should be tracked in both places. I just never got around to the second page.

5 Likes
19

Okay, we’ll now track clicks on full page search as per:

Were you by any chance performing “full page” search? If yes, please update to latest version and test again, it should be fixed now.

2 Likes
20

Awesome I’ll check and let you know…

1 Like
21

@techAPJ
Yes it works now.

Is it possible to add an export to csv feature?

2 Likes
22

Here’s what I am thinking. Please let me know your opinion.

If an Admin wants to drill into the term, they will click on the term and go to term details page, which will look like:

20
202302×558 30.4 KB

  • Clicking on the term “sso” again, admin will go to full page search with “sso” prefilled.

  • There will be a search type (header/fullpage) filter.

  • Admin will be able to see the stats/trends for different time periods (similar to what they see on dashboard) for that term on one page.

Does the above looks good?

5 Likes
23

Does unique track IP addresses or user accounts? (since users may use more than one IP)

It sounds like IP addresses here:

And user accounts here:

Our sites are only visible to people with user accounts, and that’s not the case for everyone so probably it’s IP addresses, but maybe there’s some magic going on in the background. Either way, it would be useful for us to know which of the two it is.

Sorry if this is too pedantic.

24

@techAPJ can confirm but in your case (“a requires login community”) the unique count is the unique count of users. AFAIK we use user_id and then fallback to ip

4 Likes