Search Logs Page

Now that we have search logging in place it’s time to put that data into action.

I propose creating a new page under “Admin” → “Logs” → “Search Logs” tab, like so:

The initial version of search logs page will contain following data points:

  • term: the search term.

  • searches: number of times the term has been searched for.

  • continued from search: number of times user has clicked on topic from search results.

  • most viewed topic: the topic that has been visited most for a particular term.

  • unique: number of unique searches (from different ip addresses) performed for a term.

Following filters will be available in initial version:

  • start date

  • end date

Improvements to consider in future:

  • term filter to fetch search log data for a particular term
  • topic id filter to fetch search log data for a particular topic
  • how to handle multi-word search term like “wordpress sso”
16 Likes

Nice work. For site admins very useful to know what users want. :+1:

In future if we had an option to add tags automatically based on continued from search threshold then it will become great.

“Click through” is the term that SEO wizards use.

7 Likes

Could we also get a simple Top style timespan-dropdown there?

Would like a simple way to check for current trends.

8 Likes

Looks good to me, many people have asked for it, will be awesome to have it in the product.

2 Likes

Here is a PR for this feature:

https://github.com/discourse/discourse/pull/5313

14 Likes

@sam merged the PR. The new search logs/trends page is now live at /admin/logs/search_logs. :mag_right:

3 Likes

Looks good but I think we need some refinements:

  1. [:white_check_mark:] We need to be tracking click through to user, tag and category. If you click through to any of those the search is still a success, we now only track click through to topic AFAIK.

  2. [:white_check_mark:] We should log fullpage vs mini search in the table (then we can add a “all” / “header” / “fullpage” filter to this page)

    • This allows you to isolate terms that seem to get click throughs on fullpage and dont on the mini search like sso on meta
  3. [:white_check_mark:] Left align search terms, left align most viewed topic

  4. It would be nice to be able to click through to the actual search, but its a tricky UI change cause you don’t really want to make all the terms blue and you don’t want to add an icon that repeats 100 time

  5. [:white_check_mark:] We should sort by unique descending, not by searches

  6. [:white_check_mark:] Add tooltip on unique: “unique users performing the search”

  7. We should think about ways of allowing people to track “progress” here, for example:

    • “sso”: Click through 13% (30% last week) - oh no - we changed something that made click through worse

    • This is probably most relevant in month/week time frames on stuff that is searched more than say 20 unique.

8 Likes

@techAPJ how much feature creep would it be to allow user sorting like /u?

Not much, but I feel like it’s unnecessary at this point. I can add it on my low-priority list.

4 Likes

@techAPJ

“Click through to a topic” doesn’t look like getting tracked properly.

This is what I did

  1. Searched for a term. Clicked on a topic and scrolled till the bottom of the page.
  2. Accessed /admin/logs/search_logs?period=daily
  3. I can see the term that I searched for. But it shows “0” on “click through count”, “most viewed topic” column is also blank.

I thought something is wrong with “click through tracking to topics” because, when I checked “today’s search logs”, I can see like 100+ searches, but “click through count” and “most viewed topic” column is filled for just 2 search terms. This is when I did a search myself to confirm if something is really wrong. And after my tests “like I mentioned above”, it looks like its not tracking “click through to topics” correctly.

3 Likes

PR for review here:

https://github.com/discourse/discourse/pull/5385/

5 Likes

I noticed that currently we are tracking clicks only from header search, and not from full page search. Is this intentional? (cc @eviltrout)

2 Likes

It should be tracked in both places. I just never got around to the second page.

5 Likes

Okay, we’ll now track clicks on full page search as per:

https://github.com/discourse/discourse/commit/54f5b4d4e066bb3457d7e91619737611e09cd640

Were you by any chance performing “full page” search? If yes, please update to latest version and test again, it should be fixed now.

2 Likes

Awesome I’ll check and let you know…

1 Like

@techAPJ
Yes it works now.

Is it possible to add an export to csv feature?

2 Likes

Here’s what I am thinking. Please let me know your opinion.

If an Admin wants to drill into the term, they will click on the term and go to term details page, which will look like:

  • Clicking on the term “sso” again, admin will go to full page search with “sso” prefilled.

  • There will be a search type (header/fullpage) filter.

  • Admin will be able to see the stats/trends for different time periods (similar to what they see on dashboard) for that term on one page.

Does the above looks good?

5 Likes

Does unique track IP addresses or user accounts? (since users may use more than one IP)

It sounds like IP addresses here:

And user accounts here:

Our sites are only visible to people with user accounts, and that’s not the case for everyone so probably it’s IP addresses, but maybe there’s some magic going on in the background. Either way, it would be useful for us to know which of the two it is.

Sorry if this is too pedantic.

@techAPJ can confirm but in your case (“a requires login community”) the unique count is the unique count of users. AFAIK we use user_id and then fallback to ip

4 Likes