Prioritizing content more effectively in search


(Sam Saffron) #1

When I search for the word emoji I get:

This is pretty bad cause it really is not providing much value to me. I think a smarter algorithm for prioritizing could do a much better job.

In particular, here are some changes I would suggest to fix up this instance:

  1. Allow admins to customize search priority by category. Allow each category to specify some sort of “extra weight” for a particular category. So #howto, for example could be much higher priority on meta.

  2. Allow admins to “de-prioritize” old closed topics in a category. Old closed bugs should always show up at the end of the list.

  3. Allow moderators to specify a “high priority” time window. For example, I would always like the first 2 results of the search to return hits from the last 14 days if available.

This has been discussed a lot, in particular here:

I get around this issue by adding order:latest but most end-users would have no clue about that.


(Jeff Atwood) #2

Doesn’t an implicit focus on order:latest accomplish this anyway?


(Sam Saffron) #3

Perhaps, it depends how far we go though. There are ancient results we always want to show, so they need a way to opt out.

There may even be ancient old closed results we always want first like https://meta.discourse.org/t/what-do-user-trust-levels-do/4924 which number 15 here: Search results for 'trust level' - Discourse Meta pushing it even further down would worry me.


(Jeff Atwood) #4

This feels a bit… imagineer-y to me. Ultimately you would be rebuilding Google’s PageRank, that is, the topics that get the most backlinks (from reputable, non spammy sources…) get elevated in the search results.

Because that’s sure as heck what happens when you search for “emoji” in Google.


(Anton) #5

I support adding weights to categories.

As a very minimum, I’d decrease the weight of a “chill” category - its entries bother new users quite often.


(Christoph) #6

How about the obvious:

  • incoming links
  • bookmarks
  • likes
  • views

Or are these already being weighted in the existing algorithm?

Sure, finding the perfect formula might be a bit much imagineering, but I agree that the search results could be improved.

Would the new algorithm be something that users can turn off in advanced search in order to exit the collective search bubble?

[ ] Turn off SaffronRank

(Mittineague) #7

Nothing personal Sam, but to me “search for the word emoji … not providing much value” begs the question, “what exactly would you consider to be of value?”

Kind of depends on what exactly one is looking for, no?
IMHO successful results depends greatly on ones search foo, good UI, and good search features.

As far as I’m concerned as a somewhat veteran user as opposed to a newbie, Discourse search works great. I can “sort by”, go to advanced search, and eg. search for
emoji large
emoji fontawesome
emoji tone
emoji email
and get more fine tuned sets of results.

I have the feeling what this discussion is more about is how to have search have something like Google “ads” where a site could promote certain topics for searches that use certain “keywords”

As a rough analogy, If I go to a restaurant wanting roast beef rare, I ask for roast beef rare. If I go to a restaurant knowing I want seafood, but I am undecided about what seafood exactly, I ask the waiter for his recommendation.

I think having something like an “Admin Like” could work, but I’m not sure how to go about tying it to specific words.


(Wes Osborn) #8

We have a similar situation with our Discourse instance. We use it for our Knowledge Base but also for our forum.

Just this week I had someone struggling with something and they were looking at a two year old post on the topic rather than our “official” KB article on the same subject.


(Jeff Atwood) #9

I can support this if adding search weights to categories is super super easy, otherwise it feels like the wrong approach.

We need to build Google’s PageRank properly if we are going to bother at all, we definitely have the backlink data (internally) as we track all links between topics.


(Erlend Sogge Heggen) #10

Related:


(Jeff Atwood) #14

This has definitely been discussed internally and seems like the most logical next step. Most of @sam’s complaints seem to be around certain categories (like #howto and #howto:faq ) not showing up higher in results.

Topics with more internal backlinks showing up higher in search results, is also sensible for similar reasons… but might be harder to do than a blanket category bump that site owners can tweak to taste.


(rizka) #15

At our hockey forum we have a category for live commentary of matches. The threads can gather nearly a thousand posts during a match which lasts less than three hours. (Most comments have from 1 to 10 words.) There are tons of likes and common search terms appear in threads many times. Thus, threads of the live commentary category show up high in search results with most search terms. Afterwards, the threads have some value: you can relive games and sense the emotions of fans by reading a thread. But that is all they have to offer. And most of the time when you are searching, you are certainly not looking for fans’ emotions. This is a little problem for our forum’s search function.

To sum up, sometimes instead of bumping some category up in results, you may want to drag a category down in results.


(Jeff Atwood) #16

I suggest deleting those chat topics after they are done. Similarly, I question the value of us carrying ancient resolved bug discussions around forever @sam.

A tiny handful of bugs might have relevant discussion years later, but 90% of them are just search land mines we are leaving around that will blow up anyone who happens to land on them.


(Sam Saffron) #17

Still I agree with the concept of de-prioritizing a category, I would consider de-prioritizing uncategorized on meta

Agree we can probably delete a bunch of land mines, especially ones that are messing up our top 100 search terms


(Jeff Atwood) #18

Not saying deletion is the whole solution either … but surely you can see how carrying around a bunch of ancient resolved bugs forever and ever is ultimately a bad deal for everyone involved.


(Sam Saffron) #19

Totally just it gets super hard to do automatically

The deletionist is fine with deleting 10% good content to get rid of 90% bad

The preservationist is not good with deleting 0.05% good content

And everyone is somewhere on the deletionist/preservationist scale


#20

While I think things like bug reports and game commentary do lose value over time and become irrelevant, I see no harm in keeping everything as long as it’s clearly labeled.

The idea of an archive might be the perfect balance between deleting and keeping obsolete topics. It gets old topics out of sight but they remain there for any future need (whatever that may be).

Basic concept:

1- Create Archive(s) category
2- For every category in the forum - bugs / support / lounge etc - create a matching subcategory under Archive(s) automatically.
3- Add option in wrench menu to move post to archive (filed under correct subcategory automatically)
4- Remove archived posts from live search results by default
5- Add option in advanced search options to include Archives

Bonus:
Add header to archived posts letting readers know they’re in the archives


(Dave McClure) #21

There’s already the ability to archive topics, so no need for a separate category:


#22

Ha! While typing this I had a feeling this idea was probably most likely mentioned here before with all the brains flowing around in these forums :grin:

Oh well, looks like I’m three years late to the party


(rizka) #23

I appreciate the fair suggestion, but it’s impossible. We certainly want to offer the option of reading old chats. Just having those threads at the top of search results is too much. Therefore having category weights would be yet another great improvement for us. :+1: