# Improving discovery of "your topic is similar to" relevance search
**URL:** https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220
**Category:** UX
**Created:** [November 3, 2015, 4:00pm UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220 "2015-11-03T16:00:24Z")
**Posts on this page:** 10
**Page:** 1
### Author:  [@erlend\_sh](https://meta.discourse.org/u/erlend_sh)
#### Post date: [November 3, 2015, 4:00pm UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/1 "2015-11-03T16:00:24Z")
Continuing the discussion from [Move X down to bottom to close "Your topic is similar to…" popup](https://meta.discourse.org/t/move-x-down-to-bottom-to-close-your-topic-is-similar-to-popup/35186/3):
> [@Move X down to bottom to close "Your topic is similar to..." popup](https://meta.discourse.org/t/move-x-down-to-bottom-to-close-your-topic-is-similar-to-popup/35186/3):
>
> I still struggle with this dialog, have never had a hit there when composing a topic that stopped me from composing a topic or was even too relevant.
Yeh I’ve been pondering this one a bit myself. Improving the relevance check is definitely one part of it, but I also frequently encounter new duplicate topics on Meta that I _know_ had good suggestions upon topic creation (I’ve tested it by creating a new topic and inserting their title & body).
I think an equally important side to this is **getting the user to actually look at the suggestions**. Personally I’ve become practically blind to it at this point. I don’t know how prevalent this “blindness” is, but in my case I frequent such a large portion of Meta that I feel pretty confident any new topic I make will not be a duplicate (also I tend to search before posting, but that can’t be expected of the average user IMO, that’s what the automatic topic search is there for after all). I fear this habit might have transferred with me to other forums as well, where I’m nowhere near as well versed in the forum’s content.
One idea I’m playing around with is to move the JIT notification into the preview field:
The way it could work is:
1. You start typing, and at first all you see is your preview.
2. Two paragraphs or so in, the suggestions pop up.
3. If you just continue typing however, the pop-up will be folded down to an expandable “Your topic is similar to…”. This has the added effect of maybe making the user curious: “hmmm, I wonder if the suggestions are different now”.
I don’t know how exactly the relevance algorithm works, but it would also be very nice if there was a distinction between **very** relevant topics and only moderately relevant ones. Then the relevance search could further peak user’s interest with the occasional “Your topic is **very** similar to X other topics”, referring to topics with \>80% relevance score (not a real thing afaik). Suddenly there’s an exact count for you to look at.
* * *
Not sure I’d even want this, but, further building on the hypothetical relevance score, we could also put an actual stop gap in front of “Create topic” if your topic has other topics \>95% relevant to it. You’d click “Create topic” and you’d be hit with a warning like: “These other topics seem _very_ closely related to yours. Could you please take a look and see if your topic isn’t already being discussed?”
---
### Author:  [@eviltrout](https://meta.discourse.org/u/eviltrout)
#### Post date: [November 3, 2015, 4:35pm UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/2 "2015-11-03T16:35:32Z")
I can tell you personally I would find losing the preview very annoying. I am curious to know if I am in the minority though with this viewpoint.
Additionally, we’d have to consider the case when the preview is collapsed.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [November 3, 2015, 7:32pm UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/3 "2015-11-03T19:32:30Z")
You are talking here about changing a feature that I use 0% of the time to be even more annoying
Our relevance check is super ghetto at the moment, I guess if you had a 90% match on the title with another title… Maybe… But so many people will just type “discourse is not working” getting a half decent algorithm going for forums is just super duper hard
I think we should do some testing here to see what we are returning in “real life”, how often is it helpful, etc.
First thing is that the results need to be much more useful. Also I wish I could opt out of the feature at tl2 or 3
Also see
> **[Abuse of the "New Topic" Button Vs Search Field (Repeat Posts)](https://community.smartthings.com/t/abuse-of-the-new-topic-button-vs-search-field-repeat-posts/24695)**
>
> To all this may apply to… Welcome to the SmartThings Community, this is one of the most amazing development communities around (as quoted by most long term members) Here you will find years of hard work, development, discussions and solutions for...
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [November 4, 2015, 2:37am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/4 "2015-11-04T02:37:52Z")
It is tricky because on the one hand, you have to do _something_ to ensure that people searched before creating a new, duplicate topic. And unfortunately, relevance through searches is a super hard problem.
Contrary to @sam’s observations I have had positive results when creating a new topic and getting a duplicate topic search hit from the “your topic is similar to” panel. Others have reported positive results on Twitter as well if you browse the @discourse Twitter account.
My problem is **I rarely create new topics in the first place** , and on the rare occasion that I do, I know they are guaranteed to be new topics, not duplicates. Probably the same for Sam. To be honest this is a feature for your, uh, “less experienced” users.
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [November 4, 2015, 3:13am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/5 "2015-11-04T03:13:04Z")
I do think there are some severe issues with our algorithm:
I mean **really** “Discourse in Croatian” how is that even relevant.
Or this recent question:
Only relevant hit is the actual topic that the user posted.
Its totally pot luck to get some sort of useful result and the blue color makes it terribly hard to read.
Compared this to StackOverflow that only uses title:
At stack the feature is far less intrusive, and hits better cause there is far better body of work to match by.
My recommendation:
- The blue is very hard to read … almost blinding, I prefer a much toned down styling:
- Consider a full screen topic composer for TL0 and TL1, that way we have lots of room on the screen to teach people and can follow the same pattern used at Stack for similar topics.
- Improve results a lot, currently they are just too random.
- Allow TL2 and above to opt-out of this feature
* * *
As it stands I feel the feature needs lots of love.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [November 4, 2015, 3:19am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/6 "2015-11-04T03:19:34Z")
The way we improved it at Stack was to manually drop the top 10k English words from the query terms, so you are only ever querying on really unique, rarer words rather than “how do I” matches, which are almost always bullshit.
Also to heavily prioritize title matches, but the current code uses both title and body **provided they are long enough**.
We could also scope by category, if entered.
---
### Author:  [@erlend\_sh](https://meta.discourse.org/u/erlend_sh)
#### Post date: [November 4, 2015, 9:53am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/7 "2015-11-04T09:53:36Z")
> [@sam](#):
>
> Improve results a lot, currently they are just too random.
As a very important followup to such an improvement: **Don’t pop up any suggestions unless the algorithm has found something that is indeed quite similar**. This is probably the primary reason for my blindness. Whatever I’m writing, however unique, some suggestions _will_ pop up no matter how much of a longshot they are.
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [November 4, 2015, 9:55am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/8 "2015-11-04T09:55:13Z")
Well, it’s the same sort of stuff that Google’s trillion dollar empire is based on. Search queries, expressed in SQL, probably against full text indexes. It is not black magic from another galaxy.
You might say the same thing to Google: just show me the _correct_ matches for my search 😠
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [November 6, 2015, 1:34am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/9 "2015-11-06T01:34:04Z")
> [@sam](#):
>
> The blue is very hard to read
I agree, @techapj checked in a change to tone down the dark blue a lot
---
### Author:  [@erlend\_sh](https://meta.discourse.org/u/erlend_sh)
#### Post date: [August 7, 2016, 10:56am UTC](https://meta.discourse.org/t/improving-discovery-of-your-topic-is-similar-to-relevance-search/35220/10 "2016-08-07T10:56:59Z")
Continued here:
> [@Avoid notification warning](https://meta.discourse.org/t/avoid-notification-warning/48303):
>
> Hey guys, We use Discourse for all internal communication at Dgraph. When composing posts, a lot of times I have to tag the team members using an internal group. But, Discourse then shows a warning blocking the preview pane that you’re going to notify X members, are you sure? Is there a way to avoid having that warning entirely? Similarly, for topic similarity warning, is there a way to automatically time it out? Say after 10 seconds or so, it can just disappear, instead of having to manually …