Imagine this scenario, where I want to continue reading a topic about Star Trek that I’ve previously been reading.
I go to a Discourse forum
I click the search button at the top and type in “Star Trek”.
5 different topics are presented to me, the top one is what I want
I click the top one and am taken to the topic in a completely random place.
Usually it’s way past my current read marker, which means that read marker is now destroyed.
Hmph. “Advanced search” is just as unhelpful, as I just get 50 different thread choices and I still want the top one which still takes me to a random place.
So now I’ve learnt to:
I go to a Discourse forum
I click the search button at the top and type in “Star Trek”.
5 different topics are presented to me, the top one is what I want
I right-click the top one, and copy the URL.
I paste the URL into my browser and change the /123 instead /1.
I continue to read the topic without the dreaded fear of missing an important post about Star Trek.
Is there any magic voodoo I can type into the search bar to always take me to the first post, or to my read marker?
One change I am open to is amending search so it always takes you to the first occurrence of the keyword/s in the topic vs something inconsistent like it does now.
But… taking you to first unread unconditionally, that seems very confusing. I searched for bomb and am looking at a post with no bomb in it. The one with bomb is 2000 posts down.
I obviously didn’t mean random each time, instead I meant:
Not useful
Not what I Want ™
Not at the start
Not at the end
Not where I last read
Not even vaguely related to the search term
Not where I’d place a bet on it being
On point 6: What makes post 727 rank higher than post 1? Post 727 doesn’t even contain the term Star Trek, which is what I’m actively after, whereas Post 1:
a) has it once
b) is the initial post
c) is the namer of the thread whose subject contains the words Star Trek.
Interestingly, every other search result has Star Trek in the body of the post and is highlighted bold, but not this first results of post727 in the thread I’m after. Some results even have Star Trek in the title as well. So how come they aren’t ranked higher? (Frankly, I’m glad they’re not in this case, as it’s the first thread I want, but not that post!)
Do you mean the first post in the topic, or the first post in the topic with the words “Star Trek”?
I was asking if there’s any voodoo I can type to take me to the first post in a topic. I’ve just gone and had another look at the Advanced Search page and I’ve found that the Only return topics/posts... are the very first post , which shows that yes there’s in:first. However taking that makes the results COMPLETELY different from the other ones, even though a lot of those other threads contained “Star Trek” in the first post. In the non-voodoo case most of the returned threads are very recent. In the in:first voodoo then I’m getting results from 2003 at the top. Is there any way to combine in:firstin:title and still keep the magic ordering that makes recent and relevant ones appear first?
Why can’t Discourse just read me mind and do what I want???
I feel like you haven’t been able to adequately explain what you want to me, a human, so it’s unlikely that you would be able to explain it to the computer, either. Because the computer is much more strict by nature.
So we are searching for a common word or phrase that appears hundreds or thousands of times on the site across dozens of topics, yes?
if you only want hits in the topic title then use that search qualifier
if you only want hits in the first post then use that search qualifier
if you want results ordered by most recent posts first then use that search qualifier
Is there any specific reason that one or all of the above existing options does not work for you? If not, can you explain specifically why not, with a real world example?
I’ve never able to adequately explain concepts to you, including the concept of the Android back button, which is apparently a massive mystery. But I’m dogged, so I shall keep trying.
TO WIT:
I don’t want my read marker in a thread (around post 300) to be destroyed because a crap search algorithm took me past it (to post 727), and to a post that doesn’t even contain the words “Star Trek”.
Even when I’ve read an entire thread, I don’t see the utility in being thrown randomly in the middle. Either the beginning or end are sensible places I’d expect to end up.
Frankly, I’d also like to not have to faff about in the advanced search screen just to find a thread that’s currently active, that I’ve posted in, that contains the words I want in the title. The majority of my searching and selecting-of-search-results is done in the “find” drop-down from most pages, I don’t want to have to type in endless amounts of stuff just to get the search to “behave”
So we are searching for a common word or phrase that appears hundreds or thousands of times on the site across dozens of topics , yes?
Yes, we are. So it’s baffling that the search algorithm takes me to a post that doesn’t even contain the words.
If not, can you explain specifically why not, with a real world example?
In:title doesn’t change the result, I still get a random post deep in the middle of a thread about Star Trek, when that post itself doesn’t even contain the words “Star Trek”.
in:first : The thread that was previously “the best result” for “Star trek” is now the 46th best result, even though the first post of that previously “best result” contains the words Star Trek, and the majority of the top 10 results are from 10 years ago. But hey, at least it’s not showing me post 727 that it loves so much!
Latest thread has it around 15 deep. AGAIN IT’S THAT SAME POST! What does it love so much about post 727? (In this specific case, I find it odd that the 14 threads before this one don’t contain “Star Trek” in the title, given that my search criteria is around threads.)
I can’t post an example of in:seen as I’ve now read past that famous 727 post that everyone loves so much. I mean : I’ve physically read those posts with my eyes. At the time of posting I had “read” that post in the sense that “Discourse search messed up my read position months ago and I’ve not bothered to figure out where I left off”.
Back in the old days when we migrated q23 @eviltrout noticed that when certain people would search for ultra common terms like america the whole search subsystem would grind to a halt.
Se we introduced a site setting called search_prefer_recent_posts and a setting called search_recent_posts_size
These 2 features control a special feature that is default disabled.
When enabled and you perform a search, we first search the lastsearch_recent_posts_size (default 100000) before expanding our search to your full corpus of 5 million or so posts.
This means that super common searched do not crash the system, but it also impacts the search results quite badly in certain cases.
Will always include the topic title for every post. This usually is not a problem cause our aggregation within topics always prefers the most relevant post per:
However… in your Star Trek example, your last post to mention the words “star trek” is:
And this, it appears is more that 100,000 posts ago in the system, so we are stuck with a few hundred of posts with the word “Star Trek” in them (due to the title) and ranking ends up being a little bit arbitrary cause all the results are bad, as none of the posts have “Star Trek” in the body.
My initial recommendation here is first for you to get a good feel of how search, is meant to work by asking @clay to disable search_prefer_recent_posts for 20-30 minutes and running tests.
After that… maybe you can expand search_recent_posts_size to 300000 or something like that and see if stuff feels better.
Finally, maybe we can do some extra magic here to always prefer #1 if there are no hits in the body AND search_prefer_recent_posts is enabled.
Thinking about the q23 problem maybe it makes sense just to do a title+op search first and land on post #1 if we get enough results, then fallback to expanded search
I guess I am having a hard time understanding what your actual goal is? I have re-read your posts here twice and it is still a mystery to me. Like, forget the technical details for a moment, what is your goal as a human being here? What is the purpose of these searches? What are you trying to achieve?
I tried that originally. That doesn’t change the result at all. I’m still offered, as the first result, a post that’s
In the middle of the thread I want
But way past my read marker
And doesn’t even contain the term "Star Trek". (Nor the terms "Star" or "Trek")
Can you explain to me what is “broken” about this search result?
See the above list. (Or the above posts which say the same thing)
What about it does not work for you?
As stated already: I don’t really want to faff about in the advanced search screen simply to avoid destroying my read marker. It’s easier for me to just let Discourse present me with a random post from the thread I want, then copy + paste + modify the URL myself so that I ensure I am in the thread and before my read position. The easiest way for me to do that is to go to the first post.
I guess I am having a hard time understanding what your actual goal is? I have re-read your posts here twice and it is still a mystery to me. Like, forget the technical details for a moment, what is your goal as a human being here? What is the purpose of these searches? What are you trying to achieve?
I think this is the fourth post explaining it, but I shall persevere:
I want to locate the Star Trek thread. I want to do this because:
I’ve actively been reading it multiple times in the past month
I’ve actively been posting in it multiple times in the past month
I want to read the unread posts that other people have very recently posted.
I might even want to reply, depending upon the content of the posts that I’ve just read
I don’t want to go to a random “post” in the middle of a thread, I just want to go to “the thread”.
I don’t want a search result to mess up my read marker
I want to locate that thread easily. I do not want to have to faff about in the advanced search screen messing with lots of different settings, all in the hope of trying to not destroy my read marker by being inserted into the thread in a completely random place.
Side issue: I don’t commit the number of the last post I read to memory, so I cannot pre-emptively know if a search result will take me to a post that is way beyond my read marker.
If Discourse is going to suggest a post to me as the “top result”, and this post violates the other conditions, I would at least like to be able to make the excuse, on the software’s behalf, that the post contained the terms "Star"|"Trek"|"Star Trek" and it’s therefore understandable that it took me there. Post 727 does not contain these terms. I have literally no idea why the software wants me to go to that post. (Ignoring Sam’s description of the technical details). If Discourse likes post 727 so much, why doesn’t it just marry it?
But you only read the topic once in a while? So you are basically ignoring the topic for long periods of time (Months? Weeks? Can you clarify how long?), then suddenly deciding you are interested in it again, and you can “only” find it using search?
Well, that should be easily possible using the search terms:
(Also note that we have at least one search shortcut that would help you here — l by itself means latest first. Try searching for “Star Trek” l to see what I mean.)
Still, I am unclear why you don’t use the built in topic watching and tracking facilities to read along with the topic? It sounds like your interest in this topic waxes and wanes at irregular intervals and that prevents you from using the normal notification facilities for following along with a topic? You do not actually read all the replies to the topic, can you answer with a simple yes or no to that particular question?
Simply clicking on the topic when it appears in your suggested topics (which it will, if it has new replies and you are watching or tracking the topic) would always take you to your last read position.
I think what you are running into here is an oddball scenario that you’ve created for yourself.
One thing that might help in the search clause is topics that you have posted in before which is in:posted.
Another possibility is if bookmarked topics offered a way for you to enter at your last read position@sam — that seems like the primary feature request here, because right now if you bookmark you always go exactly to the bookmark location.
I think there is merit for another ninja search operater especially for sites that are forced into the restricted search index due to being both enormous and under powered
Star Trek u
Take me to the first topic containing the words Star Trek anywhere at my last read position.
That would be a bizarre shortcut. I think something else makes more sense:
fp — only match this term in the first post
You would still have difficulty because clicking the search result would send you to the first post though.
Maybe every search result should offer a “take me to my read position in this topic” click or tap action somewhere.
Also bookmarked topics not offering a “take me to last read position” option is IMO a giant omission and the first thing we should do here.@sam I think you should assign that one immediately.
@pmusaraj can take that when he frees up, but keep in mind almost no one bookmarks anything, it is just the nature of the beast, and g b does provide this feature
I still agree with @Pod here that search gets mega weird once you run with an index subset due to the fact we index topic title with every post
