# Spooky tracking happenings đ»
**URL:** https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255
**Category:** Bug
**Created:** [April 19, 2017, 1:59pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255 "2017-04-19T13:59:28Z")
**Posts on this page:** 16
**Page:** 1
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 19, 2017, 1:59pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/1 "2017-04-19T13:59:28Z")
There are things like the following Iâve been noticing for months, but never filed a report for, because I could never reproduce them consistently. But, finally, accidentally, Iâve caught one on camera! (So please forgive the pausing and confusion reflected in my cursor movements.)
[ ](https://www.youtube.com/watch?v=cM8cPvhCCxY)
The things to look out for are:
1. Unread being empty
2. #Contribute > Feature showing as âNormalâ (despite being tracked, as shown in my preferences) and only showing as âTrackedâ on a refresh
3. A topic mysteriously appearing in unread (from days ago, from the howto category)
The 2nd point is somewhat similar to a common bug I see (but have never been able to reproduce on video) where Iâll click on a topic from new in a category Iâm tracking, but its own status wonât show as tracked unless I refresh the page. [(I wonder if thatâs related to this.)](https://meta.discourse.org/t/unread-topics-in-query-link-lists-tracking-watching/55861)
Just before recording this I was trying to reproduce that bug by tracking #Contribute > Bug, and then clicking on one of itâs topics from the unread tab. I couldnât, but upon un-tracking #Contribute > Bug, my unread tab was filled with about 20 closed topics in #Contribute > Bug from around 2015, which seems to be the same bug as point 3.
I donât understand why this could be happening. đ»
---
### Author:  [@codinghorror](https://meta.discourse.org/u/codinghorror)
#### Post date: [April 19, 2017, 8:13pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/2 "2017-04-19T20:13:39Z")
What are the repro steps?
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 20, 2017, 10:31am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/3 "2017-04-20T10:31:35Z")
There are two bugs here.
I canât consistently reproduce the first, where the correct notification level is only shown after a refresh (of a category, or a topic). Itâs been happening to me for months, but whenever I try to reproduce it, it doesnât happen.
The second, which I think is a bit more minor, but still a nuisance can be reproduced (for a limited time only) like so:
1. Clear `/unread`
2. Track a category currently set to âNormalâ
3. Go back to `/unread` and observe posts not previously in it
Iâve been able to do this with a number of categories, but only once for each. Once Iâve dismissed the posts from unread, they never appear again.
Consistently reproducing it takes a little more work, so hold onto your hats, itâs gonna get a bit crazy:
- Go to unread in a category of your choosing which has a notification level of âNormalâ, observe nothing there:
- List all the currently tracked topics in that category (and remember, or take a screenshot of them):
- Enter a topic from that category not on that list:
- Start tracking the category
- Again list tracked topics in that category
- Observe the topic you just opened in that list:
- Go to unread, and again observe that topic in the list:
* * *
**TL;DR**
So, the bug here is that previously opened topics in a category become tracked, and appear in unread (unless previously dismissed), when the category is tracked, whereas nothing happens to old not-opened-before topics.
To me, what should happen is either:
- Nothing changes about the notification status of old topics in a category when it becomes tracked, previously opened or not.
or
- All old topics become tracked when the category is tracked, but are automatically dismissed, so only new activity appears in the unread tab.
I lean towards the second solution, provided explicitly set notification levels on topics are kept.
I havenât investigated how this works with other notification levels, I imagine its the same, but I feel it would confuse things for the time being.
---
### Author:  [@cpradio](https://meta.discourse.org/u/cpradio)
#### Post date: [April 20, 2017, 11:05am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/4 "2017-04-20T11:05:44Z")
So in relation to
> [@Watching and tracking implementation for tags and categories](https://meta.discourse.org/t/watching-and-tracking-implementation-for-tags-and-categories/46987):
>
> I just revised the implementation of âtrackingâ and âwatchingâ of tags and categories: Watching categories and tags has always been a rather limited feature. When you started watching a category you would only watch topics in that category from that time on. (with an edge case that allowed you to also watch recategorized old topics) This limitation was ⊠hmm ⊠limiting. Start watching #Bug and you would not be notified on a large amount of current #Bug discussion on historic topics. TâŠ
What part is considered as ânot working as expectedâ?
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 20, 2017, 5:51pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/5 "2017-04-20T17:51:38Z")
> [@cpradio](#):
>
> What part is considered as ânot working as expectedâ?
2 parts:
The first would be topics not being properly implicitly tracked. If they were, all of the topics from #Community Building > Praise should appear in this step:
> [@LeoMcA](#):
>
> Observe the topic you just opened in that list:
It seems like that list is just showing topics with a tracking record of âtrackedâ, and thatâs not working as expected. What would be, would be showing all topics with a tracking record of âtrackedâ, and all topics which are in a tracked category. (The same would go for watching.)
Furthermore, while this makes sense:
> [@Watching and tracking implementation for tags and categories](https://meta.discourse.org/t/watching-and-tracking-implementation-for-tags-and-categories/46987/1):
>
> Topics with missing state are watched implicitly, a tracking record is created when we notify users.
It seems like the same functionality is missing when a user tracks a category. Shouldnât a tracking record also be created when there is a new post in a topic?
The second would be the appearance of seemingly arbitrary topics in unread when you track a category, (I assume a side effect of changing the tracking state of past topics from regular to tracked), which could be fixed by automatically dismissing those topics from unread.
---
### Author:  [@Mittineague](https://meta.discourse.org/u/Mittineague)
#### Post date: [April 20, 2017, 6:23pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/6 "2017-04-20T18:23:06Z")
What are your Preferences â Other
Consider topics new when
and
Automatically track topics I enter
settings?
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 20, 2017, 9:02pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/7 "2017-04-20T21:02:13Z")
> [@Mittineague](#):
>
> Consider topics new when
I havenât viewed them yet
> [@Mittineague](#):
>
> Automatically track topics I enter
never
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [April 20, 2017, 9:36pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/8 "2017-04-20T21:36:13Z")
> [@LeoMcA](#):
>
> So, the bug here is that previously opened topics in a category become tracked, and appear in unread (unless previously dismissed), when the category is tracked, whereas nothing happens to old not-opened-before topics.
This is something I found a while ago, and a thread can be found [here](https://meta.discourse.org/t/watching-tracking-old-topics/51606) (not entirely sure why it got moved from #Contribute > Bug to #Contribute > Feature).
The thread is quite confusing as I originally thought the problem was caused by the initial import from phpBB. So hereâs an attempt to summarise:
It comes down to the fact that a topic cannot be both `new` and `unread` simultaneously:
- Even though itâs implicitly tracked, topic does not show up in `unread` because the user has never viewed it
- Topic does not show up in `new` because it is beyond the threshold of whatever your âConsider topics new whenâ setting is
Itâs a hard problem to solve, because having every single old topic in a category appear in your âunreadâ tab is really not a good user experience.
âAutomatically dismissingâ them is quite expensive database-wise (a new TopicUser record would have to be created for every single topic in the category you just tracked).
We didnât really come up with a tidy idea to solve the problem, and the discussion kinda died. One option that @sam came up with was
> [@Watching/Tracking old Topics](https://meta.discourse.org/t/watching-tracking-old-topics/51606/7):
>
> Perhaps, one thing that is âcheapâ that we could do.
>
> If a topic in a tracked category is bumped, add it back to ânewâ for whatever the new duration is.
>
> Still would feel somewhat odd.
At the time, I thought this might be a good idea:
> [@Watching/Tracking old Topics](https://meta.discourse.org/t/watching-tracking-old-topics/51606/13):
>
> At the moment, the ânewâ tab is sorted by âtopic.bumped\_atâ, but filtered by âtopic.created\_atâ. Whatâs the thinking behind this? It seems a little un-intuitive that topics donât âdrop off the bottomâ of the list once theyâre no longer declared new. What would be quite nice, at least for my forum, would be to filter them by âtopic.bumped\_atâ.
But
> [@Watching/Tracking old Topics](https://meta.discourse.org/t/watching-tracking-old-topics/51606/15):
>
> The problem with a change like that is that lots of users will get very confused about how to kick stuff out of new list.
**TL;DR** : I donât have a solution, but hopefully this makes the problem clearer đ
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [April 20, 2017, 9:48pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/9 "2017-04-20T21:48:56Z")
And for the other issue in this thread:
> [@LeoMcA](#):
>
> I canât consistently reproduce the first, where the correct notification level is only shown after a refresh (of a category, or a topic). Itâs been happening to me for months, but whenever I try to reproduce it, it doesnât happen.
I can reproduce this consistently here on meta for categories. Repro steps:
1. Go to your user preferences, remove all tracked categories
2. Go to a category page, e.g. #meta, and observe that the ânotification levelâ indicator is ânormalâ
3. Go immediately to your user preferences, by clicking your avatar in the top right, then the cog
4. Add the category you just visited, e.g. #meta, to your âtrackingâ categories
5. Press âsave changesâ
6. Press the **back button** in your browser to return to #meta, and observe that the ânotification levelâ indicator is still ânormalâ
7. Refresh the page, and observe that the ânotification levelâ is now âtrackedâ
If you replace step 6 with visiting the category page by some other route, the tracking status appears correctly.
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 21, 2017, 8:15am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/10 "2017-04-21T08:15:08Z")
> [@david](#):
>
> It comes down to the fact that a topic cannot be both new and unread simultaneously:
From a practical perspective, that doesnât make sense to me. If I buy a book and put it on my shelf having not opened it, it is both new and unread.
> [@david](#):
>
> having every single old topic in a category appear in your âunreadâ tab is really not a good user experience.
Wouldnât it be possible to have an old topic (with no tracking record) only appear in unread when thereâs new activity on it?
> [@Watching and tracking implementation for tags and categories](https://meta.discourse.org/t/watching-and-tracking-implementation-for-tags-and-categories/46987/1):
>
> When a user watches a category/tag:
>
> [âŠ]
>
> - Topics with missing state are watched implicitly, a tracking record is created when we notify users.
So, just as a tracking record is created on new activity for topics in watched categories, a tracking record would be created on new activity for topics in tracked categories.
> [@david](#):
>
> âAutomatically dismissingâ them is quite expensive database-wise (a new TopicUser record would have to be created for every single topic in the category you just tracked).
Combined with the above, couldnât we just dismiss topics where the tracking record has been changed to âtrackingâ as a result of tracking the category, which would mean the same user experience, without any additional database writes? (If I understand how tracking records, dismissing and TopicUser relate.)
This way, it seems to me, we could have a mailing list experience for tracking too! đ
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [April 21, 2017, 8:20am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/11 "2017-04-21T08:20:34Z")
> [@LeoMcA](#):
>
> From a practical perspective, that doesnât make sense to me.
Sure, I agree. I was talking purely about how it currently works in discourse.
> [@LeoMcA](#):
>
> Wouldnât it be possible to have an old topic (with no tracking record) only appear in unread when thereâs new activity on it?
I believe @sam thought this would have major performance issues
> [@LeoMcA](#):
>
> So, just as a tracking record is created on new activity for topics in watched categories, a tracking record would be created on new activity for topics in tracked categories.
Iâm 99% sure a new record isnât created for tracked topics - @sam said âcreated when we notify usersâ, and tracking doesnât cause notifications. Maybe fixing this is the solution đ
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 21, 2017, 8:38am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/12 "2017-04-21T08:38:15Z")
> [@david](#):
>
> I was talking purely about how it currently works in discourse.
Do you know if thereâs an architectural issue which means the new/unread either/or has to be the case?
> [@david](#):
>
> Iâm 99% sure a new record isnât created for tracked topics - @sam said âcreated when we notify usersâ, and tracking doesnât cause notifications. Maybe fixing this is the solution đ
Indeed, I was proposing this as a solution which seems to not have any major performance issues.
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [April 21, 2017, 8:40am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/13 "2017-04-21T08:40:47Z")
> [@LeoMcA](#):
>
> Do you know if thereâs an architectural issue which means the new/unread either/or has to be the case?
I assume itâs by design - having a topic listed on the new tab and the unread tab would be a bit weird.
Taking your book analogy, while it can be both unread and new, you canât put it on your ânewâ shelf and your âunreadâ shelf simultaneously đ
---
### Author:  [@david](https://meta.discourse.org/u/david)
#### Post date: [April 21, 2017, 10:57am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/14 "2017-04-21T10:57:05Z")
Ok⊠Iâve done some testing in my dev environment. As a simplest possible repro:
1. Set `previous visit timeout hours` site setting to 0, to make testing quicker
- User A sets âConsider topics new whenâ to âcreated since I was last hereâ
- User A sets `Category1` to `tracked`
- User B posts in Category1
- User A observes the topic in ânewâ, but does not open it
- User A leaves the site
- User A comes back to the site later, and User Bâs post is gone from the ânewâ tab
- User B posts again in the same topic
- The topic does not appear in either ânewâ or âunreadâ for User A, even though they are tracking the topic
That same repro works for âwatchingâ a category as well. A notification is sent, but no TopicUser row is created (checked on the rails console). It seems that this:
> [@LeoMcA](#):
>
> a tracking record is created when we notify users.
is not happening for some reason, both for watched and tracked categories.
---
### Author:  [@LeoMcA](https://meta.discourse.org/u/LeoMcA)
#### Post date: [April 21, 2017, 11:22am UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/15 "2017-04-21T11:22:20Z")
> [@david](#):
>
> having a topic listed on the new tab and the unread tab would be a bit weird.
Would it though? We already have topics listed in the latest tab and the new/unread tabs simultaneously. And the two (new and unread) are for different purposes: new is âshow me the newest topics on this siteâ and unread is âshow me topics Iâve expressed an interest in, but havenât (finished) read(ing)â. For new to achieve its purpose it has to have unread topics in it which are new, and for unread to achieve its purpose it needs to have new topics in it which are unread.
Currently the unread tab is rather dysfunctional.
> [@david](#):
>
> It seems that this:
>
> a tracking record is created when we notify users.
>
> is not happening for some reason, both for watched and tracked categories.
Interesting, thanks for doing all this investigation! So it would seem like if this did actually happen (or more specifically, a TopicUser row was created on topic update, rather than notification, to include tracked categories), then our problems here would be solved. @sam thoughts?
---
### Author:  [@sam](https://meta.discourse.org/u/sam)
#### Post date: [November 10, 2018, 9:58pm UTC](https://meta.discourse.org/t/spooky-tracking-happenings-ghost/61255/16 "2018-11-10T21:58:37Z")
I am fine to change the semantics of âtracked categoryâ so it means, if a category is tracked it will unconditionally show up in new regardless of your new duration, it does not mean this now.