Watching/Tracking old Topics

(David Taylor) #1

I recently migrated a fairly large (150k posts) phpBB forum to discourse. A lot of our users want to be able to “track” the majority of the forum, so they’ve added all categories to their “Tracked” categories in their user settings. This works perfectly for new threads created in discourse.

However, when someone replies to an old imported topic, it does not become “implicitly tracked” as I would expect from this thread:

Steps taken:

  • User “David” has the “General” category set to “Tracked” in his user preferences
  • User “Bob” replies to an old thread that was imported from phpBB
  • “David” does not see this in his “Unread” tab

However, this works fine:

  • User “David” has the “General” category set to “Tracked” in his user preferences
  • User “Bob” creates a new thread under “General”
  • “David” sees this in his “new” tab

Some investigation on the console reveals that there is no TopicUser row associated with David for either of the topics, so I don’t understand why the implicit tracking is not working.

David’s user_id is 1, old topic_id is 8531, new topic_id is 10708

[18] pry(main)> TopicUser.where(topic_id: 8531, user_id:1).length()
=> 0

[19] pry(main)> TopicUser.where(topic_id: 10708, user_id:1).length()
=> 0
Watching and tracking implementation for tags and categories
Spooky tracking happenings :ghost:
(David Taylor) #2

This issue only seems to exist with “Tracking”.

“Watching” categories works absolutely fine - if someone posts on an old topic and I have the category set to “Watching” then I do indeed receive a notification
(Still no TopicUser row created until I view the topic, so the implicit watching is working)

(Jeff Atwood) #3

Would a prior migration or import affect this @sam?

(Sam Saffron) #4

You can’t have a topic that is both “new” and in “unread”, it is either/or.

My guess… these topics were never visited by these users, so they should show up in “new” however… you have “new” set to last 2 days or something so they end up missing.

If the users visit the topics they will become tracked.

Implicit tracking means there is no row in TopicUser, it is added first time the record is created (topic visited)

(David Taylor) #5

Thanks for taking the time to look at this.

The settings on my account are the same as the defaults for all our users - consider new when “haven’t viewed them yet”. I suspect the reason that the topics aren’t all showing up in “new” is because all the users have their “first_seen_at” field set to the date of the import, and therefore after the topic was initially created.

So I think the problem is:

  • Topic does not show up in “new” because it was created before the user first visited the forum
  • Topic does not show up in “unread” because the user has never viewed it

I would imagine this logic extends further than imported forums - any new users of a forum who decide to track a category won’t see old topics in new/unread until they’ve viewed the topic at least once. Afraid I don’t know my way around the codebase enough to find the logic for what gets included in the “new” tab, so I could be completely wrong with all of this.

Would it be better change “new” from
unviewed topics created since the user first visited
to
unviewed topics with posts since the user first visited

or maybe better would be to change “unread” from
tracked, viewed topics, with unread posts
to
(tracked, viewed topics, with unread posts) + (tracked, unviewed topics, with posts since user first visited, created before user first visited)

Old unread topic doesn’t show up as unread in tracked category
(Sam Saffron) #6

I can see the appeal on unconditionally adding “tracked” categories into the “new” filter but technically making this work could end up being quite costly performance wise.

Also, we have to keep a cap on new counts cause they are tracked in memory in the app, if you have 500k “new” topics it can brutally slow (and kind of pointless)

Honestly, I am really unsure what the best course of action is here.

Completely unread topics with recent replies should show as new
(Sam Saffron) #7

Perhaps, one thing that is “cheap” that we could do.

If a topic in a tracked category is bumped, add it back to “new” for whatever the new duration is.

Still would feel somewhat odd.

Apply "mailing list mode" per category
Old unread topic doesn’t show up as unread in tracked category
Completely unread topics with recent replies should show as new
Spooky tracking happenings :ghost:
(David Taylor) #8

I feel like adding it to “new” is wrong, because the topic isn’t really new…

Unread might be a better fit, but it does make what is currently quite simple logic get a lot more complicated

(David Taylor) #9

@sam is this something that might be fixed, or do I need to look into a workaround for my forum?

(Sam Saffron) #10

I am simply not sure about how to amend the logic here, if I add to unread then you can end up having 100s of thousands of unread on big forums

(David Taylor) #11

Would you be able to point me to where in the codebase the new/unread logic is?

(Sam Saffron) #12

Topic tracking state and topic query but this is super sensitive code

(David Taylor) #13

Now that I’ve looked at the code a bit, this would certainly be an awful lot easier!

At the moment, the “new” tab is sorted by “topic.bumped_at”, but filtered by “topic.created_at”. What’s the thinking behind this? It seems a little un-intuitive that topics don’t “drop off the bottom” of the list once they’re no longer declared new. What would be quite nice, at least for my forum, would be to filter them by “topic.bumped_at”.

Would there be any way for me to adapt the new/unread logic on my forum using a plugin, without having to hack around in the code? Or is there a best-practice way for patching the core code so that I can still apply updates?

Edit: I’ve come up with this plugin which works, but doesn’t apply the “new” badge to topics. Is the logic for that badge elsewhere?

# name: filter-new-bumped
# about: A super simple plugin to override the new_filter to use bumped_at instead of created_at
# version: 0.0.1
# authors: David Taylor

after_initialize do
	require_dependency 'topic_query'
	class ::TopicQuery
	  def self.new_filter(list, treat_as_new_topic_start_date)
	    list.where("topics.bumped_at >= :created_at", created_at: treat_as_new_topic_start_date)
	        .where("tu.last_read_post_number IS NULL")
	        .where("COALESCE(tu.notification_level, :tracking) >= :tracking", tracking: TopicUser.notification_levels[:tracking])
	  end
	end
end
Spooky tracking happenings :ghost:
Completely unread topics with recent replies should show as new
(David Taylor) #14

@sam what do you think about changing the new logic to be bumped_at rather than created_at?

I think if things are left as-is, then some clarification should be added to the “tracked categories” box in the user preferences - should really be called “track viewed topics in categories”

(Sam Saffron) #15

I don’t know, maybe add a hook there so you can easily amend it in a plugin.

The problem with a change like that is that lots of users will get very confused about how to kick stuff out of new list.

cc @codinghorror

Spooky tracking happenings :ghost:
(Erlend Sogge Heggen) #16