Natural breakpoints or "chapters" for long topics?

Feature
1

One thing we’re thinking about is how to determine natural breakpoints or “chapters” for long discussion topics – and we can eventually display these breakpoints as navigation selections on the expanded topic progress bar.

Say you have a 500+ post topic that you want to explore. How do you do that?

threadwouldntdie.jpg382×576 46.3 KB

One way is to use our “summarize topic” function, which uses our internal topic hotness algorithm to produce the reader’s digest condensed version of the topic – by stripping out 80% of the posts.

The traditional answer here is pagination, but when have you ever seen a book with chapters like this?

  • chapter 1: pages 1 - 50
  • chapter 2: pages 51 - 100
  • chapter 3: pages 101 - 150
  • chapter 4: pages 151 - 200

Chapters in a book have titles, and tend to break on meaningful turning points or inflection points in the story.

So the question is, what exactly are those natural inflection points in a 500+ post topic?

We have some ideas.

Activity-based breakpoints

Conversations tend to ebb and wane. There will be 50 posts one day, then everyone goes to sleep for the night. They wake up the next day and continue the conversation. The natural breakpoints are at lulls in the conversation:

  • Activity period 1: Monday - Wednesday
  • Activity period 2: Thursday - Friday
  • Activity period 3: Sunday
  • Activity period 4: May 2015

Time-based breakpoints

But what if the conversation never pauses, never wanes? How can you have meaningful breakpoints when the participants are in so many timezones, and the conversation so intense, that it never lets up?

Maybe you could use arbitrary time based breakpoints:

  • Time period 1: May 2014
  • Time period 2: June 2014
  • Time period 3: First week of June 2014

Hotness or Significance breakpoints

We could use our existing in-topic hotness algorithm to determine which posts in the topic had the most “heat” and display some kind of visual heatmap of the topic, a map of interesting areas in the conversation where something significant happened as measured by likes, replies, internal/external links, read time, bookmarks, and so forth on each individual post in the topic.

So those heatmap breakpoints would look like:

  • Significant event 1: three days ago
  • Significant event 2: yesterday
  • Significant event 3: six hours ago
23 Likes
2
3 Likes
3

I think this idea, the expandable progress bar, and the pure js scroll bar, are all worthy of consideration.

Perhaps even in ‘non-best-of’ mode, you could jump to the next ‘best-of’ post?

Some visualization of the activity might be a useful way to let the human nav quickly to areas of potential interest.

1 Like
4

I wonder if word frequency data might help with this. If you have a dense patch of posts using an infrequent word, you probably have some facet of a larger topic being discussed. Perhaps those words could lend themselves to more book like chapter titles.

Edit: There is a great presentation by Mark Harwood titled Revealing the Uncommonly Common with Elasticsearch. Reading the problem statement in this post reminded me of the examples presented by Mr. Harwood.

5

Personally, I feel activity based breakpoints are the first thing to explore. I think they will work most of the time.

We can also explore adding subtle hints into the stream for these breakpoints, so users can get a feel of when they happen.

5 Likes
6

A post frequency based approach would certainly be much easier to implement, but some kind of content-based topic change detection would be really cool.

These published papers after a google scholar search for ai topic change detection look very interesting, especially the second one:

3 Likes
7

Here comes the voice of reason from an average joe monkey coder so take it for what it is.

Out of these great choices, which one is the easiest solution from a coding perspective that won’t generate a defensive coding paradigm or spark incessant bitching from the WTF people?

Do that one.

Better yet the ‘topic hotness widget’ already exists and we all understand it (like when I click your picture I get the main topic and your responses minus all others). So the best option is to do nothing and keep that functionality status quo.

8

Oh, I am so looking forward to this. If you can implement it, and the expandable progress bar well, that’s one of my major gripes fixed with Discourse.

9

that would be option 4, implement standard pagination :slight_smile:

10

Activity based navigation seems the best idea. Definitely makes more sense than standard pagination (and I’m from TDWTF so that’s saying something).

If possible, I would also include “significant events”, from the hotness heatmap idea. So imagine an asymmetrical activity based timeline, within which you have clickable hotlinks to the most prominent posts in the thread.

You can even allow the moderators / power users / most active posters promote individual posts into the timeline. For example, someone derails the thread in a community where such thing doesn’t trigger branching into a separate thread (like, say, TDWTF). The derailment post is significant, but (for whatever reason) doesn’t generate enough likes to get into the map. In that case, someone should be able to click and make it a “landmark post” or whatever it’d be called.

11

Now that large breaks can be found and flagged:

Some sort of way to find those breaks from the very top of a topic would be nice.

2 Likes
12

OK, now that vertical timeline is merged into master, this is worth some consideration for future versions. What all should be highlighted in the timeline?

  • gaps, periods of long inactivity?
  • super highly liked posts?
  • periods of crazy intense rapid post activity?
  • famous people posts (blizzard employees?)
  • moderator posts?
  • topic splits or merges?
  • Friday the 13th?

I dunno, :cloud_lightning: brainstorm!

2 Likes
13

couple other ideas (the theme is me, me, me)

  • posts i’ve bookmarked
  • posts i like
  • posts that mention me
  • my posts
  • posts in reply to me
2 Likes
14

Of these I feel “super highly liked posts” is the strongest candidate by far.

1 Like
15

I could see “gaps” shown as small gaps in the blue line (that is, wherever a “x intervals later” marker would be inserted in the post stream, make a little gap in the blue line too).

4 Likes
16

The scientist in me wants simple grid ticks - boring!

I think “badge granting” posts might be good. Perhaps not the Bronze “first” badges, but the Silver and Gold.

1 Like
17

This is the only one I personally care about. I can think of numerable occasions when I’ve reached the end of a topic where I see that apparently I’ve bookmarked something in here, but I’ve no idea where that post is.

Having the possibility of enabling any of this other stuff with plugins would be great - we know our game communities would love a “famous people posts” for instance - but each community’s preference could vary greatly.¨

But hey, what happened to breakpoints and chapters?

We’re suddenly talking about any kind of timeline gimmick here. This topic started out trying to solve a very specific problem: Mega-topics.

I agree. I’m having a hard time mocking up something that makes perfect sense though. I’ll try dig around on Boingboing now that the Timeline is live there. I’ve yet to experience a mega-topic that made me go “something is very broken here” (though I did notice some tiny glitches that I’ll report in the other topic).

3 Likes
18

Pretty sure there’s a keyboard shortcut for that, isn’t there @sam? I guess not? Maybe there should be, though? It feels like power user stuff to me. Everyone else would just visit their bookmarks page and click through from there… no wondering needed.

Solution is the same: show me the most liked posts in the timeline. That’s very likely where the most interesting stuff in the conversation is. I also think showing significant gaps by subtle little breaks in the timeline per @mcwumbly is an excellent suggestion, though I don’t necessarily want to do it in this release.

2 Likes
19

Might just be my high activity level on a Like-less forum, but I’m not sure that’s necessarily the best of the aforementioned breakpoints–especially since, as I understand it, those are the posts most likely to show up in the Summarized View, anyway, right?

I almost wonder if something akin to a “sliding scale” for the timeline slider might be valuable–increase its “sensitivity” around areas of high post-activity, decrease it so it moves a little faster along areas of low activity (probably needs to include a rough date/time indicator to help elucidate what’s happening). Possibly even color variation as you scroll into high-activity areas.

I’m thinking of something like, say, a thread dedicated to a beloved, long-running TV show. During the summer and winter breaks, activity peters off, with news posts only showing up every couple of weeks as, say, casting news or cancellation rumors bubble up. As the premier nears, or in the aftermath of a particularly momentous episode, posting increases massively, and if I’m moving through the huge thread of a show way after the fact, I’d find it interesting to sorta be able to pinpoint these “inflection points” in its discussion easily. Especially if the sheer volume of posting during those moments prevents any single post from accruing a critical mass of Likes.

(To that last point, again, having spent precious little time on Like-heavy forums, I may have a skewed perception of how useful that metric would be for finding conversational turning points)

2 Likes
20

I don’t know how well it would translate, but I like the idea of some type of “heat map / cold map” indicator for posts similar to topics in topic lists.

I think it would need to be subtle almost to the point of being unnoticeable to not cause a potentially annoying visual “flashing” during rapid movement.
But AFAIK a lot of post data is already there so it might be doable.

I’m not sure how “drag” and “slip” based on clustering of post dates would work, but I like that idea too.

A problem being that the timeline is dual purpose.
On one hand it provides an indication of where one is in the topic and moves as the page moves.
On the other it serves a a way to move the page. as the slider is moved.

That is, the page moves the slider and the slider moves the page.