Discover popular conversations in your community with Hot topic sorting

announcements
,
1

Keeping up with the content in active communities can be challenging. Outside of reading everything created in your community, how can you be sure that you’re reading what’s resonating most with your members?

Introducing Hot, the newest topic sorting option that helps you discover what’s currently trending in your community based on recent likes and lively discussions. Think of it like the trusty Latest filter, but a bit smarter — Hot helps you discover content that is recent and resonant with your community members.

How does Hot sorting work?

The Hot sort algorithm blends the overall popularity (based on likes, reactions, or comments) of a topic since it was first created and its recent popularity based on how members are interacting with the topic. This means that Hot will surface newer, actively discussed topics alongside older, popular topics. Hot sorting is ideal for members in large, active communities that want to find conversations that are currently trending, which they might want to participate in.

How is it different from Top?

Unlike Hot sorting, which looks at a combination of overall popularity and recent popularity, Top sorting only looks at the overall popularity for the selected time period (e.g., week, month, etc.). Top is most useful when you’ve been away from the community for an extended period of time and want to read up on what happened.

How do I turn on Hot sorting in my community?

If you’d like to use Hot sorting in your community, head to Admin > Settings and filter for “top menu”. You can then select hot in the drop-down menu to add it to the top menu navigation options.

top menu setting

Can I make Hot my default home page?

Yes! As an admin, you can set Hot as the default sorting option in your community, as we have done here on Meta. To set this up, go to the top menu setting (Admin > Settings > Basic Setup, as shown in the above section) and use the arrows to move hot to the top of the list of sort options.

As an end user, you can change your preferred default home page by going to Preferences > Interface and updating the Default Home Page setting to your preferred option.

top menu setting

23 Likes
2

Would a sudden, significant, and recent spike in views of an older topic also cause it to be prioritized via this new sorting option? :slight_smile:

8 Likes
3

I’d certainly prefer views to be included, if possible. On my forum, we get only about 100 likes a week, but 2000 logged in user pageviews and 7000 anon pageviews. (Illustrative of a small-ish forum.)

6 Likes
4

Thanks for the update, @lindsey and co.

We’ve switched it on and will experiment.

Out of interest, does it take a while to “kick in”? At the moment there are “no hot topics”.

With all due respect, our active community has got loads of hot topics on the go at the mo. :slight_smile:

5 Likes
5

+1 " There are no hot topics" on my forum which certainly has hot topics.

5 Likes
6

Need to upgrade discourse to the latest

2 Likes
7

@simon_tomes @dfabulich you should see hot topics in your respective sites now, we initially had the feature behind a hidden flag, it got removed last week, and therefore your instance needed an update.

10 Likes
8

It’s working now, thanks.

4 Likes
9

I see hot topics. Thanks, @pmusaraj.

3 Likes
10

Hmm might I recommend not selecting this as the default view, at least not until some kind of announcement or preparation? In some forums, people will want to see what’s new. They might for example want to see new queries or new arrivals. The new hot view is useful for entertainment or for following popularity but immediately causes a problem of surfacing new content.

6 Likes
11

It is only default here on meta, we are gathering feedback on how it feels when it is default. We are not sure this is a long term default for meta or not.

We have no concrete plans on when and if we will amend the product to default to hot.

10 Likes
12

People can be quite blind and not looking at familiar thing where changes aren’t expected. Well, I am anyway.

Meaning I was wondering what is wrong here because I was totally sure /latest is somehow broken and I didn’ get what I should.

It bothered me, but not enough at the moment.

No, I didn’ see and realize that my startingpoint/homepage was changed to /hot. Now I’ve fixed for me, no big deal at all, but my weak point is that in discussing forum discussion and even annoucements are the thing when something is changed :smirk: And touching and altering my personal settings even when it is matter of forum defaults is such thing.

Back in business.

I’m not totally sure about hot, who and why it would be used. Sure, it is another option and doesn’t break anything at all, but as I see it here it is quite close to /latest but without… the latest.

Let’s give it try. And… even I don’t feel it useful somebody else sees it differently and those two aren’t conflicting each others.

But I’m wondering… would or could /hot be working solution to show to random visitors? But would it then break Google or is it totally happy with sitemap?

1 Like
13

Just one thought on that - the coverage of a Discourse forum in the Internet Archive’s Wayback Machine is already a bit patchy. I suspect making hot the default for that agent or usecase could make things worse. Or possibly better, of course - some thought would be needed.

1 Like
14

Agree - this has already bitten me three times in the past day.

In order to avoid a “me too” kind of post I will use the opportunity to point out the fact that the default home page can be overridden in your personal preferences, something I just did to avoid getting bitten for the fourth time. https://meta.discourse.org/my/preferences/interface

8 Likes
15

I see someone has reported and re-reported a problem with excessive sidekiq jobs, with no response other than mine, and that thread is down at about the 100th position in the current default Hot ordering. Not great for a support forum, I’d suggest.

2 Likes
16

It might be worth checking how the scoring is working, for that thread, which is 3 days old but has two recent responses and some likes. Should it still be down at such a low position?

17

I think “hot” can serve as an excellent default setting, providing a snapshot of key topics, which is beneficial for first-time visitors or occasional users. This can also work for support forums, since for those users, an announcement can be more important than the support question of a single user.

For those who want to stay fully informed and not miss any topics, the option to use ‘latest’ is still available. They can even switch their personal homepage to ‘latest’. Additionally, utilizing the ‘new’, ‘unread’, and tracking settings can help ensure that users remain updated on topics of interest.

.

2 Likes
18

This to me is a bit of an unfair critique, its a tough topic. Plenty of support topics to do not get “enough” eyes for whatever definition of enough.

This is an experiment, you disliking hot as a default for meta is one data point. We will collect more.

As to the algorithm:

We can adjust gravity, so we don’t “pull down” topics as quickly as we are now. I can also try to think of some way to give a tiny boost to stuff based on “last reply” as well

At the moment the algorithm is here:

AI Bot explaining hot AI

sam:

DB.exec(<<~SQL, args)
      INSERT INTO topic_hot_scores (
        topic_id,
        score,
        recent_likes,
        recent_posters,
        created_at,
        updated_at
      )
      SELECT
        topics.id,
        0.0,
        0,
        0,
        :now,
        :now

      FROM topics
      LEFT OUTER JOIN topic_hot_scores ON topic_hot_scores.topic_id = topics.id
      WHERE topic_hot_scores.topic_id IS NULL
        AND topics.deleted_at IS NULL
        AND topics.archetype <> :private_message
        AND topics.created_at <= :now
      ORDER BY
        CASE WHEN topics.pinned_at IS NOT NULL THEN 0 ELSE 1 END ASC,
        topics.bumped_at desc
      LIMIT :max
    SQL

    # update recent counts for batch
    DB.exec(<<~SQL, args)
      UPDATE topic_hot_scores thsOrig
      SET
          recent_likes = COALESCE(new_values.likes_count, 0),
          recent_posters = COALESCE(new_values.unique_participants, 0),
          recent_first_bumped_at = COALESCE(new_values.first_bumped_at, ths.recent_first_bumped_at)
      FROM
        topic_hot_scores ths
        LEFT OUTER JOIN
        (
          SELECT
              t.id AS topic_id,
              COUNT(DISTINCT p.user_id) AS unique_participants,
              (
                SELECT COUNT(distinct pa.user_id)
                FROM post_actions pa
                JOIN posts p2 ON p2.id = pa.post_id
                WHERE p2.topic_id = t.id
                  AND p2.post_type = :regular
                  AND p2.deleted_at IS NULL
                  AND p2.user_deleted = false
                  AND pa.post_action_type_id = 2 -- action_type for 'like'
                  AND pa.created_at >= :recent_cutoff
                  AND pa.deleted_at IS NULL
              ) AS likes_count,
              MIN(p.created_at) AS first_bumped_at
          FROM
              topics t
          JOIN
              posts p ON t.id = p.topic_id
          WHERE
              p.created_at >= :recent_cutoff
              AND t.archetype <> 'private_message'
              AND t.deleted_at IS NULL
              AND p.deleted_at IS NULL
              AND p.user_deleted = false
              AND t.created_at <= :now
              AND t.bumped_at >= :recent_cutoff
              AND p.created_at < :now
              AND p.created_at >= :recent_cutoff
              AND p.post_type = :regular
          GROUP BY
              t.id
        ) AS new_values
      ON ths.topic_id = new_values.topic_id

      WHERE thsOrig.topic_id = ths.topic_id
    SQL

    # we may end up update 2x batch size, this is ok
    # we need to update 1 batch of high scoring topics
    # we need to update a second batch of recently bumped topics
    sql = <<~SQL
      WITH topic_ids AS (
        SELECT topic_id FROM (
          SELECT th3.topic_id FROM topic_hot_scores th3
          JOIN topics t3 on t3.id = th3.topic_id
          ORDER BY t3.bumped_at DESC
          LIMIT :max
        ) Y

        UNION ALL

        SELECT topic_id FROM (
          SELECT th2.topic_id FROM topic_hot_scores th2
          ORDER BY th2.score DESC, th2.recent_first_bumped_at DESC NULLS LAST
          LIMIT :max
        ) X
      )
      UPDATE topic_hot_scores ths
      SET score = (
        CASE WHEN topics.created_at > :recent_cutoff
          THEN ths.recent_likes ELSE topics.like_count END
        ) /
        (EXTRACT(EPOCH FROM (:now - topics.created_at)) / 3600 + 2) ^ :gravity
 +
        CASE WHEN ths.recent_first_bumped_at IS NULL THEN 0 ELSE
          (ths.recent_likes + ths.recent_posters - 1) /
          (EXTRACT(EPOCH FROM (:now - recent_first_bumped_at)) / 3600 + 2) ^ :gravity
        END
        ,
        updated_at = :now

      FROM topics
      WHERE topics.id IN (
        SELECT topic_id FROM topic_ids
      ) AND ths.topic_id = topics.id
    SQL

    DB.exec(sql, args)

Explain the hot scoring algorithm carefully, accurately and clearly.

gpt4_bot:

The hot scoring algorithm implemented in the provided SQL script is designed to rank topics based on their “hotness,” a metric that indicates how engaging or popular a topic is at a given moment. This is achieved by considering factors such as the number of recent likes and the number of unique recent posters, along with the timing of the topic’s creation and updates. Here’s a breakdown of the process:

Initial Insertion

  • First Part: It begins by inserting new records into the topic_hot_scores table for topics that don’t already have a record. This is done by selecting topics that haven’t been deleted, aren’t private messages, and were created up to the current timestamp (:now). The initial scores are set to zero, with no recent likes or posters.
  • Filter Condition: It focuses on topics that are either unpinned (or less emphasised on pinned topics) and orders them by their latest activity (bumped_at in descending order).

Updating Recent Counts

  • Second Part: Then, it updates the topic_hot_scores table with recent activity data. This includes:
    • Count of unique participants in the topic since a cutoff date (:recent_cutoff).
    • Count of likes on posts in the topic since the :recent_cutoff.
    • The timestamp of the earliest post considered “recent” for each topic.
  • This part of the process aims to reflect the most recent engagement metrics for each topic, by catching any new likes, posters, or activity that happened after the :recent_cutoff point.

Rescoring Based on Hotness

  • Third Part: Finally, topics are rescored based on their updated metrics. Topics are first filtered to a set that includes:
    • A batch of topics most recently bumped (to ensure fresh content gets considered).
    • Another batch based on the highest existing scores (to keep highly engaging topics in view).
  • Calculation of the Hot Score: The hotness score is recalculated using the following formula:
    1. For topics created after the :recent_cutoff, the score is based on recent_likes.
    2. Otherwise, it’s based on the total like_count from the topic’s start.
    • This value is divided by the time since creation (in hours) plus two, raised to the power of :gravity—a parameter that controls how quickly topics fade over time.

Additionally, if there’s recorded recent activity (recent_first_bumped_at is not null), a secondary component is added to the score. This component is the sum of recent likes and posters (minus one, to negate double-counting) divided by the time since the recent_first_bumped_at (again in hours) plus two, also raised to the power of :gravity.

  • The updated score is stored with the current timestamp as updated_at.

Conclusion: This algorithm’s essence is to prioritize topics that are not only actively engaged (with likes and posters) but also relatively new. The use of a decay factor (:gravity) ensures that the significance of these interactions diminishes over time, allowing newer or more recently active topics to rise to prominence. The mix of considering both total and recent activity allows the algorithm to balance between long-term popular topics and fresh, engaging discussions.

7 Likes
19

(I like how well the AI is being used here for the SQL description. It’s a great use case. :smile:)

7 Likes
20

Sorry - I didn’t intend to hurt anyone’s feelings. I’m sure the leadership is happy to see a new feature completed, and the implementers are happy to have released it.

I do hope there’s room for constructive criticism.

AIUI the motivating example is a social media feed. Not unrelated, perhaps, one might look at the front page of Hacker News. The ideal over there is for new topics to have some visibility, and for active and popular topics to have some visibility. Both should fade. HN seems to do this well.

But my observation is that a support forum is a different thing, compared to a showcase or a social media feed.

Upthread you’ll see an idea that perhaps Hot is good for new uncommitted visitors, or for search results. I can see the merit in that idea.

But I would maintain that meta is mostly a support forum.

Of course.

Agree with that, of course, but I don’t think Hot addresses it. Pinned topics does a pretty good job.

3 Likes