Best Practices: Signal and Noise

We have a public Discourse forum dealing with open product development…

We get a fantastic amount of topics and replies from our heaviest users, so much that for our light / occasional users, I often get PMs about being unable to keep up… How to find the most relevant posts… How to make sure they don’t miss anything.

I would say this is a high class problem… We’ve got plenty of users and traffic, but the weekly / monthly / quarterly visitors have a difficult time keeping up.

We’ve tried several things, put haven’t really put a dent in the problem. I definitely can’t make any changes that would stifle the great engagement that we’re getting, but how are other folks dealing with this?

The “Top” posts doesn’t seem particularly helpful because the busiest topics are often not the most important for infrequent users to find out about.

On our previous forum we tried weekly summaries of the most important posts, but the summaries are were too long and unfocused (since covering so many topics).

The “Summarize this Topic” is excellent for long topics!

So far, for our forum, I’m thinking the ideal view for infrequent users would be:

  • Filtered by Group (so key staff and key contributors can be easily found)
  • Combined with individual replies that were somehow marked as “Exceptional”

Curious to hear what other Discourse admins have to say…

3 Likes

Interesting issue - and I’m sure others will have useful suggestions

Do you use tags a lot? Rather than re-publish content in newsletters, or rely on auto-curation, you could develop a set of tags that could be applied to topics that deserve additional profile - like “Recommended Reading”, or “Post of the Week” etc. You could then add links to pre-determined filtered lists of these topics that users could use to catch-up.

The tags would have to be applied manually, but this could be a moderator / team task that you could encourage the more engaged members to participate in

5 Likes

Unfortunately most of our engagement is back and forth on longer threads, with the nuggets of pure gold often mixed in very randomly…

So if tagging could be applied at the individual reply level, I could definitely see that approach working, but when the wisdom is on post # 200 and only loosely related to the topic title… :weary:

This implies a discipline problem within your community. When in doubt, more focused topics is (much, MUCH) better than a few megatopics with random “who knows, could be anything” inside.

5 Likes

The question here is, of course: what makes a post relevant? I’d distinguish between two types of answers:

  1. What the community finds useful.
  2. What the individual user finds useful.

For type 1 relevance, a good operationalization could be “most liked posts” or “most liked topics” and while I don’t know the exact formula behind the composition of summary/digest emails, I’m sure that likes weigh in rather heavily into the equation. So if people are getting, say, weekly digests, but still feel they are unable to keep up, they are either asking the impossible (“please inject relevant knowledge into my brain without me having to read anything”) or they are looking for type 2 relevance.

For type 2 relevance, watching or tracking tags/categories has already been mentioned, but the safest and most flexible mechanism is probably saved searches. Install the plugin, write a help-post about how to use it and direct people who send you those PMs to that topic.

That said, there are probably ways of improving the saved searches plugin for your use case (and probably of broader relevance). For example, wouldn’t it be great if you could specify additional criteria that need to be met before you are notified about a post, e.g. must contain searchterm and have a minimum of 3 likes? Or must contain search term and be liked by a staff member. Or … and be tagged with X.

6 Likes

This is a fantastic suggestion! I’ll definitely give that a shot! And great point on additional filtering :+1:

Our forum is really more of a community than a set of questions and answers. meta as @codinghorror pointed out is a very well organized and very disciplined forum.

Our community has more of a club atmosphere. Lots of posts are about all of the different ways that someone is great, so only ~5% of the posts are anything like an answer in the first place, let alone a great point that we need to push out to infrequent guests.

That said, for what we do, the energy of the community is far more important than the utility of the forum for infrequent visitors… putting hall monitors at the party would be the end of the party.

Just curious if any folks have similarly noisy forums and also provide ways to make it the gems hidden inside accessible for infrequent visitors.

Saved Searches is a wonderful idea :grinning:

1 Like

That’s nice, but I see how that turns up the noise level for the less active users. Perhaps you could put those kind of topics into a category of their own?

Or tag them (though the tag should probably not be called “noise” :stuck_out_tongue_winking_eye:) so that people can mute that tag.

3 Likes

Do you think we could modify our Top algorithm (or make it more customizable) to amend this? Top does also take into account things such as Likes and views, so I’m curious what kinds of topics you don’t feel are getting the ranking they deserve.

2 Likes

AFAIK, the algorithm is independent from a members account. i.e. the same for both logged in and anons.

Perhaps more weight could be given based on a members tracking status of the topics for logged in members?

I’m not so sure. It feels like doing that would be “making Top a Suggested”. And it could degrade being able to use Top as a “what topics might I have not seen that are popular with other members”.

I think having a separate Suggested page would be a better approach.

As for members finding posts within topics, wouldn’t saved search do that?

We get a fantastic amount of topics and replies from our heaviest users, so much that for our light / occasional users often PM me being unable to keep up… How to find the most relevant posts… How to make sure they don’t miss anything. This is a much different model of community than meta.

From the perspective of the infrequent visitor that’s not engaged in our community, the “noise” is spread across almost all categories and topics, so even if we had an improved version of “TOP” that focused only on the threads that had posts with the highest reference / information value, an infrequent visitor would still need to wade through the each of those entire topics, looking for the individual useful replies.

As an exercise, for the last week or so I’ve been manually quoting the most relevant replies into Linked Topics in a new reference category, but this is extremely time consuming and not really scalable compared to a “Useful Reference Information” button that high trust users had on each post.

So are, the most reasonable solutions I’ve been able to think of are:

  1. Perhaps develop a way to tag individual replies as “Useful Reference Information” and create a view that presents only that subset.

  2. Perhaps develop a way to filter Topics by User Group (currently we can only filter by User).

  3. Perhaps we can use the Retort plugin to capture some of the most common “noise” responses and channel some of that interaction away from individual replies, without reducing community engagement.

1 Like