Notes from some recent hallway usability testing

A user on our site from the other side of the country pinged me last week on Slack because they were confused about a handful of things:

We hopped on a Zoom a few days later and I watched his screen while he navigated around the site, speaking aloud about what he was looking for, and occasionally asking some questions.

A bit more background

We also made a little time just to chat about what kinds of things he was hoping to do on the site and why. One thing he’s looking to do is encourage folks to post more on our internal Discourse prior to drafting blog posts for external publishing. There are a handful of different places for people to blog (personal blogs, a more informal engineering blog for the company, and a more polished blog that marketing gets their hands on).

Aside from that, he’s just a user on the site who tries to stay on top of what discussions are going on, to stay informed and to point others to conversations they may be interested in. He’s an engineer and he still writes code, but he’s also one or two levels up in the people management side of things, so he’s got a lot of information coming at him from various sources and is always looking for ways to improve how to manage that and help others do the same.

Some of his recent posts have been sharing tips with folks about how to search Google Sites more effectively and how to choose some calendar settings that make invites friendlier for folks receiving them (e.g. turn on “Guests can Modify this event”)

Some things I observed

Happy to split out separate topics for any of these where there’s interest, but I just wanted to start by sharing these together here, with the same shared context:

Category notifications

  • He’d set one category to “watching” long ago, but forgotten about that.
  • He was confused why he was getting notifications for some topics he’d never participated in (see above)
  • After I noted it was probably due to some setting for notifications on a category, he found the notification preference by going to his preferences (not by going to the category)
  • After rediscovering that setting, he asked why the icon for getting a notification about a new topic was a reply icon. He felt perhaps it’d have been more clear if a New Topic notification looked different from a Reply notification
  • While I showed him he could also change that setting on any given category, he said, “what’s ‘watching first post’?” (indicating to me he hadn’t noticed that on the other page).
  • After I explained what watching first post meant, he said, “oh, that’s probably what I really want instead of ‘watching’”

Desktop notifications

  • He asked whether it was possible to get desktop notifications
  • I said it was
  • He found the setting in his preference
  • It said he would have to go find the browser setting to enable it
  • He said he’d look at that later
  • He pinged me on Slack later:

    I’m not sure why those notifications are disabled
    I can’t find anything online. Going to blame Chrome Canary for now

Topic map

  • He showed me a topic he wrote earlier that week
  • He showed me he was looking for ways to understand how many views it had received
    (This is what he originally pinged me about. I later connected this to the blogging workflow use case he was exploring)
  • He showed me that he could find out the view count by getting back to the topic list (see topic navigation below)
  • He was very confused about why the topic map that he saw on other topics did not show up on his own topic when he originally posted it, and just as confused about why it was there now ("It’s like it only shows up after 24 hours or something)
  • I explained it only shows up after there’s at least one reply
    Aside: Is there a way to customize that behavior? I’d now prefer to just have it always show up.

Topic navigation

  • He went to his own activity to discover some topics
  • He went to hamburger > latest to get back to latest (I always just use the logo)
  • He didn’t understand what I meant by “topic” at first, but I explained. He used the term “post” or “thread” in conversation without thinking about it. I also explained that “thread” was a forbidden term on meta, and some of the history for that, and that while I think Discourse was right to say no to threads within topics, I personally don’t have a problem with using the term “thread” and “topic” interchangeably on our forum, and find people have an easier time understanding what I’m talking about when I call topics “threads”.

Android

  • He told me he had added the app to his home screen

Groups

  • I gave him a brief demo of how some folks are using groups
  • I showed him the group user card (“wow!”)
  • The group I showed him has some rich markdown in the About page with several links
  • He suggested encouraging more groups to have wiki’s with that kind of info, and linking to the wiki from the group about section instead, to limit the number of places needed to be updated about a group.
  • I showed him the Groups page.
  • I showed him how the Activity section and how the Mentions tab has higher signal about group-related content in some cases than just what people in that group happen to be posting.
  • I mentioned that the “About” information that appeared at the top of the page and on the user card could be edited on the Manage page
  • He noted that he hadn’t even noticed the information was also at the top of the page, hypothesizing that he scanned past it because of the grey background.

User Page

  • I noted that users can add info about themselves so their user card shows similar info as what he’d “wow’d” at for groups.
  • He went to his user page.
  • I told him to click the “Expand” button to see that info.
  • He noted that he’d never clicked that before.

We’ve had some good growth in monthly active users recently, I think reaching the upper bound given the size of our organization, but many of them do not use it on a daily basis yet.

Here’s a snapshot of usage over the past year:

And here’s what the stats for this user look like (Note: I’ve significantly lowered trust level boundaries on our site because there’s a lot of trust already built into our system):

This was just one person, but I think he represents an important slice of our current users. I’m planning to do this with others in our organization occasionally to understand where we can make small tweaks to help (and to offer them some opportunity to speak directly about how they can best take advantage of the existing tools).

But perhaps these observations will also offer some insights into what things we might do in general to help improve the experience for users at this point in their adoption.

13 Likes

Why is he posting topics that never get replies? I view that as a failed discussion, or perhaps a blog entry with comments disabled?

1 Like

In this case, it did get a reply. It just took a while, and during that time, he was looking for the view counter within the topic and confused about why it wasn’t there.

And while most topics are discussions, some topic are really more internal announcements than discussions. Posting them on Discourse rather than just on email makes it much easier to find them later (via search) and to share them in other places (like Slack, where far more people are idling at any given moment).

It also allows folks to ask clarifying questions, air their complaints or add some additional helpful tips or references. Some announcements spark those responses, but not all do.

4 Likes

We have a different icon for “watching first post”, so maybe we should use that for “watching”, if it’s the first post.

I can see how that could be confusing. Unfortunately not too easy to customize at the moment - the logic is buried in the middle of a function

Personally I agree - a topic is a singular thread of conversation (singular, not plural). Using established language makes it easier to introduce people to new systems, so I don’t think there’s anything wrong with using the terms interchangeably in discussion. However, in UI/documentation, I think it’s good to pick one.

We only do this on a user’s own profile. Not sure of the history here but it might make things more ‘discoverable’ if we expand by default everywhere :thinking:.

6 Likes

Hi @mcwumbly,

Thanks! These are really good insights. I am regularly reminded of how different my experience of Discourse is vs. first time users. To me, the software is intuitive. But, I forget that is a learned intuition. What Discourse feels like when it is brand new is quite different depending on the person’s age and experience.

For instance, I have had a number of people express a sense of horror or discomfort when they drag and drop a file into the input tab and see the long, ugly URL string. It isn’t until I pointed out to them, ‘hey, it won’t look like that when you post, look over here (to the right), that’s how it will actually appear’ that they are relieved that everything is working properly.

For me, if there are ways to hide the advanced, ‘coder’ level uses of Discourse so that the average, everyday ways of using the software was more prominent and obvious, I think that will help people.

I’m also in favor of larger buttons and bigger, clearer tool tips.

For instance, for the “watching” notification:
“You will be notified of every new reply” raises questions: how will I be notified? Where?
“A count of new replies will be shown” Where? When? How can I view that?

I think these UI screen-sharing experiences with newer members are really eye opening.

1 Like

Yes, to this. For a general audience, I’ve considered biting the bullet and coding a mini-Discourse with the same approach to trust levels and threading, but with all of the markup support removed. No making text bold or italic, no headers, no tables, no image syntax, and support for images and links would be slimmed down to “attachments” and a regex for linkifying URLs. That way you don’t even need a preview pane.

No, to this. The proper way to do it would be mostly eliminating icon-only buttons (the chain and flag are particularly bad for usability) and making buttons less flat (which can easily be done in a theme).

2 Likes

Notice the person in question is a programmer:

Thread is BS programmer talk. It’s bad jargon for the sake of jargon, not to be propagated. Wouldn’t be… uh… “performant” so it isn’t … recommendant. :wink:

I don’t mind it in the context of email systems, though, which have a ton of legacy around them.

Trivially easy, just use CSS to hide the editor toolbar (or almost all buttons on it), then make the editor full width which is already supported. At that point … people are typing plain text in a box.

5 Likes