Discourse Narrative Bot Alpha Feedback

(Alan Tan) #1

@Nick and I have been working on a Discourse Narrative Bot which walks a new user through the features that Discourse provides out of the box. While the plugin is still in development stages and kind of rough at the moment, we’ll like to get early feedback and ideas :bulb: from the community so that we can build a better bot.

To test the bot, sign up for a new account on http://try.discourse.org then watch for your first PM. Please share your thoughts about the experience here :slight_smile:

:warning: This is an alpha feature, it is not ready for deployment on any live Discourse sites until we build it out further.


  • :white_check_mark: Basic localization support
  • :white_check_mark: Switch to PMs exclusively for new users
  • :white_check_mark: Random @discobot mention replies
  • :white_check_mark: Timeout replies
  • :white_check_mark: Flags, Likes, Bookmark, Search tutorial
  • :white_check_mark: “Radar Ping” to bring attention to first PM indicator
  • :white_check_mark: Scroll new PM replies into view when near bottom
  • :white_check_mark: Bot replies to any PM it gets
  • :white_check_mark: Fancy personalized certificate of completion at the end
  • :white_check_mark: Make sure bot image assets are global
  • :white_check_mark: Full screen dimming with overlaid large text to emphasize first PM action
  • :white_check_mark: Auto-select “search this topic” for the user when searching
  • :white_check_mark: Store image assets locally
  • :white_check_mark: Allow rename of bot
  • :white_check_mark: Site setting to disable bot welcome PM
  • :white_check_mark: Shorten new user narrative
  • :white_check_mark: Allow avatar change for bot
  • :white_check_mark: Allow users to choose from X available narratives (new user, advanced user, new staff)

Prevent replies to welcome PMs (auto-close)?
Discourse Narrative Bot Beta Feedback
(cpradio) #46

Okay, I had my wife go through the process. She has seen very little of Discourse and has never used a discussion site running Discourse. She only knows what I’ve shown her via the plugins I’ve created and features I’ve worked on.

It took her 2 minutes and 30 seconds to notice the pinging notification :frowning:

Once she found it, she didn’t like that the first interaction was to respond with a word that didn’t meet the minimum characters limit. She ultimately repeated unicorns twice, but maybe changing the words to phrases such as “I like unicorns”, “I like ninjas”, and “I like monkeys” would solve this.

She had a hard time finding the bookmark icon. The flag step gives the user a hint that it is hidden behind the “…” (show more) button, but the bookmark step does not provide such a hint.

Lastly, for the quote a reply, she didn’t like she wasn’t instructed to also type a response. She followed it to the letter, quoted it, and hit reply, only to be told “post cannot be empty.” She then proceeded to argue with me that it isn’t empty, it has a quote (thanks for that! :laughing:). Maybe adding “And tell me why you like that quote” would help ease any pain with that step.

Everything else she had zero trouble with, including the flag option (since it hinted at it being behind “…”). I literally blinked and she was already past that step!

Oh yeah, she was quick on the draw with Search too. She entered “bunny” and pressed enter immediately, which took her to the advanced search. Confused, she looked at me and and said “I did something wrong?”, to which I nodded, she hit back, re-read it, and then slowed herself down to do the steps again, hesitantly clicking the “In this topic” checkbox.

Once she did that though, clicking the first search result was rewarding and she found that intuitive.

(Régis Hanol) #54

Had my wife tried. She has never used Discourse before.

  • she took a couple of minutes to notice the pinging notification (like @cpradio’s wife).
  • for some reason (she doesn’t speak English well), when asked to search, she replied with “:bunny:” rather than “:rabbit:”. The bot told her that wasn’t it, so she edited the post to fix it but the bot didn’t catch that.

Overall she :heart:ed it.
She found it was much more interesting and rewarding than reading the initial PM.

(Rafael dos Santos Silva) #56

I have gone trough several iterations of in-page notifications for non-tech people. And I really mean several.

On the last one we had a bell (font-awesome bell) in red, ringing (moving from left-to-right) and pulsing (much like our ring animation, but bigger and red). And it isn’t enough for most people.

Consider something more direct like making everything except the notification be below a dark overlay (example).

(Erlend Sogge Heggen) #68

Tested on my sister, an avid email user and Word writer but not a participant on social networks or any kind of “public forum”.

  • The pulsing notification wasn’t enough to grab her attention. She clicked around on some random posts until she got bored and asked me “is there something specific I’m supposed to be doing here?”

  • She seemed to find most of the copy fun and entertaining. The most confusing line was:
    "Can you bookmark this post using the buttons ↓ under this post?"
    The ↓ just adds confusion because it can be misunderstood as the button you’re looking for. Say “directly under this post” instead. And we should definitely show a picture of the bookmarks button.

  • Post formatting step killed all the fun! It was quite simply not something she was interested in doing. She literally said “I’d rather use stuff like this” and clicked the B formatting button and successfully formatted some text. Why are we trying to teach handwritten formatting?

  • So after post formatting, i.e. 10 minutes or so in (yes, easily 10 minutes, because we show youtube videos and ask for wikipedia and image links that easily lead to longer digressions), she wasn’t that keen on continuing as she felt this had gone one for long enough already.

(Jeff Atwood) #71

OK thanks for that @erlend_sh, one of the most consistent pieces of feedback we’ve gotten is that new users can’t find the PM notification so we will go with a full-screen dimming effect to get them to look in the right place:

@tgxworld that should be your priority at the moment once the completion certificate is fully tweaked.

(I noticed that an iPad game I was playing with my son did the same thing, as it introduced new buttons it would “force” you to find them by stopping everything and dimming everything on the screen except the button it wanted you to press, so I think this will work ;))

For an additional visual hint. I’m adding the bookmark emoji to the bookmark step and removing the down arrow, unfortunately bookmark emoji varies a fair bit by Emoji type: 🔖 Bookmark Emoji

OK! Can you bookmark this post using the bookmark :bookmark: button under this post? Use the show more button to reveal additional actions.

I agree that the formatting step can be simplified, we were burying the lede there a bit with a whole lot of detail and too many words, so I did that:

Hey, can you make some words bold or italic in your reply?

**bold** and _italic_

… or push the bold B or I italic buttons in the editor.

(HTML and BBCode will also work. If you’re interested in advanced formatting, there’s a whole tutorial to try :nerd:)

(not sure why you are complaining we didn’t discuss the toolbar in this step at all, we did, I didn’t change that part, but I think we buried the lede as I said above and I agree with that.)

We did. But arguably “too many words”. And I agree too many words is the enemy.

(Jeff Atwood) #72

Completion Certificate is looking good and 99% complete

(Erlend Sogge Heggen) #75

I think the #1 most important thing we can do when we release this thing into the wild is to track time-to-response (not just first), so that we’d have data like:

  • Time to first reply: 2m 10s
  • Time to second reply: 1m 5s
  • Time to final reply: 1m 30s
  • Total time spent: 15m 35s

We wouldn’t want this number to go very high, otherwise we’d be making the same kind of lengthy, excessively front-loading tutorial that gamers love to skip.

(Sam Saffron) #78

Feedback from watching my wife run through this:

  • The spotlight effect was not enough, it just caused even more confusion: “what is going on?”, “what am I supposed to do?”. We should add some instructions.

  • While replying to the PM the spotlight effect showed up again on 2 or 3 of the replies, it was very confusing. This should be impossible.

  • The iconography was very confusing, there is no parity between emoji and font-awesome so the picture for bookmark does not look like bookmark and link does not look like link. We should whitelist it so admins can include font-awesome icons in in posts eg allow <i class='fa fa-bookmark'></i> for admins only.

  • There was one step where you have to include links from wikipedia, not everyone is great at the whole “right click” thing. The suggested links should be in code blocks as opposed to hyperlinks which makes it easier to copy paste.

  • Quoting came as a big surprise, I wonder if we can have a image in the post that demonstrates quoting eg:

  • Emoji window closing after an emoji is added was a huge surprise. “But I want to add lots of emojis”

  • The “search” example was very confusing. I think we should have 2 search step. First to demonstrate global search and another to demonstrate search within post.

For example:

Please search for “bananas” and include a link here to one of the results you find.

Did you know? You can also search within this message…

Forcing users to read their mail is confusing my users
(Jeff Atwood) #81

Yes it looks like @falco was right…

… so we’ll need to have a dialog on top of the spotlight as well cc @tgxworld plus urgently fixing any bugs with spotlight.

We can include small pictures of the icons, that would be easier than a bunch of whitelisting, and I agree the lack of visual consistency with emoji is a problem. I can make that change. Edit: this is done.

I feel like manual copy paste selection of a URL is significantly harder than tap and hold on mobile, so this would be a bit of a step backwards… I guess we could be in the business of teaching tap and hold? Sure I removed one Wikipedia bullet point …

To copy a link, tap and hold on mobile, or right click your mouse.

… and added the above in its place.

Maybe, question is how prescriptive we are going to be in the step, e.g. show a video of exactly what we want the person to do, then have them copy it? Quoting is very important though.

I deleted the other random reference to “bunny” on try2 and try so that should address the search temporarily. @tgxworld it is kinda critical to urgent that we pick a word here which is quite rare and not likely to occur on Discourse naturally. I don’t think “bunny” is that word… edit: I have changed this to capybara which will have much less likelihood of conflict with public topic search results.

(Jeff Atwood) #87

Please read this


Users will not read ANYTHING you put on the screen. The most we can do is 3-4 words with the spotlight and animation backing it up. The dialog needs to be physically near the notifications, too. For example from that iPad game I’ve been playing recently:

Left is before, right is after. Note:

  • spotlight dimmed effect
  • animated radar circles ping
  • a bit of text telling you what to do (6 words)

we need all of those things. That’s what it takes. Right now we have animated radar and spotlight… we are missing the ‘few words telling people what to do’ … so basically

(Alan Tan) #89

Ok here’s what we have as of tonight

Some love for dark theme too

(Gerhard Schlager) #91

Images and videos are an i18n nightmare if they show translatable UI elements. And our current Transifex based translation workflow doesn’t even enable translators to provide localized images. Please consider this before adding such images.

(Kris) #94

Some additional feedback from someone who’s never seen discourse (this has really come along really well!)

  1. They thought the bulleted list of ways to type emoji was a step-by-step instruction, so there was a brief moment of confusion “wait I didn’t do the other parts” and then “oh they all do the same thing”
  2. General “This is kind of long” comment
  3. They opened the “flag this post” dialogue, but then closed it to check to see if a specific option was required (then “oh I can select any of them”)
  4. They pressed the :link: button on the post in the tutorial and shared that instead of actually visiting the linked post (still worked as a step completion, but semi-missed the intention of the instructions)
  5. This was the only step that they were completely stumped on: searched Cappybara and “Why couldn’t it find anything” (didn’t click the “search this topic box”) — they were stuck until I said “reread that last line again” and then a “Ohhhh, I have to click the box”
  6. “Is this not the emoji button?” They were pressing the robot emoji in the topic title (at the top of the composer window). After a few seconds they also saw the :slight_smile: in the toolbar and said “oh wrong one” and completed the step
  7. “That was annoyingly long” but “It was a good tutorial”

(Erlend Sogge Heggen) #108

I strongly disagree. Non-techy users do not hand write their formatting and never will. In Gmail, Word and WordPress they use WYSIWYG, and in Facebook, Twitter and Instagram they use no formatting whatsoever. I still remember the first time I was exposed to BBCode. A week or two in, I honestly felt like I was halfway to programming my very own websites:

That’s how foreign it was to me. Markdown is moderately more intuitive, but it is still very much a markup language, and markup languages will never have a place in the non-techy user’s toolbelt. We’ll much sooner figure out how to implement WYSIWYG in a sane way.

What I’m arguing for is to drop the formatting lesson altogether. It’s not like - in just one tutorial step - we’re gonna convince anyone that the way they’ve been selecting text and pressing buttons all their life is an inefficient workflow. Most of the time you don’t need formatting in your post anyhow; it’s a niche feature. The only formatting feature I think of as critical is hyperlinks, and those are arguably easier to do with select&click&paste anyhow.

I’m increasingly convinced that only admins should be getting the long version of this tutorial, whereas new users get the bare necessities 3-step version where the “advanced lessons” are merely bonus levels. However there’s not much point in discussing the merits of this suggestion before we have collected some data.

(Tobias Eigen) #109

I agree with Erlend generally here but actually disagree about the tutorial step - I think we should keep it in there unless user testing shows a majority of new users get stuck. It’s valuable to give new users a taste of formatting on discourse so they don’t freak out when they have to use it.

Maybe a compromise solution is to create a follow-up lesson and mention it in the final message from discobot as possible further learning/next steps, along with a menu of commands discobot understands. E.g. “hey discobot tell me about formatting!”, “hey discobot tell me about hidden secret features!”

What this brings to mind is a wondering if at some stage there might be an admin interface for adding and managing discobot tutorials, similar to customizing templates, that we can share and turn on/off on our particular sites. I’d love to be able to create more tutorials about all sorts of site-specific things… still have my eye on the robot lawyer.

(Jay Pfaffman) #111

I hate it, but I think @erlend_sh is right on this one. I forced my students to learn Markdown in my course, and though many of them thought it was really cool, it was as he described–they thought they were into some serious computer stuff, which was fine, as the point of the course was to empower people to think that they could make computers do whatever they wanted.

It freaking kills me that Word’s interface actively discourages people from learning to use the keyboard. . . but I digress.

My vision for this tool is to make it reasonably easy to author (I haven’t had a chance to look at the code yet) so that we could use this or the wizard interface to walk people through things like configuring Google authentication and the like. I just rolled out my Wordpress and Discourse Install; I’d love to be able to scaffold people getting things configured.

(Jeff Atwood) #112

I violently disagree on this. I think I’d literally get into a fistfight over it.

So typing * is now a “markup language”? I really fail to see the equivalence between

This is [i]BBcode[/i]

which is 7 characters, 4 weirdo brackets, and an arbitrary command word (repeated!), versus

This is *markdown*

which is two common asterisks!



that’s 8 brackets, a slash, 8 extra characters that make up two arbitrary command words, and 2 asterisks.


- item1
- item2

that’s … 2 dashes and 2 spaces.

Who ever types brackets or angle brackets normally? Or a “command word”? Nobody. Who ever types an asterisk or a dash normally? Lots of people do. This is not a small difference. It is a MASSIVE difference. So saying “we are teaching them BBCode” is, qute frankly, :poop: bullshit.

While I do agree that most of the time no formatting is needed it is nice to have simple stuff like bold and italic in your back pocket, and people need to know that easy formatting is possible.

  1. First, as stated above, I challenge you (or anyone reading this, really) to complete that formatting step of the tutorial, on your smartphone, using only selection and the toolbar. It is super painful. Excruciating. So “why don’t we teach the toolbar”, well, we do in fact already do that, but typing Markdown is generally easier. Particularly on a phone or mobile device.

    Hey, here’s me trying to make something bold using the toolbar as you helpfully suggested. Beyond accurately selecting text on a touch device, which is already a difficult task in and of itself, plus the fact that I had to manually expand the toolbar to see it at all, I can’t even see the damn toolbar because the act of selecting text near the top of the editor automatically obscures whatever is above that selection thanks to the native OS selection menus on both Android and iPhone!

    So tell me again how awesome the toolbar is for mobile users, of which there are more of every passing second. No seriously, I want to hear this.

  2. Second, it is in fact a primary goal of Discourse to teach people some simple formatting. Do you have to use it? No. Is it like, required by law? No. But everyone should know a little and be able to imagine that easy formatting is possible and exists as a thing. Hell, look at Slack:

Look at all that confusing formatting! People’s heads will explode! WHY ARE YOU TRYING TO TEACH THEM ASSEMBLY LANGUAGE JEFF? HOW DARE YOU! :rage:

(Sam Saffron) #113

I wonder if the toolbar is a crutch we should eliminate for the basic narrative lesson

This has tons of advantages and extreme parity with mobile

Simply remove the toolbar during the lesson

For example people will not learn about ‘:’ autocompletion and will automatically reach for toolbar which is a shame :sadpanda:

(Jeff Atwood) #114

The toolbar option is not a problem on desktop, laptop or iPad in any way, but it is a showstopper on mobile.

All I can think of is “uh, press ENTER a few times so you can actually see the toolbar if that is how you roll”