Discourse Narrative Bot Beta Feedback

[codinghorror (Jeff Atwood)]

Discourse traditionally sends a PM to all new users which does the following:

• welcomes them to the site
• offers tips and tricks for those new to Discourse
• briefly explains trust levels and new user limits
• mentions our rules of civilized discourse

Unfortunately, it was a bit of a … wall o’ text. I’m not sure how many people actually read it, or cared. But we sure as heck sent it out to everyone that ever signed up for any Discourse… ever.

As of Discourse 1.7 we pared that PM down to a much simpler, shorter text with a link to a blog entry that contains the tips and tricks part which was the bulk of the message. That at least fixed the wall of text problem…

… but there’s another problem.

If you think about how real people approach learning a new communication tool, they certainly don’t clink links or read FAQs or read anything really. No, they jump in and use the tool, that is, they talk to real people using the tool, even if it’s only a friend or a number of colleagues they recruited to help them test it out.

For Discourse 1.8 we’re building something better than a boring old PM telling you stuff. We’re building a robot buddy that will greet all new users, a synthetic person that will reply to you and teach you stuff … so new users have a comfy, safe place after they arrive, where they can use Discourse to learn about Discourse!

We’ve been working on the Discourse Narrative Bot in alpha form since October 2016.

• This will be a bundled feature that is on by default and replaces the static welcome PM.
• If you prefer the old Discourse 1.7 and earlier behavior, the first bot message can replicate the “old” PM and ignore all replies. Just tick the “disable bot replies” preference.
• Users can completely ignore the bot and nothing is lost. In that sense it is identical to the new user welcome PM, which we’ve sent since Discourse 1.0.

What does the Discourse Narrative Bot teach new users? What we believe to be the essentials of using Discourse:

1. Bookmarking
2. Oneboxing links (linking)
3. Emoji
4. @name mentions
5. Very simple bold and italic formatting
6. Quoting replies
7. Adding a picture to a reply
8. Flagging
9. Searching

(Replying is implied, as reply to the private message is required to do anything with the bot. It also indirectly teaches users about private messaging, and notifications, of course … by generating their first notification!)

This bot has been live on try.discourse.org for several months and we’re fairly comfortable we’ve sanded away the rough edges by now. But more feedback is always welcome. If you’d like to try out the Discourse Narrative Bot right now, please go to https://try.discourse.org and sign up with a test email.

The primary focus of the bot is the new user experience, and that narrative is more or less complete. However, we are also building out a few other narratives:

• advanced user
• new moderator → defer to 1.9
• new admin → defer to 1.9

We’ll be enabling this bot here on meta.discourse.org in the next week, and we’d like your feedback as we continue to integrate the narrative bot as a bundled feature of Discourse from 1.8 onward.

• make sure bot can be renamed without issue
• bot should not show up as admin in /staff, nor shield glyph
• allow skipping steps if you get stuck
• first notification spotlight was not appearing when logging in via Facebook, Twitter, etc
• should not reply to staged users
• ignore all post length limits when talking to system users
• change track command to start
• replying in a PM does not require @discobot in front of commands
• advanced user narrative is complete with award cert
• terminate narrative if user thanks discobot
• globally ignore flags on system users
• site setting to disable all public bot responses
• rate limit help message in the same topic
• improved min / max checking for dice rolling
• badges for new and advanced user narrative completion
• add 10 random quotes as multi-language backup for dynamic quote webservice
• prevent triggering on anonymous users
• move to shorter “unknown reply” and require explicit help command
• make commands translatable
• add site setting to make first PM non-replyable (and close the PM)
• ensure bot commands inside tickmarks aren’t responded to
• add site setting for delay to first bot PM

[codinghorror (Jeff Atwood)]

I’ve merged the original 1.7 welcome message with the 1.8 bot intro message:

Some of this info was presented if the user engaged with the interactive tutorial but we need it to be front-loaded so they get it, even if they never do anything with the bot, or bot replies are disabled.

The key bits of info conveyed are:

1. How to contact staff (and lets people know yes, this is a robot) cc @ChrisBeach
2. Hints about trust levels and badges, as a new user
3. Reminder of community guidelines, this is Civilized Discourse after all

So no interaction with the bot is perfectly fine now, we meet the primary 3 goals of the new user welcome message. If they don’t interact with the bot, they don’t get offered any tips or tricks though…

Most people won’t reply to the bot so the cost is no different than 1.7, a single brief welcome PM that gives them the basics (and teaches them about notifications and PMs)

[Spriteclad (SC)]

I just tried this, and it works GREAT. An excellent way to get people started on the quirks of any Discourse forum.

Also, I feel it should take more questions after the user is done, such as “how do I make a poll?” or “how long can I edit or delete my posts?”

[rming (Ryan)]

I just came here to say that the bot provides a terrific onboarding experience. It’s playful and really encouraged me to play around with all the features. I loved watching the bot replies come in moments later. Really drove home the point that there’s some fancy and considered tech going on behind the scenes.

I particularly liked the certificate and capybara (which I’ll interpret as some kind of secret Toronto shout out).

[codinghorror (Jeff Atwood)]

@FlyingSwitzerland

What language would you like for your interface?

The best thing to do here is for the bot to mention the user’s preference page as @grodeska noted.

@mehulkar

will the bot build its knowledge base / ability to interact through some machine learning techniques?

No, the only plans we have at the moment are simple “choose your own adventure” type responses and replies. Trying to keep it simple.

(eventually yes, people should be able to write their own narrative tracks… cc @modius)

@tophee

My suspicion is that discobot distracted them away from the topic they were supposed to see first thing after accepting the invitation. Also, I would assume, as a user following an invitation to a topic, I might be a bit annoyed by the screen turning black and discobot trying to steal the show.

I hear you, and this is a valid concern… I need to hear it from a few more customers before taking action on it.

Also @rizka as for non-default site settings like edit grace period being mentioned in the bot replies, we can deal with that at a later date. It is basically trivia, and so rare to be changed from default…

[grodeska (Lawrence Grodeska)]

@codinghorror FYI, we’re going to be doing some user testing on the initial user experience for our new forum and will likely get some feedback on the Narrative Bot. Will share here when we have it.

[codinghorror (Jeff Atwood)]

I think we will tweak the badge copy a little based on this feedback:

Certified
Completed our new user tutorial

This badge is granted upon successful completion of the interactive new user tutorial. You’ve taken the initiative to learn the basic tools of discussion, and now you’re certified!

and

Licensed
Completed our advanced user tutorial

This badge is granted upon successful completion of the interactive advanced user tutorial. You’ve mastered the advanced tools of discussion — and now you’re fully licensed!

We’re also adding a @discobot fortune mode that tells your fortune. Bear in mind that quote, dice and fortune are really just parlor tricks to give the bot some personality. They’re the icing on the cake, not the cake itself.

99% of the purpose of discobot is to greet new arrivals with the essentials, and offer to play with them so they can optionally learn a few key Discourse skills, while using Discourse, e.g. the tutorial that is the first “level” of the game.

[tobiaseigen]

On my site, why are discobot (and system user) at TL4 as well as staff? We don’t use this trust level much on our site, so would prefer to keep it empty. Can I demote them to TL3 without issues or should I leave them at TL4?

[tobiaseigen]

Maybe this will be sufficient to provide as a link to folks to complete the new user track… it seems to work, though there’s still the extra step to send the message.

/new-message?username=discobot&title=Start%20new%20user&body=start%20new%20user%

I am super happy about the conversational flow of discobot’s messages and am glad it avoids making the track “feel like work” by giving lists of detailed instructions.

Time will tell how many people actually start the track - so far we’ve had it up for a week and haven’t had anyone do it except the moderators I am trying to encourage to take it.

[tgxworld (Alan Tan)]

@tobiaseigen

why are discobot (and system user) at TL4 as well as staff? We don’t use this trust level much on our site, so would prefer to keep it empty. Can I demote them to TL3 without issues or should I leave them at TL4?

I wouldn’t recommend demoting them to TL3 because I can’t be sure what might break at this moment. In most parts of our code base, the trust level check is usually accompanied by a check for a staff user. However, there are a couple of places in our code base with conditions that are purely based on the trust level of the user which I’m unsure of the outcome should you demote system and discobot to TL3. What we can do instead is to exclude non-human users from displaying in certain places UI wise.

[tophee (Christoph)]

@tgxworld

The bot basically has a new SiteSetting which allows you to configure which type of welcome post you would like to send.

This is great! Any chance that a delay can be added (e.g. send first bot message x minutes after first login) so that people get a chance to look at what they came for first, before the screen goes dark and the bot tries to steal the show?

The delay seems particularly useful in the case of topic invites because people follow the link to see the topic but then the screen turns black and the bot steals the show. I did a mass topic invite to make sure that these users see that topic first thing after signing up but it turns out that many didn’t see it and I assume that it was because of the bot. (I posted about this earlier but can’t find that post anymore).

[tophee (Christoph)]

Was just testing the new user track from the perspective of a user who gets a lot wrong and I found a few minor things (which I’m posting because I think this first experience with discourse should be as frustration-free as ever possible for the user):

Well, this first point is actually not about frustration but about education: when the bot asks you to

type **bold** or _italic_

you can just quote that part of the bot’s message and it will be happy with it. I’d say that it should push the user to type bold or italic text in the actual post itself. In fact, the same is true for the first task of posting a link. It will also accept if you quote the link from the bot’s message…

---

When asked to post an image, I first failed and then got the dog-walk.gif hint. When I then replied with an image, I had forgotten to like the post with the unicorn and the bot replied

Did you forget to like my post?

So I intuitively pressed on the post right above and waited. But nothing happened. Well, because you are meant to the post with the unicorn, not the dog-walk-hint. So I will change that last hint to

Did you forget to like my unicon post?

---

When it comes to searching for the capy​bara, the bot says:

Try searching for the term “capy​bara” in this topic

Since it’s crucial that you search in this topi, I’d change this to

Try searching for the term “capy​bara” in this topic

I also seem to remember that in an earlier version of the bot, the “Search in this Topic” option was enabled when the user would press the search icon. I though that was good but it is no longer the case.

---

Now, this one surprised me: Since I started the bot manually in a new PM and not via a reply to the welcome message, the capybara actually ended up in my own post right under the start new user command. I assume that the code says to put the capybara into the OP but I guess it would be better to put it into the first post owned by the bot, no?

---

Finally, I noticed that the post with the certificate is delivered immediately after the “Yay you found it” post. This entails that a) the “Yay you found” it post is scrolled off screen just as you start reading it and b) the bot is giving a bad example of submitting two consecutive posts in the same topic. Could the two not be combined into one post?

One could also turn the the bot’s last suggestion into a task:

If you have a physical , press ? to view our handy keyboard shortcuts.

But I assume this has not been done because it wont work in the absence of a keyboard.

[tophee (Christoph)]

@tgxworld

you can just quote that part of the bot’s message and it will be happy with it.

I don’t know if it is worth the effort trying to guard against this here. Seems very rare to me that a user would accidentally quote the message here.

Of course effort and effect have to be balanced and I have no idea how much effort it would take. I was assuming that it would be very eaay given that quotes are clearly defined parts of the post.

As for rarity, I am repeatedly surprised just what kind of mistakes people make. But in this case, think it is not so unlikely because the “Quote” button kind of invites it as soon as you select some text…

@tgxworld

I don’t think this is necessary considering that we already provide a link to the right post.

My point was mainly about sloppiness, not that it’s difficult to find out which post you’re supposed to like. None of my suggestions above is actually necessary. This is just fine-tuning to optimise the new user experience.

@tgxworld

This was actually done previous but we found that it was too difficult for most users to complete. Yea the other problem is that the tutorial wouldn’t work on mobile

How about the other suggestion then:

@tophee

Could the two not be combined into one post?

---

@tobiaseigen

Maybe it needs to be ASCII art or something?

I kind of like the idea of an ASCII art certificate but I don’t think it’s needed for reliability reasons. If the image doesn’t show up, there is some other thing going on.

Regardless: is the certificate background image customizable? (Even if logo, name and avatar keep a fixed position, this might allow for sufficient customization in cases like Tobias’.)

[downey (Michael Downey)]

I love the idea of a bot to help people figure out how the forum works.

I also loathe the loss of a personal welcome email for new users. Studies in professional community management have consistently shown that a “personal” email – even a short one! – when someone joins a community ranks consistently only behind a personal phone call when it comes to the best tactic in converting that new person into an active contributor, content creator, or (best!) collaborator in the community.

(Most recent source I could find via a quick Google search, is on page 33 of the “State of Community Management 2016” report from The Community Roundtable.)

I’d strongly recommend a scenario in which the personal welcome PM can coexist with the bot. (If that does in fact exist and I’ve missed it, my apologies!)

[HAWK]

I hear you, but before discobot did the welcome, the welcome PM was system generated anyway – it just came from a personal account. I’m not sure of the efficacy of that.

I’ve created a bit of a hybrid solution. I’ve customised the initial message that comes from the bot to incorporate the personal stuff.

And then I use canned replies to send a personal message that comes from me.

In CoPs that’s feasible. In large support communities it’s not, but those people aren’t there to be converted anyway – they just want the answer to a question.

[codinghorror (Jeff Atwood)]

Looks great! Feel free to change the avatar and name fully to match your brand; the default name and avatar are merely abstract placeholders. I have seen some use “welcomebot” for example.

[HAWK]

Is there any way of getting stats on % of users that interact with discobot? (Aside from no. that get the certified badge?)

[codinghorror (Jeff Atwood)]

I usually just visit the bot’s profile and browse its private message tab.

[dax (Daniela)]

@HAWK

Is there any way of getting stats on % of users that interact with discobot?

I think this is what you want

Discourse Narrative Bot Data Explorer Queries

In the interest of tracking user interaction with @discobot, here are are some data explorer queries. If anyone can think of a way to actually show a percentage progress through the track that would be awesome! But as far as I can see there’s no info about the sequence of states in the database To use these queries, download the one you want, and import into the Data Explorer Plugin All User Interactions all-narrative-bot-interactions.dcquery.json (1.6 KB) Shows a list of all use…