Setting up webhooks

(Erick Guan) #1

Webhooks is merged to master. Please don’t hesitate to leave comments on webhooks feedback.

Webhook is a way to notify external services about changes on the internet. It’s easier to setup, manage and develop than a Discourse plugin. Though, it does require you to program a little bit or understand its technical details.

In this #howto, I’ll set up webhook to power a Github bot for referencing pull request brought in a forum. We’ll want to see a new comment on Github pull request when a new post has a link to it.

It’s a very simple process:

  1. Publish a new post.
  2. A webhook event happens, the payload about the new post is sent to the bot on Heroku.
  3. The bot on heroku parses the payload and check Github for existing comments. Then it creates comment when it feels appropriate.

Setting up the webhook

Firstly, we need a webhook to feed our Github bot. Go to API - Webhooks panel and create a new webhook.

A webhook will simply POST the payload to the given url. https is strongly recommended. A webhook event may contain sensitive information, for example a reply to message. It may reveal information only available to staff since the payload is generated by system user. You must fully trust this URL. Here we take a heroku app as an example. (No dash allowed in the url, will fix soon.)

There are two content types available, application/json and application/x-www-form-urlencoded. We prefer json encoding.

The secret is an optional secret key for generate a signature for given webhook event. It’s shared among you and URL you’ve set. Once added, there will be a X-Discourse-Event-Signature in the HTTP header. It’s computed by sha256. It’s strongly recommended to set up one and verify as the instruction (see below).

For this example, we only make use of post event. You can definitely choose event type you need. Check Send me everything to receive every event, even the event type we will add in the future.

Triggered categories/groups are advanced features which allows you to filter some event. For example, if only you cares about Staff category, you can add Staff category so that topic event will happen accordingly.

Then, we checked TLS certificate so that we can avoid some network attacks and keep your information secure.

At last, checked active and click create!

Monitor how is the webhook

Click Go to events or click status information on the left side of webhook list.

Go to list and Edit webhook allows you to jump around. Ping is a special button to send a empty event (with no actual payload). It can help you to test your configuration. You should check out your service’s log to see if you receive a Ping event.

You can also read every event to see how it works. We shows the response status code, event id, event happened time and completion time of each event. If you want, the request/response headers and body are available. If you are debugging webhook or your service, Redeliver button can help you to send the exact request again.

Setting up a Github bot

We want to have a service to notify every mentions to the pull request. We’ll achieve that by hosting a bot on Heroku. It receives webhook event from a Discourse instance and process accordingly.

The code is available here. Simply publish this to a Heroku dyno. Then you’ll need to set up some config envs in the setting page. Check out readme of that repo.

Try it by a reply!

We’ll just need an url to the pull request.

Nailed it!

Verify and parse webhook event

Let’s dive in how it works so you can build yourself one. The flow is simple.

  1. Check out X-Discourse-Event-Id and X-Discourse-Event-Type to see whether you are interested. X-Discourse-Event also shows the internal event hook.
  2. Get payload by reading Content-Length and Content-Type.
  3. Compute sha256 HMAC of raw payload.

To be noted, X-Discourse-Event-Signature consists of sha256= and hmac of raw payload.

Webhook event type explanation

The event type is a big category of several DiscourseEvent under the hood.

Topic event

Can be restricted by categories.
Event: topic_created, topic_destroyed, topic_recovered.

Post event

Event: post_created, post_destroyed, post_recovered, validate_post.

User event

Event: user_created, user_approved.

Webhook usages

As you can see, webhook is simply a standard and configured way to notify external services. If you are looking for some kinds of extensions to Discourse based on changes (events) on the forum, consider webhooks. It will be sure to help you with a loosely coupled system. And it shall be much more stable to maintain.

Webhooks feedback
Webhooks - documentation?
Webhook for Discourse events
Wiki sync plugin
Webhooks feedback
Discourse Webhooks and Zapier
Discourse notifications
Passing back variables to Survey Gizmo
Paginating API search results
Automation based on user sign ups
Webhooks feedback
How to add new webhooks and customize webhook payload
Detect if a topic has a new post
Automated messages
(Alan Tan) #2

A post was merged into an existing topic: Webhooks feedback

(Erlend Sogge Heggen) #3

As of v1.7.0.beta4, webhooks is now available in Discourse proper :tada:

(Kevin Wildenradt) #4

@fantasticfears is there a way to set up web hooks to fire when some user notification is generated, and pass that notification along? These hooks as they are are great, but it would be natural to use this as a way to notify an external service that may in turn pass that notification along to the user. For example, ‘badge earned’ or ‘post liked’ do not appear to trigger webhooks. Do these events trigger webhooks if ‘send me everything’ is turned on?

(Erick Guan) #5

Unfortunately, there isn’t a webhook for that. I’ve seen the request for badge granted event. I’d need some time to test that event. Is there any other notification type you’d actually want to use? I believe Discourse might deliver a huge amount of user notifications.

(Kevin Wildenradt) #6

‘New trust level earned’ is one. I’m sure there are other events that Discourse will notify the user about and cannot currently fire web hooks, but those are all of the useful ones I can think of right now.

I don’t have a huge need for any one of these types of notifications in particular, I’m more saying that just like SSO is a great solution for people trying to use discourse as an extension of an existing service because it delegates authentication and account management to that service, a user notifications web hook could be a great solution for delegating delivery of user notifications to some external user notification system. The existing web hooks can notify an external service about most types of events that are important, but not all of them. A feature like this would seem to broaden the possibilities for the approach I outlined above, and if there is a plugin outlet in a useful spot in the user notification code it wouldn’t seem to be very complicated to write a plugin for.

I’m wondering if maybe there is some other existing way to do what I’m describing, and if not get a sense for how difficult it would be to implement if I found myself wanting to do so later down the road.

(Brett Bendickson) #7

Is it possible/feasible to add a webhook for user login, as well user creation? We’re considering using webhooks as a mechanism to maintain some group memberships, but having those membership refreshed on every login would be much better than just the first time. We could maintain the user list external to Discourse, and update with the “new user” webhook, but it’s not preferable.

(Jeff Atwood) #8

I was pretty sure those webhooks already existed, @fantasticfears?

(Kevin Wildenradt) #9

@bbendick there is a webhook for user creation, but none for login that I can see.

(Brett Bendickson) #10

Sorry, that was a poorly worded question. I meant, “In addition to the one that already exists for user creation, can one be added for user login?”

(Erick Guan) #11

Nope, it only happens when user created not login. But sure, there can be one more webhook for the login.

(Alan Tan) #12

Hmm is there a use case for this? For now, I think we should only support the common webhooks in core and have plugins cater for the more unique cases.

Webhooks feedback
(Erick Guan) #13

As @tgxworld said, it’s better to focus on the common now.

Though, I’ve made two more webhooks. They are user notification and session (login/logout). As a result of the decision, they will be in a plugin. (code is here) @Kevin_Wildenradt @burke.

I’ll start to build some hooks for the plugin to extend webhook as well as smaller payload mentioned in the feedback topic.
Then there will be a new dev guide of how to extend webhooks by plugin. Thanks @erlend_sh for the idea.

(Rafael dos Santos Silva) #14

A hook to filter the webhook payload would be great to allow integrations with third-party like Discord.


Should the payload of a webhook (in this case for topic_created) include the posting user’s email address? In testing I did not see this value come through.

(Erick Guan) #16

You can make a HTTP request to Discourse according to user_id in the payload. The email address of poster is not included.

(Brett Bendickson) #17

Our use case is related to SSO. We would like to place users in specific groups based on attributes from our own directory. We can do that on user creation, but that doesn’t cover maintenance. A reasonable approach would be to write a nightly job to list all discourse users and move groups as necessary, but if there were a webhook for user login, it could be maintained in real time, as necessary.

(Alan Tan) #18

Which attributes in the User model do you require? I recently added a user_edited event so that should be a better approach for maintenance.

(SMHassanAlavi) #19

Can we setup a local webhook?
I have made a webhook like this:

but the list is shownig this:

What can I do for activing this?
I ran a nodeJS app on my 5000 port to handle the requests come to localhost:5000/webhook and I tested my app by postman and it works fine but the discourse webhook doesn’t work fine for local.
what is the problem?

(Sam Saffron) #20

you are going to have to remap the port back into your container using env.