Using the REST API with other languages


(Alvaro del Castillo) #16

Sure Admir.

Using the Discourse API Ruby Gem
(Alvaro del Castillo) #17

Using the current API we have finished the discourse backend for Sibyl.

We have created a sample dashboard with the data:

The statistics are similar to the one you have in:

except the post number. We have gathered 65K posts and according to your stats there are 91.3K.

The posts (answers) gathering is a bit tricky and maybe we are loosing posts in it:

Using the Discourse API Ruby Gem
(Kiko Lobo) #19

Is there a definition of the data that the api throws back?? Say a model for a topic? What are all the fields returned?

(Apparently Archetype) #20

Not that i’ve ever found.

but everything is somewhere in this repo: GitHub - discourse/discourse: A platform for community discussion. Free, open, simple. (let me know if you make sense of it, i’m still trying to wrap my brain around it.)

I think all the serializers are here but what selects which one to use when, and how the data is fed into them (or indeed where ti comes from) i still havent found.

I’m also still looking for how exactly the message-bus feed is formed, because that’s something i integrate heavily with on my project. Havent made much progress on that though.

I’m not terribly familiar with Ruby, but i’ve got to say discourse has been one of the hardest projects i’ve ever tried to grok. Not sure if that’s my lack of familiarity with ruby or if it’s just that discourse is not terribly new developer friendly.

I’ve been documenting what i can figure out as part of my own project, but that’s unofficial documentation so it shouldn’t be taken as gospel or anything like that. <= Any documentation under api/external is describing discourse may be helpful.

(Kiko Lobo) #21

Thank you very much. Is super useful!

(Kane York) #22

There’s a bunch of action-at-a-distance in Rails, but I’ll walk through the stuff for /site.json, as it’s pretty straightforward.

  1. Read the config/routes.rb file, which defines every URL available and the controller that services the request.

  2. We see that /site is right here, and it points to “site#site”. That means it’s the site method on the SiteController.

  3. The SiteController is at app/controllers/site_controller.rb. We see that the site method is defined on line 7, and on line 8 it performs a render call.

  4. Before the render can happen, we need to call Site.json_for(guardian). guardian is defined in application_controller.rb, which is the superclass of SiteController, in a memoized method.

  5. Guardian is defined in lib/guardian.rb and lib/guardian/*.rb, and performs all security checks - e.g. guardian.ensure_can_edit_post!(post), which will throw an Discourse::InvalidAccess exception if guardian.can_edit_post?(post) is false.

  6. Site is defined in app/models/site.rb, and the json_for method is on line 77. (The self. means it’s a class method, not an instance method.) The method has a special return when login is required and the current user is not logged in, but in the normal case it constructs a new Site object and passes it to SiteSerializer. Keep this file open, we’ll need to be referring back to it when we read the SiteSerializer.

  7. SiteSerializer is defined in app/serializers/site_serializer.rb. It declares several attributes, which are fields that get returned in the JSON.

    The serializer will first check for a method with that name on itself, and use that if present, else it will call read_attribute_for_serialization on the model (the Site instance, which is called object here). The read_attribute_for_serialization method, because Site does not inherit ActiveRecord::Model but includes ActiveModel::Serialization, is just send(name) (call the method on me with that name).

    The serializer also declares a few has_many, which makes it use an ArraySerializer for that item. For the first one, categories, we look at the serializer and see that there is no method of that name, so we go to the model on line 40, where categories is a large memoized method (which doesn’t do much, as the object is short-lived). We’ve got several ActiveRecord queries, some uses of the guardian, and a few loops and maps, and (importantly) it sets a few instance variables on the category to be included in the result, which makes this unsafe to be multi-threaded. When that finishes, we pass it to an ArraySerializer, which applies a BasicCategorySerializer to each category returned from the method. (Think of this as recursing the last few paragraphs.)

    Then there’s more has_many’s, and more attributes, and we didn’t get to cover the include_{attribute_name}? method (in the serializer, returns true/false, method exists && returns false → attribute not included in result).

  8. Once the serializer finishes, we go back to the controller, which calls MultiJson.dump (it’s a wrapper around multiple JSON.stringify backends) then passes that to render json:. At this point, it’s out of Discourse code (except the after_filters and middleware) and goes through the Rails end-of-request cycle to be sent to the user.

(Daniel M ) #23

I made a bot that if a user pm’s **wikify** [discourse url] to the bot, it checks if the post is owned by that user, and if so, it wikifies it. When I get around to cleaning up the code (Java), I’ll post it on github.

(Kane York) #24

It would be better to add a site setting to the core project than to add an external service to work around an admin-only / staff-only restriction.

(Apparently Archetype) #25

massively useful information. thank you!

THAT’S what it reminds me of… I didn’t like the spooky action-at-a-distance of grails either…

/me waves cane and yells at young whippersnappers to get off her lawn!

(Apparently Archetype) #26


/me makes a note to add that functionality to SockBot2.0 too.

true enough. but intil such a setting exists this is a viable workaround. :wink:

(Daniel M ) #27

So you monitor notifications for the bot?

I prefer to use PMs- simpler api, and also works better with functions of the bot (it can copy topics from another forum into a special category, so it doesn’t make sense to mention it so it acts on another topic).

Additionally, it can reply with a success/failure message, and I don’t want TL0’s to have access to the bot (and finding the user TL would be an extra place for something to go wrong).

(Apparently Archetype) #28

yes, via interfacing with message-bus and getting the new notifications message.

If your interested most of the implementation is in messageBus.js (with some helpers in discourse.js)

the code is a right pile of poo (and there’s next to no documentation (BAD @ACCALIA! BAD!)), there’s a lot of very late night code in there. I’m in the middle of a rewrite (and upgrade to ES6) that will fix most of the painful sections of that code thanks to bad architecture. You can check out the es6dev branch to see that (although the bot is nonfunctional because rewrite isn’t finished)

Because of the way SockBot integrates with discourse there’s actually very little difference between an @mention and a PM to the bot so most modules respond to both.

Yes, i ended up having to add that. SockBot ignores TL0 outright and rate limits TL1 requests.

(Daniel M ) #29

Looks like my code!
I really should learn JS…
[orders book on Amazon, gets sidetracked and orders more stuff lo learn C#/unity and Python]

(Apparently Archetype) #30

Not anymore it isn’t!

the developers of SockDrawer are proud to announce the release of SockBot 2.0. Better, faster, stronger. We had the technology, so we used it!

We have new documentation on the workings of the bot!

we have a new npm release of the bot so installation is easier than ever!

We are on github and use it for an issue tracker should you run into any issues with the bot, drop us a line there and we’ll do what we can to help!


Did you get anywhere with the c# client?

(Admir Hodzic) #32

No ,
i give up after trying.
What I research so far

http client works ok if you want to use http protocol,
Best fend there is and fiddler

Second shot was to try communicating directly whit PostgreSQL using ef , and npgsql provider.
I did not managed to scaffold database in reversenginerig, but it is posebbile if you recreate db from scripts.

third attempt was to use thise ended whit lack of options implemented

(Mahesh Govind) #33

Could you please give me some pointers to discourse rest APIs

(Blake Erickson) #34

@vu3mmg check out this topic to get you started:

(Jagjit Singh) #35

Its very much possible for discourse’s rest api. Use ‘page’ as an URL parameter.

for example:- release.json?page=0

(warejacob) #36

Thanks for this list @Buddy, it helped me quite a bit when I was wading through the API waters the past few weeks.

I’ve built an open source .Net Core 2.0 bot (using the cookies/ username + password login method) running with ElasticSearch.

It only does a few basic things at the moment such as index site posts and it also has remindme functionality to remind users of things. The indexing along with Kibana gives some really cool data visualisation capabilities and possibly machine learning applications as well.

Time will tell where it goes, but for now, my local discourse forum seems to be liking the remindme functionality at least :+1:.