A versioned API for client side plugins

Later today I am going to be merging our new post rendering engine into master. Swapping out Ember/HTMLBars for a virtual-dom based renderer provides us with a huge performance increase, but it also has the side effect of being incompatible with how plugins were previously made.

While doing this work, it occurred to me that we made a fairly big mistake in the past by not providing a mechanism for versioning the public APIs used by client side code. We used to encourage people to just dump in whatever Javascript they wanted in an Ember initializer, which works great and is fast to develop, but it has major issues if we ever want to deprecate or update those APIs.

The good news is the latest version of Discourse has a new versioned API for plugins and extensibility. The bad news is most plugins will need to be updated.

To ease in this transition, I spent the last two weeks going through the most popular discourse plugins and updating them so that they’d be compatible with both versions of discourse. If your plugin was open source, you might have received a pull request from me! If you are a plugin author, please check your plugin against the tests-passed of discourse!

I have provided deprecation warnings for old APIs that are migrated to the new system. I’ve also tried wherever possible to always boot discourse, even if code is using old objects or functions. The odds are good that even if you coded directly against an internal Discourse object, your plugin will not stop Discourse from loading (but it will not add or change functionality!)

Using the new API

To use the new client side API, you must ask Discourse for an instance of a PluginAPI object. You can do this in an initializer by importing withPluginApi and calling it like so:

// assets/javascripts/discourse/initializers/with-plugin-sample.js.es6
import { withPluginApi } from 'discourse/lib/plugin-api';

export default {
  name: 'with-plugin-sample',
  initialize() {

     withPluginApi('0.1', api => {       
       api.onPageChange(() => console.log('user navigated!'));
     });

  }
}

The idea here is you tell Discourse you want an instance of the Plugin API object for a particular version of the API. Since this is the first release, I’ve specified the version 0.1.

The cool thing here is that Discourse can provide backwards compatible API objects, even if things behind the scene change. We could be up to version 1.3, for example, but still know how to provide an API compatible object for 0.1.

Additionally, if the API you used is so old that we can’t provide a backwards compatibility layer (likely due to a very severe refactor down the line) we just won’t call the function and your plugin code will not be executed at all. The idea here is it’s better to do nothing at all, rather than have a Javscript error that would interrupt Discourse from starting up safely.

Plugin API Documentation

The documentation for the PluginAPI version 0.1 can be found in plugin-api.gjs. I’ve documented the public surface area of the API there. The easiest thing to do when developing a plugin is to open that file and look for the latest hooks and methods you can consume. Also, don’t be afraid to ask questions on meta! All the Discourse developers read this forum daily and we can offer guidance and support.

Providing Backwards Compatiblity

New plugin authors will be able to use the withPluginApi as specified above. However, for older plugins that must co-exist with versions of Discourse that exist before the new API was added it’s a little more involved.

I have backported the withPluginApi call to the beta and stable branches of Discourse, so that call is safe to make in all currently supported releases. As mentioned previously, if no api is available for the version you requested, that code will not run, so you can use it to gate the code that works with the new API safely.

However, you’ll still need a way to run your old code, so I’ve added a noApi option to help with this. This is how I’ve ported all the plugins I’ve done so far. Here’s an example of that approach:

// assets/javascripts/discourse/initializers/backwards-example.js.es6
import { withPluginApi } from 'discourse/lib/plugin-api';

function oldCode() {
  // migrate your old plugin code here. It will only be run if no PluginAPI  is present
}

function initializePlugin(api) {
  // do stuff with plugin API!
}

export default {
  name: 'backwards-example',
  initialize() {
     withPluginApi('0.1', api => initializePlugin(api), { noApi: () => oldCode() });
  }
}

In the above example, if version 0.1 of the plugin API is available, initializePlugin will be called and can use it. If no API is present, oldCode will be called. You can paste your old plugin code there and it should work. If you want a more involved example, you can see here how I updated the solved plugin to use this approach.

Using the versioned API in a Site Customization

See this write up for information on how to use the plugin API safely within a Site Customization.

Final Thoughts

Again, I’d like to apologize for not having a versioned API in place in the past and the difficulties this will introduce. However, in the long run I believe this change will be worth it, and will drastically prevent developer pain as we continue to update Discourse.

55 Likes

This is what sets an awesome opensource project from the rest.

Thanks, and this make me much more likely to code plugins for Discourse.

17 Likes

Will you be updating the Beginner’s Guide to Creating Discourse Plugins?

1 Like

@eviltrout, previously I had initialize(container) so I could access siteSettings to enable/disable the showing of a toolbar button. In the new API, that seems to be via api.container, which is fine, but how do I do it with noApi?

It’s on my list to review it. I suspect most of the steps will stay the same but I will confirm.

You should use api.container in your new code, but you can just pass the container to your old code like this:

initialize(container) {
   withPluginApi('0.1', api => newCode(api), { noApi: () => oldCode(container) });
}

(I doubt I will ever remove container from initialize BTW. The api object just has it for convenience so you don’t have to pass it around).

5 Likes

Thanks, I thought I had tried that, but I must have had a typo or something, as I know it failed initially, but it definitely works now. :slightly_smiling:

Cool, this isn’t too bad, I shouldn’t have to do too much reworking.

I am writing a private plugin for a customer right now. I’ll try to move to the new API right now. Hopefully I can write up some reflections after that.

Right after the reading, it’s hard to understand the version “0.1” here. It’s not connected with Discourse version. Why? It just sounds wrong…

I am doing monkey patch a lot because I create a dialect which affects different controller’s behaviour. For sure, it’s rely upon on some version of client’s code which plugin version doesn’t help (?)

BTW, new decorateCooked is so nice!

Can we have a nice site_setting helper?

2 Likes

To be clear

  • this change only affects plugins that have an “initialize” ?
  • if a plugin needed to be changed to comply, there would be a Deprecation in the Ember Inspector ?

For the Checklist Plugin, I need to add click events to certain cooked content.

I’m trying to figure out what api call to use to perform that.

  api.decorateWidget('post-contents:after-cooked', function (dec) {
     var post = dec.getModel();
     console.log(post);

     if (!post.can_edit) { return; };
     // code to find checkbox syntax using jQuery selectors

occurs before it is rendered, so I’m not sure I can embed click events. I feel like maybe I’m approaching this all wrong. As I feel the above is too early, but I don’t know what call is “just right”.

Edit: Disregard, I solved it. Just needed a goodnights sleep to wake me up :slightly_smiling:

Thanks for the update @eviltrout . Topics are now rendering much faster. I can only imagine how much work was involved, especially considering your efforts to support legacy code.

I do appreciate that code injected client side via the Admin panel isn’t your focus right now, but it would save client side devs a lot of time if you could show how the following deprecated event (or equivalent) is now being generated and how we should hook into it.

var topicView =  require('discourse/views/topic').default; //get a reference to the topic view

 topicView.reopen({ //access view object
      
      didInsertElement : function(){
        this._super();
        Ember.run.scheduleOnce('afterRender', this, this.afterRenderEvent);
      },
      
      afterRenderEvent : function(){ //after view has been rendered do this:
      
        //console.dir(this);  //Use this to to get a good overview of what's available
        console.dir("Category ID: " + this._controller.model.category.id + " - Category Name: "  + this._controller.model.category.name);
        
      }
});

It’s only a token example, but help with this, coupled to the new API document should be sufficient to get to grips with the new topic architecture.

A final question. Is the new virtual DOM architecture limited to topics only, or are there plans to expands its use elsewhere?

@ccdw, depending on what you are attempting, you may want to look at the Solved Plugin, which interacts with a Topic, and to get to the first post uses

https://github.com/discourse/discourse-solved/blob/master/assets/javascripts/discourse/initializers/extend-for-solved-button.js.es6#L144-L151

dec.getModel() by itself returns a post object, which is why it then requests the topic using .get('topic')

3 Likes

Thanks @cpradio.

The trouble is the following doesn’t mean much to me when working entirely on the client.

api.decorateWidget('post-contents:after-cooked', dec => {

That’s why I gave an example of

topicView.reopen({

Hoping to see what the equivalent would now be.

Thanks anyway, I am sure the Solved Plugin reference will be very helpful once I’ve got to grips with the new post event architecture on the client.

Can you put initializer in customizer? Wrapping it like a plugin

So with the API versioning, I had another thought come to mind and bear with me. How much more difficult would it be to extend that further to let Plugins denote what version(s) of Discourse they are compatible with?

The idea being, when Discourse sees they are not compatible, it can choose to not load them and/or, execute a call on the plugin, so the plugin can disable anything it needs to to permit the Discourse instance from failing and showing a blank screen.

3 Likes

Would be great to have that feature. See also:
https://meta.discourse.org/t/compatibility-checks-and-stable-versioning-for-plugins/31523

It is the version of the API you coded against, not Discourse. This distinction allows Discourse to provide an API object that is backwards compatible. For example, you might be on a much later version of Discourse, but it can say, oh you wanted version 0.2 of the plugin API? I can build an object that responds the same way even though my internal APIs have changed.

What were you thinking? api.siteSettings?

Not exactly. It’s for client side plugins, that are written in Javascript. For example if you wrote a server side auth plugin, it won’t be affected. Having said that, most client side plugins have an initialize function. If not, how would stuff be wired up?

All the deprecations will show up in the dev console. I’m actually not sure how the Ember console works but I believe it’s meant only for Ember deprecations so they shouldn’t show up there.

I just tried your snippet and it still seems to work on the latest build of Discourse. Long term I do want to allow some customizations built that way, but I have to get around to it.

It’s only topics for now. There are some other parts of Discourse that render slowly that we might visit later, but there’s still a lot of work to do before that happens. My feeling is writing widgets for the virtual dom is quite a bit more complicated so you have to really want the performance benefit to take the tradeoff.

This is trickier than it sounds. For example, let’s say a plugin says it supports 1.3 of Discourse. Is it assumed that it will work with any later version? Because down the road we will certainly deprecate and change some APIs as we build new features.

It’s good for the lower versions though – if you say it requires 1.3 it just wouldn’t load at all on 1.2.

6 Likes

Right. But, if the only change I had to make to my plugin was change the versions support from 1.3 to 1.3-1.4 (pick your poison for how to denote multiple), then I think that is okay. It still puts Discourse in a “safe” mode when it comes to plugins, only load it when it says it is supported, thus hopefully stopping any failures that lead to blank pages.

I’m not saying this is a perfect way of doing it, nor am I saying it should be done soon. Just something I seem to keep coming back to when changes are found that break a good portion of all plugins (though this vdom was a bit forgiving – although I did have one plugin that broke completely due to refactorings in core).

Thank you for the prompt reply. No guarantees of course, but how long do you think will you continue to support that type of snippet?

Sounds good. Again, if you thinks it’s not worth exposing. It’s reasonable as the same reason you talk about the api version. The site settings may change.

But the thing actually I want to touch is the plugin enable settings. That’s something api.pluginSettings seems suitable…

What API library can’t give to me is the ability to monkey patch. For sure, Discourse may have more decorators and hook point. However, the change takes 2 years at least until plugin authors enjoy the rich hook point.

As you said, it’s tricky. I hope there’s a better way to handle monkey patching :slightly_smiling:


Some reflections when deploy a change to my plugin:

  • :white_check_mark: Before the vdom (:clap: speed, :heart: for the effort), I monkey patch components to filter something in the model. Now I decorate it. Some components is widget now because of vdom.
  • :question: api.decorateWidget can apply before and after type to the decorator. Without applying type, is it applying both?
  • :information_source: api.decorateCooked is a little bit tricky. model may exists in stream, but may not appear when composer appear. It’s useful for distinguish the post stream and composer.
  • :information_source: I still monkey patch model e.g. topic details; also controller, e.g composer.
    The monkey patch have to be inside initializer.

So sorry I can’t demonstrate by code since it’s a private plugin :frowning:

2 Likes

This is great! I’ve always been a bit anxious that the hooks and outlets I’ve been using for plugins would suddenly disappear.

Just wanted to share my experience with this playing around with it this morning.

For my ratings plugin, I had to migrate this template for the poster-name-right outlet (which no longer exists.)

I initially futzed around with trying to reproduce this logic exactly in the decorateWidget hook, i.e. getting the d-rating component, set its properties and then turning it into html. But this both didn’t work and felt wrong.

Then I remembered that I had already made a little helper to generate rating html for the topic list. So I ended up with this, which works well.

The helper looks like this.

I’m still a little uncomfortable about helper.widget.container.lookup('controller:topic') though. That feels a bit long winded.

2 Likes