Future Thoughts: Discourse's Javascript (Ember) App

(Robin Ward) #1

The Road to ES6

A few people have pointed out that they have no idea what I’m doing when converting Javascript in Discourse to ES6 module format and why I’m bothering with it. I realize that I’ve known my reasons in my head for a while but not communicated them publicly so this topic aims to do that!

I am currently spending roughly 30 minutes every day converting Discourse’s Javascript from using globals to using ES6 modules. Thanks to a custom ember resolver that is based heavily on Stefan Penner’s awesome jj-abrams resolver, I have been able to do this with a full backwards compatibility layer with the globals approach we used to use.

It’s pretty great. When our Ember application wants to resolve something, it will first check for an ES6 module. If one does not exist, it will look for a global.


Browsers probably won’t support ES6 modules natively any time soon, so our code is transpiled into Javascript that can be understood by every browser. So the question is why make the effort at all?

  • Modules allow us to explicitly express dependencies. Right now with the asset pipeline we require every file in our javascript folder recursively. But if one file depends on another, we have to manually move it upwards in a seriously ugly manifest file.

  • We won’t have to rely on globals (ie Discourse.XYZ) to access code. When everything is global any piece of code can reach in and access any other piece easily, even if logically they shouldn’t! With ES6 modules, most of the time you just let Ember wire up modules for you. If Ember doesn’t require the thing you need automatically, you have to explicitly state your intent to require it.

  • Ember has converted to ES6. So we will be able to only include the parts of Ember we want to use. That means that even if Ember grows to add many more features, we can conceivably include just the parts we want.

However, the more exciting reason is that it removes our dependency on Rails’ asset pipeline.

Over time I, like many Rails developers, have grown sour with the asset pipeline. Don’t get me wrong, I know a lot of talented people have put lots of hard work into it. My feeling was for a long time that the asset pipeline was the worst possible way to manage assets… except for every other way.

Once we have migrated to ES6 modules, we can move our entire EmberJS application to use ember-cli. If you are an EmberJS developer and haven’t tried out ember-cli yet I highly recommend you do, it’s like magic! Take a look at one of the screencasts I made to see it in action.

What I’d like to see is something like this:

  1. You check out the Discourse project from github.

  2. The root has a client and server folder, for the ember and rails apps respectively.

  3. The client project uses ember-cli for development. It will not need you to run the rails server. ember-cli comes with a proxy built in, which you could point at our http://try.discourse.org server or perhaps another public staging server we’ll set up.

    This is awesome for many reasons; so many people want to contribute to Discourse but can’t get the Rails server running. Setting up ember-cli is much easier, and even works on Windows!

  4. When we deploy new releases, our deployment process will be smart enough to not deploy the entire Rails application if the server code does not change. I really fell in love with Luke Melia’s RailsConf presentation on the topic. Deploys in seconds!

ETA for all of this?

This is a long term goal. I am converting our code slowly and it will take some time to get there. I don’t have an ETA and it certainly cannot interfere with other goals Discourse has for features. But I have made great progress! So far the vast majority of our components, controllers and views are using ES6!

It will take a few more passes but it’s going very smoothly with few regressions so far. Again this is a testament to Ember’s resolver architecture and how fantastic it is for migrating a large application like Discourse.

Be sure to let me know any thoughts, criticisms, advice and all that fun stuff as usual. Cheers!

Is it possible to work on the UI side w/o having to run rails server?
Requiring modules without using Rails asset pipeline
Separating the forum software into API and static website
Now in master: ES6 Modules + Text Rendering
(William Meldon) #2

This is great news. I’ve always shied away from the Discourse codebase because of the Ruby/Rails asset pipeline. It’s not that it’s bad or I couldn’t get it working, it just always felt like such a step backwards from my old Node toolset. And ember-cli just takes it a step further.

I think/hope this step is going to have a big impact on your contributor base. I know I’ll be more inclined to! Kudos to you for not letting Discourse turn into a legacy app before the 1.0.0 release :wink:

(Jeroen Hoek) #3

Interesting write-up. I am looking forward to seeing the Discourse code-base done with Ember-CLI. Ember-CLI is already looking really good; having a large project like Discource converted to Ember-CLI (and its Broccoli pipeline) will be quite interesting to the Ember community as a whole. I wonder how fast ember serve will respond to changes in the Discource code-base.

(Robin Ward) #4

I am very curious about this too. As I understand it, today for large projects broccoli has some performance issues, however:

  1. It’s hard to get actual numbers until I try and get the application to boot inside it. I’ve heard it might be comparable to the speed of the asset pipeline today, so it might not be any worse than the speeds Ruby is giving us now.

  2. There is a lot of work going on already to fix these issues. It is likely by the time Discourse is ready to be moved over it will already be fixed.

(Abhishek Gupta) #5

So is this thing going slow because it is not a high priority right now or it is because you are the only one doing it?

Also i would love to hear from where you started and what different scenarios you face while this “conversion” , will update that in your blog/here please? ,

(Robin Ward) #6

Other people have converted some files but it’s basically all me, at 30 minutes a day with a large project. Also I was away on vacation last week, so that’s why it’s taking a while.

I’ve made good progress though, so it is slowly and surely getting done.

I’d love to write this up when I get a chance.

(Abhishek Gupta) #7

Sounds Great ,

Also Do you see the possibility that some peoples can create their own Discourse Client , Like if we have a seperated Discourse Server, which does nothing but handles requests from a discourse client , One can Build a client of its own and Using whatever Framework he feels like, so no more Ember Dependency on Discourse Server. However writing one’s own Client seems to be hell of a task, But it open gateway to many possibilities. I hope the license allows such things.

Also i would love to see some conversions Done, and ES6 will simplify the app structure a bit more.

(Robin Ward) #8

This has always been possible. Right now Discourse on the Ruby side is a JSON API that the Javascript (Ember) application uses to make everything work. If someone wanted to make another client in another framework or even native they could do so using that API.

I’m not sure it makes sense to re-implement everything we’ve already done and have working, but if there is a good reason to, sure someone could do it. I’m not an expert on licensing but I think you are free to create your own clients as long as any changes you make to the Discourse code base are made available.

(etewiah) #9

Any updates about how close we might be to using ember CLI for the main discourse app?

I saw this on github so I’m getting excited already. The rails asset pipeline is my biggest pain point with discourse right now.


(Sam Saffron) #10

Pretty far :frowning: there is a lot of work involved in replacing the asset pipeline

Also it is work that is very hard to prioritise as many other simpler techniques can buy us tons of improvements. (eg: not shipping the composer and markdown editor with the first request and so on)

(Robin Ward) #11

I can guarantee it won’t happen this year, but I’d like to get to it next year if we can. It’s a long term plan for sure.

(etewiah) #12

Okay, thanks for the update. Was hoping it would be this year but I appreciate its a lot of work and not that much of a priority.
If I find some time, one thing I would really like to do is create a standalone admin client with ember cli - mainly as a learning project but also because I’d imagine in the longer term it would make sense to not have to include admin components with every request.
If anyone else is interested in this, send me a message - I’d be up for pair programming with someone on this.

(etewiah) #13

I am pretty keen to see the day when the ember side of discourse becomes more modular. I have a ton of ideas about things I could do with discourse but everytime I try something I find the big massive, monolithic ember app is pretty hard to modify without breaking something.

Today I got a bit cheered up when I saw this on youtube. Listen to what Yehuda Katz and Tom Dale have to say about discourse and ember engines at about 2:55 in the following video:

I know they are not working directly on Discourse but if even they envision the day of Discourse as a mountable ember engine maybe my hopes are not too far fetched :wink:

(Robin Ward) #14

I think once we port the whole site to ES6 modules it will open more doors to this kind of thing.

Having said that, our focus internally is not on this use case. Someone else from the community would probably have to champion it and stay on top of it in order for us to move in that direction.

(etewiah) #15

I saw this yesterday which give me some hope :wink:


Will be nice to collect more experiences from people who have made the journey from ember within rails to a standalone ember-cli project.

(Johnmrobinson) #16

I am a new ember developer, and thought that the discourse codebase would be a good resource for learning “best practice” approach to ember. Is this the case? or do you suggest another open source project like ghost?

(ginger man) #17

Great work so far @eviltrout here ->

Are we going to see the discourse project into client & server soon ? I saw ghost project splitting the repo recently.

(Robin Ward) #18

Maybe! I am a little backlogged right now but it’s something I’d like to try out. At some point I’d like to do a spike to see how well Discourse runs in Ember-cli. I will probably roll that into Ember upgrade work we’ll do in 1.7.

Upgrade to Ember 2.x?
(Sukima) #19

It has been about a year. Any updates on this?

(Sam Saffron) #20

Very unlikely we will migrate to ember cli in the next 12 months.

This is something we think about but are not changing any time soon.