Upgrading Discourse to Ember 4

[Johani (Joe)]

We want to update the Ember version used in Discourse.

Right now, we’re on 3.15, and we’d like to get to 4.1

We’ve set a goal to finish this work by the end of this year. Please note that this topic is a roadmap, and all the plans and estimates are tentative. This topic is not intended to house any significant discussions about particular upgrades or changes. So, let’s keep the conversation here a bit focused. Questions about how these updates will affect your site/plugin/theme are off-topic. The planned changes are 100% on the front-end (Ember app).

We’re going to be super careful about the changes we introduce. All the official plugins/themes will get updated (including any custom work CDCK has done for its customers). We’ll also send PRs to update all the popular unofficial plugins/themes.

No worries if you have a custom plugin/theme that you built for your site. We will add deprecation warnings and create the necessary announcements in a timely manner to give you enough time to make any changes required. We will also try to guide you through those changes if needed.

Let’s start with the goals. We obviously want to get to the highest version available, but we need to set intermediate goals between now and our final destination.

We plan to do incremental updates. Instead of one major update to 4.1, we’re going to break this work down into six stages. Each stage focuses on one Ember update.

Stage	Size	Updates
1	size-l	Ember 3.15 → Ember 3.16
2	size-m	Ember 3.16 → Ember 3.25
3	size-xl	Ember 3.25 → Ember 3.26 (Part 1 - General) Ember 3.25 → Ember 3.26 (Part 2 - Templates) Ember 3.25 → Ember 3.26 (Part 3 - jQuery) Ember 3.25 → Ember 3.26 (Part 4 - Octane prep work) Ember 3.25 → Ember 3.26 (Part 5 - Octane)
4	size-l	Ember 3.26 → Ember 3.27 (Part 1 - General) Ember 3.26 → Ember 3.27 (Part 2 - RenderTemplate) Ember 3.26 → Ember 3.27 (Part 3 - Legacy built-in components)
5	size-s	Ember 3.27 → Ember 3.28
6	size-s	Ember 3.28 → Ember 4.1

Choosing those specific versions is entirely based on the deprecations and amount of work they introduce - and the implied risk.

So, let’s break those down further for more clarity.

Deprecations per update

Stage 1 (size-l)

Ember 3.15 → Ember 3.16

This update only introduces one deprecation.

1. Use ember-CLI resolver rather than legacy globals resolver - until: 4.0.0 - size-l

   Discourse has its own custom resolver that currently extends Ember.DefaultResolver.

   The recommended fix is to do away with any custom resolvers and use the Ember-CLI resolver. This is easier said than done because we have quite a bit of custom logic to handle plugins and themes.

   A compromise that works is to extend the Ember-CLI resolver instead of extending Ember.DefaultResolver.

Stage 2 (size-m)

Ember 3.16 → Ember 3.25

This update introduces eight deprecations.

1. @ember/string#loc and {{loc}} until: 4.0.0 -

2. Without for - Ember’s built-in deprecate until: 4.0.0 -

3. Without since - Ember’s built-in deprecate until: 4.0.0 -

4. tryInvoke from @ember/utils until: 4.0.0 -

5. Meta Destruction APIs - until: 3.25.0 -

   We don’t use any of those, so there is nothing to do here.

6. Use Ember getter and explicitly check for undefined - until: 4.0.0 - size-s

   This is a simple deprecation. We only need to remove getWithDefault in our codebase, and there are only a few places where we use it.

7. String prototype extensions - until: 4.0.0 - size-m

   This is also a simple, low-risk swap, but we use Ember’s extended string prototype in quite a few places. From a quick check, I think we have about < 100 places where we do that between core and plugins/themes. Once we update those, we should also prevent Ember from extending that prototype with something like this.

   EXTEND_PROTOTYPES: {
      String: false
   }
   

   in our environment file.

8. Importing htmlSafe and isHTMLSafe from @ember/string - until: 4.0.0 - size-m

   There’s much overlap between this and #7 in this update, and it should be straightforward and low-risk.

Stage 3 (size-xl)

The upgrade from 3.25 to 3.26 is quite involved. This is where we do most of the “catching up.” There are sixteen deprecations in total. We’re going to break up this update into five parts - where we only do a version bump after all the deprecations have been handled.

Ember 3.25 → Ember 3.26 (Part 1 - General)

This part deals with nine deprecations that are relatively easier to handle.

1. Array Observers - until: 4.0.0 -

2. Component Manager Capabilities - until: 4.0.0 -

3. Modifier Manager Capabilities - until: 4.0.0 -

4. Optional Feature: application-template-wrapper - until: 4.0.0 -

5. classBinding and classNameBindings as args in templates - until: 4.0.0 -

   Nothing to do here; we don’t use those.

6. Transition methods of routes and controllers - until: 5.0.0 - size-m

   This deprecation removes a couple of methods from routes and controllers. It may look like it’s an involved change, but it should be straightforward. We’d only need to inject the Router as a service and call them from there instead - when required. The tricky part is that we use those in many places, and they all need to be updated.

7. Browser Support Policy - until: 4.0.0 - size-s

   ~~This should be a relatively simple change. Ember won’t support IE11 from 4.0 onwards. The nice thing here is that we haven’t supported it for a long time anyway. The only thing we need to change is to stop transpiling for IE11 in production.
   discourse/app/assets/javascripts/discourse/config/targets.js at 1472e47aae5bfdfb6fd9abfe89beb186c751f514 · discourse/discourse · GitHub

   I did some basic testing, and this change will save us about 60kb (gzip) or ~6% of our main and vendor bundles in production Ember-CLI installs.

8. {{hasBlock}} and {{hasBlockParams}} - until: 4.0.0 - size-s

   We use these in a couple of places. This is a simple, low-risk rename.

9. {{with}} helper - until: 4.0.0 - size-s

   We rarely use this, but it still needs fixing. We just need to swap them out and use {{let}} or a combination of {{if}} / {{else}}

Ember 3.25 → Ember 3.26 (Part 2 - Templates)

This part will primarily focus on deprecations involving .hbs templates. There are three deprecations that we need to focus on here.

1. Property Fallback Lookup - until: 4.0.0 - size-l

   Starting from Ember 4.0, this won’t work anymore.

   Hello, {{name}}!
   

   If we have a property in a template, we have to look it up with a preceding this like so

   Hello, {{this.name}}!
   

   We’d have to do that with all of our templates. There are ways to reduce the pain here. We can try the ember-no-implicit-this-codemod and see how far it gets us.

   I’m in favor of limiting changes to 1 file per PR. This makes it easy to review - and revert if something goes wrong.

2. Accessing named args via {{attrs}} - until: 4.0.0 size-xl

   The {{attrs}} object is going to be dropped in Ember 4.0. The change itself is very straightforward, and Ember’s example is quite nice.

   Before:

   {{attrs.foo}}
   {{this.attrs.foo.bar}}
   {{deeply (nested attrs.foobar.baz)}}
   

   After:

   {{@foo}}
   {{@foo.bar}}
   {{deeply (nested @foobar.baz)}}
   

   We’d have to do this for all of our templates. We can combine this change with converting the templates to the angle-bracket syntax. I’m a huge fan of angle brackets because they’re much closer to the standard custom web-components syntax.

   One thing that might expedite our progress here is the ember-angle-brackets-codemod. We’ll have to experiment with it and see how far it takes us. It handles the deprecation and gives us shiny angle-bracket syntax.

   Similar to #1 in this part, I also favor fixing and testing one template per PR.

3. <LinkTo> positional arguments - until: 4.0.0 - size-m

   This is also a deprecation that aims to reduce confusion. There are a few places where we use positional arguments on link-to. We can fix those like so

   Before:

   {{link-to "About Us" "about"}}
   {{#link-to "about"}}About Us{{/link-to}}
   {{#link-to "post" @post}}Read {{@post.title}}...{{/link-to}}
   

   After (with angle brackets):

    <LinkTo @route="about">About Us</LinkTo>
    <LinkTo @route="about">About Us</LinkTo>
    <LinkTo @route="post" @model={{@post}}>Read {{@post.title}}...</LinkTo>
   

Ember 3.25 → Ember 3.26 (Part 3 - jQuery)

There’s not much I can say about this that you don’t already know. New Ember apps don’t use jQuery, and it will be dropped in Ember 4.0.

In this part, we’ll focus on one deprecation.

1. Optional Feature: jquery-integration - until: 4.0.0 - size-xl

   Over the last couple of years, we’ve made a ton of progress in reducing our jQuery usage. There are still places where we need it, particularly in the composer and as a dependency for some vendor libraries we use. I don’t want to get into the details of this change here. Long story short, we should move away from using jQuery.

   However, I would like to highlight that even if we get rid of jQuery EVERYWHERE, we should still keep this option set to true until we’re ready for Ember 4.0. We need a plan to ease the transition for sites with custom themes/plugins that we do not control. In other words, let’s do the work but live with the deprecation warning about turning the option off.

Ember 3.25 → Ember 3.26 (Part 4 - Octane prep work)

In this part, we should focus on getting our files ready for Octane. We will handle two deprecations.

1. Optional Feature: template-only-glimmer-components - until: 4.0.0 - size-m

   This is a simple change in theory, but there’s implied work we need to do before we can toggle that option.

   We need to ensure that our current template-only components work with glimmer semantics. I did some testing, and our tests choked with that option turned on. Here’s an example list of some template-only components we have in core

   - app/templates/components/activation-email-form.hbs
   - app/templates/components/cancel-link.hbs
   - app/templates/components/categories-with-featured-topics.hbs
   - app/templates/components/category-name-fields.hbs
   - app/templates/components/color-input.hbs
   - app/templates/components/custom-html-container.hbs
   - app/templates/components/emoji-group-buttons.hbs
   - app/templates/components/emoji-group-sections.hbs
   - app/templates/components/empty-state.hbs
   - app/templates/components/ip-lookup.hbs
   - app/templates/components/modal-footer-close.hbs
   - app/templates/components/popup-menu.hbs
   - app/templates/components/reviewable-created-by-name.hbs
   - app/templates/components/reviewable-created-by.hbs
   - app/templates/components/reviewable-field-editor.hbs
   - app/templates/components/reviewable-field-text.hbs
   - app/templates/components/reviewable-field-textarea.hbs
   - app/templates/components/reviewable-field.hbs
   - app/templates/components/reviewable-flagged-post.hbs
   - app/templates/components/reviewable-post-header.hbs
   - app/templates/components/reviewable-post.hbs
   - app/templates/components/reviewable-scores.hbs
   - app/templates/components/reviewable-tags.hbs
   - app/templates/components/reviewable-topic-link.hbs
   - app/templates/components/score-value.hbs
   - app/templates/components/selected-posts.hbs
   - app/templates/components/subcategories-with-featured-topics.hbs
   - app/templates/components/text-overflow.hbs
   - app/templates/components/user-fields/confirm.hbs
   - app/templates/components/user-fields/dropdown.hbs
   - app/templates/components/user-fields/multiselect.hbs
   - app/templates/components/user-fields/text.hbs
   - app/templates/components/user-profile-avatar.hbs
   - app/templates/components/user-summary-users-list.hbs
   

   We’ll also need to check our Admin/theme/plugin templates and ensure everything works before shipping that option.

   Off the top of my head, I’m not exactly sure how our raw .hbr templates will fit into this. However, @david has been working on glimmer-based topic lists. So, maybe we can do away with raw templates altogether.

2. Implicit Injections - until: 4.0.0 size-xl

   We use implicit injections everywhere. We do this in an initializer like so
   discourse/app/assets/javascripts/discourse/app/pre-initializers/inject-discourse-objects.js at ac79c5efc61d259705eeb487ca21d0ec3c535807 · discourse/discourse · GitHub

   Ember is moving away from implicit injections. The preferred path forward is to convert as many of our objects as possible to services and explicitly inject them where they’re needed. Of course, there might be a few things where a service is not ideal. In those cases, we can lookup those objects directly when they’re needed like so

   getOwner(this).lookup('thing:main')
   

   Another option we have (depending on performance implications) is to wrap the Ember Classes with our own Discourse Class. We’d then use our Class throughout the app. Kind of like we do with the GlimmerComponent Class
   discourse/app/assets/javascripts/discourse/app/components/glimmer.js at fa0c796baf9a7f64a3b27823b1aa4b370a74c3eb · discourse/discourse · GitHub. This would make this task a size-l or even size-m

   In any case, this change will need some thinking.

Ember 3.25 → Ember 3.26 (Part 5 - Octane)

This is the final stretch of the 3.25 → Ember 3.26 upgrade. We will only have one deprecation left at this point, but it’s a big one.

1. Edition: Classic - until: 4.0.0 - size-xl

   Before can flick our version to Octane, I’d like to spend time converting our Classes to native Classes and our components to Glimmer components. There will be a bit of pain involved, but it’s worth it. The ember-native-class-codemod should alleviate some of that pain. We’ll have to see how far it takes us.

   There are a lot of considerations and workflow issues to keep in mind. The only thing I want to highlight is that it will follow the same flow I mentioned for templates - fix and test 1 component per PR.

Stage 4

The Ember 3.26 → Ember 3.27 update introduces twelve deprecations. I propose we split them into three parts. We will do the version bump after all the deprecations have been handled.

Ember 3.26 → Ember 3.27 (part 1 - General)

1. Reopening Classic Component Super Class - until: 4.0.0 -

2. Class-based template compilation plugins - until: 4.0.0 -

3. LinkTo @disabled-when argument - until: 4.0.0 -

   I don’t think we use any of these, so there is nothing to do here.

4. Deprecate Route#disconnectOutlet - until: 4.0.0 - size-s

   We only do this in one place. It’s the build-category-route, and it should be straightforward to fix.

5. Invoking Helpers Without Arguments and Parentheses In Named Argument Positions - until: 4.0.0 - size-s

   I don’t think we do this anywhere, but I will confirm when the time comes. Essentially, calling a helper without passing it any arguments.

   Even if we use something like that, we’d only need to add parentheses. So, this

   <SomeComponent @arg={{someHelper}} />
   

   becomes

   <SomeComponent @arg={{(someHelper)}} />
   

   note the parentheses around someHelper

6. Run loop and computed dot access - until: 4.0.0 - size-m

   We use . to access computed functions in our decorators addon. They look like this, for example
   discourse/app/assets/javascripts/discourse-common/addon/utils/decorators.js at b05fddaa7ce3968ffc70cd8d4bf290e15d06eb11 · discourse/discourse · GitHub

   We also have a stray one-off here and there. As far as I can tell, fixing those is mostly about fixing how we import them.

   So, computed.filter should be imported like this instead.

   import { filter } from '@ember/object/computed';
   

   Our version of the external vendor buffered-proxy addon uses . to access computed functions; we’d need to bump it.

   We also use . to access run functions in a few places. That said, the same fix applies. We need to update the way we import them. I think themes and plugins especially might have quite a few places where we do that with run

7. Deprecate the Ember Global - until: 4.0.0 - (#size ?)

   Ember won’t be available in the global context after 4.0. It’s hard to estimate the amount of work/impact here without a deeper look. That said, I know @cvx has been doing tons of work to get rid of this pattern.

Ember 3.26 → Ember 3.27 (part 2 - renderTemplate)

This part will only focus on one deprecation.

1. Deprecate Route#renderTemplate - until: 4.0.0 - size-l

   In a nutshell, we can’t use named outlets in Ember 4.0. So, this won’t work.

   {{outlet "thing"}}
   

   We use renderTemplate in almost 30 places in core alone. The upgrade itself seems pretty straightforward. We can use {{#in-element}} and a plain empty HTML element as a placeholder for what we used to render in named outlets.

Ember 3.26 → Ember 3.27 (part 3 - Legacy built-in components)

This part will focus on built-in legacy components, and it will handle four deprecations.

1. Importing Legacy Built-in Components - until: 4.0.0
2. Built-in Components Legacy Arguments - until: 4.0.0
3. Built-in Components Legacy HTML Attribute Arguments - until: 4.0.0
4. Reopening Legacy Built-in Components - until: 4.0.0

I didn’t add sizes to these because… it really depends. Let me explain.

Legacy built-in components like Checkbox, TextField, TextArea, and LinkComponent will be removed in Ember 4.0. We use them in quite a few places, and we also use some deprecated patterns on them.

Ember offers an upgrade path that allows us to keep using them, but we have to import them differently. However, they will not receive any updates from Ember and will remain frozen. I’m hoping that we can do away with all of them; however, that might be a bit involved. This change will need more discussion when the time comes.

Stage 5

Ember 3.27 → Ember 3.28

This is a size-s since it’s only a version bump. 3.28 is the last LTS release in the 3.x development cycle. It doesn’t introduce any new deprecations after 3.27, and it’s a good version for us to stop on for a few weeks while things settle down.

3.28 LTS is supported through August 2022 (both bug fixes and security patches)

This “break” has a few benefits.

1. It gives us more time to see if any issues arise
2. when we ship a stable version, it should be on 3.28
3. it gives us time to make any announcements we need to make regarding self-maintained themes and plugins
4. it gives us time to ensure that the jQuery → no jQuery is as smooth as possible.

Stage 6

After a few weeks pass, we can finally bump our version to Ember 4

Ember 3.28 → Ember 4.1

We can now toggle the optional jQuery integration off as the last deprecation from the 3.x cycle.

This update introduces two minor deprecations.

1. Deprecate Ember.assign - until: 5.0.0 - size-s

   We don’t use that in core, but we’ll need to check themes/plugins. In any case, it’s a simple rename.

2. AutoLocation Class - until: 5.0.0 - size-s

   In theory, we’d only need to change locationType: 'auto' to locationType: 'history' in our Ember environment file, and it should just work.

Workflow

As I mentioned at the start, we will be very careful with these updates. We will test/fix/pin all the official plugins/themes with every update, and we’ll also send PRs to popular unofficial plugins/themes.

The goal here is not to slow down development or create headaches. So, the PRs should will be strictly-scoped, with one change per PR and nothing too large.

In an ideal world, all of the changes would happen in the background without interrupting anyone else’s work. This is why we plan to keep the PRs short and sweet. Also, we’re not too fond of mixed patterns. So, we don’t want to get stuck in an interim state on a per-file basis. A component is either classic or Glimmer, and a template either uses curly braces or angle brackets, nothing in between.

I hope this road map was clear. Like I mentioned in the beginning, this is just a general top-level overview. If anything is unclear, incorrect, or doesn’t sit right with you, please let us know.

[isaac (Isaac Janzen)]

@Johani

String prototype extensions - until: 4.0.0 - size-m

Core: All have been

• DEV: Remove ‘underscore’ string prototype extensions
• DEV: Remove ‘decamelize’ string prototype extensions
• DEV: Remove ‘dasherize’ string prototype extensions
• DEV: Remove ‘classify’ string prototype extensions
• DEV: Remove ‘capitalize’ string prototype extensions
• DEV: Remove string prototype extensions

Plugins:

discourse-events

• DEV: Remove ‘camelize’ string prototype extensions

discourse-data-explorer

• DEV: Remove ‘capitalize’ string prototype extensions
• DEV: Remove ‘dasherize’ string prototype extensions

---

[featheredtoast (Jeff Wong)]

@Johani perhaps not something blocking Ember 4 directly, but mixins might be worth considering on the roadmap as well:

In addition, some new framework classes, such as Glimmer components, do not support Ember mixins at all. In the future, mixins will be removed from the framework, and will not be replaced directly.

[merefield (Robert)]

Do we have a feel for when the Topic Lists will move to Glimmer Components and dump raw templates?

[david (David Taylor)]

Tentatively hoping to get to it in the next 3-6 months, but that’s not set in stone.

Right now the primary focus for our “modernising JS” team is getting Discourse onto Ember 4.x+ (3.28 is now EOL).

[SimonBiggs]

Hi @david,

Out of interest, what would be your recommendation on a theming side of things. We’re investigating undergoing a significant retheme of Discourse (simplifications, make it more “social media like”, less developer focused, use posts with comments instead of threads).

Given the amount of change pegged on the front-end of Discourse over the next 6 months, is this something we should potentially wait before we attempt to do?

Cheers,
Simon

[david (David Taylor)]

Hi Simon, it’s difficult to give a definitive answer here given the uncertainty on the timeline.

At CDCK we are still developing new themes for customers against the existing version of core. Any big changes (e.g. a rewrite of the topic list) will be opt-in initially, so you’ll have time to adapt things.

In general, you’ll have an easier migration path if you use “recommended” APIs like plugin outlets, and avoid overriding any templates.

[SimonBiggs]

Thanks @david, that is helpful.

Cheers,
Simon

[merefield (Robert)]

How are we going to deal with model modification given Octanes Glimmer and “data-down, actions up” approach?

We have a challenge with Plugin outlets I observe, where previously we had 2-way binding but if we attach a Glimmer Component to an outlet, we no longer have that option.

2-way binding via plugin outlets is an established pattern, where in some cases we want to update the model passed via the plugin outlet.

I noticed this recommendation with the Ember docs:

Introduction - Octane Upgrade Guide - Ember Guides

Notably:

" The second option is, you could run the ember-native-class-codemod for all remaining components. This will turn them into components that import from @ember/component , retaining all the same APIs that classic components have, but just represented in a Native Class syntax."

Any thoughts here gratefully received.

[david (David Taylor)]

The two-way binding change refers to re-assigning arguments, but you can still mutate them.

For example, this is not allowed in Glimmer components:

this.args.topic = blah

But this kind of thing:

this.args.topic.title = "blah"

is still possible.

In fact, I don’t think re-assigning arguments is currently possible in Plugin Outlets because of the way we use a {{hash}} to pass in the args. So I’m not expecting any changes on this front.

Many official themes/plugins are already using Glimmer components as plugin outlet connectors, and the current docs on meta describe how to do that.

Glimmer components do provide an improved developer experience, and improved performance. But it’s worth noting that there is no immediate rush to convert from classic components to Glimmer components. Classic components are still supported in Ember 5.

The most important thing right now is to be resolving any deprecation messages in themes/plugins. We’ll be posting more about upgrade strategies over the next few weeks/months, but we’re making good progress on getting core ready for the upgrade. There’s even an experimental Ember 5.3 branch of Discourse which we’ve been running on an internal instance for the last few weeks with great success!

[merefield (Robert)]

@david

The two-way binding change refers to re-assigning arguments, but you can still mutate them.

Oh! That’s very interesting thanks!

[merefield (Robert)]

Understandably there’s a lot of scope in the upgrade and I recognise this is all very hard to give a timeline for, but is there any movement yet on the Topic Lists?

[david (David Taylor)]

There sure is! @cvx is actively working on it, and there is already a site setting “experimental glimmer topic list groups” if you’d like to try it out.

However we haven’t started exploring the customizability aspect of this yet, so please do not try to build any themes/plugins against it. We hope to work on that in the next few weeks.

[merefield (Robert)]

Excellent progress!

Yeah, keeping the customisation options as open as possible would be really appreciated.

We see a lot of requests for very different layouts for the Topic List Item to that which is regular.

[merefield (Robert)]

I’ve noticed the new deprecation notices, e.g.:

"Use the value transformer topic-list-columns and other new topic-list plugin APIs instead. "

Will there be a communication on this (perhaps I missed one? )?

[david (David Taylor)]

Yup, we should have docs up in the next week or so!

They’re not ‘proper’ deprecation messages yet - we’re logging them using console.debug instead of console.warn, so they aren’t even visible in the default chrome devtools configuration. (cc @cvx)

[david (David Taylor)]

Here we go @merefield

Upcoming topic-list changes - how to prepare themes and plugins

As part of our ongoing quest to standardise the rendering systems across Discourse’s codebase, we’re replacing the implementation of the topic-list. Previously, this used a ‘raw handlebars’ (.hbr) approach, and could be customized via template overrides and raw-plugin-outlets. The new implementation of the topic-list uses modern Glimmer components, and has been built from the ground-up to to be customizable in sustainable ways. The new implementation is now available behind the glimmer_topic_l…

[pfaffman (Jay Pfaffman)]

@david

so they aren’t even visible in the default chrome devtools configuration. (cc @cvx)

OOh. Maybe I want to know about that. How do you see them? Does console.debug not make the linters unhappy?

I think this is part of my answer:

you have to set the log level to Verbose in the Devtools topbar while in console to see debug messages; log messages are visible by default

[david (David Taylor)]

@pfaffman

set the log level to Verbose in the Devtools topbar

Yup!

@pfaffman

Maybe I want to know about that.

The reason we made them debug is because we were making sure everything was ready before opening the warning floodgates. It’s just that @merefield was too observant and found them anyway

Now that the topic is published, we’ll be upgrading them to normal deprecations imminently

[pfaffman (Jay Pfaffman)]

But maybe in my own development work I want to be using console.debug rather than console.log. As a rule the things I’m doing, only I care about.