Frontend on Prod Site Down -- Ember/Test error

Support
1

My discourse instance front end is completely blank and wont load after the new upgrade. This is not a good thing, and I am a bit panicked as users are unhappy. Backend seems to be up.

This happened after the newest upgrade today, on a digital ocean droplet running a container. I used launcher to rebuild the app.

Any help is deeply appreciated

Console error reads:

app.js:31 
Uncaught TypeError: Cannot read properties of undefined (reading 'registerAsyncHelper')    at s.callback (vendor-suffix.js:58:1)
    at s.exports (loader.js:106:1)
    at s._reify (loader.js:143:1)
    at s.reify (loader.js:130:1)
    at s.exports (loader.js:104:1)
    at s._reify (loader.js:143:1)
    at s.reify (loader.js:130:1)
    at s.exports (loader.js:104:1)
    at s._reify (loader.js:143:1)
    at s.reify (loader.js:130:1)
    at s.exports (loader.js:104:1)
    at requireModule (loader.js:27:1)
    at n._prepareInitializer (app.js:24:1)
    at app.js:81:1
    at Array.forEach (<anonymous>)
    at n.start (app.js:73:1)
    at HTMLDocument.<anonymous> (start-app.js:4:1)
    at discourse-boot.js:30:1
    at discourse-boot.js:31:1
3 Likes
2

All the HTML is loading fine, but something with the ember app is not working…

looking at the code where it is blowing up, it seems like the Ember.Application.extend is throwing an error for some reason

  var Discourse = Ember.Application.extend({
    rootElement: "#main",
    customEvents: {
      paste: "paste"
    },
    Resolver: (0, _resolver.buildResolver)("discourse"),
    _prepareInitializer: function _prepareInitializer(moduleName) {
      var themeId = moduleThemeId(moduleName);
      var module = null;

      try {
        module = requirejs(moduleName, null, null, true);

        if (!module) {
          throw new Error(moduleName + " must export an initializer.");
        }
      } catch (error) {
        if (!themeId || (0, _environment.isTesting)()) {
          throw error;
        }

The culprit seems to be this conditional: !themeId || (0, _environment.isTesting)()

1 Like
3

Ugh…it was a plugin…not sure which one it is, but after some digging around here I suspected that might have been an issue…removed cakeday, signatures, data explorer, discourse-video-upload, and encryption. Going to introduce them one at a time at a later date and see if I can pinpoint the culprit. But for now, the app is back up and I am not touching it

Terrible experience. Feeling pretty guilty right now…should probably have a staging clone process for these upgrades in the future. Lesson learned. Or at the very least I will always snapshot before upgrade so I can restore on failure

3 Likes
4

The problem is Discourse encrypt. You can fix it by putting it last in your yml file and selecting the commit from before it was first broken (signature verification was broken the other day), like this:

  - git clone https://github.com/discourse/docker-manager.git
  ... then your other plugins ...
  - git clone https://github.com/discourse/discourse-encrypt.git
  - cd discourse-encrypt && git checkout 12639de

As you can see, I put the Discourse encrypt plugin last, just in case the cd and git checkout commands would affect subsequent commands (which I don’t think they would).

And yes, that is the commit hash of the last version which definitely works okay.

My site is now working fine again. And Discourse Encrypt has the green ticks again.

6 Likes
5

@TomSSL thank you for this breakdown of the issue

@sam I am going to drop in an application tomorrow for the Full Stack Engineer position. Figure while I am chasing broken plugins, I am going to try to join the team and work on plugins :slight_smile:

I would just DM you, but this instance wont let me.

3 Likes
6

See Current Openings | Discourse - Civilized Discussion :slight_smile:

1 Like
7

Please don’t leave this config in place for a long time - your instance won’t receive future updates to the Encrypt plugin. That will very likely cause things to break in future.

We are aware of the signature verification issue, and are working on a fix and have pushed a fix.

However, this Ember.Test error is unexpected. @Mycobee can you confirm what version of Discourse you’re running? How did you run the update? Via the /admin/upgrade UI?

4 Likes
8

Yes, this is vitally important and I forgot to mention it (it was nearly 4am my time when I wrote this and I had just got things working again and thought I’d look in here to see if anybody else had the same problem). This is only a temporary fix to get things working again. Thanks for the reminder @david.

That fix didn’t work for me when I updated everything at 00:00 UTC this morning (ten and a half hours ago). I only updated my instance then because I saw signature verification had been fixed. Then I had to do the temporary fix described above. Is it feasible that my usual update didn’t pick up the latest code for some reason?

2 Likes
9

You were still seeing the signature verification :x: issue? Or did you run into the Ember.Test issue?

Also, I just checked in with the team, and it sounds like we have identified the Ember.Test problem, and will have a fix very soon. We’ll update here when it’s available.

2 Likes
12

I was getting a blank front page and had to disable all plugins (in safe mode) to get the site to load at all. I do weekly updates at around 9:00 UTC on Mondays and this was an extra update to fix the signature verification issue. However… there were quite a lot of other changes picked up as well, of course. And the version bumped to 2.9.0beta3 in that period, too.

As you will already have seen (but maybe somebody hasn’t), here is the diff for Discourse Encrypt between the version I have running now and the latest version as of this message.

Also, it didn’t really take until almost 4am to fix. It was all fine shortly before 2am, but then I had to tell people what had happened (most of our users are in the US).

2 Likes
13

This issue should be fixed by this commit in Discourse core:

So, once you’ve updated Discourse to the latest version, it should be safe to update discourse-encrypt to the latest version.

5 Likes
14

That worked indeed, the upgrade now goes without issues.

3 Likes
17

@david I upgraded to 2.9.0.beta3 which caused the issue.

Regarding the upgrade process, I usually do this via the /admin/upgrade UI. This case was different.

I only upgraded docker-manager from the upgrade UI. After this, I went had to handle some other things, and came visited the /admin/upgrade path a few hours later to complete the upgrade process.

I was met with a text page telling me to go to the host, change to the /var/discourse directory, git pull and rebuild the app with ./launcher…So that is what I did, and then the chaos ensued.

While the stress and nerves were high, I found something on this forum about uncaught TypeError being related to a plugin, so I ripped out everything non-essential from the app.yml plugins and rebuilt the container. At that point, the ember app began working again.

1 Like
21

This topic was automatically closed after 2 days. New replies are no longer allowed.