The State of JavaScript on Android in 2015 is... poor

Using workers doesn’t solve anything here. You’re still hitting all the deopts while rendering the HTML code.

5 Likes

I ran the benchmarks on my phone, a OnePlus 2 (all screenshots here) and it beat the “absolute best known Android score”. Those were pretty bold claims, since I doubt this to be even close to the “absolute best” android phones out there. Anyway, just one observation: Chrome stable is slow, and I can’t even exaggerate how slow it is. I ran the benchmarks more than a couple of times per browser, and perhaps 10 times on Chrome stable: it crashed twice (after taking the screenshots, it crashed about 4 more times), and when it completed it ranged from 1000ms to 2000ms.

Chrome Latest Stable: 1105ms, it’s embarrassing… it almost always crashed on my phone (only completed the test 4 times out of almost 10).

Chrome Beta: 286ms This was the second fastest browser I tried, although on one run it went up to 450ms. 286ms was the fastest, the rest also being all under 320ms, with just one run going up to 450ms.

APUS Browser: 281ms This was the fastest, and its slowest score was actually 299 so I was surprised.

Firefox (stable): 345ms I was actually expecting it to be much worse, judging by @codinghorror’s comment

Firefox (beta): 375ms was not too bad, considering.

Next Browser: 311ms I had never used this browser, and only ran the tests on it since @Codemobile mentioned it. If it scored 236, perhaps the rest might also perform better on the Note5.

Dolphin: 593ms meh.

I guess Chrome Beta for now is a better option for those of us that want Chrome…

7 Likes

“How long before phones get 10x faster? How long before they’re 100x faster?”

If desktop CPU performance is any indication[1] 10x single core performance should take one or two decades. And 100x 3 to 4 decades. btw, 64x for desktop is around the one-atom-transistor scale. So anyone’s guess if we can actually reach that.

[1] Past 3 years about 13% YoY. Intel thinks it can maybe do this the next two years also. And hopes it has its next hat trick (EUV) ready after that. EUV was supposed to be operational in 2014.

Data: A Look Back at Single-Threaded CPU Performance (7-'15) - Album on Imgur

One thing I noticed on your benchmarks and the test runs I did on Android is the extremely high percentage error. Perhaps something is not right here?

Also noticed it, pretty worrying when compared to the Safari mobile screenshot @codinghorror uploaded with ~18% error. On the desktop I get ~30%/80ms on Chrome, and ~3%/52ms on Safari (and ~300%/90ms on Firefox) so definitely Safari is doing something well.

1 Like

beating a dead horse at this point, but aside from creating more maintenance… if you’re looking to be utilized globally apps aren’t a good idea

via Opera

6 Likes

For people who see a white page on the benchmark site on alternative browsers (say, Midori, or Ubuntu’s “Browser”). It needs Local Storage enabled.

3 Likes

I got 239ms on the Next Browser with my Note 5. I think this all has much more to do with the browser and not the processor. I also feel like this test is not really a reflection of real life performance. This is the reason benchmarks need to be done on real life scenarios… Like using a 3d game to benchmark a graphics card. Everyone keeps telling me it is because I am using Chrome on iOS… But guess what… Safari is a horrible browser and does a very poor job of displaying Webpages. Why in the world would I care what kind of benchmarks it gets? The article does not state what browser a as used… I actually assumed it would be chrome because it would be very odd to compare tests again each other when every variable was changed.

I walked past a developer a few days ago trying to use aws sns in safari on osx… It looked like someone had shaken his monitor and everything just settled in random places… He was so used to websites not rendering correctly he didn’t even notice. I told him to open it in Chrome… It rendered perfect. Safari is the new IE… Only it has always been just as bad. Though I get this isn’t a discussion of what is a good browser… But you can’t strap a jet engine to a motorcycle and say “see, this apple jet motorcycle is way faster than the Google Commuter Van is we put them on the salt flats and set them in a straight line”

3 Likes

Hmmm… You know Chrome @ iOS is a skin* above Safari, because Apple doesn’t allow other web browsers?

*Skin with a lot of cool stuff, like sync.

5 Likes

Got 242ms on Chrome beta. The error thing is interesting… And I will look into it. I am genuinely curious what the real benchmarks are for these devices. I use both iOS and Android every day… Don’t mind either… And have never noticed either slow with JavaScript.

Ok… After some digging I have learned that Apple actually uses ember-js for some of their core sites… Thus it is likely they have some very unfair optimizations in place for this test. I propose we find some other tests that more accurately reflect Chrome on Android vs Safari on iOS. I will go first…

Google’s old JavaScript performance test site:

Note 5 Chrome: 9141
Note 5 Chrome beta: 8444
IPhone 6+ on Safari: 9758
iPhone 6+ and Chrome: 515

What? Can’t blame that slowness on the CPU for ios chrome… More likely because of Apples restrictions on development in ios. Sadness.

1 Like

Can GPU power be used in JavaScript? Could it be possible? Many android devices have great GPU.

I’m reading this to see if it’s possible.

1 Like

@codemobile, you’re linking to a deprecated benchmark. Google Octane is the newer and correct version that was shown in the screenshots in the very first post in this topic…

@eyko thanks for benchmarking all the various “other” browsers on Android (Firefox, etc), as you can see, they’re all slower. Chrome Beta will usually give the best results. (Also I think there is something deeply wrong with your “Chrome Stable” install, I’ve never seen it crash for me on Nexus 9 or Nexus 7 in many times of running this benchmark.)

286ms in is certainly in the ballpark for a just-released Android device with the latest CPUs – for example, the Nexus 9 has the highest Geekbench Android single core CPU score and consistently achieves 300ms. But it was released in November 2014, so not exactly new.

It is certainly possible that with further V8 optimizations the Nexus 9, or a very new Android device released in 2016, could achieve 150ms in this benchmark – that’d put it roughly on par with the iPhone 6. I do wish and hope that can happen in 2016.

I have yet to see anything close to that to date, though.

2 Likes

Ember and Angular appear to be quite a bit more resource intensive and are just plain bigger, code-wise than several alternatives. This becomes especially noticeable on lower-end mobile devices and slower connections.

Ember load times being nearly 5 times slower than React and Ampersand the following link points to contains a useful graph of some performance testing for comparison: https://github.com/facebook/react/issues/4974#issuecomment-144721245

Obviously that’s load times, versus benchmarking runtime as you’ve done here. But load time is certainly an indicator of resource use and has drastic effects on perceived performance.

React, Ampersand, and Backbone all perform very well in comparison with Ember.

The filament group has also done research here: Researching the Performance costs of JavaScript MVC Frameworks | Filament Group, Inc. (see spreadsheet linked from their post for all the data).

Of particular note is their summary which I’ve pasted below:


Results
The data is available for inspection, but the following stood out to us:

  • Ember averages about 5 seconds on 3G on a Nexus 5 and about 3 seconds 3G in desktop Chrome.
  • Angular averages about 4 seconds on 3G on a Nexus 5 and about 3 seconds 3G in desktop Chrome.
  • Backbone appears to be the only framework with workable baseline performance on all connections.
  • Looking at the difference between the Nexus 5 and the Chrome desktop over 3G suggests that the execution time required to get the application on screen plays a significant roll in the overall render start performance for Angular and Ember.
5 Likes

I don’t know what a good Octane score is, but I achieved a three-test average score of 7303 on my Moto X PE 2015

Safari 9 seems to have improved (okay, just 5ms on my laptop, but
consistently) so perhaps it’s just down to browser’s JavaScript engines and
their optimisations and not CPU architectures (are there any similar tests
with other frameworks?)

@codinghorror Single core performance is a problem with any JavaScript application on any platform. Considering the target browser support for Discourse, I’d argue that single core performance shouldn’t even be taken into consideration.

Run the same type of tests using Angular 2.0, you’ll see what I mean.

I’m obviously doing something wrong, I am comparing iPhone 6s+ 128GB with an Android. The Android seems to be twice as fast?

Baseline Render
iPhone 6s+ = 155.39s
Android = 77.49

Baseline: Handlebars
iPhone 6s+ = 156.29
Android = 80.64

My iPhone 6S gives me an average 3-run score of 16198.

It may be slowER than iOS, but it’s fast enough to have a decent user experience if one puts one’s mind to it.

A forum site is not supposed to be CPU-bound (unless the monetisation relies on quiet bitcoin mining, huh?)

Perhaps the framework is too heavy, and perhaps the memory footprint needs looking at.

Ultimately, you can drive the JS memory footprint to zero by rendering incoming data and immediately throwing away everything but the generated DOM (keep ids in DOM and retrieve those whenever you need to talk to server).

People got x86/DOS emulators running in bare Chrome at a decent speed, so it’s more a matter of engineering mastery than bad platform.

1 Like

@codinghorror A solution might be to render Discource on the server and send it to the client side, enriching it with JavaScript logic afterwards. I know @tomdale has been working on Fastboot (https://github.com/tildeio/ember-cli-fastboot). But from the looks of it it is definitely not ready for production apps yet.