Chrome 59 headless support

(Kane York) #1

It would be possible to start using Headless Chrome for smoke tests instead of PhantomJS.

The bare-bones implementation would be to run:

% CHROME_PATH=/opt/google/chrome-beta/google-chrome-beta
% $CHROME_PATH --headless --disable-gpu --dump-dom

(pointing to the localhost:xxxx url) and check that the topic list shows up.

This command takes several (~8?) seconds on my machine on the first run, then is very fast (~1sec) afterwards.

More complicated integration tests - such as typing a topic and posting it - could be done with the debugger port.

Making this change will probably speed up the CI workflow due to the more optimized JS implementation.

(David Taylor) #2

Looks like phantomJS may cease to be maintained because of this!topic/phantomjs/9aI5d-LDuNE

(Erick Guan) #3

PhantomJS API supports scripting for tests, not sure how Chrome team will support them.

(Kane York) #4

qunit - what our current tests are written in - has support for a Chrome Remote Debugging backend. I’m not sure if they’ll work out of the box due to the fixtures stuff, though. And I’m not sure if it’s possible to write true integration tests with just the qunit API.

Ideally would be a Ruby interface to the remote debugging port, to coordinate tests of things like a thread live-updating over the message bus when a new post is made.

# Set up
topic = create_topic(user1, ...)

# Action: Create a post in the thread
## or make this into a helper function that manages the composer'#topic-footer-buttons button.create')
browser_2.type_post('#reply-control .d-editor-textarea-wrapper textarea', "A reply to the topic that satisfies minimum length requirements")
th = do ||
end'#reply-control .submit-panel button')

# Check: Post is created on the server
expect(topic.posts.count).to eq(2)

# Check: post is delivered to other browser
new_post = topic.posts.last
th.join # browser.wait_for_message_bus
expect(browser.find_element("article[data-post-id=\"#{}\"]")).to #not be nil, I forget how to spell that

(David Taylor) #5

I’ve had a play with getting the qunit tests running in headless chrome, and it seems to work alright. The performance improvement isn’t as dramatic as I’d hoped - on my computer phantomjs runs the core tests in 227s, and headless chrome runs them in 201s.

I’ve written a JS script, which depends on a couple of NPM modules: chrome-launcher and chrome-remote-interface, which provide an interface to the Chrome Remote Debugging system @riking mentioned.

I’ve then given the rake qunit:test an optional third parameter to use chrome instead of phantomJS. Set the USE_CHROME environment variable to use chrome instead of phantomJS

So, assuming you have chrome and node/npm installed, using it works like this:

npm install chrome-launcher chrome-remote-interface
USE_CHROME=1 rake qunit:test

In summary, there’s not much advantage over PhantomJS at the moment, but that will change if PhantomJS stops being developed in future.

(Sam Saffron) #6

10% faster is a pretty big advantage! great work!

Next step is adding this into our docker test image and docker dev image.

(David Taylor) #7

Having a look at this at the moment. It looks to me like PhantomJS is currently included in the base image.

Is there any reason for this, and if not shall I move it to only be in the dev/test images? Seems to me like unnecessary bulk for the image used in every install of Discourse. On the other hand, it’s only 22mb, so not a big deal ¯\_(ツ)_/¯

(Sam Saffron) #8

Yeah it was kind of “historical” cause you can not run the smoke test without it. But I am open to just moving this stuff out, we will just have to be careful with our CI.

I think for the time being just add chrome to the dev/test image and we will move to that.

(David Taylor) #9

Good news: These two PRs combined allow use of chrome in discourse_test by setting the USE_CHROME environment variable: :slight_smile:

Bad news: Chrome is huge - it adds 415mb to the image :face_with_thermometer:

(Eli the Bearded) #10

Incidentally, I noticed when I tried to build my own base image this week, that docker build died on that ADD. The file is not included in the repo.

(David Taylor) #11

There’s some magic that occurs to obtain phantomJS

The build.rb script executes the download_phantomjs script on the host, which spins up a docker container, which I’m guessing builds a custom version of phantomJS.

So if you run that download_phantomjs script first it should work :slight_smile:

Build docker base image fail
(Eli the Bearded) #12

The comment at the top said “Run by Sam” (or similar), so I just skipped to a manual docker build. Guess I should have read further. :blush:

(Sam Saffron) #13

A lot of the phantom mess can go very very shortly, cause we don’t want to carry it for too much longer.

(David Taylor) #14

@sam I should probably mention - the chrome stuff is currently off by default, so if you want to try it in CI you need to set USE_CHROME=1 :slight_smile:

(David Taylor) #15

Something I just stumbled across is Puppeteer, which is created and maintained by the Chromium devtools team. It bundles a version of chromium that skips all the UI dependencies, bringing it in at just under 100mb. Seems like a far tidier option than having to install chrome separately and use a third party library for the API.

It requires node 7 though, so not an option for Discourse quite yet (discourse_docker ships the LTS version, which is version 6). Maybe once version 8 LTS is released in October it could be an option.

The API looks very similar to the chrome-remote-interface module I’ve used, so hopefully will be trivial to switch over once the time comes :slight_smile:

(Sam Saffron) #16

More than happy to upgrade to node 7, this is not an issue

(David Taylor) #17

Oh cool, in that case I’ll have a quick look at it next week. What are the critical things that depend on node/npm in Discourse? (so I can check they don’t break)

Am I correct in saying most of the critical stuff is run in mini-racer instead?

(Sam Saffron) #18

At the moment we package assets in node, but we can easily migrate to mini racer now that I added explicit support for disposing contexts

(David Taylor) #19

I’ve made a PR to install node8 in the base image

I also had a play with puppeteer, and it is much easier/cleaner than what I had before. I’ve published an NPM module to do exactly what we need - I figure it might be useful for other projects using QUnit. So once node has been upgraded it should be as simple as

npm install -g qunit-puppeteer
qunit-puppeteer http://localhost:3000

The npm install installs puppeteer as a dependency, which in turn pulls in a recent version of chromium :slight_smile:

Then I think the next step would be trying to figure out how this can slot into AutoSpec…