Chrome 59 headless support

It would be possible to start using Headless Chrome for smoke tests instead of PhantomJS.

The bare-bones implementation would be to run:

% CHROME_PATH=/opt/google/chrome-beta/google-chrome-beta
% $CHROME_PATH --headless --disable-gpu --dump-dom https://meta.discourse.org

(pointing to the localhost:xxxx url) and check that the topic list shows up.

This command takes several (~8?) seconds on my machine on the first run, then is very fast (~1sec) afterwards.

More complicated integration tests - such as typing a topic and posting it - could be done with the debugger port.

Making this change will probably speed up the CI workflow due to the more optimized JS implementation.

9 Likes

Looks like phantomJS may cease to be maintained because of this

https://groups.google.com/forum/m/#!topic/phantomjs/9aI5d-LDuNE

5 Likes

PhantomJS API supports scripting for tests, not sure how Chrome team will support them.

6 Likes

qunit - what our current tests are written in - has support for a Chrome Remote Debugging backend. I’m not sure if they’ll work out of the box due to the fixtures stuff, though. And I’m not sure if it’s possible to write true integration tests with just the qunit API.

Ideally would be a Ruby interface to the remote debugging port, to coordinate tests of things like a thread live-updating over the message bus when a new post is made.

# Set up
browser.log_in(user1)
browser_2.log_in(user2)
topic = create_topic(user1, ...)
browser.navigate("/t/#{topic.id}")
browser_2.navigate("/t/#{topic.id}").wait!
browser.wait!

# Action: Create a post in the thread
## or make this into a helper function that manages the composer
browser_2.click('#topic-footer-buttons button.create')
browser_2.type_post('#reply-control .d-editor-textarea-wrapper textarea', "A reply to the topic that satisfies minimum length requirements")
th = Thread.new do ||
  browser.wait_for_message_bus
end
browser_2.click('#reply-control .submit-panel button')
browser_2.wait!

# Check: Post is created on the server
topic.reload
expect(topic.posts.count).to eq(2)

# Check: post is delivered to other browser
new_post = topic.posts.last
th.join # browser.wait_for_message_bus
expect(browser.find_element("article[data-post-id=\"#{new_post.id}\"]")).to #not be nil, I forget how to spell that
4 Likes

I’ve had a play with getting the qunit tests running in headless chrome, and it seems to work alright. The performance improvement isn’t as dramatic as I’d hoped - on my computer phantomjs runs the core tests in 227s, and headless chrome runs them in 201s.

I’ve written a JS script, which depends on a couple of NPM modules: chrome-launcher and chrome-remote-interface, which provide an interface to the Chrome Remote Debugging system @riking mentioned.

I’ve then given the rake qunit:test an optional third parameter to use chrome instead of phantomJS. Set the USE_CHROME environment variable to use chrome instead of phantomJS

So, assuming you have chrome and node/npm installed, using it works like this:

npm install chrome-launcher chrome-remote-interface
USE_CHROME=1 rake qunit:test

https://github.com/discourse/discourse/pull/5054

In summary, there’s not much advantage over PhantomJS at the moment, but that will change if PhantomJS stops being developed in future.

10 Likes

10% faster is a pretty big advantage! great work!

Next step is adding this into our docker test image and docker dev image.

4 Likes

Having a look at this at the moment. It looks to me like PhantomJS is currently included in the base image.

Is there any reason for this, and if not shall I move it to only be in the dev/test images? Seems to me like unnecessary bulk for the image used in every install of Discourse. On the other hand, it’s only 22mb, so not a big deal ĀÆ\_(惄)_/ĀÆ

Yeah it was kind of ā€œhistoricalā€ cause you can not run the smoke test without it. But I am open to just moving this stuff out, we will just have to be careful with our CI.

I think for the time being just add chrome to the dev/test image and we will move to that.

4 Likes

Good news: These two PRs combined allow use of chrome in discourse_test by setting the USE_CHROME environment variable: :slight_smile:

https://github.com/discourse/discourse/pull/5062

https://github.com/discourse/discourse_docker/pull/369

Bad news: Chrome is huge - it adds 415mb to the image :face_with_thermometer:

8 Likes

Incidentally, I noticed when I tried to build my own base image this week, that docker build died on that ADD. The file is not included in the repo.

There’s some magic that occurs to obtain phantomJS

The build.rb script executes the download_phantomjs script on the host, which spins up a docker container, which I’m guessing builds a custom version of phantomJS.

So if you run that download_phantomjs script first it should work :slight_smile:

https://github.com/discourse/discourse_docker/blob/master/image/build.rb#L98

https://github.com/discourse/discourse_docker/blob/master/image/base/download_phantomjs

3 Likes

The comment at the top said ā€œRun by Samā€ (or similar), so I just skipped to a manual docker build. Guess I should have read further. :blush:

1 Like

A lot of the phantom mess can go very very shortly, cause we don’t want to carry it for too much longer.

6 Likes

@sam I should probably mention - the chrome stuff is currently off by default, so if you want to try it in CI you need to set USE_CHROME=1 :slight_smile:

4 Likes

Something I just stumbled across is Puppeteer, which is created and maintained by the Chromium devtools team. It bundles a version of chromium that skips all the UI dependencies, bringing it in at just under 100mb. Seems like a far tidier option than having to install chrome separately and use a third party library for the API.

It requires node 7 though, so not an option for Discourse quite yet (discourse_docker ships the LTS version, which is version 6). Maybe once version 8 LTS is released in October it could be an option.

The API looks very similar to the chrome-remote-interface module I’ve used, so hopefully will be trivial to switch over once the time comes :slight_smile:

8 Likes

More than happy to upgrade to node 7, this is not an issue

3 Likes

Oh cool, in that case I’ll have a quick look at it next week. What are the critical things that depend on node/npm in Discourse? (so I can check they don’t break)

Am I correct in saying most of the critical stuff is run in mini-racer instead?

At the moment we package assets in node, but we can easily migrate to mini racer now that I added explicit support for disposing contexts

4 Likes

I’ve made a PR to install node8 in the base image

https://github.com/discourse/discourse_docker/pull/370

I also had a play with puppeteer, and it is much easier/cleaner than what I had before. I’ve published an NPM module to do exactly what we need - I figure it might be useful for other projects using QUnit. So once node has been upgraded it should be as simple as

npm install -g qunit-puppeteer
qunit-puppeteer http://localhost:3000

The npm install installs puppeteer as a dependency, which in turn pulls in a recent version of chromium :slight_smile:

https://github.com/davidtaylorhq/qunit-puppeteer

Then I think the next step would be trying to figure out how this can slot into AutoSpec…

4 Likes