Chrome 59 headless support

It would be possible to start using Headless Chrome for smoke tests instead of PhantomJS.

The bare-bones implementation would be to run:

% CHROME_PATH=/opt/google/chrome-beta/google-chrome-beta
% $CHROME_PATH --headless --disable-gpu --dump-dom https://meta.discourse.org

(pointing to the localhost:xxxx url) and check that the topic list shows up.

This command takes several (~8?) seconds on my machine on the first run, then is very fast (~1sec) afterwards.

More complicated integration tests - such as typing a topic and posting it - could be done with the debugger port.

Making this change will probably speed up the CI workflow due to the more optimized JS implementation.

9 Likes

Looks like phantomJS may cease to be maintained because of this

https://groups.google.com/forum/m/#!topic/phantomjs/9aI5d-LDuNE

5 Likes

PhantomJS API supports scripting for tests, not sure how Chrome team will support them.

6 Likes

qunit - what our current tests are written in - has support for a Chrome Remote Debugging backend. Iā€™m not sure if theyā€™ll work out of the box due to the fixtures stuff, though. And Iā€™m not sure if itā€™s possible to write true integration tests with just the qunit API.

Ideally would be a Ruby interface to the remote debugging port, to coordinate tests of things like a thread live-updating over the message bus when a new post is made.

# Set up
browser.log_in(user1)
browser_2.log_in(user2)
topic = create_topic(user1, ...)
browser.navigate("/t/#{topic.id}")
browser_2.navigate("/t/#{topic.id}").wait!
browser.wait!

# Action: Create a post in the thread
## or make this into a helper function that manages the composer
browser_2.click('#topic-footer-buttons button.create')
browser_2.type_post('#reply-control .d-editor-textarea-wrapper textarea', "A reply to the topic that satisfies minimum length requirements")
th = Thread.new do ||
  browser.wait_for_message_bus
end
browser_2.click('#reply-control .submit-panel button')
browser_2.wait!

# Check: Post is created on the server
topic.reload
expect(topic.posts.count).to eq(2)

# Check: post is delivered to other browser
new_post = topic.posts.last
th.join # browser.wait_for_message_bus
expect(browser.find_element("article[data-post-id=\"#{new_post.id}\"]")).to #not be nil, I forget how to spell that
4 Likes

Iā€™ve had a play with getting the qunit tests running in headless chrome, and it seems to work alright. The performance improvement isnā€™t as dramatic as Iā€™d hoped - on my computer phantomjs runs the core tests in 227s, and headless chrome runs them in 201s.

Iā€™ve written a JS script, which depends on a couple of NPM modules: chrome-launcher and chrome-remote-interface, which provide an interface to the Chrome Remote Debugging system @riking mentioned.

Iā€™ve then given the rake qunit:test an optional third parameter to use chrome instead of phantomJS. Set the USE_CHROME environment variable to use chrome instead of phantomJS

So, assuming you have chrome and node/npm installed, using it works like this:

npm install chrome-launcher chrome-remote-interface
USE_CHROME=1 rake qunit:test

https://github.com/discourse/discourse/pull/5054

In summary, thereā€™s not much advantage over PhantomJS at the moment, but that will change if PhantomJS stops being developed in future.

10 Likes

10% faster is a pretty big advantage! great work!

Next step is adding this into our docker test image and docker dev image.

4 Likes

Having a look at this at the moment. It looks to me like PhantomJS is currently included in the base image.

https://github.com/discourse/discourse_docker/blob/master/image/base/Dockerfile#L100

Is there any reason for this, and if not shall I move it to only be in the dev/test images? Seems to me like unnecessary bulk for the image used in every install of Discourse. On the other hand, itā€™s only 22mb, so not a big deal ĀÆ\_(惄)_/ĀÆ

Yeah it was kind of ā€œhistoricalā€ cause you can not run the smoke test without it. But I am open to just moving this stuff out, we will just have to be careful with our CI.

I think for the time being just add chrome to the dev/test image and we will move to that.

4 Likes

Good news: These two PRs combined allow use of chrome in discourse_test by setting the USE_CHROME environment variable: :slight_smile:

https://github.com/discourse/discourse/pull/5062

https://github.com/discourse/discourse_docker/pull/369

Bad news: Chrome is huge - it adds 415mb to the image :face_with_thermometer:

8 Likes

Incidentally, I noticed when I tried to build my own base image this week, that docker build died on that ADD. The file is not included in the repo.

Thereā€™s some magic that occurs to obtain phantomJS

The build.rb script executes the download_phantomjs script on the host, which spins up a docker container, which Iā€™m guessing builds a custom version of phantomJS.

So if you run that download_phantomjs script first it should work :slight_smile:

https://github.com/discourse/discourse_docker/blob/master/image/build.rb#L98

https://github.com/discourse/discourse_docker/blob/master/image/base/download_phantomjs

3 Likes

The comment at the top said ā€œRun by Samā€ (or similar), so I just skipped to a manual docker build. Guess I should have read further. :blush:

1 Like

A lot of the phantom mess can go very very shortly, cause we donā€™t want to carry it for too much longer.

6 Likes

@sam I should probably mention - the chrome stuff is currently off by default, so if you want to try it in CI you need to set USE_CHROME=1 :slight_smile:

4 Likes

Something I just stumbled across is Puppeteer, which is created and maintained by the Chromium devtools team. It bundles a version of chromium that skips all the UI dependencies, bringing it in at just under 100mb. Seems like a far tidier option than having to install chrome separately and use a third party library for the API.

It requires node 7 though, so not an option for Discourse quite yet (discourse_docker ships the LTS version, which is version 6). Maybe once version 8 LTS is released in October it could be an option.

The API looks very similar to the chrome-remote-interface module Iā€™ve used, so hopefully will be trivial to switch over once the time comes :slight_smile:

8 Likes

More than happy to upgrade to node 7, this is not an issue

3 Likes

Oh cool, in that case Iā€™ll have a quick look at it next week. What are the critical things that depend on node/npm in Discourse? (so I can check they donā€™t break)

Am I correct in saying most of the critical stuff is run in mini-racer instead?

At the moment we package assets in node, but we can easily migrate to mini racer now that I added explicit support for disposing contexts

4 Likes

Iā€™ve made a PR to install node8 in the base image

https://github.com/discourse/discourse_docker/pull/370

I also had a play with puppeteer, and it is much easier/cleaner than what I had before. Iā€™ve published an NPM module to do exactly what we need - I figure it might be useful for other projects using QUnit. So once node has been upgraded it should be as simple as

npm install -g qunit-puppeteer
qunit-puppeteer http://localhost:3000

The npm install installs puppeteer as a dependency, which in turn pulls in a recent version of chromium :slight_smile:

https://github.com/davidtaylorhq/qunit-puppeteer

Then I think the next step would be trying to figure out how this can slot into AutoSpecā€¦

4 Likes