Developing Discourse Plugins - Part 6 - Add acceptance tests

Previous tutorial: Developing Discourse Plugins - Part 5 - Add an admin interface


Did you know that Discourse has two large test suites for its code base? On the server side, our Ruby code has a test suite that uses rspec. For the browser application, we have a qunit suite that has ember-testing included.

Assuming you have a development environment set up, if you visit the http://localhost:4200/tests URL you will start running the JavaScript test suite in your browser. One fun aspect is that you can see it testing the application in a miniature window in the bottom right corner:

The Discourse application is built with a lot of tests that will begin running when you visit the /tests URL. So it may be helpful to filter your tests by the plugin you are working on. You can do that in the interface by clicking Plugin dropdown and selecting your plugin:

Adding an Acceptance Test in your Plugin

First, make sure you have the latest version of Discourse checked out. Being able to run Acceptance tests from plugins is a relatively new feature, and if you don’t check out the latest version your tests won’t show up.

For this article I am going to write an acceptance test for the purple-tentacle plugin that we authored in part 5 of this series.

Adding an acceptance test is as easy as adding one file to your plugin. Create the following:

test/javascripts/acceptance/purple-tentacle-test.js

import { acceptance, exists } from "discourse/tests/helpers/qunit-helpers";
import { click, visit } from "@ember/test-helpers";
import { test } from "qunit";

acceptance("Purple Tentacle", function (needs) {
  needs.settings({ purple_tentacle_enabled: true });
  needs.user();

  test("Purple tentacle button works", async function (assert) {
    await visit("/admin/plugins/purple-tentacle");
    assert.ok(exists("#show-tentacle"), "it shows the purple tentacle button");
    assert.ok(!exists(".tentacle"), "the tentacle is not shown yet");
    await click("#show-tentacle");
    assert.ok(exists(".tentacle"), "the tentacle wants to rule the world!");
  });
});

I tried to write the test in a way that is clear, but it might be a little confusing if you’ve never written an acceptance test before. I highly recommend that you read the Ember docs on acceptance testing as they have a lot of great information.

In each test we write, we need to assert something. In our test, we want to make a few assertions to check whether the tentacle is hidden initially and then shown only after clicking the button.

We want to define a set of actions to be taken before an assertion is made. To do that we use the await keyword. By using that keyword, we wait for the execution of each asynchronous helper to finish first.

Our first action of importance is: await visit("/admin/plugins/purple-tentacle");. This tells our test to navigate to that URL in our application. That URL was the one that displays the tentacle.

After visiting the page where the purple tentacle button appears, we want to check if we can see the button on the page exists and that the tentacle image doesn’t exist yet.

That is done by the following assertions:

assert.ok(exists("#show-tentacle"), "it shows the purple tentacle button");
assert.ok(!exists(".tentacle"), "the tentacle is not shown yet");

P.S. the previous version of the purple-tentacle plugin didn’t have the #show-tentacle element id in the handlebars template. Check out the latest version to follow along!

Once those tests pass it’s time to test the interaction.

The next command is await click('#show-tentacle'); which tells our testing framework that we want to click the button and show the tentacle.

After we simulate a click on the button, we can check whether the tentacle appears by asserting:

assert.ok(exists(".tentacle"), "the tentacle wants to rule the world!");

Not too bad is it? You can try the test yourself by visiting http://localhost:4200/tests?qunit_single_plugin=purple-tentacle&qunit_skip_core=1 on your development machine. You should very quickly see the purple tentacle appear and all tests will pass.

If you want to run the plugin qunit tests on the command line using PhantomJS, you can run

rake plugin:qunit['purple-tentacle']

(where purple-tentacle is the folder name of your plugin)

Debugging your tests

As you write your plugins, your tests can help you identify issues in your plugin. When you’re developing your tests or if you make changes to your plugin’s code, the tests may fail. To help understand why, Ember has some nice helpers: pauseTest() and resumeTest().

To make use of them, add await pauseTest() within your test code where you would like the test to pause. Now, when you run your test in the browser, the test will automatically pause at the point you added `pauseTest(). This will give you a chance to inspect the page or view any errors to help debug for issues.

Where to go from here

I hate to sound like a broken record but the Ember documentation on testing is excellent. You might also want to see how Discourse tests various functionality by browsing the tests in our javascript tests directory. We have quite a few examples in there you can learn from.

Happy testing!


More in the series

Part 1: Plugin Basics
Part 2: Plugin Outlets
Part 3: Site Settings
Part 4: git setup
Part 5: Admin interfaces
Part 6: This topic
Part 7: Publish your plugin

29 Likes

OMG, have I put off writing tests for this long?!

A few things
I noticed you put an “id” into the template file that wasn’t there in the previous tutorial
I added 3 more tests to deal with the “Hide Tentacle” button I added
I needed to be logged in it is an Admin page
I needed to use port 4000 for my VirtualBox Vagrant set-up

Happily, quick success !

5 Likes

I’ve noticed a push to implement testing for plugins lately and I completely agree that this is important. I’ve been terrible at writing these in the past and want to get caught up.

I was considering the setup of Travis CI on a plugin repo as I noticed the ability to use Docker with Travis. But having never touched Travis before, is it possible to do this with a plugin? It would be nice to submit PRs to my own repo and have the tests run automatically.

5 Likes

how can I run single ember tests on the browser?

http://localhost:3000/qunit?module=Acceptance%3A%20Purple%20Tentacle

I mean, if I have another acceptance test called “another test”, how would I run it? Thank you!

If you click “rerun” beside a test, you’ll get a URL that is just for that particular test that you can refresh over and over.

You can also run by module by clicking in the “All Module” box and checking off just the modules you want to run.

2 Likes

This tutorial needs updating to replace ok(...) with assert.ok(...).

Have put the necessary changes in a PR - I’m happy to edit the OP if it’s made into a wiki :slight_smile:

https://github.com/eviltrout/purple-tentacle/pull/2

2 Likes

OK it is wikified! PM me if anything else needs this treatment any time.

1 Like

A note after writing some basic acceptance tests for the Elections Plugin:

  • If you use the default loggedIn option, you’re logged in as eviltrout with the properties in this session fixture

  • You can login as a custom user (necessary if you’ve added custom user properties) by defining your own session fixture and setting it as the current user using the beforeEach hook. See here for an example.

@eviltrout is there a better way of using a custom user for acceptance tests?

5 Likes

What we should probably do is change the acceptance method in discourse to take an argument for the user to log in as. Then your test could say { loggedIn: true, logInAs: angus }

If you’ve got time for that PR we’d accept it!

4 Likes

q: how to run the ruby tests tests for a plugin?

i am trying to run the tests for discourse-github-linkback, but i have no idea how.

After some stackoverflow i tried this (and i got some errors):

~/discourse$ bundle exec rspec ./plugins/discourse-github-linkback/spec/lib/github_linkback_spec.rb 

An error occurred while loading ./plugins/discourse-github-linkback/spec/lib/github_linkback_spec.rb.
Failure/Error: CoreExt::ActiveSupport.without_bootsnap_cache { super }

NameError:
  uninitialized constant GithubLinkback
# ./plugins/discourse-github-linkback/spec/lib/github_linkback_spec.rb:3:in `<top (required)>'
# ------------------
# --- Caused by: ---
# NameError:
#   uninitialized constant GithubLinkback
#   ./plugins/discourse-github-linkback/spec/lib/github_linkback_spec.rb:3:in `<top (required)>'
No examples found.

Randomized with seed 60987


Finished in 0.02025 seconds (files took 3.36 seconds to load)
0 examples, 0 failures, 1 error occurred outside of examples

Randomized with seed 60987

Note that i see the plugin in the local discourse dev admin page, so i hope it is correctly installed (into ~discourse/plugins/

Simplest way, run bin/rake autospec, and save the file

1 Like

From what i see, doing cd ~/discourse and then running

runs all the tests for discourse.

On my ubuntu 18.04 VM with i5 6500, this has been running for half an hour already.
It feels wrong to run the whole tests suite when i only want to run the tests in that one plugin.
(this is especially so when i probably will modify my tests again, at least a few times)

Did i do something wrong?

Let me guess, you are symlinking your plugin vs checking out directly to the path under plugins

nope,

tbp@tbp-linux-dev:~$ cd ~/discourse/plugins/
git clone -b allow-all-user-repositories-wildcard https://github.com/TheBestPessimist/discourse-github-linkback
tbp@tbp-linux-dev:~/discourse/plugins$ ll
total 40
drwxr-xr-x 10 tbp tbp 4096 mai 13 21:48 ./
drwxr-xr-x 20 tbp tbp 4096 mai 13 21:18 ../
drwxr-xr-x  6 tbp tbp 4096 mai 13 20:29 discourse-details/
drwxr-xr-x  6 tbp tbp 4096 mai 13 21:48 discourse-github-linkback/
drwxr-xr-x  6 tbp tbp 4096 mai 13 20:29 discourse-local-dates/
drwxr-xr-x  9 tbp tbp 4096 mai 13 20:29 discourse-narrative-bot/
drwxr-xr-x  7 tbp tbp 4096 mai 13 20:29 discourse-nginx-performance-report/
drwxr-xr-x  5 tbp tbp 4096 mai 13 20:29 discourse-presence/
drwxr-xr-x  3 tbp tbp 4096 mai 13 20:29 lazyYT/
drwxr-xr-x  9 tbp tbp 4096 mai 13 20:29 poll/
tbp@tbp-linux-dev:~/discourse/plugins$ 

Can you confirm you tried saving the spec file once it was running, cause it is meant to interrupt stuff and start running the plugin spec?

since the initial bin/rake autospec is still running, there was/is no “save spec file” message yet.
also, by running bin/rake autospec --help, i saw no --save option.

I didn’t understand what save the file meant, so my full command was bin/rake autospec > spec.rb <- probably this is wrong

I meant in Vim or whatever you use, save the spec file, there is a watcher on it that will invoke the spec runner once it is saved.

Also see: How do I run only discourse/plugins/poll/spec?

1 Like

Finally i did it :^).

With the help of that link, i ran the tests by using this command:
bundle exec rake plugin:spec["discourse-github-linkback"]
and, ofc, i was in ~/discourse when doing that.

Thanks for all your help @sam!

(1st PR for discourse-plugin babyyyyy :tada:)

8 Likes

My Qunit-Tests aren’t executed or at least they don’t show me any output.
They finish with the following:

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#Build-times-out-because-no-output-was-received

Travis.CI Result - Travis CI - Test and Deploy Your Code with Confidence
Test-File - https://github.com/kokoro-ko/discourse-humble-box

I am not that skilled with CI yet and haven’t found my mistake by now.

Do your tests pass if you visit /qunit on your development environment? How about if you run rake plugin:qunit[discouse-humble-box]?

2 Likes