I’ve been doing a lot of experimenting with QUnit today, and have had another look at this issue.
discourse_dev
The issue was indeed with the LOAD_PLUGINS environment variable. When using the bin/docker/rake
script, the LOAD_PLUGINS variable doesn’t get passed through to the container, hence why I couldn’t get it working with the discourse_dev
image.
discourse_test
I also experimented with the discourse_test
image. It is my understanding that the discourse_test
image is used to test master
before merging into tests-passed
(according to this). In its default configuration:
docker run -it --rm discourse/discourse_test
it does not seem to run any plugin tests at all (either rspec or qunit). I am assuming that’s how broken core plugin tests managed to end up in tests-passed. I managed to get it to run both qunit and rspec plugin tests by running
docker run -e "LOAD_PLUGINS=1" -it --rm discourse/discourse_test
Maybe that’s something that could be used internally to make sure core plugins get tested properly? Or alternatively the LOAD_PLUGINS
variable could be set in the Dockerfile?
Travis
Travis is currently running plugin rspec tests, but not qunit tests. I made a PR for turning on plugin qunit tests here:
https://github.com/discourse/discourse/pull/4971
Running that currently results in 11 qunit failures from a number of plugins: (Cakeday, Canned Replies, Details, and Spoiler)
QUnit Fails
Module Failed: Acceptance: Cakeday
Test Failed: Anniversary emoji
Assertion Failed: Error: Unhandled request in test environment: /t/11.json?track_visit=true&forceLoad=true&_=1499806929081 (GET)
Expected: true, Actual: false
Assertion Failed: TypeError: undefined is not an object (evaluating '$posterIcons[0].title')
Expected: true, Actual: false
Test Failed: User is not logged in
Assertion Failed:
phantomjs://webpage.evaluate():49
Module Failed: Acceptance: Canned Replies
Test Failed: Inserting canned replies
Assertion Failed: Error: Unhandled request in test environment: /canned_replies?_=1499806929085 (GET)
Expected: true, Actual: false
Assertion Failed: ReferenceError: Can't find variable: equal
Expected: true, Actual: false
Test Failed: Editing a canned reply
Assertion Failed: Error: Unhandled request in test environment: /canned_replies?_=1499806929088 (GET)
Expected: true, Actual: false
Assertion Failed: Error: Element .reply-selector #canned-replies-combobox not found.
Expected: true, Actual: false
Test Failed: Creating a new canned reply
Assertion Failed: Error: Unhandled request in test environment: /canned_replies?_=1499806929091 (GET)
Expected: true, Actual: false
Assertion Failed: ReferenceError: Can't find variable: equal
Expected: true, Actual: false
phantomjs://webpage.evaluate():49
Module Failed: Acceptance: Details Button
Test Failed: details button
Assertion Failed: it should contain the right output
Expected: Before
[details=Summary]
some text in between
[/details]
After, Actual: Before
[details=Summary]
some text in between
[/details]
After
phantomjs://webpage.evaluate():49
Module Failed: Acceptance: Spoiler Button
Test Failed: spoiler button
Assertion Failed: ReferenceError: Can't find variable: ok
Expected: true, Actual: false
These are legitimate fails, which I can reproduce running the tests suite locally. I’ve already fixed all but one of the “discourse-details” tests, and the “Spoiler” test should be an easy fix. The others may need more investigation.
Edit: discourse-details PR here and spoiler alert PR here