Setting up plugin continuous integration tests on Travis CI

(David Taylor) #1

Following on from Beginner’s Guide to Creating Discourse Plugins Part 6: Acceptance Tests and some brief mention here and here

I’ve got a travis setup working which pulls the latest version of Discourse, ‘installs’ the plugin, and runs all the tests. It does this using the discourse_test docker image, so the environment should match up almost exactly with production systems. Any changes to dependencies, including things like ruby/postgres versions should happen seamlessly as the team rebuild the docker image.

Setting it up

  • Download this gist, and commit it as .travis.yml in the root directory of your plugin’s repository.
  • Head over to, and sign up with your GitHub account
  • Once you’re signed in, go to your profile page and enable builds for your plugin
  • And that’s it! Every time you make a commit or a PR, tests will be run against the tests-passed version

To make sure that new discourse releases don’t break your plugin, you can try setting up Travis’s CRON beta to run the tests every day, regardless of whether you’ve changed the plugin. My setup looks like this:

All available environment variables are listed at the top of docker.rake:

QUnit tests won't pass in discourse_dev docker image
(Vinoth Kannan) #2

Can you please tell why you allow_failure on stable branch instead of beta branch?

(David Taylor) #3

Ideally I don’t want to allow_failure on any of the 3 release branches. Unfortunately the stable branch of discourse currently fails its own tests (see all the red crosses here) and has been failing since at least 6th Feb. That’s kinda ironic since it’s supposed to be the stable branch :wink:

So once that’s fixed I’ll turn off allow_failure

(David Taylor) #4

Getting plugin CI tests working on travis is now super easy with the recent changes to the docker_test rake task. I’ve update the gist in my top post. discourse-chat-integration currently takes about 3.5 mins to load the image, setup, run rspec tests & qunit tests, so pretty quick by travis standards.

@team could someone please make the top post a wiki so I can update it? :slight_smile: Also, would you accept PRs to add travis to official plugin repos?

(Angus McLeod) #5

@David_Taylor This script didn’t work for me out of the box. Is there any other config it assumes?

It fails in Travis with

The command "docker run -e "COMMIT_HASH=origin/tests-passed" -e "SKIP_CORE=1" -e SINGLE_PLUGIN=$plugin_name -v $(pwd):/var/www/discourse/plugins/$plugin_name discourse/discourse_test:release" exited with 1.

(David Taylor) #6

Hmmm… I don’t think there should be anything required beyond the Travis config above.

There’s a working config on discourse-chat-integration: Travis CI - Test and Deploy Your Code with Confidence

Do you have a link to a failing Travis job you can spare?

(Angus McLeod) #7

Yup: Travis CI - Test and Deploy Your Code with Confidence

(David Taylor) #8

Looks like it’s failing at the rubocop linting stage - roughly line 710 in the log.

If you don’t want to lint the code, and just run tests, you can add -e SKIP_LINT=1 to the docker run command.

All the environment variables available are listed at the top of docker.rake:

(Angus McLeod) #9

Ah. I had assumed they were warnings rather than errors that would stop the script. Thanks for taking a look.

(David Taylor) #10

No problem :slight_smile:

The docker script is designed to “fail quickly” - if any of the steps fail, it will exit with an error, and won’t bother with the rest of the steps.

This is good from an efficiancy point of view, but it can get quite annoying when debugging!

The length of the logs it produces is also obscene, which doesn’t help :confounded:

(Angus McLeod) #11

Sorry to keep peppering you with questions, but I’ve found that when a plugin includes a gem, like my locations plugin does, gem install [gem] will throw a permissions error when Discourse attempts to install it in the Docker container. Any thoughts on how to set the permissions right inside the container?

See: Travis CI - Test and Deploy Your Code with Confidence

Note that lib/plugin_gem.rb handles the installation of plugin gems (I think).

(David Taylor) #12

Because we’re mounting the plugin code as a volume in the docker container, it gets all of its permissions/owner/group from the host system (travis). The UID of the travis user, and the UID of the “discourse” user inside the container are not the same, and so the discourse container does not have write access to the plugin directory.

I would never recommend this in a production system, but since this is just tests, I think it is safe to do a chmod -R 777 on the plugin files before running the tests. I wouldn’t call it a fix, but it does work around the problem.

I submitted a PR to your repo with the 1 line fix, which works for travis builds on my fork :slight_smile:

I’ll add a note to the OP here in case anyone else has the same issue