How could a CI workflow for unofficial plugins be implemented?

I invite you to share thoughts about possible CI workflows, which would run tests on community supplied unofficial plugins.

Some communities depend heavily on plugins, without established maintenance:

Since testing plugins depends on a discourse installation as test bed, I wonder how a community supported CI workflow may look like.

Some plugins include specs, like @angus implementation of activitypub, which, as I understand, enable tests for CI.

Currently, I think about two possible ways to improve testing of unofficial plugins, depending on specs / tests, included in the plugin source:

a) to build some machinery, which aides site maintainers to run tests in a staging environment

b) to have some service, which forks a predefined test-image to report tests run on the last published code of the plugin.

What do you think?

Did I miss some already established workflow?

This may be useful:


In addition to CI with backend and frontend tests, which most Pavilion plugins have now, we have a plugin manager system of which CI is one part. This is how it works



Yes, and to add to what @angus has shared, you can find our daily compatibility dashboard here:

Which displays status of checks for compatibility of Pavilion Plugins (and sometimes others) against tests-passed and stable. This is updated every day, automatically.

This of course relies on tests, including smoke tests, specs (backend) and qunit (front end) tests.

Our subscription plugin (Custom Wizard) has the most test coverage, as you might imagine, but some of our free plugins include a good suite of both back-end and front-end tests (e.g. Locations).

Writing tests is good practice and certainly Pavilion has become yet more disciplined in this space as we have matured as a business.

Critically tests also document your intended functionality, which is extremely important especially during compatibility updates or refactoring.


Impressive. Could you point to the code, which implements this?

1 Like

It’s an architecture as per @angus’ diagram, with multiple repos involved, but this is the status server:

It’s also an approach:

  • Never implement a fix without checking for a test, and if a suitable test doesn’t exist, add one.
  • Preferably develop the test first, show that it fails, then fix the problem and confirm your new test passes.

That way your coverage builds over time …


  • Retrofitting tests can be risky as you might misinterpret what the code intended to do … better than doing nothing though …
1 Like

All official plugins have specs that you can use as examples. The discourse-plugin-Skelton plugin includes github Actions to run tests at every commit, and I think, daily.


Do I understand correctly?

a) This is available via GitHub actions: plugins with proper specs / tests using GitHub actions will have a badge on GitHub, if all tests pass and the status of CI actions is readable by API.

b) Except discourse official plugins and pavilion plugins, there exist no automatic overview for administrators, if used plugins will work in the version intended for update?

Searching for metadata on plugin compatibility, I found Pinning plugin and theme versions for older Discourse installs via a .discourse-compatibility file.

As I understand, this is a solution to the reverse problem: discourse to old for a plugin.
How to treat the other way around: plugin to old for discourse?

Could /admin/upgrade warn about plugins, failing tests for an intended upgrade?

We originally set out to and provided an unbranded version of our Plugin dashboard where 3rd party authors could put forward their plugin to be included. This didn’t gain any traction, in part because the population of 3rd party plugin developers is very small indeed.

Third party Discourse plugin creation is pretty niche and third party providing plugins with good test coverage is very niche! :sweat_smile:

A lot of the freelancers in this space have either joined Pavilion or CDCK (or, in time, both!)

So in the end we decide just to collapse our dashboard into our branded community site.