Plugin compatibility is an issue that has bitten me a couple of times. Yes - I have been using unsupported plugins. But no - these shouldn’t be able to take down my forum when I innocently hit the upgrade button on Discourse core.
Firstly, I accept that the hard-working core Discourse team have enough on their plate, and they shouldn’t have to support myriad third party plugins. This would slow core development down and create all sorts of awkward developer coordination bottlenecks.
However, there may be a simple and universal solution here that doesn’t require ongoing effort from the core team. I propose that on hitting the upgrade button, a new docker image is built in the background, and within it, acceptance tests of all plugins are run before the switch-over happens from old to new docker image (apologies if I have mis-stated any technical terminology). If any tests fail, the upgrade is cancelled and the admin is alerted to the failing tests. No site downtime. No errors seen by users.
With this in place I can be confident that upgrading the forum is not a “Russian roulette” operation.
I agree the user experience right now is pretty bad - plugins break regularly, and uninstalling one-by-one is incredibly tedious when using the default install.
I don’t think it’s a case of responsibility, more about making the Discourse product more resilient to change. I don’t think anyone would expect the core team to maintain 3rd party plugins.
I think a good first step towards making things better would be to add some kind of “disable plugin” button to Docker_manager. Then a future thing could be to make disabling broken plugins automatic based on acceptance tests.
I think it should be possible to implement that so that it only requires restarting the server, rather than rebuilding… but not 100% sure of that
I would propose something simple in the way WordPress / WooCommerce does it…
… no mythical perhaps complex smoke tests…
This blog post outlines the recent introduction specifically for WooCommerce related plugins:
When an upgrade check is performed Discourse could advise / give helpful warnings to prevent plugin related errors that could lead to broken features / downtime.
A plugin predefined file should contain two Discourse version number values:
“Requires at least”
“Tested up to”
“Tested up to” Discourse version number which indicates the version of Discourse the developer of the plugin has tested the plugin against.
The blog post goes into benefits of requiring the plugin developer to update this version number and ultimately reducing the immediate work required by the plugin developer to update the plugin because the user is warned prior to upgrade.
So that’s where to put it. And then docker_manager would have to pull that file for each plugin? That seems like a drag, but also seems like a safe solution. Then people could see on the upgrade screen that a plugin hadn’t been tested and they’d know to wait or ask the developer.
This could reduce substantially the number of “Rebuild broke” . . . “Disable plugins” topics.
I don’t think the version numbers declaration will work in my use case. I stay on the latest discourse code, upgrading every couple of days. Continuous deployment is great. No Big Bang changes. And version numbers are largely meaningless.
I wouldn’t expect all plugin developers to manually test against every single commit of discourse-core and manually declare version compatibility on an ongoing basis.
Surely it’s easier for the upgrade process to automatically run a test suite for all plugins, and then to cancel the upgrade if any of the tests fail?
Plugin developers should be keeping their code compatible with Discourse core on an ongoing basis.
I don’t want to have to choose between a) moving forward and losing key features of my site because of plugin incompatibility and b) staying put and failing to get new security updates in discourse core.
An automated testing approach would help plugin developers because they’d be made aware at exactly the point at which their plugin loses compatibility with Discourse core, so they (or others via a GitHub PR) are able to diagnose the incompatibility more easily and fix it.
Automated tests are not “mythical” or hard. Most plugins already have them. They’re just not currently being run when a Discourse instance is upgraded.
I work in the finance industry. If automated testing wasn’t possible, we wouldn’t be able to iterate our software. Simple as that. Luckily automated testing is easy, and it’s something that enables us all to iterate confidently without unexpectedly breaking the system as a whole.
For the record we always smoke test every official plugin (and run the tests the plugin provides) during our CI process which happens before meta is deployed on every commit.
I don’t see it as the job of the installer to be running the tests is a continuous integration server that should be doing this for everyone. But I am not sure how much extra work I want to take on for the ecosystem right now. Long term I am fine to have a digital ocean instance that is constantly testing plugins against latests commits.
For starters, the only way of enabling a plugin now is by doing a full rebuild, so that is a killer here. We would need to build a system that allows you to enable/disable plugins from /admin which requires a whole bunch of work.
I think the first task here regardless is “enable / disable” buttons in /admin/upgrade that knows to talk to the server and restart it. (note, none of this work is really reusable for our internal hosting something that makes me not super enthused about doing it now)