Experimenten met Model-Based Testing

AI-generated summary

larrytheliquid introduces Colimit, a tool for model-based testing, and proposes using it to test Discourse. They explain that model-based testing generates numerous user flows from a high-level specification, potentially uncovering hard-to-find bugs. Since Discourse is open-source and complex, it’s a good candidate for testing with Colimit, which can adapt to different testing styles (API, integration, smoke tests). Larry asks the community to identify areas with high bug rates or low test coverage that would benefit most from this approach, and seeks feedback on whether model-based testing would add value given Discourse’s current testing state.

Hi Discourse Community! :slight_smile:

I’ve been working on tech (Colimit) that helps people apply Model-based testing to test their web/browser/DOM frontend, API backend, or even ORM-models or view-models. If you’re not familiar with the idea, you basically define a high-level spec of what your app is supposed to do in terms of possible actions that can be taken, and from that you get hundreds or thousands of sequences of valid user actions (i.e. user flows) that can intelligently crawl a large state space of possibilities and uncover hard-to-reach bugs compared to hand-written test cases.

I’m coming out of the phase of using Colimit on small examples and thought of Discourse as a possibly great candidate to try it out on next, because it’s both a sophisticated app that’s run in production but is also fully open source, making it easy to tinker with. The cool thing is that Colimit is written in an abstracted way so you can reuse the same specification/model (by writing adapters) to do different styles of testing like e.g. the Discourse API tests via the gem, versus the integration tests using Capybara, vs the smoke tests using Puppeteer, etc.

Before getting started, I was wondering if anyone from the Discourse community can think of any areas of the app that are currently major sources of bugs or have less coverage that could be more valuable to focus time on?
Also, just out of curiosity I’d appreciate any replies from people that think this sounds interesting and sees potential value for testing the Discourse project, or if the opinion is that the current state of testing is already good enough (in which case there is still at least possible maintenance value from being able to replace manually written tests from those that could be generated automatically from a model).

Thanks!

1 like