Why are integration specs written in QUnit?

Development

AI-generated summary

The discussion revolves around why integration specs are written in QUnit instead of RSpec. rimian lists the advantages of using RSpec, including running phantomjs directly, removing abstraction layers, and having a nice DSL. However, sam argues that RSpec would require additional abstraction layers and that QUnit tests are not the bottleneck. tgxworld discusses the difference between acceptance tests and integration tests, and how acceptance tests can be slow and hard to write.

The conversation also touches on the terminology used, with mcwumbly suggesting the term “end-to-end tests” to disambiguate tests that go through both client and server code. sam uses the term “smoke test” for tests that glue everything together and test a real production instance.

rilimian points out that RSpec/Capybara tests are framework-agnostic, allowing for tests to be written against different application architectures. He understands that the application already has acceptance tests in QUnit and is not criticizing the existing work, but rather seeking to understand the reasoning behind the choice.

1

I’m wondering what was the reason behind writing integration tests in QUnit when RSpec is available?

The advantages of RSpec would be:

  • Runs the phantomjs driver directly rather than through a shell command
  • Removes the layer of abstraction on top of QUnit (really a unit testing framework)
  • RSpec has a nice DSL that is easy to use. Not much ruby experience needed.
  • Far less boiler plate code (javascript)
  • RSpec can run single tests or groups of them (tags, etc) with options like --fail-fast
  • Headless browser tests run faster
  • Native output to stdout and correct exit code
  • support for creating fixtures and transactional tests.
  • There are already some integration specs in spec/integration

The list goes on. I’ve written complex AngularJS applications with RSpec integration tests. It’s pretty simple really.

2

Do you mean RSpec as in

3

I am actually a fan of having a set of acceptance (high level) tests that allows us to test the entire stack at the same time.

Note that acceptance tests are not the same as integration tests.

Right now, our acceptance tests for the front end uses fixtures to stub the server’s response and that requires us to update it whenever we change our backend API. Furthermore, I find that acceptance tests for complicated scenarios are really hard to write as I have to stub every single request made to the server. That to me introduces alot of friction when writing acceptance tests. My ideal scenario will be to have fixtures be created server side and we just assert for behavior of the app as we interact with it via Capybara (for example). One caveat though is that acceptance tests are usually much slow from my experience. Somehow using capybara to click around and waiting for elements to appear takes quite a bit of time.

With all that being said, writing acceptance tests for the entire stack will not be possible if we ever decide to migrate to ember-cli.

4

This is just a test runner. I mean something like this: discourse/spec/integration/category_tag_spec.rb at main · discourse/discourse · GitHub

Also see: http://betterspecs.org/

5

I am super confused here, there is no “ruby libphantomjs gem” any driver is going to have to run the phantomjs and pipe commands to it. Only way around this is to avoid the need for a real dom and use something like mini_racer to run the specs, which is what I do in the pretty text spec, but is far more difficult to engineer at the application level.

Feel free to correct me if I am mistaken

The only way to do this would be by adding a different (and probably) slower layer of abstraction on top. Cause we got to run the app somehow when specs are running and forward requests to it.

Personally, these days, I really hate rspec and much prefer minitest. I usually prefer qunit tests to rspec. Our bottleneck is friction around running the qunit specs, not the framework itself.

Anyway… in general our focus on refining our current toolset vs “flipping the chess set and choosing checkers instead”.

I strongly recommend you try out rake autospec, try editing a spec see what happens, make it break, make it work and so on. I would like that type of workflow for qunit, it would make me happy.

6

I’m curious to know which tests do you classify as integration tests. We have alot unit tests for our Ember models, controllers, components and widgets which can’t be run and written in RSpec.

7

I see the acceptance tests are stubbing out http requests to the rails app. So, it’s not correct for me to call them integration specs because they don’t test the entire stack. You can however write the acceptance tests in RSpec or mini test.

8

ahh I’m probably confuse because of our different interpretation of the terms being used here. I was under the assumption that you were referring to acceptance tests as integration tests because it will then be possible to write those in RSpec using Capybara. However, majority of our Javascript tests are unit tests and I don’t see how we can replace Qunit with RSpec.

9

I realize I may be stepping into a quagmire with regards to terminology, but I like the term “end to end tests” to refer to and disambiguate tests that go through the client and server code together.

10

I usually use the term “smoke test” for those.

discourse/spec/phantom_js/smoke_test.js at b3965eb06994fbe11b5a945efc99b136714b3fe3 · discourse/discourse · GitHub has got to pass for a build to be stamped with test passed.

We have “integration” tests on the server that look at end-to-end server behavior. And “integration” tests in qunit for end-to-end client behavior.

But the “smoke test” is the only test we have the glues everything together and tests a real production instance.

11

Another nice thing about RSpec/Capybara is that these tests are framework agnostic. As long as your application has a DOM (or in the case of ReactJS, a virtual DOM), you can write tests against it. So, theoretically, you can swap out your application architecture and your integration (or end-to-end) specs should pass.

I’ve been criticised in the past for writing RSpec/Capybara tests against AngularJS because that’s not the way the AngularJS community like to write integration tests. Same for EmberJS apps. But developers have to learn a new test framework for every different application and often these frameworks are not as mature as RSpec, Capybara, Cucumber, etc.

I understand this application already has acceptance tests in QUnit and I’m not criticising the excellent work that’s been put into this project. I just wondered what the reasoning was and possibly I did not fully comprehend how this application works.