Our poor old test suite is order sensitive

(Sam Saffron) #1

I just added order = 'random' to our spec suite and noticed a pile of failing specs. Will look at cleaning much of this Monday.

Just wanted to make sure that we all know that not working in order random is really considered a bug and something we should fix it indicates tests are fragile.

We also want to make sure we eliminate all cases of before :all in rspec, its a feature that will be removed and is pretty broken anyway (often causing the random order fail issue)

cc @adkron

(Jeff Atwood) #2

The way to repro is to specify a manual seed, from GitHub:

How about seed 55513 it has 76 failures? #5026 had 5 failures. I think between those two we can have a good start.

(Jeff Atwood) #3

I believe we have resolved this; if anyone else still sees order sensitivity in tests that’s a good thing to open on GitHub actually.

(Jeff Atwood) #4