I’ve been having some problems with failing tests in the discourse_dev docker image, and I assume they’re down to the addition of multisite tests.
When creating a new container, or running RAILS_ENV=test rake db:migrate
in an existing container, the problems start with a missing database:
ActiveRecord::NoDatabaseError: FATAL: database "discourse_test_multisite" does not exist
Fixing this is simple enough: once I’ve given the discourse user sufficient permissions, I can create the necessary database with RAILS_ENV=test rake db:create
.
If I then run RAILS_ENV=test rake db:migrate
again nothing errors out, so everything seems fixed.
At least until I run the rests:
Finished in 9 minutes 54 seconds (files took 6.19 seconds to load)
7414 examples, 14 failures
Even if I nuke my docker container and all its data, I still eventually get 14 failing tests*.
Sometimes the first test run will succeed with all but one test, which will then succeed upon rerunning it, which is when the 14 tests will fail.
These failing tests all seem be down to there being more rows in the database than expected, and it also seems like the discourse_test
database isn’t getting properly cleared:
discourse_test=# select id from categories;
id
------
1
1042
522
(3 rows)
Which would match some of the extra rows causing tests to fail:
7) CategoryList category order fixed_category_positions is disabled returns categories in order of activity
Failure/Error: expect(category_ids).to eq([cat2.id, cat1.id])
expected: [1201, 1200]
got: [1201, 1200, 1, 522]
So I imagine, aside from creating the database, there’s some step I’ve omitted to mark the discourse_test
database for truncating after running tests.
@tgxworld any idea what’s going on here/what I’m missing?
Full(ish) autospec output
discourse@discourse:/src$ rake autospec
If file watching is not working, you can force polling with: bundle exec rake autospec p l=3
Skipping JS tests, run them in the browser at /qunit or add QUNIT=1 to env
Running Rspec: spec
Press [ENTER] to stop the current run
Randomized with seed 60487
[...]
Failures:
1) pool drainer can correctly drain the connection pool
Failure/Error: expect(old).to eq(1)
expected: 1
got: 0
(compared using ==)
# ./spec/components/freedom_patches/pool_drainer_spec.rb:11:in `block (2 levels) in <main>'
Finished in 10 minutes 24 seconds (files took 5.66 seconds to load)
7414 examples, 1 failure
Failed examples:
rspec ./spec/components/freedom_patches/pool_drainer_spec.rb:8 # pool drainer can correctly drain the connection pool
Randomized with seed 60487
The following specs have failed:
./spec/components/freedom_patches/pool_drainer_spec.rb:8
Running Rspec: ./spec/components/freedom_patches/pool_drainer_spec.rb:8
Seeding uncategorized category!
Run options: include {:locations=>{"./spec/components/freedom_patches/pool_drainer_spec.rb"=>[8]}}
Randomized with seed 32607
.
Finished in 0.03986 seconds (files took 3.14 seconds to load)
1 example, 0 failures
Randomized with seed 32607
Running Rspec: spec
Randomized with seed 35771
[...]
Failures:
1) Guardian topic featured link category restriction uncategorized forbids featured links if uncategorized forbids it
Failure/Error: uncategorized.save!
ActiveRecord::RecordInvalid:
Validation failed: Slug is already in use
# ./spec/components/guardian_spec.rb:2513:in `block (4 levels) in <main>'
2) Guardian topic featured link category restriction uncategorized allows featured links if uncategorized allows it
Failure/Error: uncategorized.save!
ActiveRecord::RecordInvalid:
Validation failed: Slug is already in use
# ./spec/components/guardian_spec.rb:2507:in `block (4 levels) in <main>'
3) Guardian can_edit? a Topic not archived returns false when the category is read only
Failure/Error: expect(Guardian.new(trust_level_3).can_edit?(topic)).to eq(false)
expected: false
got: true
(compared using ==)
# ./spec/components/guardian_spec.rb:1213:in `block (5 levels) in <main>'
4) Guardian can_edit? a Topic not archived returns false for trust level 3 if category is secured
Failure/Error: expect(Guardian.new(trust_level_3).can_edit?(topic)).to eq(false)
expected: false
got: true
(compared using ==)
# ./spec/components/guardian_spec.rb:1225:in `block (5 levels) in <main>'
5) Category topic_create_allowed and post_create_allowed works
Failure/Error: expect(Category.topic_create_allowed(guardian).count).to be(5)
expected #<Integer:11> => 5
got #<Integer:13> => 6
Compared using equal?, which compares object identity,
but expected and actual are not the same object. Use
`expect(actual).to eq(expected)` if you don't care about
object identity in this example.
# ./spec/models/category_spec.rb:75:in `block (3 levels) in <main>'
6) Category security lists all secured categories correctly
Failure/Error: expect(Category.secured).to match_array([uncategorized])
expected collection contained: [#<Category id: 522, name: "Uncategorizedc7b55971bfbdd86848b2ecf179fa95d9", color: "AB9364", topic_id... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
actual collection contained: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
the extra elements were: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
# ./spec/models/category_spec.rb:138:in `block (3 levels) in <main>'
7) Category update_stats for uncategorized category updates topic stats
Failure/Error: expect(@uncategorized.topics_week).to eq(1)
expected: 1
got: 0
(compared using ==)
# ./spec/models/category_spec.rb:502:in `block (4 levels) in <main>'
8) CategoriesController reorder reorders the categories
Failure/Error:
expect(list.categories).to eq([
Category.find(SiteSetting.uncategorized_category_id),
c1,
c4,
c2,
c3
])
expected: [#<Category id: 522, name: "Uncategorizedc7b55971bfbdd86848b2ecf179fa95d9", color: "AB9364", topic_id... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
got: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
(compared using ==)
Diff:
@@ -1,6 +1,7 @@
[...]
# ./spec/controllers/categories_controller_spec.rb:126:in `block (3 levels) in <main>'
9) Site omits categories users can not write to from the category list
Failure/Error: expect(Site.new(Guardian.new(user)).categories.count).to eq(2)
expected: 2
got: 3
(compared using ==)
# ./spec/models/site_spec.rb:47:in `block (2 levels) in <main>'
10) Scheduler::Manager can disable stats
Failure/Error: expect(ActiveRecord::Base.connection_pool.connections.reject { |c| !c.in_use? }.length).to eq(1)
expected: 1
got: 0
(compared using ==)
# ./spec/components/scheduler/manager_spec.rb:73:in `block (2 levels) in <main>'
11) CategoryList security properly hide secure categories
Failure/Error: expect(CategoryList.new(Guardian.new admin).categories.count).to eq(2)
expected: 2
got: 3
(compared using ==)
# ./spec/models/category_list_spec.rb:19:in `block (3 levels) in <main>'
12) CategoryList category order fixed_category_positions is enabled returns categories in specified order
Failure/Error: expect(category_ids).to eq([cat2.id, cat1.id])
expected: [898, 897]
got: [1, 898, 897]
(compared using ==)
# ./spec/models/category_list_spec.rb:102:in `block (4 levels) in <main>'
13) CategoryList category order fixed_category_positions is enabled handles duplicate position values
Failure/Error: expect(first_three).to include(cat4.id)
expected [1, 899, 900] to include 902
# ./spec/models/category_list_spec.rb:110:in `block (4 levels) in <main>'
14) CategoryList category order fixed_category_positions is disabled returns categories in order of activity
Failure/Error: expect(category_ids).to eq([cat2.id, cat1.id])
expected: [904, 903]
got: [904, 903, 1]
(compared using ==)
# ./spec/models/category_list_spec.rb:123:in `block (4 levels) in <main>'
15) CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Failure/Error: expect(category_ids).to eq([cat1.id, cat2.id])
expected: [905, 906]
got: [1, 905, 906]
(compared using ==)
# ./spec/models/category_list_spec.rb:128:in `block (4 levels) in <main>'
Finished in 9 minutes 51 seconds (files took 6.83 seconds to load)
7414 examples, 15 failures
Failed examples:
rspec ./spec/components/guardian_spec.rb:2511 # Guardian topic featured link category restriction uncategorized forbids featured links if uncategorized forbids it
rspec ./spec/components/guardian_spec.rb:2505 # Guardian topic featured link category restriction uncategorized allows featured links if uncategorized allows it
rspec ./spec/components/guardian_spec.rb:1209 # Guardian can_edit? a Topic not archived returns false when the category is read only
rspec ./spec/components/guardian_spec.rb:1221 # Guardian can_edit? a Topic not archived returns false for trust level 3 if category is secured
rspec ./spec/models/category_spec.rb:49 # Category topic_create_allowed and post_create_allowed works
rspec ./spec/models/category_spec.rb:129 # Category security lists all secured categories correctly
rspec ./spec/models/category_spec.rb:501 # Category update_stats for uncategorized category updates topic stats
rspec ./spec/controllers/categories_controller_spec.rb:101 # CategoriesController reorder reorders the categories
rspec ./spec/models/site_spec.rb:43 # Site omits categories users can not write to from the category list
rspec ./spec/components/scheduler/manager_spec.rb:108 # Scheduler::Manager can disable stats
rspec ./spec/models/category_list_spec.rb:12 # CategoryList security properly hide secure categories
rspec ./spec/models/category_list_spec.rb:100 # CategoryList category order fixed_category_positions is enabled returns categories in specified order
rspec ./spec/models/category_list_spec.rb:105 # CategoryList category order fixed_category_positions is enabled handles duplicate position values
rspec ./spec/models/category_list_spec.rb:120 # CategoryList category order fixed_category_positions is disabled returns categories in order of activity
rspec ./spec/models/category_list_spec.rb:126 # CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Randomized with seed 35771
The following specs have failed:
./spec/components/guardian_spec.rb:2511
./spec/components/guardian_spec.rb:2505
./spec/components/guardian_spec.rb:1209
./spec/components/guardian_spec.rb:1221
./spec/models/category_spec.rb:49
./spec/models/category_spec.rb:129
./spec/models/category_spec.rb:501
./spec/controllers/categories_controller_spec.rb:101
./spec/models/site_spec.rb:43
./spec/components/scheduler/manager_spec.rb:108
./spec/models/category_list_spec.rb:12
./spec/models/category_list_spec.rb:100
./spec/models/category_list_spec.rb:105
./spec/models/category_list_spec.rb:120
./spec/models/category_list_spec.rb:126
Running Rspec: ./spec/models/category_list_spec.rb:126
Seeding uncategorized category!
Run options: include {:locations=>{"./spec/models/category_list_spec.rb"=>[126]}}
Randomized with seed 34576
F
Failures:
1) CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Failure/Error: expect(category_ids).to eq([cat1.id, cat2.id])
expected: [1043, 1044]
got: [1, 522, 1043, 1044]
(compared using ==)
# ./spec/models/category_list_spec.rb:128:in `block (4 levels) in <top (required)>'
Finished in 2.14 seconds (files took 2.15 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/models/category_list_spec.rb:126 # CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Randomized with seed 34576
The following specs have failed:
./spec/models/category_list_spec.rb:126
Running Rspec: ./spec/models/category_list_spec.rb:126
Run options: include {:locations=>{"./spec/models/category_list_spec.rb"=>[126]}}
Randomized with seed 27151
F
Failures:
1) CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Failure/Error: expect(category_ids).to eq([cat1.id, cat2.id])
expected: [1045, 1046]
got: [1, 522, 1045, 1046]
(compared using ==)
# ./spec/models/category_list_spec.rb:128:in `block (4 levels) in <top (required)>'
Finished in 2.2 seconds (files took 2.02 seconds to load)
1 example, 1 failure
Failed examples:
rspec ./spec/models/category_list_spec.rb:126 # CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Randomized with seed 27151
^Cdiscourse@discourse:/src$ rake autospec
If file watching is not working, you can force polling with: bundle exec rake autospec p l=3
Skipping JS tests, run them in the browser at /qunit or add QUNIT=1 to env
Running Rspec: spec
Press [ENTER] to stop the current run
Randomized with seed 42750
[...]
Failures:
1) Category update_stats for uncategorized category updates topic stats
Failure/Error: expect(@uncategorized.topics_week).to eq(1)
expected: 1
got: 0
(compared using ==)
# ./spec/models/category_spec.rb:502:in `block (4 levels) in <main>'
2) Category security lists all secured categories correctly
Failure/Error: expect(Category.secured).to match_array([uncategorized])
expected collection contained: [#<Category id: 1042, name: "Uncategorizedcf1b1a1d6c22e08611cdfaef2f8862cf", color: "AB9364", topic_i... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
actual collection contained: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
the extra elements were: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
# ./spec/models/category_spec.rb:138:in `block (3 levels) in <main>'
3) Category topic_create_allowed and post_create_allowed works
Failure/Error: expect(Category.topic_create_allowed(guardian).count).to be(5)
expected #<Integer:11> => 5
got #<Integer:15> => 7
Compared using equal?, which compares object identity,
but expected and actual are not the same object. Use
`expect(actual).to eq(expected)` if you don't care about
object identity in this example.
# ./spec/models/category_spec.rb:75:in `block (3 levels) in <main>'
4) CategoryList security properly hide secure categories
Failure/Error: expect(CategoryList.new(Guardian.new admin).categories.count).to eq(2)
expected: 2
got: 4
(compared using ==)
# ./spec/models/category_list_spec.rb:19:in `block (3 levels) in <main>'
5) CategoryList category order fixed_category_positions is enabled returns categories in specified order
Failure/Error: expect(category_ids).to eq([cat2.id, cat1.id])
expected: [1195, 1194]
got: [1, 1195, 522, 1194]
(compared using ==)
# ./spec/models/category_list_spec.rb:102:in `block (4 levels) in <main>'
6) CategoryList category order fixed_category_positions is enabled handles duplicate position values
Failure/Error: expect(first_three).to include(cat4.id)
expected [1, 1196, 1197] to include 1199
# ./spec/models/category_list_spec.rb:110:in `block (4 levels) in <main>'
7) CategoryList category order fixed_category_positions is disabled returns categories in order of activity
Failure/Error: expect(category_ids).to eq([cat2.id, cat1.id])
expected: [1201, 1200]
got: [1201, 1200, 1, 522]
(compared using ==)
# ./spec/models/category_list_spec.rb:123:in `block (4 levels) in <main>'
8) CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
Failure/Error: expect(category_ids).to eq([cat1.id, cat2.id])
expected: [1202, 1203]
got: [1, 522, 1202, 1203]
(compared using ==)
# ./spec/models/category_list_spec.rb:128:in `block (4 levels) in <main>'
9) CategoriesController reorder reorders the categories
Failure/Error:
expect(list.categories).to eq([
Category.find(SiteSetting.uncategorized_category_id),
c1,
c4,
c2,
c3
])
expected: [#<Category id: 1042, name: "Uncategorizedcf1b1a1d6c22e08611cdfaef2f8862cf", color: "AB9364", topic_i... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
got: [#<Category id: 1, name: "Uncategorized", color: "AB9364", topic_id: nil, topic_count: 0, created_at:... default_view: nil, subcategory_list_style: "rows_with_featured_topics", default_top_period: "all">]
(compared using ==)
Diff:
@@ -1,6 +1,8 @@
[...]
# ./spec/controllers/categories_controller_spec.rb:126:in `block (3 levels) in <main>'
10) Guardian can_edit? a Topic not archived returns false when the category is read only
Failure/Error: expect(Guardian.new(trust_level_3).can_edit?(topic)).to eq(false)
expected: false
got: true
(compared using ==)
# ./spec/components/guardian_spec.rb:1213:in `block (5 levels) in <main>'
11) Guardian can_edit? a Topic not archived returns false for trust level 3 if category is secured
Failure/Error: expect(Guardian.new(trust_level_3).can_edit?(topic)).to eq(false)
expected: false
got: true
(compared using ==)
# ./spec/components/guardian_spec.rb:1225:in `block (5 levels) in <main>'
12) Guardian topic featured link category restriction uncategorized allows featured links if uncategorized allows it
Failure/Error: uncategorized.save!
ActiveRecord::RecordInvalid:
Validation failed: Slug is already in use
# ./spec/components/guardian_spec.rb:2507:in `block (4 levels) in <main>'
13) Guardian topic featured link category restriction uncategorized forbids featured links if uncategorized forbids it
Failure/Error: uncategorized.save!
ActiveRecord::RecordInvalid:
Validation failed: Slug is already in use
# ./spec/components/guardian_spec.rb:2513:in `block (4 levels) in <main>'
14) Site omits categories users can not write to from the category list
Failure/Error: expect(Site.new(Guardian.new(user)).categories.count).to eq(2)
expected: 2
got: 4
(compared using ==)
# ./spec/models/site_spec.rb:47:in `block (2 levels) in <main>'
Finished in 9 minutes 53 seconds (files took 6.03 seconds to load)
7414 examples, 14 failures
Failed examples:
rspec ./spec/models/category_spec.rb:501 # Category update_stats for uncategorized category updates topic stats
rspec ./spec/models/category_spec.rb:129 # Category security lists all secured categories correctly
rspec ./spec/models/category_spec.rb:49 # Category topic_create_allowed and post_create_allowed works
rspec ./spec/models/category_list_spec.rb:12 # CategoryList security properly hide secure categories
rspec ./spec/models/category_list_spec.rb:100 # CategoryList category order fixed_category_positions is enabled returns categories in specified order
rspec ./spec/models/category_list_spec.rb:105 # CategoryList category order fixed_category_positions is enabled handles duplicate position values
rspec ./spec/models/category_list_spec.rb:120 # CategoryList category order fixed_category_positions is disabled returns categories in order of activity
rspec ./spec/models/category_list_spec.rb:126 # CategoryList category order fixed_category_positions is disabled returns categories in order of id when there's no activity
rspec ./spec/controllers/categories_controller_spec.rb:101 # CategoriesController reorder reorders the categories
rspec ./spec/components/guardian_spec.rb:1209 # Guardian can_edit? a Topic not archived returns false when the category is read only
rspec ./spec/components/guardian_spec.rb:1221 # Guardian can_edit? a Topic not archived returns false for trust level 3 if category is secured
rspec ./spec/components/guardian_spec.rb:2505 # Guardian topic featured link category restriction uncategorized allows featured links if uncategorized allows it
rspec ./spec/components/guardian_spec.rb:2511 # Guardian topic featured link category restriction uncategorized forbids featured links if uncategorized forbids it
rspec ./spec/models/site_spec.rb:43 # Site omits categories users can not write to from the category list
Randomized with seed 42750
^Cdiscourse@discourse:/src$
* one time it was 15. I’m not sure why.