In theory the test database is supposed to be automatically cleaned up after every test run. So it should never end up in this state. But… if test processes get terminated badly somehow, I think it’s possible for that cleanup (technically, the restoration of a pg savepoint) to be skipped.
So this shouldn’t be a normal occurrence. Personally, I’d only bother with the drop/create/migrate if I start getting weird failures.
But, if you’re ok with the wait, there’s no harm in recreating your test database regularly 