Tests have a shelf life. Boris Beizer famously wrote about the pesticide paradox a few decades ago:
Yesterday’s elegant, revealing, effective test suite will wear out because programmers and designers, given feedback on their bugs, do modify their programming habits and style in an attempt to reduce the incidence of bugs they know about.
A test may start out providing excellent information about gaps in expected behavior, particularly a broadly scoped test against a legacy system. In these early days, there are boundless new edge cases and obscure errors lurking in dark corners waiting to be discovered. But at some point the bug reports stop. Not because the system is finally “bug-free” but because the test can’t do anything new. At best, it has become a regression test. More likely it is an expensive set of checks that probably served a great purpose at the time, but that time is now over. It must be destroyed!
Of course, it could be irresponsible to dismantle a test suite without a succession plan. Regression tests do have their place, but that place is not within an aging integrated suite (integrated tests are a scam!). Just as a good leader delegates appropriate tasks across a team to build organizational knowledge, you can delegate regression testing responsibilities to unit tests which succinctly specify the intended behavior and fail for one reason.
The broad test did a good thing — it demonstrated a gap in your understanding. Now you can do a good thing by codifying that understanding for the future health of your code. Let the old test retire, secure in the knowledge that it served honorably for as long as necessary. You can thus reclaim the time previously spent running and maintaining it to come up with a new way to find as-yet-unknown problems and avoid the pesticide paradox.
Pingback: Long haul: two years later | WriteAsync .NET