If you’re a fan of 90s kids shows, you probably remember the Animaniacs. To say it was a kids show is probably a mis-classification because its jokes had layers; and a fair amount of political humor subtly placed strategically throughout. Anyway, one of the the recurring segments was “Good Idea, Bad Idea”. The one I remember went something like this:
Good idea: Playing Catch with Grandpa (picture two kids throwing a ball with grandpa)
Bad Idea: Playing catch with Grandpa (picture two kids tossing Grandpa back and forth).
The subtlety in the joke comes from taking the same sentence and interpreting it two vastly different ways. We see the same thing with Test Driven Development and its somewhat malignant little cousin: unit testing.
TDD practitioners will often have codebases that achieve close to 100% test coverage.
Developers that write unit tests will try to achieve 100% code coverage; believing the code coverage number is why software is able to be delivered faster and with fewer defects.
But, much like tossing grandpa, it’s a horrendously bad idea to strive for 100% test coverage and an even worse idea to try to do so through unit tests (as opposed to TDD).
One reason for this is that better than 90% of the libraries out there were not developed with Test Driven Development. They may have unit tests, they may have certain sections that were developed through TDD, but it did not become a part of the core identity of the system.
Because of this, you need tortured hacks to test certain libraries, and these tests themselves are a smell that’s something off.
In the above linked example; the poster does something that I’ve done and seen other developers advocate for in the past — it’s a mistake we all make at least once, but it’s a mistake that hurts our ability to deliver better software, faster. The mistake is to try to bend a non-testable class to the idea that everything should be unit tested, and therefore the hacks are necessary to meet that goal.
here’s a few ways the above test and code could go wrong:
- It doesn’t actually test what the OP wants, which is some guarantee that the third party code will do the right thing under certain conditions.
- Rather, it tests the structure of the code and is brittle to library changes.
- The OP wants to test, in isolation, the idea of broadcasting push notifications; but that is an activity that is meant to be E2E tested. No one cares that a push notification was sent, they care that it was received. Testing sending is like checking that your backups work without actually trying to restore the data. It’s the restoration that’s the important part, not the backing up.
In this case, I would have forgone the unit test entirely; and ensured my code was created through Test Driven Development; and would have inserted scar tissue between my code and this library; to ensure the system remains resilient, even if the structure of the library changes. I would have set up dummy accounts for E2E or integration tests (depending on which was more valuable in the medium term), and solely tested this library through black-box testing like an end to end or an integration test.
I’m beginning to open up availability for the spring for training and advising .NET software project teams on how to adopt Test Driven Development. If your team is going to be starting a new project in the next year, hit reply and drop me a line and let’s chat to see if an in-person course on Test Driven Development is right for your team.
My course on Test Driven Development for .NET software project teams goes deeper into this approach and is meant to capture these same sorts of real-world problems developers run into when adopting Test Driven Development. To sign up to receive course updates, and to get a subscriber’s only discount that isn’t available any other way, go to course.doubleyourproductivity.io and fill out the subscription form.