Month: February 2020

Test Driven Development (TDD) is a wonderful way to ensure your team can write better software, faster. But, much like anything else that’s easy to learn but hard to master, the “three steps” for TDD are really six, and the practices represent a fundamental shift in how you build software.

That doesn’t happen overnight, or even over a 4-day TDD course.

One of the toughest parts of TDD is realizing that when you try to apply it to an existing code-base for new features, you run into pain. Lots of pain. That pain exists because the existing software wasn’t written to be testable — it was written to work, and probably written under a deadline. Since the team is on a deadline, and Test Driven Development has ‘test’ in the name, we can de-prioritize it, right? We’ll get to that later.

And so it goes. Team gets a new project, Team doesn’t know how to develop software that is easily testable, and team tries TDD, and the first time they run into a deadline, TDD goes out the window.

It, as they say, happens.

I happen to know that TDD is very well-suited for tight deadline situations — much more so than the alternative (writing tests after the code is written, or not at all), and part of my mission in helping teams build better software, faster is to demonstrate that this is possible. But it requires changing how developers think about a problem. It requires inverting what is useful to what the business finds is useful.

You see, when faced with a new project, Developers will pick the technologies first. You’ve very likely had a ‘Sprint 0’, where developers researched databases, front-end frameworks, and technology stacks to figure out what they were going to use to solve the problem, and you’ve also probably dealt with stakeholders who see this as a necessary evil. It is an evil, but it’s not necessary when you use TDD.

TDD is meant to flesh out what a system does, not how it does it. The tech stack is the how. The business cares about the what, and they want to see that ‘what’ as quickly as they can.

The whole purpose of TDD is to figure out the what as quickly as possible, and defer decisions on the how for as long as possible, if not longer. You see, TDD gives your business the confidence that your team is writing code that solves your problem and shows demonstrable features quickly, all while defering the decision of ‘how’ until much later in the process.

This also has the ancillary benefit to the development team of punting on the decisions that can paint them into a corner until they know enough to know whether or not they need to even paint that corner.

—-

if you’re interested in an online course that teaches you and your team how to build software using TDD and avoid the problems I talk about above, sign up at course.doubleyourproductivity.io.

If you want a more personal touch, and you’re interested in a custom engagement where I help you and your team solve these problems uniquely to fit your business and culture, reach out and let’s chat.

In the intervening months between the problems of The Second Time I tried TDD and when I actually embraced it as a viable software creation methodology, I would use it for small components, or for parts of the system that had no external interfaces (think Third-party HTTP API endpoints, Database, or the UI), but I would avoid using it in cases where I needed to integrate with a third party API. The mechanics of Mocking and Stubbing (especially Mocking) felt unstable. Unhelpfully the “Mockist” TDD mindset was “if you’re having a problem mocking or stubbing, it must be you,” and the “classicists” had already cast aside mocking and stubbing.

If you’re me, you’re wondering who the heck all these people are and why their battles feel like a religious war (and programming has enough of those already). Let’s ignore them for now and move backwards *slowly* towards the car. So there I was, lamenting the fact that I couldn’t resolve the words spoken by TDD’ers (You can write your whole appliction using TDD!, or “You’re not professional if you don’t use TDD!”) and the pain I felt trying to do so. And then I had the good fortune to attend PyCon 2013 (The company I was a part of at the time had an amazing conference budget – basically you could spend 2500 a year on conference attendance, few to no questions asked) and I was able to watch Gary Bernhardt’s Boundaries talk, live.

Gary’s talk has a simple concept behind it: Push all of your core decision making code into a functional-style that works with POCOs or the language’s primitives. The code that has to talk to the outside world is your ‘shell’, and that can be written in an imperative style; with all of the modern day associations we make (like third party HTTP APIs, Database calls, and the UI).

This talk changed everything for me. Patterns that I had been trying to implement but partially failing now made sense as to why. The problem of mocks and stubs started to fade away, and I realized I could use TDD to build an entire application — or at least everything inside of the boundaries. It was then that Test Driven Development started to change for me. It was then I felt the logjam break.

I’ve been using this approach since 2013 to build applications, and I’ve learned quite a bit along the way. My course, Test Driven Development for .NET Project Software Teams shows you how to implement the “Functional Core, Imperative Shell” style in .NET applications, as well as teaches the architectural and practical issues you’ll come across and how to make it so you really can deliver better software, faster, through TDD. If you’re interested, sign up to hear about course progress and to get a special subscriber’s discount no one else will receive.

In the last email, I talked about the abject fear I felt trying Test Driven Development (TDD) as a neophyte in an organization well versed in it, and about how TDD didn’t make sense to me that it could be useful.

Imagine going from fear to that’s it?

TDD has three steps (Red, Green, Refactor) that are really six steps but we like things to appear simple so we don’t scare people off:

1. Write Test describing the smallest discrete piece of behavior you can
2. Write Code that fails that test (RED)
3. Run the test, find out it fails (hopefully for the reason you think it will fail!)
4. Write the simplest code you can to pass the test for its failure reason
5. Run the test again and witness it passing (GREEN)
6. Refactor the code you wrote (if needed) (REFACTOR)

I spent day 2 of using TDD going through this exercise, making incremental progress on my work, with my pair sitting next to me and us switching off at each test. I would write the test, they would write the code that passes the test, and vice versa.

By the middle of about Day 2, we got to the hard stuff, the boundaries stuff. The stuff that every working programmer will run into quickly but that beginning tutorials on TDD try to ignore: Databases, user input, or frameworks that weren’t written with testability in mind. How do you write a test that is supposed to run fast (on the order of milliseconds) but you have to mock out a database or the front-end web server?

It’s like accidentally passing the bunny slopes on the ski-lift and the next stop is the double black diamonds.

I wish I could tell you this story had a happy ending, but it didn’t. I put down TDD for another few weeks due to these events, and I felt like it couldn’t be useful or help me be a better programmer. You know that part in the hero’s journey story arc where the hero is at their low point? That’s where I was.

Next time I’ll talk about about what got me out of this low point, and how it can help all teams trying to build better software, faster.

—

I’m putting together a course on TDD for .NET Software teams, a course meant to help teams cross the chasm I describe above. If that’s your jam, sign up to receive details and be the first to know when the course is ready (and of course the requisite early subscriber discount you won’t be able to get any where else).

I can still remember the first time I tried Test Driven Development (TDD).

At the time, it took a lot more finagling to get Tests to work; and that didn’t help the giant, crushing, feeling of inadequacy I felt.

I was working at a great company in the DC area with a well-respected tech team, and they were rah-rah XP and Scrum; with XP practices such as Pair programming and TDD headlining their efforts.

When you go into a situation where everyone around you is used to doing a thing and you’ve never seen it before… let’s just say I was pretty scared. Today we know it as ‘imposter syndrome’ but back then I was just… scared.

And then we started using TDD. My Pair said, “Ok, make the tiniest change you can make to pass this test”. I can’t remember it exactly, but it was something along the lines of returning the correct quote type (an enumeration) based on the request, and so I started to figure out what sort of behavior it’d need in different circumstances, and wrote way more code than I needed to.

My pair politely said, “stop. Literally write the simplest code that passes this test.” and he politely wrote something like,

return 42;

I was… aghast. How the heck was this useful? We certainly wouldn’t go to production with “return 42;” in our code base, right? As I’d later find out, of course not — but TDD wasn’t the practice of writing the end result first; it is the practice of discovering and discerning the end result through small incremental steps that provide confidence with every evolution.

I went home that night unsure if I’d ever use TDD again. It didn’t seem like it accomplished much, and it seemed too simple to help make better software.

Next time I’ll talk about Day 2: The Tests Strike Back

Someone on the internet mentioned that if your organization values test, it doesn’t matter if you write them before the code is written or after the code is written.

I respectfully disagree.

Test Driven Development is a discipline I’ve talked about before. Briefly, you write the test first; only write enough code to pass that single test; and then you refactor the code to clear out any duplication moving forward (I particularly like the Rule of Three in deciding whether to refactor).

Unit Testing is somewhat the opposite. You have the code already written, and someone, be it yourself or a pointy-haired boss, says, “That should be under test!” So you write the test(s) that cover the code’s responsibilities, and you call it a day.

Non-obviously, they have different — and sometimes opposite effects on your team and codebase.

Unit tests incur pain to write: Systems that are created without testability in mind are harder to write tests for than systems that are written to be tested. This is a truism, and generally accurate.

TDD requires discipline to write: Ensuring each piece of behavior is specified in a test first is an uncommon way for a builder to think about building.

Unit tests are brittle, since they are written around the scar tissue of the system, they’re affected by structural changes in the system. TDD style development is not.

TDD requires a change to how you think about designing software. Unit tests do not.

I could go on and on, but these attributes affect the entire system and the people who depend on it; and are not merely cosmetic choices.

Test Driven Development (TDD) is a method of development where the test for each piece of functionality is written before the functionality is written; and then iteratively the functionality is fleshed out only within the limits of the tests written for it.

Put another way: No test, no feature, and no feature beyond the tests written.

There are deeper mechanics behind that idea; (mechanics commonly referred to as red, green, refactor).But, that’s the gist of it.

Test Driven Development helps with the following problems:

• Murky problem space
• Ensuring architecture is easy to change when the business needs new functionality not considered at initial design time
• confidence for your development team in making future changes without regression bugs
• developing software more accurately and quickly

And a non-reason to adopt TDD, but often stated: 100% test code coverage

Most times teams try to bolt on TDD and get through problem #1 and #2, and never make it over the hump for problem #3, but the long term benefits are #4 and #5. Imagine if a year from now you could ask your team to add new functionality and not have to worry about whether or not there is a subtle bug that destroys your customer’s confidence in your work. That’s pretty powerful.

But, TDD isn’t useful in all contexts, for all teams, for all reasons. Teams that have a slow rate of requirements change or feature change (averaging less than 1 feature per person per unit of release time) won’t benefit as much as teams that have a higher rate of change. Teams that don’t have the infrastructure to support Continuous Improvement (Automated builds) won’t benefit until they have CI in place.

TDD isn’t a short term win for productivity or speed of development; it takes several iterations of concerted practice to get even small teams to the point where it changes how they think of developing software in a way that enables better TDD practices; but if teams stick with it, and have guided help during the rough times with TDD, the medium and long term benefits to the business are worth it.

TDD isn’t a short term productivity hack, it’s a good way to reduce cost of change, cost of unplanned development work, and the cost of a lack of confidence in making changes to your system when the customer wants something new or different.

There’s an interesting question over on the Project Management Stack Exchange site, that asks:

A sprint has just ended. All stories, but one were done by the software engineers. Should the sprint be labeled as ‘failed’ because of the story that was not completed? What exactly is a failed sprint?

My answer:

I am personally adverse to the idea of ever saying a sprint ‘failed’.

Scrum uses the term ‘inspect’ 27 times, and ‘adapt’ 16 times over the course of the guide.

Scrum also has no notion of ‘failure’, and the only reference to failure in Scrum is listed here:

Failure to include any of these events results in reduced transparency and is a lost opportunity to inspect and adapt.

Scrum’s purpose isn’t to give you confidence that you will deliver every single story you create, in the sprint they were assigned to. That’s a management artifact that we assign to scrum because we’re human and we like to believe we’re in control.

In fact, in the agile manifesto, they say something very key to determining ‘success’ or ‘failure’:

We are uncovering better ways of developing software by doing it and helping others do it

That doesn’t sound like the manifesto of someone that has all the answers. We are uncovering better ways of developing software — notice the present tense. Software development is an emerging field where hidden constraints of people and machines are uncovered every day. It’s going to happen to you and your team, and it happening — and thus putting a story in jeopardy of being delivered — isn’t a failure, it’s natural.

Scrum’s purpose is to give you short iterations to try to deliver working software in that will be used by the customer, and then adapt to changing circumstances as they come up. “Failure” in the eyes of scrum would be to not change when the circumstances change, to not ‘inspect and adapt’, as it were.

In fact, Scrum doesn’t use the term ‘commitment’ any longer in the Scrum guide — they’ve changed it to forecast, for the exact reason I list above. We may get upset at the weatherperson when they get the forecast wrong, but we still have to have our umbrellas ready if they are wrong.

That’s Scrum. Having your umbrella ready — not firing the weatherperson because they got the forecast wrong.

If your team has inspected and adapted; if they’ve responded to changing circumstances, if they have adapted — then they haven’t failed. They’ve done precisely what agile methodologies and what Scrum dictates.

Developers love to abstract things. Creating an abstraction is a lot more fun than writing glue code.

Developers write a lot of glue code, and get tired of it easily. Glue code is kinda like that email you copy and paste, and just change some details of for each person you send it to.

Would you get tired if all you did all day was join two lego bricks together? I would.

But, modern programming is a lot of joining lego bricks. Plug this piece into that, find another piece if it doesn’t fit, repeat.

What if you could create a lego block joiner? Then all you’d need to do is make it so that lego brick joiner could join any two lego bricks, and then you could spend your days on the beach sipping mai tais.

That’s the allure of an abstraction. Joining the bricks isn’t fun, but creating something that’s able to join legos is.

If you have five lego bricks to join together, you may not want to create a lego-brick joiner. But, programmers may only be joining five lego bricks right now, but they’re thinking of those potentially tens or hundreds of lego bricks they might need to join in the future. Next quarter, next year, you know, the future.

Does your business need a lego-brick joiner? The programmer will tend towards yes, even if it may not, because the joiner is a lot more fun than the idea of joining lego bricks for the rest of their days.

When you’re in your next standup, or sprint planning, or team meeting, and a programmer says they want to ‘abstract’ something, or ‘create an abstraction layer’, think of it as an investment. If you don’t need a lego-brick joiner, or you’re only doing that thing a dozen times, maybe you don’t need one.

Even if they are pretty cool.

When we say “Zero Code”, we mean software that allows you to accomplish your unique business process or goals through creating your own custom application without writing any code. Microsoft Access is perhaps the oldest modern example of this. Excel is considered a modern day data-store for the ‘no code application’ movement, and even to this definition be considered zero-code, as long as you don’t think of excel formulas as coding. It’s also highly likely several parts of the business you support run on excel.

The “Zero Code” I mean here is related to the ‘no code’ movement; but it’s more on that promise that we’ve had since time immemorial that somehow complex software applications could be created where their users could do complex things without having to understand a scripting language, DSL, or other programming languages.

If you’re an enterprise software team delivery manager, chances are your business champion is going to start asking you (if they haven’t already) if your team can make this application ‘zero-code’ or ‘code-less’ or ‘drag and drop’.

My heart started beating faster just writing that sentence. I’ve been there, I’ve inherited the scars from the attempts — across teams of all skill levels and in some major business domains.

Put another way, zero-code is both the holy-grail and the tarpit of software development. The allure is unmistakable: Who wouldn’t want to be able to change the application when the business needs change — without code? Software Developers are expensive, they take longer than the business needs, and their solutions are often buggy — imagine if you could do away with that as a business champion. Wouldn’t you?

In the movie The Matrix, Morpheus (Laurence Fishbourne) describes the matrix as a world with rules; some can be bent, others can be broken. That is software, no doubt — but Zero-Code software tries to rewrite the rules of the world and provide mechanisms to rewrite the rules of the world while it’s in use. Put another way it’s akin to trying to switch out the seats in your car while you’re driving it at 65MPH down the highway. Getting Zero-code right requires a lot of scaffolding, planning, and development, and it’s still inadequate for most businesses.

If I haven’t been plain enough: You don’t need Zero Code, and you’ll end up jeopardizing ever delivering by pursuing zero-code at this point in time (and for the foreseeable future). I do believe ‘zero-code’ is possible in the next 20 years; but it requires a method of thinking and solving problems that we still haven’t bridged with our needs for data integrity and consistency.

Instead of Zero-code, focus your developers on solving the business problem as simply as possible, and ensuring the business domain is accurately reflected in the software. Zero-code is alluring for developers as well, and we aren’t always in a good place to combat against it.

Your business does change — but it probably doesn’t change so often as to necessitate a 5-10x increase in development time to provide a zero-code implementation.

This blog post and metaphor brought to you by that stakeholder I had on a project who was convinced that software should be ‘configurable’ without code changes and that such configuration was as simple as allowing the steering wheel on a car to be adjustable.