“Build One to Throw Away (You will anyway)”

When I read that line from The Pragmatic Programmer in the 2000s, it shocked me.

What do you mean throw software away? I just got it working!!

20 years after its initial publication, we still are adverse to throwing code away.

Why?

Some examples of this:

  • Your team uses a GenericRepository<T> that somewhat works. It works for your entities that have the same ways to get them (GetById, GetAll, Find), but it fails for when you need child discrete entities (think of getting all orders in the system for a report; or all orders that contain a certain product) without getting their parent entity (in this case, the “Customer”), and then you have this weird thing where you try to add another method to the GenericRepository to work around this because we’ve invested in the GenericRepository<T> pattern. (This is a real example I’ve encountered multiple places)
  • A developer comes up with a new way of doing something in your system that contravenes your established convention, but is more readable and maintainable. People on the team who are afraid of change start to bring up lots of reasons why you can’t possibly do it that way, and that this change should be ‘researched more’ (note: If you’re asking someone to ‘do more research’ without a specific deliverable and a specific question you want answered, you’re really just softly scuttling the idea. Also, asking someone to ‘do more research’ when they did the research and your question doesn’t have the answer you want is also scuttling the idea).
  • About six months into your project, you realize that a relational database wasn’t the best choice of data store for your project. Or that ArangoDB is just too niche to get support for, and instead of saying “We need to stop now, our foundation was for a house and we’re building an office building”, you plod on, introducing ever more complicated caching and retrieval mechanisms to work around the database.


I could go on. Over my career, I’ve heard and experienced stories of this that all come back to the same general idea:

Once it works, we don’t want to change it.

or put another way:

We get invested in how we solve problems, and don’t want to learn new things.

All of this is normal, and it the prime reason why we don’t change precisely when we need to. There is inertia to the old ways of thinking and doing business.

But remember the second part of the quote: You will [throw it away] anyway.

I suppose on a long enough timescale that’s a tautology, so I’ll focus on the short term.

What modules have you rewritten over the past year? Why did you rewrite them?

What changes did you make that your architecture hadn’t accounted for? What did you do when you found this out?

There’s a saying You’ll never know less about your problem than you do right now.

Tomorrow you’ll know more than today, and so on.

If your hesitance to throw away what you built is that you’ll lose work; you’re right. You’ll lose work created with less understanding than you have right now.

You’ll lose work created (sometimes) on false premises.

The upside to throwing away work is that it gives you room to correct those false premises.

The upside to using TDD is that while you may throw away the work, you’re generally not throwing away the customer’s view into your world; allowing you to correct your thinking and verifying it’s correct without disrupting your customer.

That’s very powerful, and a powerful reason to adopt TDD. You’re going to throw away your work, would you rather throw it away and replace it with confidence? Or without confidence?

An example of a non-brittle test

For the TDD course, I’ve been iterating through the course material; implementing it as a I go through each lesson.

In the most recent example, I wanted to show how a test could go from failing to passing, without changing any test code; and without changing the API.

Now, this is normal in TDD. The API is what the end consumer of your process will see (whether that’s another process or an external consumer).

For instance; here is the API for my budget right now:

Budget b = new Budget("name", startDate: new DateTime(2020,05,01));

the API for my budget takes in two things for its creation:

1. A name (how will you refer to this?)
2. A date the budget should start

The Budget exposes two operations:

b.Add(new BudgetItem(/*...*/));
b.CumulativeSpent(effectiveDate: new DateTime(2020,07,01).Date));

1. Adding a budgetItem to the budget with a .Add() method
2. Telling me how much as been spent as of a certain date (The “effectiveDate” in this example).


Now, here’s the test, and the test didn’t change from when it was failing (and it has been failing for a few days, I ignored it because I got ahead of myself in writing it) until when it was passing:

[Test]
public void BudgetShouldAccountForTotalsAcrossMonths()
{
  BudgetItem i = new BudgetItem("b1", 1.23M, new DateTime(2020, 05, 01).Date, new OncePerMonth(), new DateTime(2020, 05, 01).Date);
  Budget b = new Budget("Budget That Shows across Months", new DateTime(2020,1,1).Date);
  b.Add(i);
  var totalSpent = b.CumulativeSpent(new DateTime(2020, 07, 01).Date);
  Assert.That(totalSpent, Is.EqualTo(3.69M));
}

Ignore the BudgetItem’s API for now. It’s hideous and going to be refleshed out (that’s where the next LiveStream will focus on, using what I’ve learned about the domain and the APIs I’ve created to create better APIs)

So how did I make the test pass?

By making a one line change inside the .CumulativeSpent() call:

Importantly, and the takeaway from this email, this is how change happens. Not by affecting what the consumer of your code does to execute your code — we don’t want to change tests. If we have to change tests when we change code it’s a sign our tests are too coupled to how the code does its job. We don’t care if there’s a mechanical turk manually adding up each budgetItem, all we care is that it gives us the cumulative spent.

In short, this change had me like:

P.S. If you aren’t already a member of the course list, and watching the livestream (or getting more updates on the course, like this one) interests you, add your email to the list at https://course.doubleyourproductivity.io. I’ll send out an email to that list when I’m about to start the livestream (it’ll be either this weekend or early next week, since my wife is still working on finishing grad school in the evenings).

When did On-Base Percentage Come About?

If you’ve watched (or read) Moneyball, then you know that On-Base Percentage (OBP) in Baseball was a second tier stat for the a long time until the advent of Sabermetrics being used more widely as a way to gain advantages as a team by relying on undervalued players instead of superstars.

Wikipedia says OBP became an official statistic in 1984, but when was it created?

August 2, 1954.

30 years earlier!

Life Magazine published an article “Goodby [sic] to Some Old Baseball Ideas” written by Branch Rickey included an equation for On-Base percentage. You can read that article here. (In the article, OBP is called “On Base Average”).

Here’s what the calculation for OBP/OBA looks like:

Why did it take 30 years for an objectively better way to measure getting on base to become ‘official’?

Why did it take another 20 for it to form the basis of Sabermetrics and take the baseball world by storm?

Old habits die hard.

Can you imagine how the course of history for some teams would have changed if they had paid closer attention to something that was sitting under their noses for 50+ years? The baseball world would look completely different.

But, this isn’t a baseball blog. It’s a blog about how to double your productivity, focused on strategies for software teams to produce better software, sooner.

TDD has been around for 20+ years. How many teams have you been on that have practiced TDD? For me, the answer is one. Out of all the teams I’ve been on across all the industries and sized companies, I’ve only been on one team that practiced TDD before I arrived.

I’ve been with seven different companies in my career and twelve or thirteen teams, and there was one that practiced TDD before I arrived (incidentally, it’s also the same team where I first saw the value of TDD).

TDD and OBP share a lot in common as measures. They measure what is objectively important; and they are relatively simple measures. They focus on an outcome-based approach to measurement, instead of a heuristic based approach (unit tests and static analysis are heuristics), and there’s a binary result: Either your code is easy to change with confidence or it isn’t.

Either you got on base or you didn’t.

Now that doesn’t mean getting on base is easy, or that making your code easy to change is easy; but the outcome is essential to software teams: Being able to make changes with ease and confidence. Being able to deliver on time, without regression bugs, and without fear.

Can you imagine how different the software world would be if all teams could do that, right now?

Special Thanks to Ben Orlin and his book: “Math with Bad Drawings” where I found out this tidbit about OBP (Pages 222-226).

What to do instead of writing unit tests (5/5)

At this point, if you’re still with me after four posts on the subject, you may wonder:

How the hell does any change ever happen?

How does any codebase ever get better?

That’s a fair question, and if we’re all putting on our big boy sweatpants and being frank with each other; it’s really hard.

Why would I do all of this ?

Why would I:

  • Figure out the part of the codebase that life would be great if I could get past these unit tests that are hard to write and have questionable value
  • Figure out that part’s cyclomatic complexity
  • Figure out whether its value is visible or invisible to business stakeholders
  • Try to sell changing that module to the business

Today, we’re going to talk about the final part, putting everything together.

At this point, you are ready to dive into the code and make changes.

I’d recommend, if this is the path you’re intent to go down, to write characterization tests to fully understand what the bits are doing. That is an entire series unto itself; so I won’t cover it here, but the idea is to write a test against the outermost public API for the thing you’re touching (warts and all) and have that series of tests help you understand what’s at play when you make a change.

You should also write contract tests that ensure the shape of data you’re sending to the database or to the user doesn’t change.

And then you dive in. The book Refactoring by Martin Fowler is useful here; as it details step by step what to do in a given situation. You start to put parts under test and factor out changes, and you do it, again and again.

But those are the technical steps that come at the long end of the steps I mentioned above; and those steps I mentioned above are important.

For what? Why would I do all that?

Well, to be blunt: If you’re not developing software through TDD, this is the road you’re on for the rest of your career.
You’re on a never-ending cycle of pain in your codebase, using metrics like cyclomatic complexity to triage issues (as well as whether it visible or invisible, and who it affects), and trying to sell change to your business.

That’s going to be your life.

Inertia doesn’t just apply to physics; it also applies to making change, especially change to codebases that are working and in production.

The appetite for something breaking is very low, and the unforseen benefits of refactoring a module are too hard to quantify to get over that hurdle.

In short, if you work at a median company doing median work, the deck is stacked against you.

It is a much larger subject than one post can do justice to (or even 100 posts), but the art of selling the need to change is crucial to being able to improve your codebase if you make tests an afterthought.

The short question is: Do you want the pain associated with writing tests after? Do you want to go through these steps for the rest of your career?

If not, you have two choices: Adopt TDD, or ignore unit tests entirely.

Selling the Change

If you work on a self-organized empowered agile team (Scrum or otherwise) and therefore you don’t need to sell change, then you can skip this post. You’re already in the place where you are empowered to fix problems in the code base (your white whale, that we’ve been spending the last four posts talking about).

How do you know if you’re in a self-organized empowered agile team?

If this white whale we’ve been tracking is on your backlog (and it should be), then talk to the Product Owner and team about moving it up into the next sprint.

If there’s actual trust and agency there, then you’ll be able to work on it, just on saying, “This is causing us pain and we need to fix it.”

If that doesn’t work for you, then I am sorry to be the one to tell you this, but you’re probably not on a self-organized empowered agile team; even if your organization uses one or more of those words mashed together.

That’s ok, because you can still sell change, even if you aren’t empowered to make change.

So, you’ve found this part of the codebase that is your white whale, you’ve used metrics to determine how bad it really is objectively; and you’ve ascertained whether it’s a module that affects the customer, or just the health of the system (“just”, as if that’s any less important), and you want to dive in and fix it. There’s just one teensy, tiny thing you have to do before that.

You’ve got to sell that change.

In a business that understands the value of being able to move nimbly, you wouldn’t have to sell very hard, if at all; but unfortunately not all businesses value the expertise software developers bring to the table. There is a if it works, ship it mentality that causes long-term problems for software projects.

What is the outcome your business will see if they let you spend x time on this?

Will they be able to see features and changes sooner?

Will this change open up new features the business/customer has been wanting, but unable to receive due to architectural limitations?

Will the system be faster?

Will developer time maintaining that part of the system (or the system in general) go down due to this change?

Will the team feel better when this part of the system is better? Will it improve morale?

It’s important to speak in terms that whomever you’re selling this change to cares about.

For instance, for business people, there are five major reasons to make change:

1. Increase Revenue
2. Lower Costs
3. acquire new customers
4. Retain Customers
5. Expand into new Markets

Which one (or more) of those five will your fixes enable? How will it enable them?

It’s also important to understand the person or people you’re selling this change to. They’re thinking about these things; but more importantly, they have a set of goals for this quarter that are on their mind. What are those goals? If you haven’t talked to them about those goals, you’ll want to, and you’ll want to see if your changes help further those goals.

You’re not trying to sell just anyone on your change — you want to sell a specific person. In your organization, you know who this is, whether they’re a shadow leader or the titled leader. They’re the one you need to sell your change on. There’s a saying in Basketball Play the man, not the ball. Basically that means that the mechanics of basketball only get you so far; but to actually win, you have to know who you’re playing, and how they play.

This is true in every aspect of your life. This information shouldn’t be used for a zero-sum win; but rather finding a way to win that lifts everyone up. To do that, you have to understand what drives the person you’re talking to and trying to sell the change on.

Now, there’s no way for me to do justice to the art of selling in an email or blog post; literal volumes have been written on the subject; but the point of saying this is that even for software developers: we must sell our changes to other humans in order to make things better.

We as developers like to think of software development as a purely technical task; but that’s not the case. Of course in an ideal world we’d be able to pursue positive change in our codebase because we need it as developers; but as long as the world isn’t ideal and you’re not in a self-organized empowered agile team, you’ll have to sell this change to someone.

Next time we’ll talk about what to do once you’ve sold the change.