Mutating Objects makes Testing Harder

In my course on TDD, one of the sections I’ll spend a lot of time on is how to write code so that it’s easier to test. This is critically important when using TDD or wanting to adopt Test Driven Development.

Object mutation takes many forms, but I’m going to focus on one in particular today: Using private members to retain state and returning that updated state partially.

Here’s an example (C# pseudocode):

public class Order {

  private decimal stateTax = 0.06M;
  public decimal purchasePrice = 0.0M;
  public void calculateTax(decimal purchase) {
    purchasePrice = purchase x (1.00 x stateTax);
  }
}

[Test]
public void testTaxCalculatedCorrectly() { 
  Order o = new Order();
  o.calculateTax(5.00M);
  Assert.That(o.purchasePrice, Is.EqualTo(5.30M));
}

The problem with something like this that the object’s state changes throughout the lifetime of the object, making it more difficult to test and narrow down changes. A better way to handle this (and this is something I go into a lot more depth in the course) is instead of modifying the copy you have, create a new copy, apply the changes to that new copy, and return it:

public class Order {

  public Order calculateTax(decimal purchase, decimal stateTax) {
    Order o = new Order();
    o.stateTax = stateTax;
    o.purchasePrice = purchase x (1.00 x stateTax);
    return o;
  }
}
[Test]
public void testTaxCalculatedCorrectly()
{
  Order o = new Order();
  Order output = o.calculateTax(5.00M, 0.06M);
  Assert.That(output.purchasePrice, Is.EqualTo(5.30M));
}


The performance wonks among us are going nuts, I know. Newing up a new object? Returning that? What?

The benefit here is three-fold; and these two characteristics have large ripple effects for TDD:

  • the object we passed in is not changed or mutated
  • The amount of setup we need to do to test is controllable by the test itself; which is in this case effectively a user input interface
  • We now have documentation around the properties we need to calculate tax; and we have a flex point for changing how we store tax should th eneed arise later

This is one of the principles Gary Bernhardt talks about in his talk, Boundaries, and it helps make your code more testable and easier to reason about at any given state.

I go through this and other techniques in my course on TDD for .NET Software Projects. Sign up at https://course.doubleyouproductivity.io. You’ll receive periodic updates (like this email), and you’ll get a subscriber’s only discount no one else will receive.

TDD & Agile &

I never bought into the ‘agile will save us all’ philosophy that permeates the True Believers. It’s not because I don’t believe them — I do. Operating according to the agile manifesto is a step up. It’s a big step up. Scrum provides a framework for operating with agility, and I’ve been a part of organizations that were successful and had adopted Scrum. I don’t believe they were successful because they adopted scrum, but I believe their culture allowed them to adopt scrum, and scrum helped them fix some of the blindspots in their culture. I refuse to use the term synergy out of principle (unironically), but they do feed off each other. Culture and scrum that is. I’m not sure how synergy and irony interact, but anyway.

One of the reasons I distrust the ‘agile will save us all’ philosophy is because it has to cross a giant chasm: People and Power. Particularly the command-and-control management that exists in a lot of shops. Embracing agile means pushing decisions down to the people on the team. Embracing scrum means codifying that and ensuring the team is cross-functional, self-organized, and is empowered to change everything in its purview (including whether or not they adopt scrum, which is itself, wild).

In development shops and internal IT projects, the budget and the deadline drive everything. When your budget and deadline driven, agile isn’t a good fit (if you aren’t deadline driven, it works, and if you aren’t budget driven, it works; but put both of those together and it becomes difficult to adjust), because normally projects have relatively fixed scopes. Iron Triangle? we don’t need no stinking iron triangle.

Anyway, I’ve done it again, so I’ll come back to the point: If your organization hasn’t truly embraced the tenets of agile and operates with self-organizing, cross-functional, fully empowered teams, then it’s probably because the culture isn’t ready yet. That’s too bad, because operating in tight feedback loops with constant input is a great way to develop software.

You want to know another system that gives you tight feedback loops and constant input that you’re building the right thing, but doesn’t require you changing your organization’s culture?

Test Driven Development.

You can’t control what the VP does. You can’t control the culture’s inability to start with the transparency needed to make scrum a success, but if you’re the manager of a development team or a director of software development, you can effect what your developers learn and how they develop software, and TDD is a pretty good step in the right direction of better software, faster.

I’m using the lessons I’ve learned over the years in real world projects to teach how to use TDD to deliver .NET software projects faster and better. If you’re interested in receiving updates about the course, including when it’s about to drop, sign up at https://course.doubleyourproductivity.io. As a special bonus for signing up, you’ll receive a subscriber’s only discount you won’t get anywhere else.

The Technical Debt Boxes in My Office

About four or five months ago now, I embarked on a project to re-insulate the attic and put down a platform. The house was built in the fifties and apparently attic insulation was optional; there is about 3-4 inches of insulation in our attic (it should be 2-3 times that amount, if not more). We store a lot of stuff up there, so I basically need to build a floor in my attic.

So I incurred the real-life version of technical debt. I decided to stick all of those boxes in my office (which at the conclusion was filled floor to ceiling with boxes) and there’s still half the attic the go. So I switched tactics. I wouldn’t take all the boxses out and go through them, I’d only go through half , pare down, put the floor system down on one half of the attic, move the boxes from the undone half to the done half, and then do the other half, and then put the boxes in my office back up.

Good plan, right? (It’s ok if you need to read through that a few times).

I could have purchased a POD, and just thrown everything in that, and that would have been more expensive (around 250 for pickup/dropoff + 30-90 a month), So it would have been 500-600 for the project depending on how long it took me.

But, just like the technical debt we incur in our projects, I could avoid paying that amount if I did my plan above. The only catch was I couldn’t use my office for five-six months.. Now at the time I didn’t know that, I thought it’d be 4 weeks, tops, but here we are 5 months later and I still can’t use my office, and the floor system still isn’t in place, on either side of the attic.

So what happened? Why isn’t the project done? Well, more important stuff got in the way. Like Thanksgiving, Christmas, a vacation, another project for my third daughter’s first birthday, and generally, life.

So now I’ve got a few problems. The technical debt I incurred is coming back to bite me since it’s tax season and I need to get to that computer in my office, and it’s going to be spring soon, and all of my weekends will be full (2 school age kids).

So while I saved 500 dollars by not using the POD, I’ve incurred more problems than I otherwise would have by doing the right thing in the first place (well, the right “wrong” thing — I really should have buckled down and completed the project before anything else).

This sounds like every tech debt situation I’ve ever seen (or been a part of). You think, “Eh, we can save time by taking shortcut X”, only to realize shortcut X led to a cliff that you now gotta scale, and no way to easily go back because you took Shortcut X.

Accruing Technical debt can help you go faster or omit certain necessary activities in your software project, but they’re not free. You’ll pay in some form or fashion for making those decisions, and the hope is what you’re going to pay is less than what you would otherwise.

Test Driven Development lowers the cost of choosing not to incur technical debt; and it helps you detect the problems that technical debt causes early on, before you’ve spent four months with an office full of boxes.

Want to learn test driven development and apply it to your next software project? Do you want to get around those real-world problems when adopting TDD? My course on Test Driven Development for .NET Software project teams tackles the real world difficulties teams feel when they adopt TDD, and helps you with strategies to deliver better software, faster. To sign up to receive course updates and to receive a subscriber’s only discount you won’t get anywhere else, go to course.doubleyourproductivity.io.

Test Driven Development is only a piece of the puzzle

The way I talk about TDD, you’d think it’d cure all ills, beget world peace and generally make everything better.

Eh, no.

It makes a few pains better, like if you’re working on a software project or your business is around delivering software on a deadline, it’s going to help you deliver your project on time, on budget, and give you the ability to respond to customer change, faster and with more confidence that you won’t break something.

It buys you peace of mind that you wouldn’t have otherwise. If your software team practices TDD, they will be able to make changes more rapidly and more confidently.

But it’s not an overnight transformation, and it’s only a piece of the puzzle.

The easiest piece to talk about next is your build and integration pipeline. If you don’t practice continuous integration (or at least have automated builds), you won’t be able to respond as quickly when the tests tell you something’s wrong.

If you don’t use TDD from the beginning of your project, you’ll have architectural considerations to worry about: parts of the system that weren’t built to be tested and therefore need help to be able to test them and ensure they’re doing what they’re supposed to.

Likewise, TDD doesn’t solve architecture at a system level; it makes the insides easier to change quickly, but it says nothing about your interfaces to the outside world — like that legacy billing system your system needs to integrate with.

If people are working in one-person silos and you don’t see the value of pairing to increase knowledge transfer and lessen the blindspots, then you’ll get less value out of TDD than you otherwise might have.

If programmers try to mold TDD to their thinking, rather than allow TDD to bend how they think about the problem, they won’t see as much value.

If blind-faith is put into TDD and the system is not evaluated as a whole, as a living organism that responds to external stimuli (office politics, deadlines, budgets, developer skill, designer skill, architectural skill, customer crankiness), then you won’t get as much value out of TDD (the word is “systems thinking”, but I like to think of software as an actual sentient being as it better explains some of the weirdness that happens in software).

If TDD becomes an end unto itself, if it is not pursued because it’ll help the customer, then you won’t get as much value out of it.

My point to all this is: TDD is a piece of the puzzle, and to really unlock its potential, you have to view it that way, and to see what pieces it touches and make sure they’re ready for what’s next.

I’m putting together a video-based online course that will explore these topics while teaching TDD for .NET based software project teams. I believe there’s value in a course that expands beyond the rote mechanics of TDD and embraces how it’s used in the real world to solve real problems. If that sort of thing is up your alley, sign up to receive course updates, occasional emails about TDD, a special subscriber’s only discount that no one else will receive.

That would have been *perfect*

Ever think of a witty retort an hour too late? Me too.

Ever think of the best way to handle a sticky interpersonal issue after you already handled it… suboptimally? Me too.

Ever spend three days writing a seed data SQL script and realizing it would have been perfect for TDD? Me too.

Let me back up.

In an engagement, the system and constraints were such that the configuration necessary for the software to run was stored in the database, and due to the “going fast” nature of the project, the necessary putting together of the script so that other environments (staging, production) could operate was held off until very late in the project (a scant few weeks before delivery).

This is, as they say, what it is.

However, the timing of that meant that by this point there were dozens if not hundreds of rows across tables and databases that needed to be recorded and put into production so that the application would work out of the box. It was also possible the target database already had values in it, and it’s also possible we’d be adding new ‘seed’ data as the project went on (guaranteed, in fact). So we needed a way that would handle this.

There are a few ways to handle this:

  1. Write a script that contains these values by looking through the databases and manually constructing the script.
  2. Generate a script that can run idempotently (in case some of those values are already in the target environment) by writing a program that targets the tables with these values and emits idempotent SQL. (Idempotent is a fancy word for re-entrant which is a fancy word for being able to run the same thing over and over again and getting the same result each time, without cascading side effects
    (As a simple example, i=i+1 will always increment i, no matter how many times you run it. i = 10 will always assign 10 to i, no matter how many times you run it. The latter is idempoten, the former is not).
  3. Use a program like SQLCompare to ‘diff’ the databases and have it generate the script.

There isn’t much difference between #2 and #3 except that if we write our own program, we can target tables, or target uniqueness (inside baseball: The unique values were often strings, since this was configuration data, and business domain uniqueness was not the same as database uniqueness values (Primary Keys).

With Writing our own script generation program, we can also omit auditing data, as that wouldn’t need to be captured or maintained on the seed data. A common complaint with SQL Compare is that it retained the auditing columns as well.

So I chose #2, and started off. The requirements: Write a program that generates SQL that can target specific databases and specific tables and get out the unique occurrences of configuration data and put that data into a idempotent set of inserts that won’t overwrite the data if it exists on the target table, but will fill in the gaps where it doesn’t exist.

Did I mention this problem is ripe for TDD? Did I also mention that I failed to use TDD to solve this problem?

Why? I felt like it was ‘too small’ for TDD, and that I’d be able to make quick work of it without TDD, and that it was a one-off program.

And then I got into the problem a bit deeper, and realized there were rules I had forgotten about. Like in SQL certain types of values need to be escaped, and others need to have quotes around them, and what about quotes inside of values that contain quotes? All of this made for a more difficult implementation than I was expecting.

So I did it. In the end, I pivoted languages for business reasons, but I got it done, without TDD.

And then I realized another problem I had created by not using TDD: The only documentation was whatever my variable names were and whatever I wrote as documentation.

Well named tests and examples of executing the code with expected inputs would have given a maintenance programmer a leg-up on understanding what the program did and why it did it, and unlike documentation after the fact, would have helped me go faster and document the code at the same time.

Even after using TDD for all this time, there still times I don’t use it when I should. This was one of those times. IF you want to learn how TDD can help you do all those things I listed above (and more), sign up to get details about my course: TDD for .NET Software Project Teams, which is a TDD course that helps you learn and implement TDD in real-world software projects, using the same real world constraints we have day to day. One of the perks of signing up is that you’ll receive a subscriber’s only discount that won’t be available anywhere else.