Moving Quickly vs. Moving Nimbly

There are adherents to TDD who believe that the test is the document for the change.

There are also people who believe that valuable software happens if you just focus on the code.

I am not saying those overlap in a perfect circle, but you should draw your own conclusions.

For the rest of us, we want to document changes for others, without becoming a warehouse of documentation that no one reads.

There are many sources of documentation, and each has its purpose.

There’s PR/MR (Pull/Merge Request) documentation, that documents why you made the changes in your branch, what reviewers and testers should be concerned about, and areas you believe need extra attention.

There’s the documentation you put into a user story / ticket; where you want to capture the relevant details of why you’re doing this story; the ins and outs with other parts of the system; the high level details that this story encompasses; and the desired end state.

There’s the documentation you put into code: the code comments where you explain why you did what you did in an area where you couldn’t use proper naming to disclose your thinking. There’s also the tests you write, whether they’re characterization tests to detail how the component works and why it works that way, or the contract test that details what your APIs expect and how they work, or a end-to-end test that details the path the user takes through the system, that ties a user action to the actions the system takes to carry out that user request. There are also development driven by tests; that detail the particular ranges and expectations for each line of code.

But, none of these documents explain why a system is changing to a particular implementation, or new technology, or changing from an event-sourced system to a transactional system, or why angular was chosen over react (or vice versa), or how that affects the business, or how the business affects system changes.

Think about all the factors that go into any large change in a system:

  • Political context (there’s a possible re-org, and you want to show value before that happens so you don’t get re-organized right out the door)
  • Team context (the team doesn’t really understand the Reactive programming pattern, and so implementing RxJS or NGRX will take too much time, or the people on the team just don’t like to write unit tests)
  • Financial context (we have three weeks budgeted to fixing this part of the system)
  • Business context (there’s a sales deadline in two months, so whatever we have has to be done in one month so sales can meet their quotas with this new feature)
  • Technical context (We can’t upgrade to .NET Core because of legacy library X, so we have to remove/replace that library before we upgrade to .NET core, and that upgrade requires a new license agreement that has to go through procurement)
  • Architectural context (Our system is required to produce reports within 5 minutes of a transaction happening, so any solution needs to fit in to that architectural standard)

They may be uncomfortable to capture, but you need to capture them, because in 6 months you’re not going to remember why you made the decision you did, or all the ins-and-outs that caused that decision, but you need to, because if you don’t you might reverse course in 6 months only to lose both ground and faith from your business stakeholders.

You capture these decisions in what is known Architectural Decision Record; a document that contains all of those contexts I listed above; and for each record, you should fill out each section, even if it feels uncomfortable to do so. If you say, “well, this section is going to be blank so I’ll skip it”, then think hard about that context before skipping it. There’s always a business factor at play, or a social factor at play; it just may be uncomfortable to talk about; or it may be “because we want to maintain the status quo politically”, but it’s there.

They are also good to have listed because they give you areas to ask stakeholders about. Asking your boss, what political considerations are involved? is a necessary (albeit uncomfortable) question, because you want to do the right thing that gets the right result; and politics is how two or more people make decisions. Asking your manager about the financial considerations is just as crucial; because if you don’t ask, you’ll get surprised when you only have three weeks when you thought you had six (BTDT).

Besides those contexts that are each of their own sections, there are also other sections for the ADR:

1. Change desired
2. Reason for Change
3. Contexts (from above)
4. Approach A
– Pros
– Cons
– impacts on those contexts and considerations
5. Approach B
– Pros
-Cons
– Impacts

6. Approach C
– Pros
– Cons
– Impacts
7. Approaches that didn’t make the top 3 but were discussed (you can bullet these, don’t need to go into detail)
8. Decision
9. Follow-up date on this decision and the follow-up recorded impact

This is the template I use; there are others, and you should mix and match your ADRs to fit your business. The most crucial part I’ve found is the actual context around the decision, because those are meant to capture the aura around a potential change; and that’s usually the first casualty of memory.

I pick three approaches because there’s research that says you should offer choice, but not too much; and if you can’t find yourself a third choice, remember: doing nothing is also an option.

These documents sometimes take weeks to fill out; that’s OK. The important part is that they take as long or as short as needed to capture the decision and receive alignment from others as to why the decision is being made. That will vary on the team and the change needed, but is one way to align the team to move nimbly, not just quickly.

The next Right thing

How many times have you been in a meeting where someone on the team says, “And we should make this configurable; just in case we need it”, or “this will scale better if we do x“.

Yea, that about sums it up.

As much as we hate to admit it, we can’t predict the future. Outside of the highly trained epidemiologists, the wider world didn’t anticipate that COVID-19 would have the effect its had. Almost none of us would have predicted we’d be at home for at least two weeks, or that schools would be shut down for the rest of the year, or that toilet paper would be the hottest commodity on the planet.

Let’s say you had some inside knowledge in January and your epidemiologist friend had told you that COVID-19 would be bad, and you trusted that person. Would you have stocked up on toilet paper? Would you have told your teachers and your schools that they needed to make sure their distance learning plans were set?

Probably not — even if we can see the first order effects well, it’s those second and third-order effects that get us, every time.

The mistake we make in building software is that we try to predict those second and third-order effects. We try to anticipate where events are heading, or the effect that each feature will have, and we suck at it.

Slack is currently rolling out their new UI. This is a platform that’s seen a skyrocketing increase in usage due to the Stay-at-home orders; and they’re also in the middle of a (prior planned) UI change. My experience tells me this is a bad idea and will result in lots of pain and an ultimate partial or full reversal, but should you want to believe me, see my last paragraph.

As I said, it’s easy for us to make predictions, but it’s hard to see the future.

So why do we do it?

For one thing, it feels good to feel like you ‘got it right’. It’s proof that you are smart. Now it’s utterly silly for us to hang our hats on this notion, but we do it, all the time, with all the attendant pains it brings.

Contrast this with building software through Test Driven Development, where you never try to predict the future. You just do the next right thing, and the next right thing, and so on, until you have gotten to the point where you can attempt larger changes, because you know there are tests protecting you from making an oops.

It’s a completely different way of thinking about software; and while it doesn’t forgo designing up front; it does minimize the dangers of trying to future proof before you have a clear understanding of what that means, and what the effects are.

In your next design meeting, even if you aren’t practicing TDD, try this out. Try to focus on the next right thing. Or as Einstein once said, “I never think of the future. It comes soon enough.”

It’s hard to focus right now..

It’s really hard to focus right now.

In addition to trying to help others adopt TDD, I also work with clients to produce software for them. I’m notionally supposed to be writing features for that client right now; but much like everyone else, I’m running at about 20% efficiency.

This culture and this team I am working with has never adopted TDD, and few people on the team understand unit tests.

This is normal for our industry, and from experience, I know it’s contributing to the lack of concentration for me and for others.

With the kids at home, we (my spouse and I) get interrupted often, and that leaves us pockets of work time, and if you’re a developer, you know how hard it is to stay in ‘the zone’. I refer to that ‘zone’ in How to Destroy Programmer Productivityand if you could use a little baseball bat humor right now, I hope that link helps.

For the course I’m building, it’s easier to jump back into that every night than it is my daytime client work. The biggest reason is that because I have a failing test awaiting me when I go back to the course, I know exactly where I’m at. I don’t have to spend any time wondering what to do next; the failing test tells me what to do.

I only have to concentrate on the next right thing (sorry, Anna).

Contrast that with client work, where Unit tests and TDD approaches to building software vary among the team and culture. There, it’s harder to jump back into it. I don’t have an easily testable architecture and infrastructure waiting for me. I have my notes, and I have comments in the code; but I don’t have that part I rely on — the tests, which are especially useful during times like these. It’s a small thing, leaving a failed test to come back to, but when your attention is split among so many things, it’s extremely helpful to have.

How are you handling this? Hit reply and let me know. I’d love to hear from you. All we have is each other (especially since in most places — like where I live — we can’t even hang out with our IRL friends).

Intro to Hexagonal Architecture (video)

I’ve been setting up my home office to shoot the videos for my forthcoming course: Test Driven Development for .NET Project Teams. Besides writing scripts and assembling materials, I decided to shoot a short video to see the state of the set-up. Instead of something that couldn’t be useful to you (like me randomly talking for 15 minutes) I put together an intro to Hexagonal architecture (aka ports and adapters), which is a necessary component to ensuring you can build an application through Test Driven Development.

You can watch the video here: https://www.youtube.com/watch?v=SRSbwW19B6o&feature=youtu.be (At some point I might publish that link; but right now it’s a bit of a screen test, and may be re-shot later).

I would love your feedback on this, from small things like video aspect ratio, to the bigger things (like a concept was explained unclearly), and what you liked as well (and why!). Leave a comment below. I read every piece of feedback.

end note: This video was stream of consciousness (the slides were added after the fact to smooth out some of the rougher parts), so keep that in mind.

It’s a hack… but it works.

Last week I needed to finish the attic — I was putting down more insulation and creating a raised platform so we could store our many, many, possessions in the attic. I say this with a sense of ennui, as I feel like adding more space to store things encourages storing more things (kinda like expanding highways causes more traffic). But anyway, since my house was built in the 50s, there are quite a few gotchas in putting this platform down. Things like dealing with rafters that have faux headers (used to ensure the structure didn’t ‘shift’ during construction before the roof was put on), or random wooden beams that hang off into space.

So, at one point, it was either put a corner into a void or spend another hour working on the attic.

So I chose the hack.

The hack was that since this part of the attic would (at most) hold one box, and never a human, I could get away with letting the stud hang off the end of the ceiling beam.

This was a hack, and I can only imagine what someone who sees this in 40 years says.

I can’t really leave a note for them; so I guess they’ll think the previous owner was an unqualified hack.

They’d probably be right; but they’re also forgetting the context I was operating in — finishing the attic during a global pandemic so I could have my office back.

The good news is that with TDD and unit tests, you can explain your hacks to people who come after you. Comments work, though they often lie because they can’t be guaranteed to stay in sync with the code. That’s their chief problem. They lie.

The cool thing about your unit tests (and TDD is even better at this) is that they have to stay in sync with what the code does, otherwise they’d break. They can’t lie (though if poorly named or shaped, they can mislead).

The next time you need to make a hack in code; think about producing a characterization test that explains why the code does what it does; and indicates in what circumstances the code (and its tests) would be safe to remove. This may be a more brittle test; but given that it’s documenting a hack, that’s probably a good thing.

(If you’ve read the last post titled “Chesterton’s fence”, you’ll probably notice that this post is a corollary to that one. I’d like to think it’s worth your time if you haven’t already read it).

Chesterton’s Fence

Chesteron’s fence is a principle that, in essence, states the following:

Never take down a fence until you understand why it was put up in the first place.

Apologies to G.K. Chesterton, who put it so much more eloquently in The Thing (the book, not the movie).

This principle manifests itself in our codebases all the time.

Ever try to delete code you thought was unused? Ever bring down production doing so? I have.

Ever refactor code without tests around it (or even with characterization tests around it) and realize later that you missed a code path? I have.

Ever change a setting in a database, only to realize the system depends on that setting, even though it was marked as unused? I have.

All of these are Chesterton’s fence examples. Put simply, it’s really easy, and almost certain, for us to not understand the system once it’s released into the wild. It grows on its own, adapting to changing circumstances, and a lot of times those circumstances make the code look sort of like this:

Any time a new developer comes into the system, they may be tempted to remove that gate. After all, it’s not doing anything, right?

Well… How do you know?

The trouble is, you don’t and you can’t know with certainty, not without Test Driven Development. Delete some code and if all the tests still pass, the code wasn’t needed.*

Funny enough if you use TDD, you’ll be regularly deleting code enough to not have to worry about Chesterton’s fence.

There is immense value in this approach for your team. When production bugs are as easy as deleting a required setting that people thought wasn’t being used, it’s critical to ensure you have a style of development that keep these freak, costly accidents from happening in your code base. Yes, your code may just be an internal financial dashboard for an insurance company; but if that dashboard goes down, can your insurance agents still sell insurance?

*there are a lot of ins, outs, and what-have-yous with this (like using a FauxO style of TDD so you don’t have to necessarily worry about lots of integration test paths; but still having integration tests to cover the boundaries, like network IO, disk IO, and the user interface)

P.S. I offer virtual team TDD immersion training for .NET teams. During this time when your team may be experiencing a lull due to global events, think about using this training as a way to keep your teams sharp and ensure you can deliver software reliably and consistently to your stakeholders. Learn more at https://www.doubleyourproductivity.html/paid.html

Abstractions Carry a Cost

Abstractions have a cost. Your project, team, and you will carry the cost of that abstraction for its lifetime.

We don’t like to say that, because as developers we love abstractions, but every single abstraction carries a cost, and that cost affects everyone that develops, tests, or uses the application.

Most of the time, we believe (or it is demonstrable that) these abstractions provide more value than they cost.

Sometimes we do this without any evidence that that’s the case.

I was reviewing a codebase recently and most of the codebase was T4 template generated. Every controller and model was generated from a T4 template; with logic inside those templates that through user input determined what was generated and how it was generated.

I have no doubt when these abstractions were introduced, the team thought they’d save lots of time.

One year later, that application is no longer maintained; and in fact, within a few months of release it was no longer maintained.

What sort of value did they get out of a complex abstraction for only 3 months of maintaining it?

It’s hard to tell, but my immediate guess is “not much”.

Abstractions have a cost.

In another project I worked on, the firmware for Jewelbots, there were almost no abstractions. While I had the privilege to have an SDK to use for the System-On-Chip we sourced for our Jewelbots, there were breaking changes every release, and the little abstractions that were present would wildly change. I was so close to the metal that I would have wished for an abstraction or ten.

Where’s the balance?

First off, if someone says “this abstraction will save us X time”, you should ask them to prove it before implementing it system wide. That seems a bit adversarial, so maybe there’s an easier way?

We talk about software design as this abstract thing; divorced from the day to day realities of software development. We treat software as if we’re in the frontier west, living out of wagons and running from one gold rush to the next. We’re so busy adding on what we might need that we forget what we do need: Software that we know works. Software that we know we can change without having to rewrite it because it’s too complex to change. What we’ve forgotten is that while we treat goldrush software as an ideal, we’re asking billions of people to put their trust in our software. We build abstractions that don’t make things better for our customer, and only make things marginally better for ourselves, while ignoring the concepts that will help both ourselves and our customers.

If you’re willing to spend 3 months on code generation, or 3 months on some fancy Entity-Attribute-Value machination that makes it ‘easier to add new features without a database update’ (God help us all), you should be willing to build your software through Test Driven Development. Not only will your software come out better designed, and with fewer bugs, but you’ll also achieve a balance to the abstractions you want to create; by creating abstractions that are easily testable (and thus, easy to change).

Taking the Guesswork out of Estimation

I don’t know about you, but I am terrible at software estimates.

I’m even worse if I haven’t done that thing before, whatever that *thing* is.

As a recent example, a client (for the near term I still accept project work in .NET and Angular on the side, while growing this business) needed me to implement an editor for their workflow (editor here means a WYSIWYM editor like a markdown editor).

Complicating matters is that Angular is very particular about how you can interface with third party libraries; so you end up needing to use or create a wrapper.

Complicating matters further, if you use a design system like the Material CDK, you also want to wrap that wrapper in something that implements the Material design constraints.

All this for a content editor!

If I didn’t know this, it would have thrown off my estimate a lot. And logically, it’s not something that should ever happen. It’s illogical to require these hoops to implement a text editor in angular, and if you’re not intimately familiar with the issues at play, (as a business person probably would not be), it’s easy to wonder why it should take more than a day to implement a text editor in Angular. Or, as this XKCD brilliantly illustrates:

So what to do?

If your system hasn’t been created through TDD, you’ll get some measure of success in estimating by making the change and seeing what tests break. In the case where you have good test coverage, you lower the amount of parts you have to guess will change, and guessing is what makes estimation so hard.

In a system created through TDD, it’s even easier as your tests will not be coupled to the specific things you’re tied to (like a text editor), rather they’ll be tied to the operations your system makes on that text editor; allowing you to swap out editors and *knowing* for certain if it affects the operation of your system.

I offer team based immersion training for TDD; both online and in person, where you’ll learn just how to do this. To get started, visit https://doubleyourproductivity.io/paid.html.

“We don’t need TDD, our project isn’t worth it”

In 2018 I converted our garage into a bedroom, laundry room, and entryway. I enlisted the help of tradespeople (a mason, plumber, and electrician) to handle the exterior, plumbing moves, and electrical needs (respectively), but I did the rest myself.

It took me a year from plan to being finished, and I finished the day before Thanksgiving.

It was hard, possibly the hardest non-tech thing I’ve ever done. I know the software world pretty well, having worked across tech stacks, industries, team types, and methodologies — I feel like I understand technology. It is not a surprise to say I definitely do not understand the home remodeling world.

To get ready for this fact, I did a ton of research. I looked up building codes and realized an entirely different lexicon was at play (somewhat related, but the word “shed” doesn’t exist in our county’s building code — the appropriate word is “accessory structure”), I watched Youtube videos, and I asked people who had done this work before.

As it stood, I still failed inspection a few times, because my research hadn’t taken into account some unique nature of the space’s transition. For instance, I should have probably outsourced the subfloor construction; since I needed to raise and level the floor about 1.5 inches at one end, and 5 inches at another. I made it work, but I’m pretty sure a professional would have done it faster and it would have been better constructed.

Why didn’t I engage a professional? I mistakingly thought I’d save money by not contracting that part out. That somehow, a floor that I will walk on for the rest of the time we live in this house didn’t need the care of a professional. It was Good Enough Of course, as I found out later, I just hadn’t done enough research. But at the time, I spent around 2 months (I could only devote nights and weekends to it) raising and leveling the floor. If I had engaged with a professional, it would have been done in less than a week. This is about the only regret I have about that project. The floor isn’t quite level in one corner, and because I didn’t insulate what I raised, it’s a bit hollow sounding and a little colder than it should be.

You ever have a software project like this? One where you think “we’ll do it right this time”, but that one (or more) stakeholder says “this will be good enough”, and you acquiese? After all, how bad could it be?

It couldn’t be any worse than a floor with an unlevel corner that dips down a little bit, and a chilly floor in the winter, right?

We treat internal software projects (or sometimes, consulting projects) just like I treated the floor in my garage renovation, and contrary to the beliefs we hold, it shows. It affects us.

For your next garage renovation software project, think about whether you want it to be like your last few projects. Do you want there to be an unlevel floor? Do you want to dislike the floor in the winter? If not, then you’ll want to focus on practices that will help you deliver a project on time, that users can use without asking “What happened to this floor?”

P.S. I offer virtual team TDD immersion training for .NET Software teams. This training helps your team learn Test Driven Development, together, and ensures the entire team develops a shared understanding of the processes, reliable outcomes, and aims of TDD for your codebase. To get started, visit https://www.doubleyourproductivity.io/paid.html to learn more and on how to book a session.

“You Got the Job, Now what?” (Part 2) – Webinar

I appeared on an Apex Systems sponsored webinar titled: “You got the job, now what?” (Part 2). This webinar aired on February 25, 2020.

Part 1 is also available for your perusal (sorry for the reg-wall; I don’t control that).

In Part 2, I joined two career experts to talk about what happens after you join a new company, and how to be and bring your best self to that table.

I really enjoyed this segment, and learned quite a bit. I hope it helps you too.