I bandy about the term ‘agile’ on my email list a lot. While it may seem like I use it to mean “practices we adopted but didn’t solve our problems”, it also has both a specific set of things it means, and at the same time is a too broad categorical list of things executives want it to mean. To say it’s a process or a methodology is being too generous, as it’s a process in only the most hand wavy sense of the word.
It’s a set of values and principles, and we’ve somehow created from that a central thesis that it is useful as a definition of a development methodology, and is somehow prescriptive in what it means.
“We’re agile” has come to mean “We’re moving fast and breaking things to get new, not-thought-out untested features delivered this quarter to make our executives look good at the industry conference”. Or, “No we don’t have a plan, we react to whatever is happening now”, or “Our development team runs at 100% utilization because idle developers cost us money, and developers are expensive.”
But, those archetypes of modern agile aside, even if we were to take the agile values and principles at face value, you’d be unable to articulate an actual process you’re supposed to follow if you adhere to these principles and values.
If you hired a consultant, and they said to you, “Value People and interactions over Processes and tools”, I hope you’d slap them silly (after asking them what they possibly mean by that in a practical sense). Does that mean runbooks are out? Style Guides? Checklists? Of course not, they’d exclaim! Then what does it mean? And at this point they’d probably just repeat what they said, and both of you could share an awkward silence. What does it mean? Seriously, I’m asking. At best it means “don’t let following the book keep you from throwing the book out when it needs to be thrown out”, and that both vague and, to be fair, not a bad fortune cookie fortune. When would you throw out the book? Under what circumstances? When it’s inconvenient? When it actively detracts from what a particular user wants? When it gets in the way of the quarterly profit? When? The problem with this sort of ‘value’ is that it sounds good but doesn’t actually define its parameters, making it terrible as part of an actual process definition.
Likewise, taking at (not really) random an agile principle: “The best architectures, requirements, and designs emerge from self-organizing teams.”
I have to ask what sort of hate the inventors of agile have for the rest of us that they imposed their context and view of the world on us and we just said, “Yea, that sounds right”.
The best architectures, requirements, and designs demonstrably do not emerge from self-organizing teams, because those self-organizing teams have constraints they don’t control. It doesn’t matter if the four of us on a team get to pick what we work on, or what tech-stack we use, or whether we make it a relational database or a no-sql database. We’re not the Avengers. We don’t have superpowers that obviate the need for resources and budgets. (As an aside, in the MCU movies, shield doesn’t seem to do anything other than get in the Superheroes way; whereas for the non-superheroes among us, we’re always dependent on some sort of bureaucracy to support our work).
Normal teams don’t control their budget, the market, the board of directors, or the C-suite that is hunting for a quarterly profit. It is both defeating to claim there is such a thing as a self-organizing team (and have it be true) and downright dangerous to think that with any sort of seriousness. There will always be external constraints, and putting the idea that the team, by virtue of being ‘self-organizing’ can somehow navigate an entire corporate culture, and know what is right for the customer, product, and business, is foolhardy — at best.
It is also stupidly dangerous because it isolates the ‘development team’ from the business in which it operates with a blue flag that says “Just let us organize ourselves!”. The flag may as well have a snake and say “don’t tread on me” (It’s election season here in the USA).
Now, I understand the desire behind that principle: That if you let a team figure out what they need to do and how they’ll be more bought in and do a better job than if you had some VP or director standing over their shoulder indicating what they can and cannot do and how. And if you say, “well it doesn’t actually mean any of that, it only means that they get to choose how they get the job done”, I’d like to sell you a bridge. Agile pretended that somehow multi-million dollar budgets were in the control of the team, or head-count, or technology choices, or what sort of features get worked on. As a principle it’s unrealistic, and even if it were completely realistic and attainable, would be about the worst way to run a software team as can be imagined. Human Hierarchies and interdependent groups exist because that’s the ‘best’ way we’ve figured out how to work for at least since the middle-ages. The builders are not also the dreamers, and the dreamers are not also the folks with the money, and you need all three to get anything done.
Agile is a set of values and principles; it’s not a well-defined (or even an ill-defined) process, and treating it as such will at best lead folks to misunderstand each other and lead to bad outcomes. It’s a set of values and principles you can use to create a process, but it’s not a process in of itself. As much as I dislike the results of Scrum, it’s at least a process.
This post originally appeared on my daily email list. I’m reproducing it here in vain hopes you’ll join said email list. The box to join should be below. Someday I’ll have this process automated so my daily emails pop up here too, but the cobbler’s kids and all that.
In the real world, it’s a culture of micromanagement, pointless and energy draining meetings, spinning on a dime to cater for whatever the changed requirements are this week, and prioritising burn down over quality and maintainability of code.
It has become one of the biggest red flags in job listings.
“Agile” was never (ever) intended to be a process nor a methodology. It merely defined a set of techniques for engineers to use in order to continue delivering in the face of changing requirements and schedules – within a true methodology – and engineers had already been using them for years – it just didnt have a name.
As you imply, it sacrifices two of the most important (and difficult) aspects of software development in favor of delivering buggy code quickly: requirements and testing. And quality has suffered… immensely.
I agree with your points. I will be saving this to my “problems with Agile” folder that often I send out to co-workers – I guess an attempt to sway their line of thinking away from this dumpster fire of a philosopy/methodology/process. You really succinctly cover the issues. I have never been a fan of Agile. I have been leading successful tech projects (hw, sw, fw, rf, mech, dsp) on-and-off since ~1989. I find that people use Agile to justify starting projects with no planning and “winging it” along the way – no requirements, no requirements decomposition, no understanding of the timeline of tasks (start-to-start, finish-to-finish, finish-to-start, etc., i.e. what needs to be done before what and when task relationships), poor design planning, the list goes on and on. Or, they think that abiding by the Agile principles can make anyone a technical leader or tech project manager. There are some good things about Agile – I tend to “waterfall” the larger detailed project plan, then use an Agile approach on the larger “sub-project” tasks – but every Agile implementation I have seen was so sloppily implemented that the project stretched on and on, the same tasks being done over and over (pushed from one Sprint to the next), and the development ended up with so much technical debt, developers would leave the project to get away from it.
Whatever “agile” started as (and I agree it is “…a set of values and principles”, “Agile” is a business. It was/is – a great set of practical ideas coopted by corporations (both that sell it and that use it) into formalized product and tool for management. (Which is very annoying because it really can work well.)
Having said that, even in its corrupted state it is better (even if only marginally) than the previous, heavy software processes.
Just so long as it also isn’t coopted on the other, equally impractical, side of developers who use it as an excuse for old-style “cowboy programming” – i.e. no real design, no documentation, etc. (because that’s not “agile”!) – as if that makes more sense.
i can summarize it: the word agile is an adjective, not a proper noun. use it as such.
In Agile, the “team” is supposed to include all the stakeholders – including the people who control the budgets and resources, those who have quarterly goals to meet, and those who represent the customer. In practice, this happens so rarely that most people — the OP included — aren’t aware that it’s part of the Agile ideal.
That is an excellent observation. The Software Development Life Cycle (SDLC) includes a lot more than just software design, coding, testing, and deployment. What comes before, and afterwards, has to include all the factors you mentioned.
I deviate from the “stakeholder” context, though, because that usually becomes a point of delay and introduces non-technical “stakeholders” trying to influence technology decisions beyond their expertise.
Instead, as a project manager/team manager/software engineer, I interview all those affected, especially when it comes to budgets and contractually-obligated deadlines, to determine the financial and time constraints and negotiate them where necessary. I expect the non-technical folks to tell me their requirements. I draw the line when they, however well-meaning and innocent, they venture into technology “advice”.
That approach only works when the project is managed by one person with both the technical and business expertise, and the negotiation skills to lead, paint clear pictures, and keep all involved headed in the right direction, pulling together.
My two cents on this. My understanding is that the Agile approach araised from a fedup situation of development teams because the incredible amount of bureaucracy that the watefall and V-Cycles had produced. Let me give you some illustrations:
– I wrote a program and I released it after passing all the defined and accepted testing and validation procedure. From this moment on the release had ownership of an entity running following its internal principles. Two or three days after I noticed I had a typo in a notification or error message so I fixed it just by replacing a character in a hard coded string, agree it’s not a good practice to have hard coded strings but if I had them in an outer file to the code it would had been the same as the release concerned a group of files, so I was obliged to submit a request for modification code that would been approved by somebody unable to understand the programming language and that applied an approach inherited from industry, sometimes aeronautics, and so I had to wait almost one month before the substitution of an “e” by an “i” was materialized in front of people using my software. This is one of the cases when people and their interaction are to privilegiate against process and procedures.
– I was working in a support and advice activity and one of my customers were developing applications in Cobol. At one moment they reported me an issue that appearead to be a bug. I went then to the development team and discussed the issue with them and they acknowleged that it was a bug in the product but the bug was located in a runtime library written in another language than Cobol. The sustainer was a subcontractor and he told me that his contract with our firm entitled him to work on maintenance issues one month a year and that the contractant had agreed with him to work on the maintenance issues only after the 30th of November, we were in April. The guy at the sw team indicated where the bug was located and provided me with the source code and the procedure to build the runtime library. I fixed the code, I built the library and the bug gone away. The problem was that because it has not been done by the official owner of the product my fix was only valid if the customer wished accepting it. The manager of the customer firm refused and asked for an “official” release. I explained him that in this case he had to wait until the 1st of December at least and then wait for the approval to the Global Quality team which was going to be planned by the end of January of the next year. The developers of the customer firm were willing working with my fix and indeed they did it, but their product was not “official” because the stance of their Executive Director. So we had a solution to a very trivial problem but the official path avoided using it. Once again Process and Procedures were favored instead of People and their interaction.
– I wrote once a thin layer of code released as a DLL (it was MS OS/2 it could had been Windows NT or any other Windows derived from it or even an so library for Linux). I did it following the strict guidelines provided by the Team Leader of our development team, he had made this decision after an accurate analysis with an expert on Microsoft products both Windows and early OS/2. Originally my interface had no code but just interfaces like in Java or other OO program, although it was C code, so the functions were empty and only returned a passed structure. Progressively I started writing code to comply with specific features requested. Our Project Manager asked me to write a document about my interface layer. I told him that the easies way would do to provide with the source code for the developers willing understanding what was going on but he had committed to other managers in order to provide them a document explaining a so basic architecture that it had no real added value. I wrote the document I put some irony in it and I enjoyed writing it. I wonder if the readers noticed the irony in the document, I suspect they archived and classified the document without never having read it because every time a developper contacted me, I provided them, with the due agreement of my Team Leader and his mangers the source code of my interface and they were more than happy to have it. In this case you sed that Exhaustive Documentation took over the Working Software Release.
– I wrote exhaustive documentation for inexistent products becaue the doxa at the time was that a product had to be finished on paper before starting to code the firs line. I spent days and weeks writing them and when we arrived to the Decision Revue meeting I noticed that more than half of those who should agree on go/no go decision were reading my documents released one or two months ago for the very first time at the very previous minutes before the meeting for decision started. In this case why writing documentation that is never read?
– The last for the road. I was managing after sales service at European lever and we had agreed on a process that was the most economically pertinent but that hurt some convictions of some Country Managers in our subsidiaries in the countries. One of them had a better experience than us in the services to home facilities like TVs, DVD readers, DVD recorders, HiFI equipment and so on. He challenged our process since the inception and took advantage of the minor incident to highlight that our process wasn’t appropriate for his local situation. He had a case and we had an important meeting appointed somewhere in Paris area. Before I traveled for the meeting my boss gave me strict instructions to not accept any change to our overall policy and that because the challenging Country Manager was more skilled than me I would just listen and take note of his remarks but not agree with any change in our approach. The meeting lasted 3 hours and I did like my boss asked me to do, I kept on my position, I tried being the more emphatic possible and at the end we were in a ping-pong game when we refused completelly to change our point of view. I was sensitive to the need of changes for his particular case I woud have willed saying that I agreed with him but that changing everything for his relatively minor market share was not accurate. Two other gurs not being “R” in the RACI matrix unblocked the situation by saying that we had understood his problems and that it would be good that I came to his country to make the appropriate decisions with the field people that were having difficult times with our process. This was enoug to him and I was satisfied too, I really wished doing something that could alleviate him and his staff. So after saving mutually our faces I came to my boss and told him that I considered it was appropriate to adapt or modify our process to his particular situation. My boss agreed, in reality he had agreed since a while but he was working to make accept the changes to his own bosses. I realized then that if I could stand holding a strong position withtout giving up it was because I was not going to be submitted to the same conflict befer maybe 6 monts or one year. If I had to work with this country manager every week or every day keeeping this postion would be impossible or becoming a very painful situation. In Agility the daily interaction with your stakeholders is promoted because for most of humans is impossible keeping a statuquo like this one when you have to face interaction day to day with somebody disagreeing with you. So this daily interaction is intended to force people to find a mutual understanding.
– I don’t think that Agility well managed will close developers to any other people in an organization. You are suppossed to show your work in progress in order to recover the feedback from future users and it’s intended to break the silos architecture in organizations. I fear that the industry had privileged too much a kind of immature programmers, notice programmers not developers, of the style of those having spent 80% of their life since being teenagers without doing something else. Developers per se tend being too much focused on code and technical issues than on any other thing, they have to be because this is what is expected from them and why there were choosen for this activity, but putting more focus on no-life coders doesn’t help at all, these people don’t mind about budgets, easyness of usage, change management required to succesfully introduce a product that change work habits, or future maintenance. As far as they are face to a screen and conding on keybard they are glad and satisfied. So it’s equally other stakeholders responsibility to establish the links with developers to build trust and confidence and having a positive regular exchange with them. Per my experience is possible but it requires open minded and good will people.
I think that first you have to understand that many of the Agile principles were defined in reaction to this phase when people not being developers and not caring about developers defined methods, process and approaches based on the industry that that hey knew better than the SW development and second that the main problem with Agility is that it requires that people be better than they are: alterity, empathy, not bound by hierarchy and process, not fearing saying “I don’t know” or “I disagree”, able to organize and take into account things rather far from our day-to-day activity and that’s why it fails, just because people are not that good.
The Agile Manifesto and Agile Principles are not a development process, as you say. However, they are recommendations for what to include in your process. “Individuals and interactions, over processes and tools” means that good people. in a collaborative environment, will get you further, faster than will following an elaborate process. It means *trust your people*. This is why corporate lackeys cannot understand agile, because the last thing they would think to do is trust the experts they pay so much money for. (And honestly, WTF?)
Documentation is a weak substitute for collaboration. Self-directed, highly collaborative teams work out their process and their product together, so they understand all of it. Self-directed teams are more effective than a strongly hierarchical team doing the same thing, because a self-directed team takes advantage of every team member’s knowledge. It’s not about whether all the stakeholders are on the team. It’s about doing the things your team has the authority to do as efficiently as possible. This is another thing corporate lackeys don’t get about Agile. They want to push away the responsibility, while keeping authority for themselves.
The authors of the Agile Manifesto had the good fortune to work at skunkworks’ and small practices where the corporate lackeys were absent or under control. The authors are also all consultants, willing to earn big bucks teaching you how to control the corporate lackeys of your enterprise in order to get work done.
The problem comes when the lackeys themselves hire the Agile consultants. They see the Agile Manifesto, but don’t get what it means. They beg the consultants for a nice tight process they can control, so they can make developers responsible without giving them any authority. And to their shame, too many consultants are willing to do just that.
As a process, Agile offers just two improvements over the traditional waterfall process; iterative development to make feedback available sooner, and continuous release, to allow value to be extracted from the first features completed. If no one collects feedback and uses it to modify the process or update product features, iterative development has no value. However, it can still be abused by management lackeys to make developers’ lives a hell of endless sprints, with weekly requests to work uncompensated overtime to “catch up” to a schedule that was never meant to be more than an estimate.
The moral of this story is that there are two Agiles; the one smart developers dreamed to build better software, and the one managers abuse to exert control and extract uncompensated labor.
Agile started out as an idea with potential merit, but it was hijacked by people who love pointless meetings and drawing on the kanban board. Most of the developers I know hate Agile because they waste too much time in meetings with clueless people trying to justify their existence. As the commenter above mentioned, agile is one of the biggest red flags in a job description; the others being design patterns and TDD, but I digress.