Back to The Future of Agile Software Development
This essay is inspired by Ken Schwaber’s unSAFe at any speed blog. In brief, Ken cautions against returning to the RUP practices disguised as agile framework, and urges his readers to keep their commitment to the Agile Manifesto.
While I’m very much with Ken on the subject of heuristics, thinking for yourself and sticking to the principles of agile, there’re some more deeper things to that than management vs. software developers, or RUP vs. agile oppositions. Make yourself comfortable, as it’s going to be an interesting 7+ min read, or whatever more time you might need to contemplate it.
The Origins of RUP and waterfall (’80s–’90s)
Let’s look back at RUP, waterfall and other such unified processes, you name them. Where are their roots , and when did they actually make sense? It’s in the ’80s-90s of the last century, maybe even in the ’70s. The unified processes are the legacy of industrial and agricultural age production methods from the earlier times of the 20th century. Back then, in the ’80s-90s, it was very natural to think of software development as yet another field of production, not any different from hardware or commodities. Besides — and it’s an important thing to note — the pace of changes in the industrial and agricultural environment has always been slow. No changes for 50+, or even 100+ years in some cases.
First, everyone thought that software production is going to be like that as well. No changes, hence no changes in the market/production environment, hence no problem-solving skills required. Well, if the problems related to an unchanged routine activity have been solved once and for all, all you need to do is follow some rules and commandments. No new problems -> no changes -> rigid processes. With software development, it all went along in the same fashion with the boom of outsourcing in the late 90′s – the early 00′s. I used to work in an outsourcing company, by the way, and I remember how we’ve been luring the prospective clients back then by neat RUP diagrams, as we tried to convince them that their project will be done on time and within budget.
Next, with no changes in the environment, rigid processes are supposed to work fine. Well, fine it was until some moment. It appears that by 2001 (maybe it was related to the notorious Y2K problem, along with the others), but there came an understanding to thinking people in the software industry that something is wrong. Rigid boilerplate processes did not seem to be working to create viable software in the changing environment. This critical mass of changes found a small outlet, as the lava first appears only as a tiny stream, in the 2001 agile manifesto, and Ken Schwaber was one of those who signed it.
I see the Agile Manifesto as the soul’s cry of creative software engineers who wanted to say it out loud, that they are aware of this problem, they claim this awareness to the whole world, and say that they’re not “code monkeys” but thinking people who have their solution to this problem in mind. They wanted to stand out and share their beliefs about better ways of developing software. They just couldn’t live with that awareness and not sharing it anymore. I haven’t studied the biographies of the agile manifesto founding fathers but for some reason I’m sure they’ve had enough of enterprise softdev companies running in the Procrustean bed of the 20th century industrial ways.
The Agile Manifesto and what came next (2001 — ?)
After 2001, the whole decade up till now, has been filled with the buzz about agile adoption, fading or getting louder at times. Most likely, the agile manifesto has had its first ardent followers among bootstrapped softdev start-ups and some small web service shops. Or, among the software engineers who simply felt they can’t do their work well within the old paradigm. Just anyone who was not a mere “human resource”, not a cog in the rigid process but a thinking person who faced the real problems in their biz/softdev life-style, and naturally had no other way to go on as to be able to solve those real problems. That’s what the essence of agile is. Agile problem-solving in the fast changing environment.
Let’s flip the phenomenon called “agile” and look at the other side of the coin. As agile methodology got its entourage of followers, agile coaches, trainers, various certification programs, it started showing some signs of rust. Like an idol worshiped just out of tradition, because everyone seems to have forgotten how it appeared originally, in the first place. Now, consider this. The industrial-style companies of the late 90′s are still there. They still have their problems. They’re still the same as in the late 90′s, or even late 80′s. They have budgets to spend and outsourcing teams to run. But since RUP and waterfall have publicly been coined as malpractices, these enterprise companies must have jumped for joy as they saw someone approaching them with the stuff called SAF (Scaled Agile Framework). As far as I can see, SAF has the same essence as RUP (correct me, if I’m wrong) but now it wears a nice glittering gift wrap with the big word “AGILE” on it. The terms are different from RUP, though. It’s called SAF, but the essence is the same: it’s a prescriptive framework for industrial/agricultural style production in software development.
Such companies will indeed be glad to open their checkbooks to something like that, as Ken says in his blog. That’s what they need, and they will run fine with it, as long as the global market and business environment lets them live this way, keeping the patterns of production where they don’t need problem-solving skills in software development as the ultimate prerequisite for running their businesses.
Now, speaking of the global market environment. Of course, the distinction is not black-and-white. There can be companies who are enterprise, who are changing, transitioning, transforming their businesses, etc. Everyone is human, and everyone has their own problems. SAFe indeed might be able to solve the problems of some companies. Of course, there can be a hybrid of enterprise thinking and creative thinking (the Japanese look like the most obvious example of that to me).
”Agile” as buzzword and “agile” as continuous problem-solving
I want to emphasize that there’s agile as “agile, the buzzword”, the silver bullet that many heard of, the idol that is supposed to heal all the ulcers if you worship it. Usually, people resort to this kind of agile if they:
a) don’t have time, or guts (sorry), to dig deep, study and learn from their own experience and move on. Well, it’s one of the most common misconceptions of the humankind: a belief that your particular problem, in your particular business/market context has a ready made solution out-of-the-box, coming from some 3d party. The ready-made answers can only come to a purely technical problem, like the sequence of steps to assemble a wardrobe purchased from IKEA. I’m continuously amazed at the questions people ask on some forums, as they believe that the problem in collaboration, or production, or in a process peculiar to their organization can be resolved by someone’s quick online answer. There’s no prescription for such problems, period. Only the experience-based flow and thinking, iterations and corrections in response to the feedback from your particular business/production/market environment are viable.
b) if they’re the folks who muss RUP. They still need something like it. Budgets to spend, reports to write as they operate in their narrow neat silos, untouched for one reason or another (so far) by changes since the late 90′s or maybe even earlier.
The 2nd kind of agile — and I don’t want it to be a rusty buzzword — is about problem-solving out-in-the-trenches. Take a living organization that operates in a very fast-changing business/market/production environment. Well, I don’t think that such organizations can afford thoughtless copy-pasting of abstract prescriptions. By the way, here’s an interesting brief summary of Agile 2013 conference. In short, the best thing about this conference, according to the author, was mingling with the folks who have the same problems, but the conference gave no actual answers to those problems. I published a post called Agile Conferences: Look To No Epiphany on a similar subject about 4 years ago as well.
The logical conclusion? People. People. People.
If you can’t afford blind copy-pasting of prescriptions to solve the real problems in your organization, then PEOPLE are your biggest asset. That’s the real agility. Only people can be agile. Not a process, in the very true sense. Agile people can tweak the process as they need to solve their problems that appear in the fast-paced environment where they operate. The people who make one team together and are able to solve problems continuously, because a changing environment always entails new and new problems. If in a more narrow software development terminology we have the term called “continuous delivery”, then my version of the buzzword definition would be:
Agile is continuous problem-solving.
Now, what do you think the future will bring? More of those 20th century industrial/agricultural softdev corporations? Or more of those indie companies with their universally knowledgeable people, who are, well, agile and alert because otherwise they won’t survive in the changing environment? This is an interesting subject in itself, and it goes far beyond this essay.
People are people. They have their problems. They live. Life changes. Agility is a way to survive, live, enjoy those moments, and create some nice software stuff that will make the lives of other peoples easier, maybe even nicer, and will help people solve their problems, eventually.
So, that’s where the answer to the question ” What will become of agile in the next few years?” is. It’s a global economic/business market, and what this market needs more: agility as real agility, or enterprise-industrial style enterprises with their rigid homeostasis untouched by the changes. Why and how the rigid homeostasis can be untouched is not to be discussed in a blog on agile software development :) Maybe I’ll write about it elsewhere, some other time.