Why Self-Organization Is a Luxury

This content is syndicated from Edge of Chaos | Agile Development Blog by Olga Kouzina. To view the original post in full, click here.

I continue exploring deep reasons behind various phenomena in agile software development. It’s amazing, how often the universal principle of entropy manifests itself in the  relatively short ~ 10-year mainstream history of this movement.  It looks like I have another eye-opener.

Entropy in software development

What is entropy, in simple terms? This is the principle of bouncebacks as in a pendulum. If there’s too much emphasis on one side, a social or ideological pendulum bounces to the opposite point, as if to compensate this overdoing. We have too many decisions to make and we’re tired of that? Let’s invent the panacea, called Big Data. We got tired of deadlines in waterfall software development, and even Scrum didn’t help, because we got tired of those time-boxed iterations? All hail Kanban now, because it has no deadlines. We are tired of managers treating us as numb tools? We want to break free, so we delve into the tempting freedom of self-organization and no managers at all. I sketched the visual for this pendulum, to make the metaphor clearer.

Agile IT Pendulum

Now, the universal law of bouncebacks is not a problem per se. It’s a given. We can’t get away from seasons changing; and they say that if you laugh too much, you will soon cry. Anything in this world has a bounceback, if overdone. What all those bouncebacks have to do with problems of my company, you might ask? That’s what I’m getting at. There’s a composite of forces that start a trend or a movement. If the critical mass of companies are having similar problems, then the pendulum gains the powerful momentum to the opposite side. That’s how it happened with agile. Some companies hopped on this vessel as early adopters, some have not yet accumulated enough energies for that move, some are in the process of swooshing with the pendulum swing. Each and every company has their own unique place in this swing, the unique combination of energies and driving forces. The way I see it, the art of technical leadership has nothing to do with catching the swing and rushing headlong where everyone else is rushing. To follow the “join everyone else” call is the easiest thing to do, and a certain component of luck might help here as well, if a company has caught the early tide flow, and benefited from the swoosh. The hardest thing is to sense where your organization is now, in which particular point of the swing, and how standing there influences your org’s productivity and ability to deliver software to your customers.

I’ve spent enough years in software development, with customers, developers and managers, and I speak from personal experience. It feels good swinging along with the pendulum as it bounces from the left to the right, as on the picture. I felt that the agile movement and self-organization in a team is what I really needed at some point of time. However, later on I started connecting the dots in the bigger picture and saw some signs that self-organization and several other values declared in the agile manifesto are not universal laws. They rather signify the points of bounceback. There’s no universal proof that self-organization, or face-to-face communication yield the highest productivity for any company. Keep in mind that agile manifesto signatories are software craftsmen. They’ve postulated the agile principles to declare that software development is a craft, and they are a noble guild that wants to serve customers. By declaring those principles, they mentioned nothing about constraints, times-to-market, expectations of stakeholders, and financial goals.

Is it a human instinct to deliver projects on time?

Some complexity theory studies compare the laws by which organizations function with those of colonies of ants or flocks of birds (e.g. check this article dated 2009). According to those theories, ants and birds know instinctively how to self-organize to fetch food, or to fly safely to their destination, and the researchers somehow concluded that IT companies must follow the same pattern. I’ve shared this belief until recently, when a realization struck me. I beg your pardon, but is it our natural human instinct to deliver a release on time? Or to a meet a harsh deadline? Can you imagine the Great Wall of China built by a self-organized team? Of course, no. Why am I asking this? Well, software developers are not ants and birds, and, speaking of instincts, their basic high-profile instinct as software craftsmen is to craft what they do to the point of perfection. If given endless time, they would craft it at their own pace, be it a UX design, or a piece of code, and that’s where they indeed are able to self-organize and run a flat organization as e. g. Valve or 37 Signals do. Lucky is a software craftsman, and lucky is a team that can afford the luxury to release at their own pace, with this deep feeling inside that they have crafted this piece to perfection. However, we live in the world that is full of constraints. A pizza-sized team of craftsmen wants to grow big, because they now want to come up with yet another awesome software. This move requires more hands, and coordination with stakeholders, internal and external ones. The noble castle of software craftsmanship is invaded by merchants, who want deadlines met and sales goals hit.

So, when the harsh reality of the market hits the noble craftsmen, they follow inertia and think that self-organization (or flat structure) will let them achieve those very non-instinctive deadlines and sales goals. Looks like that’s what happened to the guys from Everpix who did not have practical guts to comprehend that they have to take merchants into account, and kept having their heads in the clouds. If they were smart enough to sense in due time that the pendulum wants them to swing backwards, down-to-earth, to pragmatism and discipline, Everpix might have still been alive delivering some nice service to people. That didn’t happen, however.

It’s time now to take a pragmatic look at the eulogies they sing to flat organizations and self-organized teams. The opposition of hierarchy and flat goes first in the list of pendulum bouncebacks (see the sketch above). As for the other three pairs, I’ve covered them elsewhere. The flat company structure is related to the concept of laissez-faire leadership. While this is an excellent approach to leading a company from the standpoint of individual learning and skills acquisition of team members, it does have some faults if what a company needs most of all at this very moment is maximum productivity. Quoting the article that I linked to above: “Researchers have found that this is generally the leadership style that leads to the lowest productivity among group members.” Again, this leadership is a bounceback from rigid hierarchy. It does allow people to take decisions and contribute to the activities that are beyond their direct responsibilities, but people have only so much energy, and it strongly depends on individual working styles. It could be that someone who has a lot to contribute to good decisions is not OK with the combined need to make decisions and to produce deliverables. Organizing and decision-making is work, the chore that drains energy from the performance component of such an individual. If a UX designer or a senior developer who is at the same time responsible for the implementation of a feature or a design spends too much time in decision-making activities unrelated to their main work, this is a serious hindrance to productivity.  Oh, and don’t forget that personal discipline and responsibility is a must for that style, as well as high cross-competence in all aspects of software development. What happens then if team members lack competence, and what they do is learn?  The momentum spent on learning is subtracted from production.  Here’s a visual to help explain the idea quickly:

Time Learning Productivity

The background colored shapes show how each of the components would take from the other two, if overdone.

Balance learning, time and productivity?

There is a way, though, to get the best of productivity and keep the learning in place. But it does require smart thinking behind the daily setup of company operations. Back to the pendulum, the tech leader has to keep the company somewhere around the pivot point, with no rush to extremes. Someone has yet to develop the balanced mix of agile and hierarchical management. This mode of pivotal operation implies razor-sharp intensity and focus in anything and everything, the foresight in knowing and defining which issues need whose attention, and when, and setting the boundaries as far as to the point of worshiping individual productive flows, that of decision-makers and that of performers. It takes some hard thinking, and taking hard choices: can we afford hitting 5 meetings with 7 people to discuss this issue? Is it worth it? Does it really matter for productivity that everyone reconciles and has the same viewpoint on any given problem? How the skills and competence of people can be applied optimally, to keep them working and have them learn new things? As I read stories about companies that seem to be successful with flat structure, I noted their discipline and foresight with work-related interactions. So, if tech leaders want to copy-paste the laissez-faire style because everyone says that it’s cool, they need to be realistic in the assessment of where their company is in the pendulum, and which priorities overrule at this very moment. Hands-off leadership, which is the same as self-organization, is a luxury that not everyone can afford.

I hope to share more thoughts on the modus operandi of tech leadership in the future.

Related articles:

Back to The Future of Agile Software Development

Kanban as Multiban?

Prioritization and Big Data? Think Human Nature

Why Is It Right to Write?

Leave a Reply

What is 10 + 3 ?
Please leave these two fields as-is:
Please do this simple sum so I know you are human:)

There are 101 ways to approach anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”


“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”


“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”


“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”


“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”


“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”


“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”


“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”


“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”


“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”


“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”