Self-Organization and Complexity

This content is syndicated from Brad Appleton's ACME Blog by Brad Appleton. To view the original post in full, click here.

In my previous blog-entry I talked a little about how self-organization is a key aspect of software agility. In this blog-entry I'd like to explore in more detail just what "self-organization" really means.

Self-organization comes from complexity science and the theory of complex adaptive systems (CAS). A system is "self-organizing" if, left to its own devices, it tends to become more organized over time. This is in stark contrast to the concept of entropy from the laws of thermodynamics whereby a closed system tends to move to a state of increasing disorder in the absence of outside influences.

In a complex adaptive system where self-organization occurs, we necessarily have an open system rather than a closed one. The theory goes that if a complex system possesses the necessary emergent properties in an appropriate fitness landscape, then the system will yield emergent behavior, exhibiting system-wide "patterns", that increases the "order" or organization in the system. Hence the system has become self-organizing as a result of this emergent behavior & structure.

Some of the emergent properties of self-organizing systems can include:
  • Autonomy,
  • Adaptability,
  • Decentralization,
  • Dynamic reconfiguration,
  • Positive & Negative Feedback
  • Cooperation & Information exchange
The theory of complex systems suggests that self-organized systems are:
  • better in selecting optimal state and local decisions
  • robust
  • effectively adapt to environment and use feedback loops
  • often come up with emergent and unexpected solutions
  • self-regulated and better cope with perturbations
According to James Highsmith,
“There's no hierarchy of command and control in a complex adaptive system. There's no planning or managing, but there's a constant re-organizing to find the best fit to the environment. The system is continually self-organizing through the process of emergence and feedback”

The Emergent Phenomena Research Group at Brywn Mawr says:
"In self-organizing systems, global order (i.e., complex aggregate structure/behavior) results (emerges) from the local behaviors of simple agents following simple rules specifying conditions under which those agents act (interact) in such a way that the results of the agents' actions are fed back as input to the operation of the rule system at some future time step. "

In The Science of Self-organization and Adaptivity, Francis Heylighen writes:
Self-organization is basically the spontaneous creation of a globally coherent pattern out of the local interactions between initially independent components. This collective order is organized in function of its own maintenance, and thus tends to resist perturbations. This robustness is achieved by distributed, redundant control so that damage can be restored by the remaining, undamaged sections. ...

Self-organizing systems are characterized by: distributed control (absence of centralized control), continual adaptation to a changing environment, emergent structure from local interaction, robustness/resiliency (able under change and can quickly repair, correct, adapt/adjust), non-linearity, feedback (both positive and negative), self-sufficiency and closure/coherence. ...

Organizational closure turns a collection of interacting elements into an individual, coherent whole. This whole has properties that arise out of its organization, and that cannot be reduced to the properties of its elements. Such properties are called emergent.

Every self-organizing system adapts to its environment; Systems may be called adaptive if they can adjust to such changes while keeping their organization as much as possible intact.

From Software Development is Complex Adaptive System. No Doubt
Emergence is the way complex systems and patterns arise out of a multiplicity of relatively simple interactions. Small actions of agents lead to unexpected emergent system behavior and it is impossible to predict system behavior based on the behavior of an individual agent. Small errors pile up and may cause huge problem.

There's no hierarchy of command and control in a complex adaptive system. There's no planning or managing, but there's a constant re-organizing to find the best fit to the environment. The system is continually self-organizing through the process of emergence and feedback.

A system in equilibrium does not have the internal dynamics that enables it to respond to the environment and will slowly (or quickly) die. A system in chaos stops functioning as a system. The most productive state to be in is at the edge of chaos where there is maximum variety and creativity, leading to new possibilities. A set of simple rules allows edge of chaos and powers creativity, flexibility and success.

And last but not least, from Conditions for Self-Organizing in Human Systems :
Self-organization is the spontaneous generation of order in a complex adaptive system. It is the process by which the internal dynamics of a system generate system-wide patterns. Some of the emergent patterns in a self-organizing system are coherent, and others are not. Coherence is a state of the system in which:
  • Meaning is shared among agents.
  • Internal tension is reduced.
  • Actions of agents and sub-systems are aligned with system-wide intentionality.
  • Patterns are repeated across scales and in different parts of the system.
  • A minimum amount of energy of the system is dissipated through internal interactions.
  • Parts of the system function in complementary ways.
System-wide patterns in which the parts are aligned and mutually reinforcing (coherent) are more stable than other self-organized patterns. Because of the mutually reinforcing dynamics of a coherent pattern, the effort required to change the pattern is greater than the effort to maintain it, so coherent patterns are more stable than incoherent ones. When the system reaches a state of coherence ... tensions within the system are reduced, and the available energy of the system is aligned and focused on system-wide behaviors, rather than diverse and disruptive behavior of individual agents or sub-system clusters. ...

In human systems, the process of self-organizing is particularly important. Teams, institutions, and communities include individuals or groups of individuals that function as agents in self-organizing. As the agents interact, patterns of behavior emerge over time.These patterns form and reform spontaneously and continually at multiple levels within the system. Individuals work together to form teams. Ethnic identity groups establish relationships and micro-cultures. Functional departments engage with each other to do the work of the organization. At all of these levels, agents interact naturally to form patterns of system-wide behavior. ...

The CDE model is a set of the three conditions for self-organizing of human systems: Container, significant Difference, and transforming Exchange. The path, rate, and outcomes of self-organizing processes are influenced by these three conditions, which are co-dependent such that the function of each of the conditions depends on the others in nonlinear interactions in the system. A change in any one of the conditions results in a change in the other two over time.

Also see the following:
In my next blog-entry I'll talk specifically about self-organizing teams. Some specific characteristics and/or results of self-organization that I'll be delving into more deeply in subsequent blog-entries are:
  • Swarm Behavior (a.k.a. Swarm Intelligence)
  • Collective Intelligence (a.k.a. Collective Mind)
  • Social Creativity
  • Group Coherence

Leave a Reply

What is 6 + 8 ?
Please leave these two fields as-is:
Please do this simple sum so I know you are human:)

There are 101 ways to approach anything.
To find the best way, sometimes you need expert help

What People Say

“Kelly revolutionised the way our digital department operated. A true advocate of agile principles, he quickly improved internal communication within our teams and our internal clients by aligning our business and creating a much enhanced sense of transparency in the decisions the business was making. Kelly also introduced a higher sense of empowerment to the development teams...”


“Kelly’s a leading program director with the ability to take charge from day one and keep strong momentum at both a program and project level driving prioritisation, resourcing and budgeting agendas. Kelly operates with an easy-going style and possesses a strong facilitation skill set. From my 5 months experience working with Kelly, I would recommend Kelly to program manage large scale, complex, cross company change programs both from a business and IT perspective.”


“Kelly is an extremely talented and visionary leader. As such he manages to inspire all around him to achieve their best. He is passionate about agile and has a wealth of experience to bring to bear in this area. If you're 'lucky' he might even tell you all about his agile blog. Above all this, Kelly is great fun to work with. He is always relaxed and never gets stressed - and trust me, he had plenty of opportunity here! If you get the chance to work with Kelly, don't pass it up.”


“Kelly is an Agile heavy-weight. He came in to assess my multi-million $ Agile development program which wasn’t delivering the right throughput. He interviewed most of the team and made some key recommendations that, when implemented, showed immediate results. I couldn’t ask for more than that except he’s a really nice guy as well.”


“Kelly and I worked together on a very large project trying to secure a new Insurer client. Kelly had fantastic commercial awareness as well as his technical expertise. Without him I would never had secured this client so I owe a lot to him. He is also a really great guy!”


“Kelly came to the department and has really made a huge impact on how the department communicates, collaborates and generally gets things done. We were already developing in an agile way, but Kelly has brought us even more into alignment with agile and scrum best practices, being eager to share information and willing to work with us to change our processes rather than dictate how things must be done. He is highly knowledgable about agile development (as his active blog proves) but his blog won't show what a friendly and knowledgeable guy he is. I highly recommend Kelly to anyone looking for a CTO or a seminar on agile/scrum practices - you won't be disappointed!”


“Kelly was a great colleague to work with - highly competent, trustworthy and generally a nice bloke.”


“Kelly was engaged as a Program Director on a complex business and technology transformation program for Suncorp Commercial Insurance. Kelly drew on his key capabilities and depth of experience to bring together disparate parties in a harmonised way, ensuring the initiate and concept phases of the program were understood and well formulated. Excellent outcome in a very short time frame. ”


“I worked with Kelly on many projects at IPC and I was always impressed with his approach to all of them, always ensuring the most commercially viable route was taken. He is great at managing relationships and it was always a pleasure working with him.”


“I worked with Kelly whilst at Thoughtworks and found him to be a most inspiring individual, his common-sense approach coupled with a deep understanding of Agile and business makes him an invaluable asset to any organisation. I can't recommend Kelly enough.”


“Kelly was a brilliant CTO and a great support to me in the time we worked together. I owe Kelly a great deal in terms of direction and how to get things done under sometimes difficult circumstances. Thanks Kelly.”