A State Of Radical Complexity

When the banks collapsed in the last financial crisis, there was a phrase bandied about, that some were ‘too big to fail’. This week i’ve been pondering something of the opposite: are some organisations ‘too big to succeed’. I’m not thinking specifically of their headcount, their geographical spread, or the physical weight of their buildings, but rather more the radical complexity of their networks, the unknowable nature of their knowledge, and the sheer inertia of their formal hierarchies. Possibly good organisations, probably doing good work, but ultimately doomed to fail through their inability to understand their true dynamics.

The work around ‘The Landscape of Trust’ has helped me to understand the granular nature of social systems: the networks of trust, pride, and respect that are held outside the formal structure, but fully at the heart of the functioning social systems that we inhabit. Global organisations are made up of many of these granular structures: defined not by functional structures, not by formal team layouts and Org charts, but rather by bonds of strong social ties and connections. These are high functioning social units, that may, coincidentally, overlay formal units of organisation, but the capability is held within the social structure, or at least largely so.

The formal organisation may own the tooling, the technology, the physical space and the engineered product, but the social system owns the trust, the innovative thought, the creativity, and the momentum.

I guess that there are two states: one of complexity, and one of radical complexity. Perhaps a tipping point between that which is simply complicated, and that which is unknowably intertwined.

Even within known systems, people delude themselves as to the mechanisms by which they are effective: we believe that we know more people than we do, we believe that we are better connected than we are, and we hear confirmatory signals that we prioritise over weak voices of dissent from within these sub-optimised structures. There is great interest at the moment in ‘fake news’, but that is barely to scratch the surface: confirmation bias allows us to delude ourselves that we are more wise than is wise to believe, and probably lies behind much consensus decision making in large organisations.

Consumer psychology teaches us some of this: we tend to intuit a response, or a course of action, then socialise it and gather support. Our selective attention and confirmation bias tends to direct us to reinforce this initial view as we rationalise our action before the event, and even post hoc rationalise it to align with emergent observation.

Essentially we tend to believe in what we do, and continue to believe it, whilst failing to recognise that our belief is a matter of blind faith, not logical deduction. Organisations are remarkably emotional entities.

Of course, these features tend to work quite well: we muddle along, making some good decisions, and a few bad ones, we make general progress, which, although perhaps not optimised, can still cope with the odd bump in the road.

But at what scale?

Technology has allowed for the radical scaling of organisational effectiveness: no longer limited to branch networks and postal based admin systems, we can achieve effect at a scale and size never contemplated before. But it’s not limitless scale. There are limits to the connections that we can hold, the knowledge that we can glean, the ability of even the smartest individual to comprehend every aspect of a system.

In the Age of Domains, we professionalised, we solidified the pyramids of power and formalised the manifestation of organisation: we build the standardised organisational design and management principles, based upon visibility, clarity, and solidity. These domains had the benefit of being clear and visible, they were scalable, at least in terms of physical organisation: if what you wanted was to know where all the nuts and bolts were, then this structure worked well. Formal hierarchies, and domain based knowledge, allow us to quantify and understand organisations within that one dimension.

But in the Age of Engagement, the Social Age, much of the value of the organisation has migrated into the social structure: sense making networks, social filtering, social moderation, social capital and reputation based authority, agility, cognitive surplus, crowd sourcing and hacking, social movements, and so forth. The capability, our ability to innovate, to scale, the change, all this is intimately tied up in the social system.

Don’t get me wrong: the formal system still exists, and is still vital. Under a model of Dynamic Tension, we recognise both: the formal system holds all the visible assets, the hierarchy, and the control, and the social system holds the trust, the pride, and the momentum. We cannot afford for one to collapse the other: if the formal system triumphs, it looks safe but lacks agility, and if the social system triumphs, we have fun, but struggle to achieve effect at scale.

Organisations are excellent at scale, social systems are excellent at connection and sense making. We need both.

But they scale differently.

The mechanism of scale in formal organisations is to build wider spans of control, more functional units, more systems, more infrastructure, more oversight, more rules, more contracts, more visible structure.

The mechanism of scale in social systems is different: more tribes, rituals, customs, perhaps more belief, greater networked power, broader reputation, perhaps greater unrest, possibly fragmented coherence, speciation, specialism. I’m not altogether sure: the clearest examples we have are nations, broadly connected national entities that often present greater divergence than conformity, and often struggle to overcome political entropy.

Possibly politics is the answer: as we scale, we become more political, and politics (a convenient terms for structural differences in opinion) becomes a gradually more potent force.

In physical systems, friction builds as you speed up, leading to the reentry systems of rockets glowing red hot as they streak through the atmosphere. The same appears to be true in social systems, or at least in some social systems.

There are exceptions: social movements, which tend to be one dimensional, can gain consensus more easily. We can all get behind an opposition message of ‘we wish that this thing was better’, even if we cannot gain consensus on what the better thing should be.

In political systems, this is the difference between opposition voices and those that sit in power. It’s easier, in some ways, to hold opposition, because you can build a network of broad consensus against the ruling elite.

These political forces may form the friction of radical complexity.

As the formal system scales, it does not particularly build radical complexity: it’s complex, but a type of complexity that can be managed through technology. The emergence of geographical information systems allowed for better estate management, and new stock, picking, and inventory control technologies, the mechanisms of fleet management, these allow us to know ‘what’ is where, and ‘who’ can do ‘what’, at least within the formal system.

The social system, as it scales, builds political potential, friction within the systems. Just look at organised religion, another analogy for social systems at scale: they build, schism and fracture, sometimes failing. You don’t find many mithraists around these days.

Not all systems are infinitely scaleable, although the absolute limits of scale can be hard to measure, and indeed, the radically interconnected nature of the system may make measurement effectively impossible.

Physical properties we can measure: there is a limit to how high you can build a brick wall before the weight of bricks on the top crushes the weight of bricks at the bottom. It’s high, but not infinitely high. Similarly, to get a rocket into orbit, you need to overcome inertia and gravity, so you need a big rocket, but big rockets weigh a lot, so you need a bigger rocket to lift the weight of the rocket. That’s why, on the Apollo missions, the initial Saturn V rocket stack weighed thousands of tons, whilst the landing capsule that returned to earth weighed just hundreds of kilos. The massive complexity was required to achieve a seemingly small return. And as the system gets bigger, it requires relatively more complexity for respectively smaller returns.

Indeed to stick with a space analogy, we see the new Victorian disruptors achieving space flight for a small fraction of the cost of NASA, because they lack the established cost, complexity, oversight and organisation of the larger entity.

This is a feature of large organisations: the constraint that they face is often not imposed upon them by competitors or native market forces. It is, instead, grown from within. They engineer there own constraint.

As organisations grow, they build both their formal structure, and the political potential for stagnation. They become complex, and then radically complex.

We are not at the endgame in many cases: some techno-evangelists pin their hope on machine learning systems and artificial intelligence, harbouring a belief that these things will be able to measure and understand this new complexity. And to some extent, they will: machine learning algorithms will be able to map interaction and even knowledge flow within social networks, they will be able to map one dimension of connectivity and community. They will provide visibility of that which is currently hidden within the system. But only one layer.

The final feature of radical complexity concerns the layers: whilst there is only one formal system, the social system itself is multi layered, and complex in it’s own right.

To put it another way, there is only one Org chart, but there are many, many, overlapping, contextual, conflicting, and contrasted, social networks. Some are semi formal and visible, whilst many are private, unknown, internally complex, and invisible. We run a risk of having almost enough knowledge to be truly ignorant. Sure: formal technologies will permeate these networks, but one lesson we should have taken from the Age of Domains, with the emergence of measurement as a mechanisms, of control, was that we can only measure what we understand. We can measure on formal systems, but we may not even know that the other communities and spaces exist.

We can delude ourselves with ease that we have full understanding, but true understanding may forever, at least in functional terms, be beyond us. Short of modelling the whole of society, we will lack broad understanding of the multi layered social system, and even if we had enough hubris to try, the system is multi layered and contextual, so it flows around and under our feet.

Is there a tipping point? Are some organisations too large to simply be complex, but instead tip into radical complexity?

I would suggest, almost certainly, that this is the case. They have, after all, often grown by the mechanism of acquisition and merger, another name for colonisation and conquest. And nation stage are certainly, demonstrably, unsettled, impermanent, and radically complex, deeply political, and subject to entrenched political dissent.

A key feature of organisational complexity is the nature of organisations themselves: they are mechanisms for accreting system, process and control, with barely any capability for dis-engineering it once it’s purpose is served. I can think of no formal organisational system that is characterised by becoming simpler as it scales.

Perversely, some social systems do become simpler with scale: they find a clarity, and one that is occasionally demonstrated to amplify fast. Arguably a notion like the Arab Spring was such a scaling social effect.

Formal systems focus on control, and dampen out simplicity, whilst social systems, are inherently complex, but carry within them some potential for sense making and simplicity. Which is strangely perverse, but speaks to the forces of amplification, magnetism, and sheer unpredictability of social systems, as well as to the aforementioned forces of confirmation bias and socially moderated decision making.

What can be done about this?

Well, that’s a complex question in itself: the best bet i have is that of the Socially Dynamic Organisation. For me, the answer lies in reconciling the two parallel pillars of the organisation, achieving that dynamic tension between formal and social systems. But can that be done at infinite scale? Possibly, but only if we develop a new capability of unwinding complexity within the formal structure. If we are able to remove hard wired, formal complexity, then we may conceivably build social connectivity and dynamic ability within the new, connected system.

Are some organisations too big to fail? The evidence would be against that. Are some organisations too big to succeed? In their current form, be they government systems, global banks or pharmaceutical companies, emergent tech empires, or military structures, quite possibly, yes. Or at least ‘yes’, unless we are willing to consider radical new models of organisational design, and redistributed models of power and control.