Moving Away From Strict Hierarchies

By | AgileBusiness | No Comments
Hierarchy

There’s been a reasonable amount of focus over the last few years questioning the value of strict hierarchical structures in business and asking whether, particularly in the context of today’s digital-empowered and networked operating models, this traditional approach to organisation needs to change. 

It’s a good question to ask. Despite the fact that the context in which business operates has changed substantially due to the impact of digital technology, most businesses are still structured in ways that made more sense in an industrial age where control, efficiency, scale and minimisation of deviance were all important. But does that really make sense in a world that is increasingly characterised by horizontality, networks, data and value flows, systems thinking and platform business models?

The challenges inherent in hierarchy as an organising principle has already been challenged by some. A broad meta-study of 54 prior studies (covering analysis of over 13,000 teams) conducted by Lindred Greer, Bart de Jong, Maartje Schouten, and Jennifer Dannals at Stanford Graduate School of Business (‘Why and When Hierarchy Impacts Team Effectiveness: A Meta-Analytic Integration’) found that the net effect of hierarchy on performance was broadly negative. Whilst some expert-based hierarchies helped improve team performance, many others were dysfunctional.

As I wrote in my book, the digital era has brought with it a need for a level of continuous innovation, team and individual autonomy, customer-centric approaches and strategic and tactical adaptability and responsiveness that is simply not served well enough by strictly hierarchical organisation. Functional silos act against the ability to create joined-up, exceptional customer experiences. They hamper the ability to collaborate quickly. They get in the way of cross-functional design and innovation. They limit flexibility of job activity. It’s no accident that (often self-organising) small multi-disciplinary teams have become the engine of change in many forward-thinking companies. With near universal access to information, the old ideas of leadership characterised by the perception that all the answers and solutions exist at the top of the organisation and flow down, have become grossly outdated. 

Yet there are challenges at the other end of the organisation design spectrum too. Some brave businesses have taken a radical approach to the adoption of flatter structures using methodologies like Holacracy right across the company. Yet in spite of Holacracy being around for over a decade there are still very few examples of it being applied successfully at scale.

But what if there was a way to successfully balance the benefits that can accrue from hierarchy (efficiency, clarity of authority, concentration of specialist expertise, clear lines of communication, simplified career path) with those of a more fluid, evolving structure (agility, adaptiveness, cross-functional collaboration and innovation, speed of delivery)? What if we challenged these long-standing orthodoxies about the optimal way to organise our teams and reinvented organisation design around a more nuanced understanding of where hierarchy is beneficial, and where it is not? What if we could design an organisation that could successfully balance these extremes, adeptly manage the interplay between them, and bring a new level of fluidity to structures that enabled far greater organisational agility?

I believe that this is very possible. Yet my contention is that few businesses seem to be thinking big enough about how we rewire structures for the world in which we now find ourselves. Traditional orthodoxies need to be challenged, but what does the new normal look like?

That’s what I’m interested in exploring over the next few months both here and over on my personal blog. As always, feedback, thoughts and ideas will be welcomed.

For more like this, order your copy of Building the Agile Business Through Digital Transformation, or you can join our community to access exclusive content related to the book.

Why Corporate Innovation is so Hard

By | AgileBusiness, Digital Disruption, Disruptive Innovation, Organisational Structure | No Comments

I loved this piece by Tim Harford in the FT about why corporate innovation fails not least because it emphasises one of the more under-acknowledged challenges with generating disruptive innovations in large organisations.

Harford references J F C Fuller’s ‘Plan 1919’ from the First World War, a pioneering and ambitious strategy to use new British tanks to roll over the German trenches and strike a decisive sledgehammer blow to the German army that would end the war. Fuller, Chief Staff Officer of the nascent Tank Corps, originated a plan to amass 5,000 heavy and medium British tanks, 3,000 of which would be used to penetrate German defences along a 90-mile front supported from the air. 800 faster-moving medium tanks would then proceed to attack the German Army’s string of headquarters miles behind the trenches to disrupt the command structures. A further 1,200 medium tanks, supported by artillery, airpower, cavalry and truck-mounted infantry would then move rapidly to penetrate far behind enemy lines.

Fuller’s lightning thrust plan was revolutionary. Tanks had until then only been used to open up gaps in the enemy trenches through which foot infantry could advance a few miles. But Fuller was proposing a new form of mechanised warfare that could end the attritional stalemate of trench warfare and focus on disorganising the enemy. He wrote: “Tactical success in war is generally gained by pitting an organized force against a disorganized one.”

Fuller’s biographer Brian Holden Reid called Plan 1919 “the most famous unused plan in military history” and yet as Harford notes, Fuller had actually created an entirely new military strategy that would be studied by the Germans and implemented to devastating affect in 1940. Fuller had in fact invented Blitzkreig.

The great irony was that after Fuller’s plan failed to see the light of day in the First World War, many nations still neglected to see the value in his strategy and believed that tanks should be used in small pockets to support infantry. The Army even went as far as stopping the publication of Fuller’s book for several years yet Heinz Guderian, the mastermind behind Hitler’s blitzkrieg, still managed to read Fuller’s work after the war and used it to great impact.

The story is an excellent analogy for why organisations so often look at the new through the lens of the old and ignore the kind of ideas and concepts that can be truly transformational, even if those concepts are originated inside their own organisations (Xerox Parc’s personal computer with mouse and graphical user interface, Steven Sasson’s first digital camera for Kodak, Sony’s Memory Stick Walkman, the IBM Simon, the first touchscreen phone). In his piece about why companies squander great ideas, Harford quotes Joshua Gans, author of The Disruption Dilemma and economist at the Rotman School of Management:

“Disruption describes what happens when firms fail because they keep making the kinds of choices that made them successful.”

In other words, companies get stuck in the strategies and thinking that have made them successful before.

And perhaps there is an under-acknowledged reason why this happens. Harford’s article references a 1990 paper by Rebecca Henderson and Kim Clark (Architectural Innovation: The Reconfiguration of Existing Product Technologies and The Failure of Established Firms) in which the authors distinguish between the components of a product and the way that they are integrated into the system, setting out four types of product innovation:

  • Incremental innovation: this may strengthen the core components of the product but it also maintains the existing linkages between them (an example would be improving the performance of a car component like a driveshaft without impacting the way in which the car is put together)
  • Modular innovation: which may change the fundamental technology of the component but still doesn’t change the way in which the system links together (like an automatic transmission)
  • Architectural innovation: this may change the design but whilst the components may not change significantly the way in which they link together does (like front-wheel drive transmissions)
  • Radical innovation: which is the most extreme, and involves changing both the technology of the components and also the way in which they link together (electric vehicles for example)

Henderson and Clark make the case that ‘architectural’ and ‘radical’ innovation can more fundamentally challenge the existing organisational structure and processes which makes it more difficult for incumbents to respond:

‘…architectural innovations destroy the usefulness of the architectural knowledge of established firms, and…since architectural knowledge tends to become embedded in the structure and information-processing procedures of established organisations, this destruction is difficult for firms to recognise and hard to correct.’

Incremental and modular innovation are less challenging to established structures since the system doesn’t fundamentally change. Radical innovation establishes a new dominant design and a new set of design concepts embodied in components that are linked together in a new architecture. Architectural innovation involves the reconfiguration of an established system, linking together existing components in new ways. So it can be harder to perceive and make sense of since it involves similar component parts to the problem that are put together in new ways and so have very different relationships with each other. When an organisational structure and information flow has grown up around the old system, it becomes very difficult for the company to respond in suitable ways. The structure gets in the way.

Henderson uses the example of IBM to demonstrate how a company can respond well even to radical innovation if it fits the structure that already exists. IBM successfully dealt with significant developments such as the semiconductor, the integrated circuit, the hard drive, and the shift to mainframe computing since it was not dissimilar in structure to producing mechanical tabulating machines. Yet when it came to the PC revolution IBM’s initial success came only from going against existing strengths and the advantages of its extant structure and eventually internal politics rose up and the PC division struggled to cope and was sold off. In the First World War, says Harford, the invention of the tank did not fit existing systems and structures for fighting the war and so the real potential to use it in a very decisive way was missed.

Reading Harford’s piece reminded me of Noah Brier’s write up of Conway’s Law. In 1968 programmer Melvin Conway wrote that:

‘organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations.’

In the context of software for example, the designers of two different component modules need to communicate with each other in order to ensure that the modules work effectively together. So the interface structure of software naturally needs to show congruence with the social structure of the organisation that created it.  This similarity between organizations and designs he called homomorphism, noting that ‘the very act of organizing a design team means that certain design decisions have already been made, explicitly or otherwise’.

American Computer Scientist Fred Brooks noted the wider application of this concept in management theory, observing in his early book on software engineering (The Mythical Man-Month):

‘Because the design that occurs first is almost never the best possible, the prevailing system concept may need to change. Therefore, flexibility of organization is important to effective design.’

The key points are that when we’ve started to design any system the choices that we’ve already made can fundamentally affect the final output, that organisation structure and design/architecture are intrinsically linked with the former impacting and constraining the latter (rather than the other way around), and that organisational flexibility is required to organise quickly to fix mistakes and adapt.

Noah Brier builds from this idea, referencing the work of Harvard Business School professor Carliss Baldwin whose work around the mirroring hypothesis shows that mirroring between technical dependencies and organisational ties becomes evident as a way of preserving scarce cognitive resources in solving complex problems:

‘People charged with implementing complex projects or processes are inevitably faced with interdependencies that create technical problems and conflicts in real time. They must arrive at solutions that take account of the technical constraints; hence, they must communicate with one another and cooperate to solve their problems. Communication channels, collocation, and employment relations are organizational ties that support communication and cooperation between individuals, and thus, we should expect to see a very close relationship—technically a homomorphism—between a network graph of technical dependencies within a complex system and network graphs of organizational ties showing communication channels, collocation, and employment relations.’

In other words mirroring makes outputs and new products easier to understand because it aligns nicely to current ways of organising.

It may be that in some instances (particularly in the context of incremental or modular innovation) mirroring will result in effective innovations and outputs. But with more radical, architectural innovation that challenges the architectural knowledge, information flows and structure of an organisation, mirroring can be counter-productive.

Rapidly changing contexts or emergent capabilities characterised more by unknown unknowns do not suit rigid systems and ways of working that mirror existing ways of doing things. Instead we need to work back from the system design we need into a structure that can reflect it. Rigid and deeply hierarchical structures may be good at delivering incremental and modular innovation and supporting optimisation and efficiency, but do not lend themselves to the flexibility required for adaptive, emergent problem-solving. These types of challenges require small, cross-functional teams that can move quickly, unconstrained by ingrained systems, architecture and thinking.

We need to fundamentally redesign our organisations to reflect both of these needs.

For more like this, order your copy of Building the Agile Business Through Digital Transformation, or you can join our community to access exclusive content related to the book.

Why We Reject New Technology

By | AgileBusiness, Digital Disruption, Disruptive Innovation | No Comments

Thermometer

There’s a great account in the Scientific American focusing on why new technologies that can make our jobs easier are somehow often rejected, using the adoption of the thermometer as an exemplar.

At the end of the sixteenth century Galileo Galilei invented the first device that could measure temperature variations – a rudimentary water thermometer. Around 120 years later Gabriel Fahrenheit came up with the first modern mercury thermometer. The Dutch physician Herman Boerhaave thought that the device had great potential and proposed that measurements using a thermometer could be used for diagnosis and to improve treatment.

Yet despite its evident utility it took over hundred years for use of the thermometer and the discipline of thermometry to become widespread. Prior to the mercury thermometer, Doctors would largely use touch to determine whether the patient had a high temperature or was suffering from a fever. This qualitative approach was regarded as being able to capture a rich amount of information, more in depth than any tool could generate, and for many years was seen as a superior approach to using thermometry.

In spite of the prevailing inertia to adopting this new technology, a group of researchers persisted in attempting to turn the relatively idiosyncratic opinions and descriptions from Doctors into reproducible laws but it was not until 1851 when a breakthrough happened. In a transformation piece of work (published as “On the Temperature in Diseases: a manual of medical thermometry”) Carl Reinhold Wunderlich recorded temperatures in 100,000 patient cases, and successfully established not only that the average human body temperature was 37 degrees, but also that a variation of one degree above this constituted a fever, which meant that the course of illness could be better predicted than by touch alone.

Thermometry represented a giant leap towards modern medical practice. Patient expectation changed and by the 1880s it was considered medical incompetence not to use a thermometer. But why did it take so long to become widely adopted practice? The original thermometers were large, cumbersome devices and the tool developed over many iterations but this still doesn’t explain its slow advance.

The Scientific American article notes how easy it is to reject technology that we don’t understand, or technology whose successes we’ve had nothing to do with. Perhaps our fear is that in its success, new technology will detract from our own utility. More likely, slow adoption of technology comes down to what Andy Grove (of Intel) used to call the ’10x’ rule, referring to the idea that a product must be at least ten times better in order to overcome barriers to adoption and switching costs because people tend to underestimate the advantages of a new technology by a factor of 3 while simultaneously overestimating the disadvantages of giving up old technology by a factor of 3.

But as the piece also goes on to point out, the subtlety is actually in how we combine the best of the old with the best of the new – describing how a children’s hospital in Philadelphia had used quantitative algorithms to identify particularly dangerous fevers. The algorithms proved better at picking out serious infections than the judgement of an experienced doctor. But when the two were combined it outperformed either in isolation:

‘It’s true that a doctor’s eyes and hands are slower, less precise, and more biased than modern machines and algorithms. But these technologies can count only what they have been programmed to count: human perception is not so constrained.’

Similarly, at the 2016 International Symposium of Biomedical Imaging in Prague, a Harvard team developed an AI that could detect cancer cells amongst breast tissue cells with 92 percent accuracy, almost as good as the trained pathologists who could pick out 96 percent of the biopsy samples with cancer cells. Yet when artificial and human intelligence were combined 99.5 percent of cancerous biopsies were identified.

Technological change rarely means forgetting all that we know. More often it is helpful to frame it in that thought of combining the best of the old with the best of the new. Perhaps the key lesson here is that a fixed mindset (one where you believe that success happens at the expense of someone or something else) does not help the adoption of new technologies. When we can see the bigger picture, and adapt existing knowledge and skills to combine the best of the old with the best of the new, we always progress more. Growth mindsets win.

For more like this, order your copy of Building the Agile Business Through Digital Transformation, or you can join our community to access exclusive content related to the book.

Image source

Fixed and Growth Leadership

By | AgileBusiness, Leadership | No Comments

Stanford Psychologist Carol Dweck’s concept of fixed and growth mindsets is just a brilliant way of expressing some of the leadership attributes that are most essential in modern, agile businesses. We touch on it briefly in the book, but it’s worth exploring further.

There are, says Carol (in ‘Mindset: The New Psychology of Success’), key differences in how we view our personality. A ‘fixed’ mindset is founded in the view that your intelligence, character and creative ability are static and so cannot change in meaningful ways. Success is an affirmation of those inherent attributes which compare favourably to other fixed standards. Avoiding failure and striving for success become critical ways to maintain the feeling of being skilled, smart or accomplished, and so pursued at all costs.

In contrast a ‘growth’ mindset relishes challenges and sees them as an opportunity to learn, and failure as an opportunity to grow and improve.

The context of Dweck’s research is mainly focused on children, students and how they learn but I think there are strong parallels to defining what successful organisational cultures and leadership look like in the modern world.

In Dweck’s research (summarised in the video above), children with fixed and growth mindsets demonstrated very different approaches and goals. Those with the latter recognised the need for effort, work and practice in order to improve. Their goal was to learn at all times and at all costs. Conversely those students with fixed mindsets were afraid to try new things in case it made them look dumb. Their goal was to look smart at all costs and to avoid tasks that might show deficiency.

More than this, there was a key dynamic in the relationship between ability and effort. Those with fixed mindsets believed that if you have the inherent ability then you don’t need to put in the real effort. Any setbacks simply reveal their limitations and so they try to avoid deficiencies or failure at all costs, and have no way to effectively handle real difficulty. Growth mindset children however, believe in improvement through effort and practice, and relish hard challenges as an opportunity to learn.

Dweck says that this difference is a fundamental reason behind so many students not reaching their full potential. With the universal need for continuous improvement, and more than ever for rapid and constant learning, I believe that these different mindsets and cultures are a fundamental reason why so many organisations fail to reach their full potential. An organisational mindset that rewards leaders for looking smart and never admitting when they don’t know the answer or have made a mistake, is a culture that will not support learning. A business that is too focused on outputs to the detriment of how those results will be achieved is one that will struggle to find new and potentially exceptional alternatives.

Dweck has shown that these mindsets can be transmitted through words and actions. If we are to embed a culture of continuous learning in an organisation we need to be very attuned to the behaviours that we support and those that we discourage. With children, Dweck has shown that praising intelligence rather than effort encourages a fixed mindset from a very early age. It can turn students off from learning. Instead, praising the process, the strategies, or the effort leads to a desire to persist, to experiment, and to learn at all times. We need to take the same approaches with our teams.

These behaviours are very subtle and yet the value system that we create within organisations are hugely powerful determinants of success and failure. In the modern world we need to regard every initiative as an opportunity to learn and we need to recognise the importance of behaviours that support a growth mindset and culture.

For more like this, order your copy of Building the Agile Business Through Digital Transformation, or you can join our community to access exclusive content related to the book.