Where Agile Goes Wrong

Agile methodology has become the cornerstone of many large digital-first transformations. Enterprises have adopted this approach in new technology projects, software maintenance, legacy migration, and innovation hubs.

But is agile really up to the job? Not always.

Chief information officers often find that agile doesn’t scale or even hinders efforts to break through the digital ceiling. Infosys surveyed 1,000 business and IT executives in December 2020 to understand the benefits, challenges, and opportunities of practicing agile. The responses underlined three common, interrelated issues that keep this methodology from becoming a transformational tool.

First, many agile teams overemphasize the rituals rather than embracing a new mindset. Just 14% of business executives ranked Agile ways of working as a top investment priority for 2021.

Second, the role of product owner is critical for Agile to deliver business impact. 46% of CIOs and 74% of business CXOs prioritize investment in product management capabilities, the highest investment for 2021 by a high margin. But finding strong product owners with the right outlook and mix of skills is incredibly tough.

Finally, to be truly effective, the entire organization needs to be aligned with the agile mindset and its methods. Our research found that adopting this mindset across a wider number of teams leads to greater agile success, (including superior business and IT outcomes and greater growth amongst peers), and its absence makes Agile teams less effective. In fact, an holistic agile mindset is perhaps the most fundamental of the three problems, since enterprises that solve this challenge will find it easier to overcome the first two barriers.

Holistic agile requires practices, principles and mindset to extend beyond the technology department

A holistic agile approach requires that the practices, principles, and mindset extend far beyond the technology division. Product sales, marketing, finance, operations, and human resources (HR) all should work in sync within a clear agile framework. And firms are taking note. A majority of firms have scaled Agile outside the technology division, according to our research. (Figure 1)

Figure 1. Agile is being scaled across the business

Source: Infosys

Of course, this is a predictable trend. A key tenet of agile is to involve end users and business stakeholders in the creation of new products or systems.

On closer examination, however, we found that agile projects often use a proxy or delegated subordinate for a business or an end user. For instance, they may use customer or end-user interviews rather than having a customer group present throughout. Or they may have a junior member of a business division, with great enthusiasm but no power, involved daily.

In fact, there is some anecdotal evidence that technology-led agile projects actively diminish the role or value of business and user input in order to accelerate timelines. This is usually a response to a lack of engagement or focus from the business side. It’s true that it can be difficult to maintain the attention of senior business stakeholders or end users, not least because of the start-stop-pivot progress that is often inherent in an agile project.

What’s at stake?

Getting this methodology to scale across an organization is likely the biggest challenge that large, traditional enterprises face today when implementing agile. But it’s one that is crucial to their survival and ongoing competitiveness — a 20th-century operating model is not going to succeed in the 21st century.

The darlings of this new era — Google, Amazon, and Netflix — are inherently agile in their operational structure. They and their digital-first peers are the ones that set the standards for competitiveness in product innovation, customer responsiveness, and shareholder value. In our research, we found a cohort of companies that fit this description (we termed them “Sprinters”); they have been practicing Agile a shorter time than others, though have been able to scale it with much greater business and IT success.

Amazon is perhaps the best example of doing Agile at scale. Each of its multiple product lines has teams that combine web developers, product owners, and pricing and merchandising specialists. These teams have significant autonomy, including the ability to change things in their product lines. There also are centralized guidelines and services (most famously the AWS cloud platform), but each team has the capability to sense and respond to environmental changes largely independently.

Compare this with a typical large enterprise established in the 20th century. That period’s mantra of industrialize, standardize, and optimize has led to organizations where many core services are centralized and disconnected from product owners. Finance, technology, HR, marketing, manufacturing, and engineering often sit in their own high castles responding to a queue of requests from the other castles.

This structure is efficient for a market environment that is stable and predictable. However, today’s market is anything but.

The long road to holistic agile

Changing the fundamental structure and mindset of an enterprise is not going to happen overnight. Agile needs to prove itself as a method that does more than speed up and improve the quality of technology projects; it must link directly to business outcomes and sustainable growth. And it will take more than one or two great projects to drive a sustained holistic organizational change.

Agile is more than just measuring speed, and must be tied directly to business outcomes and growth

Transformation leaders must build a structure that consistently measures progress against business outcomes and incorporates lessons that test and evolve practices continuously. But of course, this pushes against another perennial problem faced by large enterprises: Due to the complexity, age, and poor connectivity of processes and underlying systems, it is difficult to gain real-time insights into business outcomes. This can make it impossible for an agile project to get rapid feedback on whether it is delivering better customer conversion, revenue growth, improved cash flow, or employee utilization.

Improved automation, data analytics, and dashboarding can help. But it’s clear that in order to truly measure projects by business outcomes, companies need upfront investment in measuring and monitoring.

Yet, as with any transformation, the biggest mountain to climb is that of culture change. In our research, we found that culture and mindset is where most get stuck on their journey, listing legacy thinking and culture as the top two most prevalent challenges. (Figure 2)

Figure 2. Culture and mindset continues to be a challenge for firms

Source: Infosys

Further, clients report that even after many years of investing in agile methodologies, adoption and understanding across the organization are patchy.

This culture problem is often the result of inconsistent investment or training and is sometimes caused by the need to integrate different agile practices after mergers or acquisitions. Largely, the problem resides with the diversity of interpretations that have resulted from a complex evolution and fragmentation of agile principles and practices.

We believe that to succeed in a cultural transformation, a fine balance has to be struck between adherence to an agile toolset and adoption of the mindset required to work in agile (Figure 3). We use the term toolset to describe those that adhere closely to the recognized tools and practices associated with Agile. Mindset is the cultural and organizational traits and values that are most aligned with the Agile Manifesto.

Balancing adherence to an agile toolset and the right mindset is critical to agile success

This mindset can include business structures and operating models that support better agile project success. For instance, businesses should organize multidisciplinary teams around products rather than around disciplines such as marketing, sales, HR, and digital technology. In fact, organizing teams around products is a lever that improves both business performance (up to 14 basis points) and IT performance (up to 8 basis points), according to our analysis.

Figure 3. A framework to understand the balance between toolset and mindset in agile

Source: Infosys

In agile, practitioners are the ones who typically follow methodologies rigorously, sometimes to the extent that they alienate other members of their organization. The strength of practitioners, however, is that they enable consistent methods to be scaled across an organization. But they may end up focusing too much on tools and rituals over outcomes and value.

At the other extreme we have “exemplars,” who work in a naturally agile way through multidisciplinary teams that are close to the customer and business outcome. These are typically seen in small businesses, startups, creative agencies, or other organizations where value is placed on speed, innovation, and empathy with the end user. In these organizations, hierarchies tend to be flat and structures and roles are flexible. However, exemplars can often lose sight of the rigor needed to scale, measure, or improve their practices.

To achieve true transformation, leaders must navigate a careful path that balances the best of both archetypes, minimizing each set of drawbacks. Ideally, an organization would have both structural flexibility and the rigor of processes, practices and tools [with strong outcome orientation] in order to achieve holistic agile at scale.

Starting down the road

Every organization’s path to holistic agile needs to be designed for its specific circumstances. But there are some guiding principles that can set enterprises on the right course.

Apply agile to the entire product life cycle

The practices, tools, and principles of agile should not be restricted to the engineering or technology elements of a product. Everything from customer service to distribution, finance, and sourcing should be included in an agile structure to increase the opportunity for better business outcomes.

Integrate teams; don’t rely on proxies

Teams must consist of decision-makers from every discipline. Too often, proxies are either guided by more senior executives or, worse, committees of executives. Large enterprises need significant organizational shifts to create teams of decision-makers. But freeing up those decision-makers and speeding up decisions is the crux of any agile transformation effort.

Measure business outcomes, not story points

Too often, agile projects are measured by tactical key performance indicators such as speed, quality, throughput, and cost. Although almost half of those surveyed use customer journeys and identify associated value streams to a strong extent, the success of projects is largely tied to cost savings. By these measures, agile projects tend to do very well, but it is time to raise the bar.

Organizations must start measuring the success of projects on business upside (customer retention, conversion, growth, net promoter score, user feedback, revenue) and by efficiency delivered (employee and asset utilization, product returns, margin, cash flow, cost of sales).

Embrace transparency; reward accountability

Measuring business outcomes is difficult for two reasons. For most, the data — if it is being collected at all — is not readily available or accessible to everyone in an organization. This can be solved by investments in data integration, analytics, and visual methods for conveying complex information, processes, and data. Yet this only exacerbates the second difficulty, which is that increased transparency threatens to expose poor performance and bad decisions. This problem needs to be countered with cultural change techniques. Failure should not be stigmatized and accountability for mistakes should be rewarded.

Incremental budgeting aligned with business value

Project finance is too often designed to fit in with the quarterly reporting cycle that most businesses adhere to religiously. “Use it or lose it” mechanisms also encourage behaviors and forecasts that rarely have any grounding in a project’s business benefits. Project budgets must be funded incrementally and in line with specific, measurable business value. Budget holders must simultaneously take more risks with smaller, more frequent sums rather than the current situation, where large budgets tend to be spent after all risk has been removed. Our analysis found that incremental funding can increase both business and IT outcomes by as much as 7 basis points.

Train, train, train, and hire

Agile skills development tends to be patchy in many organizations — either limited to small cohorts of specialists or provided only on an ad hoc and intermittent basis. On one hand, it is helpful to have a small group of specialists, or high priests, who can own a business’s agile transformation and guide people and projects. However, this should be complemented with thorough, ongoing, businesswide training to bring the entire organization up to speed with the concept and practice of agile.

Continuously adapt; avoid rigid adherence

Agile is a set of principles, backed up by practices, frameworks, methods, and tools. There is no right way to do anything; focusing too much on the lore of agile leads to a loss of focus on outcomes. Also, as an organization matures and adopts more agile practices, new needs will emerge and new lessons will be learned. Any organization’s agile approach must be continuously adapting and evolving rather than being designed once and implemented.

We’re under no impression that any of the above is easy. But the world — and industry — are on the cusp of a new-era business style and structure. And central to this is agile methodology. Those that commit long term to a transformational investment in their culture, technology, and operating model will rise rapidly as the 21st century’s most successful live enterprises, able to respond and adapt in near real time to pressures on people, processes, and technology.

Download