Making Spotify-Inspired Agile Work

Scaling Agile is a big investment for information technology (IT) practitioners this year. According to Infosys Knowledge Institute research, 38% said that Agile scaling frameworks were one of two top priorities, compared with 29% for cloud. One of those scaling patterns is Spotify-inspired Agile. Both smaller digital natives and larger, slower incumbents are copying the music streaming behemoth’s lean Agile model in a bid to achieve a quantum jump in both business and engineering outcomes.

To put Spotify-inspired Agile to work, these companies have made innovation a key part of their DNA. They develop their capabilities so that business and technology act as a single organism, able to quickly adapt to external market influences. This means changing the way interactions evolve and updating their technology stack with DevOps and microservices. But more than that, it requires that companies fundamentally change their structure — reorganizing employees into autonomous units such as squads, chapters, and tribes — that deliver capabilities in line with the wider organization.

But there is a problem. A new operating model cannot easily be retrofitted to existing organizational norms. Businesses that apply Spotify-inspired Agile as is — without understanding the true philosophy that defines it — will never achieve the return on investment they expect from Agile initiatives. Companies need a deeper appreciation for what the Spotify-inspired Agile pattern is about and how it can be adapted and customized on a case-by-case basis.

Care must be taken to understand the true philosophy of Spotify-inspired Agile in order to achieve significant value

We begin this paper by discussing the Spotify-inspired Agile philosophy across six elements, and then we recommend a method for applying it at scale for large businesses. These six elements have been deduced from our work implementing Spotify-inspired Agile. Typically, any Spotify transformation will involve maturing these six elements in turn.

Element 1: Autonomous structures and cross-pollination

Spotify-inspired Agile is lean and can quickly scale to meet every product development life cycle. Lean startup groupings bring the organization together under a common mission — creating an autonomous culture with fast decision-making, competence building, and knowledge transfer.¹

Figure 1. Guilds, tribes, chapters, and squads are key to Spotify-inspired Agile

Source: Infosys

The structures in Spotify-inspired Agile consist of the following:

• Squads are vertical groupings of people working toward a common mission. A squad is cross-functional, self-organized, and committed.
• Tribes are groups of interrelated squads focused on a specific common objective. Each tribe enables discovery, build, and support functions — aligning with work to be carried out through the use of groups known as chapters.
• Chapters are horizontal groupings of people within a tribe who possess a similar set of skills, such as developers, quality assurance staff, and Agile coaches. Each chapter’s mission is to develop competence and knowledge sharing within the tribe.
• Guilds are made up of people within each tribe and act as a special-interest community for overarching business strategy. Anyone from a tribe can join a guild at any time.

Each of these structures defines parameters for reaching the state of nirvana referred to as “definition of awesomeness.” These structures mature through a scorecard-based inspection and evolutionary approach for adaptation.

The tangible benefits of this structure include:

• These structures create a community-based model within the organization, thereby promoting transparency, alignment, and better employee engagement.
• The squads, chapters, and tribes operate like lean startups — with the ability to self-manage, make quick decisions, and enable rapid experimentation.
• Squads and chapters overlap in a matrix that increases the fluidity of decision-making and develops agility.

Element 2: Embracing a lean startup culture

The key to Spotify-inspired Agile is the creation of a lean startup culture. To stay ahead of the curve, continuous experimentation is used in a “conceive, implement, learn, adapt” model of engagement. Failures are viewed as learning sessions, and customer feedback is used to either preserve the status quo or to pivot quickly to improved product design and operation. Two practices have become standard in lean culture and are used by Spotify-inspired Agile in their operating model.

The DIBBs (data, insights, beliefs, and bets) model uses data from market trends to develop insights and create a “belief” hypothesis that will be proved or disproved by each “bet” the team makes. With new data, DIBBs may change and lead to different features.²

In a lean startup culture, continuous experimentation is used in a “conceive, implement, learn & adapt” model of engagement

Think it, build it, ship it, and tweak it are the four stages used by Spotify-inspired Agile to develop features:³

• Think it: Is the feature worth building? Will it result in tangible value to the customer? Design thinking is used at this stage to arrive at a portfolio of features that will be built in the next stage.
• Build it: Here, the impact is more important than product velocity. With that in mind, a set of features are built into a minimum viable product (MVP) that is good enough to release to the market.
• Ship it: To test the product, the MVP is shipped to a small number of users, who will then validate or disprove the original hypothesis.
• Tweak it: Changes are made to the product by observing and measuring customer adoption. Actionable metrics are important.

The Spotify-inspired Agile culture creates an enterprise that ideates and delivers quickly based on market trends. It also works closely with the customer and can recover from setbacks quickly by understanding what customers want through customer adoption and usage trends. Further, the enterprise acts like a live organism, continuously innovating and pivoting quickly in case product impacts vary greatly from the original hypothesis.

Element 3: Adopting mature agile practices

With Spotify-inspired Agile, high-functioning squads build valuable and high-quality products quickly. However, doing Agile well takes time. This requires business mastery and engineering excellence, such as the ability to use extreme programming practices. Agile coaches are also needed to enhance technical, Agile, and business domain processes while helping teams transform quickly.⁴

Mature Agile practices achieve:

• Self-managed and self-organized tribes, squads, chapters, and guilds.
• High release frequency in small iterations.
• High-quality coding and defect-free deliverables.
• The ability to plan and address feature development, defects, technical debt, and risk in a more efficient way.

Element 4: Purpose-driven leadership with minimum viable bureaucracy (MVB)

Spotify-inspired Agile leaders follow a set of core tenets that accelerate decision-making, leading to less hierarchy, greater autonomy and innovation, and faster product delivery. Leaders share authority, making tribes, squads, chapters, and guilds responsible for their work. This moves away from traditional management styles prevalent in the past century and gives the whole organization a higher sense of purpose. In this paradigm, leaders should foster team partnerships focused on shared values and purpose. This in turn will enable the right support structures for decision-making.

Below are some benefits of this type of leadership:

• Employees work for the good of the team, satisfying their need for a higher purpose.
• Authority is delegated, leading to higher levels of motivation and MVB.
• Decision-making is quicker, and teams innovate rather than wait for approval.
• The model creates a culture where the leader serves each team rather than micromanages work.

Element 5: Speed facilitated through technology

Mature Agile teams are one thing. But if they don’t have the right technology, projects will stall. With Spotify-inspired Agile, DevOps is a key lever that helps teams deliver based on market trends. Here, short innovation cycles lead to a reduced time to market. Optimized DevOps includes Agile life cycle management, automation, continuous integration, continuous deployment, and cloud infrastructure. Other tools necessary for mature Spotify-inspired Agile DevOps include road-mapping, work item tracking, source control, configuration management, test automation and deployment, and post-production support tracking.

By integrating these tools at all stages — including ideation, creation, deployment, and operation — lead times decrease and product development flow is smooth and reliable. In fact, better technology also:

• Makes configuration and source control more efficient.
• Allows continuous integration.
• Enables faster release rates.
• Makes fault-finding faster.
• Increases mean time to repair.
• Enables improvements through use of tool dashboards and a consolidated view of the product flow cycle.

Element 6: Aligning teams through lean governance

Giving tribes, chapters, and squads so much autonomy shouldn’t be a one-time change. In fact, continuous iteration is needed to make sure the Spotify-inspired Agile pattern works properly and to ensure work is aligned with an organization’s vision. To help, practitioners have developed the Spotify-inspired Agile rhythm model (Figure 2).⁵

Figure 2. The Spotify-inspired Agile rhythm model

In Spotify-inspired Agile, “North Star” goals are two-year targets set by the management. Strategy teams create bets, which are aligned with the goals and scheduled for completion in six to 12 months. Further, each company bet is broken up into bets at the tribe level, with a squad or a chapter backlog assigned. This aligns their work with the North Star goal.

In this paradigm, not only is the rhythm important, but a lean governance model also is necessary. That includes streamlining planning, investment tracking, and product development. This governance model is made up of product owners, tribe leads, chapter leads, and Agile coaches. Together, they form an in-tribe steering committee that governs and funds different tribe bets. These committees meet every month to review and plan. Further, members from each committee are part of an executive steering group chaired by senior managers to monitor and approve each tribe’s plans. These plans are themselves based on company bets aligned with the North Star goal.

The benefits of this approach mean:

• There is alignment in squad or chapter ways of working with the overall organizational bets.
• There is an alignment of tribe and company bets.
• All bets are properly conceived based on a business case.
• All backlog items are ranked based on tribe-level bets.
• Each bet is based on a hypothesis that is derived from understanding the market at a single moment.
• Bets can change based on market impact analysis.

Challenges to the Spotify-inspired Agile philosophy

However, doing Spotify-inspired Agile well is not a copy-and-fit exercise. Different industries will have different constraints, and the as-is state of the firm, including how much product work is currently being done and the scale of business misalignment, will have an impact on implementation (Table 1).

Table 1. Challenges to adopting Spotify-inspired Agile

Making Spotify-inspired Agile work

Companies can gain insights into how to customize Spotify in large, often nebulous organizations by breaking up the transformation into discrete stages. See our paper “Hybrid Scaling of Agile: The Need of the Hour” for a complete discussion on how to customize any Agile scaling pattern to the needs of the business through what is known as “hybrid scaling of Agile.” Using this approach, an evolutionary model has been derived for making Spotify-inspired Agile work in many organizations.

An important part of this evolutionary approach is the formation of short-term goals for the business. These goals are aligned with an overarching change management vision rather than going for an all-out change process. This means that teams are empowered in stages and change can be measured discretely after successive iterations.

In this way, Spotify-inspired Agile implementation is tied to pivot themes — with each theme related to the six elements of Spotify discussed earlier. Each element is then “matured” in the organization through a set of short iterations, MVCs, wherein each Spotify element is adapted and customized to the specific organizational road map and strategy. In this paradigm, when Spotify elements don’t fit the organization, suitable customizations can be applied and tested, after which lessons are learned and new knowledge is applied to the next MVC (Figure 3).

Figure 3. An evolutionary model for tailoring Spotify implementation

Source: Infosys

The model has the following stages:

• Enterprise assessment to define vision for change — This first step focuses on understanding the problem an organization is trying to solve and defining the vision of transformation. It does this by studying the as-is state across market, business, and product development agility. The results help a firm realize a vision and need for implementing Spotify. This vision is then communicated to all levels of the organization, and change agents who will help implement this vision are identified.
• Fitment analysis — How change can be implemented is then mapped out. This involves choosing the elements of Spotify to concentrate on and identify a pilot project. The outcome here is arriving at a set of Spotify elements that can be implemented as is and those elements that can be implemented through the necessary adaptations.
• Defining the theme and MVC — Once the elements are determined, they are tied to a theme. This theme then acts as the foundation for changing and maturing different Spotify elements. Run as multiple iterations, these MVCs change the scope of each element, and in the case that an element can’t be applied as is, suitable customizations are made. Also, success is defined when each theme is created and measured at the end of each MVC iteration to ascertain maturity.
• Inspect and adapt — To fulfill each theme, certain criteria for success must be met. If they are, work pivots to the next theme. And if this is not successful, another MVC is carried out to achieve further theme maturity.
• Improve business agility — Once MVCs have been run and each theme matured to a satisfactory conclusion, the flow lead time is measured for each tribe. Based on this measure, the Spotify elements that need to be tuned or implemented next are worked out — the outcome thus bringing us full circle to where the theme for the next iteration is mapped out. If all elements are implemented, the procedure loops out and results in the target operating model. The lessons from this stage can then be used as the basis for scaling to other programs.
• Refine and scale to other accounts — Lessons from the previous increments are used as the basis for scaling Spotify to other accounts or lines of business. Typically, this process requires assessing the suitability of Spotify elements and running theme increments and MVC iterations until business agility is optimized across tribes.

Case study: Spotify-inspired Agile at an insurance client

One insurance company was planning to embark on a new line of business. However, staying close to the customer was a problem. Because of existing structures, legacy culture, and processes, lead time to market was as long as seven months, and responding to customer demands lacked dynamism. A new operating model was needed, one where first-mover advantage could be found by studying customer needs and responding quickly with new products and services.

The strengths of Agile-based development and a lean Spotify startup culture were suggested as a way to improve speed. Further, the company was interested in scaling Agile across its IT portfolio and improving knowledge transfer among teams.

After assessing culture, organization, processes, and leadership style, executives found that slow release cycles were caused by siloed teams that lacked synergy. Further, technology was a problem, with tooling unable to work in a continuous integration-continuous deployment paradigm. These lessons fed into a change vision document and identification of change agents.

The evolutionary Spotify-inspired Agile pattern was chosen to achieve the change vision through successive MVCs. Further, a fitness analysis of the six Spotify-inspired Agile elements was carried out, themes were designated, and a pilot project was chosen. The Spotify transformation was conducted in four increments, each one maturing different elements within a theme (Table 2).

Table 2. Themes and elements of the insurance client’s Spotify transformation

Though several elements could be implemented directly, much of the Spotify-inspired transformation needed further customization (Figure 4).

Figure 4. Customizations carried out in each Spotify-inspired Agile theme

Source: Infosys

Since legacy applications were highly monolithic, Spotify-inspired Agile squads were not fully independent. The first theme, therefore, managed dependency and integration problems, resulting in Scrum of Scrum and integration meetings. Further, to minimize red tape and to improve business agility, discovery squads were incorporated within the tribe. These squads refined the product backlog and ensured defect-free deliverables through continuous exploration.

In the second theme, the firm focused on lean startup culture and autonomy and customizations involving product planning and governance. Product increment planning in particular was already at work within the client’s operating model but was further modified into the Spotify-inspired culture to ensure business continuity. Product owner chapter meetings resolved dependencies between features that cut across different squads.

The result was the reduction of feature lead times from seven months to 11 weeks . Further, release frequency increased to seven weeks, and high-performance squads achieved a commitment reliability above 85% and productivity improvement of above 50%.

Moving ahead with Spotify-inspired Agile

Spotify-based Agile is not a salve for all business problems; it needs to be adopted with care. Though it worked for a music streaming company, significant modifications are required to implement it at scale in more risk-prone businesses. In our experience, the following recommendations have been shown to make Spotify-inspired Agile work at scale in various large enterprises:

• Some enterprises try to mimic Spotify’s structures with no focus on business outcomes. Instead, the framework often needs to be customized to align with the prevailing business and operating models. This may mean blending practices from other Agile scaling frameworks such as SAFe.
• Spotify-inspired Agile encourages adaptability, often a stretch for firms that are driven only by predictability. This aspect must be considered during implementation of Spotify.
• The transformation must look beyond product development and encourage more business-IT alignment in areas besides the product development life cycle.

Spotify-inspired Agile must be customized to prevailing business and operating models, often blending practices from other Agile scaling frameworks

• The prevailing application architecture is important when implementing Spotify-inspired Agile. Centralized, monolithic architecture leads to complex dependencies among teams, slowing down innovation and overall agility. In such cases, customization is needed to adapt processes to the technology environment.

A typical Spotify alignment journey has three stages — incubation and learning, evolution, and stabilization (Figure 5).

Figure 5. The Spotify-inspired Agile alignment journey

Source: Infosys

Incubation stage — Spotify-inspired Agile elements are implemented. Impacts are assessed through themes, and suitable MVCs are implemented to address gaps so that the business can deliver and ship fast. The inflection point is the stage where the company identifies significant customizations, shifting operating models from standard Spotify-based Agile approaches.

Evolution stage — Elements are suitably customized and have evolved to fill the gaps identified at the inflection point. This enables continuous discovery and delivery. Now the model starts to stabilize in terms of delivery predictability.

Tipping point — The model stabilizes and develops consistency in lead time and achieves optimal business agility.

Agile adoption requires sponsorship from leaders who are all-in for the journey ahead, can bring together business and technology, and will align the whole organization with the North Star vision. As part of this, Spotify-inspired Agile frameworks need considerable adaptation, with time spent analyzing prevailing structures, processes, patterns, and culture before a journey is mapped out.

Infosys has been there before, and we are working with a wide number of clients in adapting Spotify-inspired Agile and other scaling patterns to the needs of big business. We recommend that organizations start with a low risk technology pilot before migrating to other areas. This enables experimentation and improvement before fully implementing Agile across the partner ecosystem. Better business outcomes will result, measured by greater customer satisfaction, increased efficiency and a faster, more resilient enterprise.

References

1. Scaling Agile @ Spotify with Tribes, Squads, Chapters & Guilds, Henrik Kniberg & Anders Ivarsson, October 2012
2. Spotify Rhythm: How we create focus, Henrik Kniberg, June 1, 2016, Agile Sverige
3. See Ref 1
4. How Agile Coaches Help Us Win – the Agile Coach Role at Spotify, December 12, 2013, InfoQ
5. See Ref 2

Download