Introduction
In 2016, a leading global bank used DevOps to transform its IT operations. Over a period of 18 months, software release frequency increased by over 40%, and there was a 70% reduction in testing effort.
In the first article in this series, DevOps was defined as a methodology that enables rapid, agile development of software and scalable, reliable operations1. We also summarized the benefits of this approach.
This article describes how the traditional operating model was redesigned so that DevOps teams could group together to work agilely. The dialogue among all software delivery teams, IT operations staffers and business stakeholders had to change significantly to ensure that the Agile operating model was successful2.
How it all started
Every DevOps transformation must ensure that changes to the IT landscape are aligned with overarching business goals. These goals then become the catalyst for change that team leaders work toward.
The bank envisioned three goals to catalyze this IT transformation:
- Significant cost optimization through direct savings and absorbing headwinds
- Sizable improvement in IT quality and service stability
- Greater than 10% improvement in key performance indicators
The bank developed six capabilities to drive its transformation
The first two capabilities — operational model redesign and enterprise agility — are the focus of this article.
The need to redesign the operating model
Most incumbent banks are not geared for an agile way of working and DevOps. Technical engineers and other IT roles are grouped under lines of business in the traditional organizational structure. These business streams are supported by service functions for building and running applications (see Figure 1).
Figure 1. Products and services were traditionally delivered by applications managed by multiple lines of business
With this design, products or services (such as payments) are delivered through applications managed by multiple business groups. This discourages collaboration between teams, encouraging them to form silos. Teams feel they have to raise barriers between themselves to communicate with each other (referred to as “support models”).
The bank enlisted Infosys support to rethink this operation for its South American operations, so that it could support a DevOps model of application life cycle management and redefine 15 business-critical services (covering over 100 applications).
According to Gartner research, organizational change is key to allowing DevOps to flourish: “People-related factors tend to be the greatest challenges — not technology.”3
At the bank, the entire operating model was reconfigured, with agility as a primary design principle for its new DNA.
The new, improved operating model
The new operating model took full advantage of DevOps.
Along with bringing people together in one unit and removing silos, DevOps requires software development practices that combine software development with technology operations to shorten the development life cycle. Business objectives are closely aligned with every stage of the process. Features, fixes and updates are delivered in a cyclical fashion, and each change improves on the one before it. This iterative feedback loop ensures rapid feedback, and quality is kept front and center.
The core principle for the redesign was, simply put, “What you build, you run!” Small DevOps teams of approximately 15 people each were created.
Figure 2. In the Agile operating model, self-governed teams deliver a product or service
In Figure 2, each line of business houses multiple agile teams, known as Pods, with support functions integrated into daily activities.
These self-contained, cross-functional product teams have complete ownership and autonomy to “plan, build and run” innovative solutions for the business. Support personnel no longer have barriers like ticketing systems and operation-level agreements. Rather, they are part of the DevOps team. By partaking actively in an application’s life cycle, support personnel have joint ownership in the team’s outcomes, which encourages collaboration and, ultimately, accountability.
What exactly is a Pod?
- Self-sufficient and self-sustainable agile team that has the autonomy to build and run its product or service, along with complete accountability for its performance
- Cross-functional and cross-skilled resources collectively working to achieve the sprint and release goals (sprint and release are terms associated with the agile way of working)
- Consists of 10-20 people, depending on the complexity and number of applications or services delivered
- Drives the DevOps model in a seamless fashion with an all-inclusive structure of business development, testing and operations
- Teams work on a single, integrated product backlog, compared to multiple project teams working on multiple project or application backlogs
Figure 3. Diverse and skilled teams of 15 people deliver products aligned to the agile way of working
These agile, multiskilled and self managed teams work together toward common goals on a daily basis.
A focus on business outcomes is ingrained into agile teams from the start. This allows product owners to deliver the value of DevOps in a manner that makes business sense, refining the teams’ understanding of customer value on an ongoing basis to evolve capabilities and further enable organizational change4.
Since teams comprise people in both business and technology, workflow and communication become more transparent. An intrinsic understanding of the product or service also develops.
Building and fixing everything together in a series of sprints accelerates customer feedback and reduces friction, cost and risk.
Operations-as-a-Service as the new normal
Development and operations do not overlap in a traditional operating model. One team designs, develops and tests the application or service, and another set of individuals manages the operations and infrastructure component.
With Operations-as-a-Service (OaaS), the software environment is deployed into production automatically. OaaS also monitors and maintains the system, applying patches and security updates when they are needed.
With the new agile operating model in place, the operations engineer (see Figure 3) performs the tasks and activities once managed by the run and operations teams in the traditional model.
The agile team’s integrated product backlog brings together tasks such as monitoring, incident resolution, problem investigation and configuration management. The product owner then prioritizes these tasks for the operations engineer to complete.
Mature agile teams
A mature team is highly agile and demonstrates high levels of business engagement, primarily through the product owner. This level of collaboration is not easy to achieve and can take many months for individuals to learn to work together effectively.
The bank leveraged the disciplined agile framework to help foster this working culture. This framework provides a blueprint for lightweight guidance at the team level. Teams are driven to succeed as objectives and targets are transparent throughout the process. Progress for each sprint is measured collectively through KPIs such as increased deployment frequency, increased deployment success rate and a faster mean time to restore (MTTR).
Kanban and scrum delivery methods were made available to the agile teams within the bank enterprise, and the teams could choose the most appropriate delivery approach — or even a combination of multiple delivery models (e.g., scrumban).
How was the operating model developed?
A combination of hand-picked bank employees and Infosys agile DevOps experts formed the program deployment team. Each of the team members had elements of the expertise, experience, and agility to transform the bank’s culture, navigating the complicated landscape.
Figure 4. Bank staff shadowed the Infosys team to learn and optimize agile ways of working
The external team was chosen based on their respective fields of knowledge: agile coaches, DevOps automation experts, test automation experts, KPI consultants and service management experts. Each lead or expert was tasked with their own workstream to help bank members learn the ropes, through shadowing and on-the-job learning.
This structure enabled the transformation to progress quickly and infuse new ways of working throughout the teams. Extended members of the team also quickly became knowledgeable about the new working culture. This ultimately helped sustain transformation post Infosys exit.
True agility in the operating model requires a DevOps philosophy to take root. This is demonstrated as collaboration and cross-functional thinking, and must be embedded in day-to-day activities.
There must also be a willingness to fail, with buy-in from key stakeholders within the company supporting the long-term transformation. Unfortunately, this was not initially the case at the bank as the DevOps transformation began (Article 5 in the series describes how this was rectified).
DevOps is not a one-time rethinking of the operating model, but a successively improving change in culture and capabilities.
With that in mind, refrain from creating a separate DevOps team that works separately from the rest of the organization. Silos should be removed within IT as DevOps is rolled out, and business leads should be invested in the results of the process5.
The next article looks at the DevOps tooling framework that was used within this operating model. The concept of DevOps as a practice is introduced, along with progressive DevOps tools that improve on the quality of code reviews.
References
- https://www.infoworld.com/article/3215275/what-is-devops-transforming-software-development.html
- https://www.mckinsey.com/business-functions/digital-mckinsey/our-insights/beyond-agile-reorganizing-it-for-faster-software-delivery
- https://www.gartner.com/smarterwithgartner/the-secret-to-devops-success/
- Ibid
- https://blog.turbonomic.com/blog/on-technology/implement-devops