Platform engineering: How to modernize engineering culture

Insights

• Expert coders are in short supply, and many are suffering from burnout. The shift to DevSecOps has improved efficiency and speed, but increased coders’ workload.
• Tooling sprawl is also a problem, and executives rarely have a way of tracking cloud costs.
• What’s needed is an effective way to manage product and engineering complexity using an internal development platform (IDP) that curates the best technologies for each project and abstracts the underlying complexity to meet stringent service level agreements.
• This platform provides developers with a suite of tools, including codebases, testing scripts, frameworks, and maintenance software, freeing teams to focus on core product development.
• Good platform engineering, however, requires dedication and perseverance, and a balance between simplicity and flexibility, along with a change management approach that reduces big bang efforts with micro improvements.
• Infosys’ LEAD platform is one such IDP, and offers a range of features across five modernization patterns, including cloud native development, cloud modernization, database modernization, and DevSecOps adoption.
• Platform engineering is the next phase of DevSecOps, a conduit rather than a replacement. Firms that adopt this team topology create products faster and become more innovative, spurring top line growth.

Technology firms are in a tough spot, with rising job cuts and tightening budgets. Meta fired 10,000 workers in March, on top of the company’s 11,000 job cuts last November.

Even in this climate, expert coders are in short supply, and many are suffering from burnout. The shift to DevSecOps (short for development, security, and operations) has improved organizations’ efficiency and speed, but increased coders’ workload. They now do more coding, security, deployment, monitoring, and infrastructure provisioning. Senior developers often bear the brunt of this shift.

Generative AI tools such as Codex and ChatGPT are valuable resources, but more should be done to help those on the frontlines of product development.

Tooling sprawl is a growing problem as developers choose their own tools. DevOps teams spend big on third-party software and cloud services, and executives do not track costs. Cloud financial management goes along way, but it is not enough.

Developers need an effective way to manage product and engineering complexity. They need a self-service, automated platform that provides easy software integration and deployment. This platform should curate the best technologies for each project and abstract the underlying complexity to meet stringent service level agreements (SLAs).

Platform engineering catalyzes digital transformation

Enter platform engineering. This is a complement to, and not a replacement for, DevOps, and has trickled down from organizations such as Google, Airbnb, and Spotify. Here, a dedicated platform engineering team builds an internal development platform that supports the engineering life cycle and provides continuous integration/deployment/testing as a service (Figure 1). The platform provides developers with a suite of tools, including codebases, testing scripts, frameworks, and maintenance software, to meet aggressive SLAs. It minimizes cognitive load, freeing teams to focus on core product development, such as understanding business and domain requirements.

According to Puppet’s 2023 State of Platform Engineering survey, 94% of respondents agree that platform engineering helps their organizations realize the benefits of DevOps. They admitted improvements in system reliability (60% agree), greater productivity and efficiency (59%), and better workflow standards (57%). Around 30% of respondents said that platform engineering serves the entire organization, making it a catalyst for enterprise-wide engineering cultural transformation. This is even more attractive for smaller organizations that lack resources to build expert DevOps and site reliability engineering (SRE) teams.

Figure 1. The typical components of an internal development platform

Why platform engineering is the need of the hour

But it’s not just burnout and tool sprawl that are driving the transformation. In the age of cloud-native, multicloud, container-orchestration, and software-defined networking, knowing what to build, deploy, and maintain has become an art. Internal development platforms that provide tooling for Kubernetes abstraction are crucial to deal with these challenges. Platform engineering teams and internal development platforms also drive the adoption of better standards, such as those developed by OpenTelemetry (for observability), OpenAPI (for describing and documenting APIs), and Software Package Data Exchange (SPDX) for software bills of materials. These platforms enforce quality by setting guardrails for developer self-service.

The rise of software and engineering trends such as DataOps, MLOps, and data mesh architecture, which all rely on self-service, shift-left security, and everything-as-code, has forced firms to use platform engineering best practices and internal development platforms, even if they don’t know it is platform engineering that they are actually doing.

Gartner predicts that by 2026, 80% of software engineering organizations will establish “platform teams as internal providers of reusable services, components and tools for application delivery.”

When platform engineering works well

Gartner named platform engineering as a top tech trend for 2023 and included it in its hype cycle. However, as with all innovations, platform engineering may become a golden cage, rather than a well-trodden golden path (or “paved road”) to supplement and guide the core engineering team. Platform engineering should provide an intuitive and easy developer experience, not one that forces them to change their workflow.

Many organizations have created a static portal of open source and vendor tooling, which they mistakenly believe is an internal developer platform. This portal does not follow important platform principles such as “standardization by design.” Other teams finesse a similar trick, calling something platform engineering when what they’ve really built is a user interface with service catalog capabilities. These portals don’t get to the core of underlying issues such as application configuration and infrastructure orchestration.

Hard won insights from the trenches of platform engineering show that forcing a poor platform on developers will be met with malicious compliance: Every time the platform fails the team, they’ll blame the platform and the team building it. This will end up fracturing DevOps teams, rather than the harmony needed in crucial moments of a big project.

Good platform engineering, just like adopting DevOps, requires dedication and perseverance. It needs a good balance between simplicity and flexibility, and a change management approach that reduces big bang effort with micro improvements. This is the approach we are taking at Infosys as we move to become an AI-first organization.

So how can firms do platform engineering well and ensure this tech trend delivers on its promises?

• Internal development platforms can be built from both open-source and commercial tools. Some tools are mature and already in use, whereas others are new and yet to establish themselves. There are hundreds to choose from, so platform engineering teams need senior software architects with considerable domain knowledge for decision making.
• The platform must be considered as a product, with developers as the customer. The best platforms are built iteratively by the platform team, based on feedback from developers. As we discuss in our paper, Product-centric value delivery: a new strategy, shifting to a product-focused organization implies a stable platform engineering team to support long-term operations.
• The ideal development platform for one company may not be right for another, and even within the same company, different development teams will have different requirements. Therefore, the platform topology should balance central co-ordination with flexibility and autonomy for developers to choose what they need to build their application. The platform shouldn’t overly restrict engineering organizations (or units) in terms of the technologies, tooling, methodology (e.g., GitOps vs ClickOps) or interfaces (CLI vs UI vs API) they want to use.
• A good internal development platform reduces the amount of backlog coupling i.e., diminishes the number of dependencies across the work queues of multiple teams. Specifically, it should provide self-service capabilities across provisioning, configuration, management, and operations.
• Getting developers to use new tools is difficult, even if the product is excellent. This requires evangelism, consulting, and metrics to show developers the benefits. A dedicated knowledge base that suggests which tools and technologies are mandatory, along with those that are default but can be overridden, can also help. It also means starting small and building slowly over time, using the micro-is-the-the-mega approach to change management. Firms should harvest already-proven solutions from application teams and try joint ventures to create and test capabilities with the teams that will use them.

Infosys’s LEAD platform

Infosys’s Live Enterprise Application Development (LEAD) platform brings advanced capabilities. It offers a range of features across five modernization patterns, including cloud native development, cloud modernization, database modernization, legacy modernization, and DevSecOps adoption. Features span the complete technology stack and help development teams across architecture, development, testing, SRE, and deployment (Figure 2).

Figure 2. Technology stack of Infosys’s LEAD

LEAD abstracts the complexity of underlying technology to boost developer productivity. The platform helps coders unlock information from legacy systems, simplifies their decision making, all while reducing teams’ dependence on niche skills. It can save up to 40% of developers’ effort in developing customer-centric products, and enables up to 25% faster time-to-value. Going further, deep insights into all types of technical debt provide high quality code from the genesis of coding projects. The platform also integrates with Agile ALM tools for customer insights that improve sprint velocity, release predictability, and product quality.

A US-based investment management company used LEAD to transform its legacy applications to cloud-native microservice based applications. The platform accelerated time to market by 25% through standardization and enforcement of cloud-native best practices; built-in end-to-end observability and automated provisioning of application environments; and code quality/security scanning as part of CI-CD pipelines.

Evolving DevOps toward a better engineering future

Platform engineering is an emerging discipline, and it is what happens when DevOps engineers pull down the siloes and talk about the issues in their organizations. It’s the next phase of DevSecOps, a conduit rather than a replacement. Firms that adopt this team topology create products faster and become more innovative, spurring top line growth. According to the State of DevOps 2023 report, platform engineering directly correlates with DevOps evolution: 48% of highly evolved DevOps teams (as measured by four metrics – deployment frequency, change lead time, mean time to recover, and change failure rate) use internal platforms. Mid-level DevOps teams only use platform engineering 25% of the time, and low-evolution DevOps teams just 8%.

“Highly evolved firms make heavier use of internal platforms for their engineers, enabling developers to access authentication (62%), container orchestration (60%), and service-to-service authentication (53%), tracing and observability (49%), and logging request (47%) services via self-service,” says the report.

It’s early days for platform engineering, and there is much value to be unlocked from its growth and adoption.

Download