Modernizing enterprises: The power of generative AI in code migration

Insights

• Maintaining an organization’s diverse tech stack is resource-intensive and difficult to govern, requiring specialized teams with expertise in different technologies.
• Relying on manual or tool-based code migration is a problem, especially since the number of developers with skills in older languages such as Cobol is dwindling.
• Large language models (LLMs) can automate and optimize the code migration process, maintain operational efficiency, reduce costs, and ensure software compatibility.
• But care should be taken when using generative AI at scale. Organizations should tailor and supervise the technology, ensure adequate prep, and employ managed services to reduce complexity and risk.

Modern enterprises are complex, interconnected systems. Over 88% of these systems are legacy, according to Infosys Knowledge Institute research in 2022, and there are more than 220 billion lines of Cobol code still used in production around the world.

Maintaining this diverse tech stack is resource-intensive and difficult to govern, requiring specialized teams with expertise in different technologies. Modernization, which relies on code migration — the process of moving software code among environments, platforms, or languages — has become essential to simplify this spaghetti of different systems and codebases, supporting faster, more efficient business operations.

However, most businesses still rely on traditional approaches to code migration, including:

• Manual migration: This labor-intensive method involves human programmers manually rewriting code from one language to another. While it offers fine-grained control, it's slow, error-prone, and struggles to scale with larger codebases.
• Tool-based migration: Organizations also use licensed tools designed for code migration that operate on rule-based systems such as Microsoft Azure DevOps, AppDynamics (Cisco), and similar tools from Amazon and Google. While they provide automation and customization, these tools have limitations when dealing with complex, non-standard code patterns, mainly due to the fact that the migration rules and application logic are pre-defined (i.e. are rule-based) by the developer for specific data structures. This often results in an ineffective migration process that requires substantial manual intervention.

Relying on manual or tool-based code migration is a problem, especially since the number of developers with skills in older languages such as Cobol is dwindling: “Finding the Cobol engineers — how many people know that?”, says Amin Venjara, chief data officer of 75-year-old payroll-processing company ADP.

Modernization also requires experts proficient in both source and target technologies. This is challenging. In our Modernization Radar, 51% of respondents cited a lack of skills and talent as a bigger pain point than the risks of disruption and cost.

Business performance can also be at risk during the transition from older to new technologies, despite parallel development efforts. Modernization also risks loss of functionality and data.

Perhaps the steepest hill to climb when bringing executives on board a legacy migration project is that modernization is expensive. Return on investment (ROI) is also hard to measure, and technical teams often don’t speak the language of the business, instead talking about technical issues without outlining the benefits of modernization, including efficiency and architectural compatibility with important technologies like AI, and APIs.

The truth is that businesses investing in the right modernization approach can expect over 200% ROI over three years.

Make room for the new kid in town

One way to tackle these challenges is with generative AI. Large language models (LLMs) can automate and optimize the code migration process, maintain operational efficiency, reduce costs, and ensure software compatibility.

“As we get better and better at using generative AI for code migration, we’re going to see some really nice gains for people that have either stalled or been putting off a lot of their digital transformation work,” Fiona Tan, CTO of Boston-based Wayfair, an American e-commerce company, recently told the Wall Street Journal.

As of 2024, 63% of businesses are trialling generative AI for tasks like code generation and maintenance.

Compared with existing tools, generative AI can parse the code and can accelerate the conversion of frameworks and languages such as Cobol into Java.

By understanding syntax and structure and leveraging datasets of programming languages (to deal with legacy code, LLMs have to be trained on the specific codebase), these models eliminate human error and accelerate modernization without altering core architecture. This process also optimizes the code for enhanced performance on the new platform or technology.

At one North American airline, the codebase didn’t allow it to add modern functionalities to its tech stack, and dependency on this codebase was considerable. To get the benefits of modern technology as well as maintain greater control, the airline conducted a proof of concept for migrating to Java on a hyperscaler platform. The process included using generative AI for tech stack analysis and migration requirements. Multiple models and prompts were evaluated to check the quality of the code conversions, and only the best translation was integrated into the target tech stack.

In another implementation, a Dutch energy company was running on a codebase that limited its flexibility to integrate further technology for competitive advantage. The approach taken here used generative AI to set the framework for the migration and followed that with the actual code migration exercise. LLM-enabled bots were used to generate documentation from Cobol and Job Control Language (JCL) programs, and generative AI was further used to rework the documents into requirements and to understand the application architecture, which is an essential starting point for dealing with legacy code. Generative AI-infused testing applications also generated test cases from the document requirements. Once the code was translated, the code was deployed manually against the test cases so that the new code met the functional requirements extracted from the Cobol.

Harvesting business benefit with generative AI: Eight recommendations

But care should be taken when using generative AI at scale. The following eight recommendations hold for large enterprises starting out on their own projects:

1. Tailor and supervise the technology: The code migration process should be tailored to navigate diverse technological landscapes, ensuring that there is a smooth and accurate transition between languages and platforms. The process must also be closely supervised to mitigate against hallucinations, a feature of generative AI.
2. Bring your people with you: At the North American airline company, feedback management was needed to bring skeptical team members along with the modernization project.
3. Use rule-based programs to complement generative AI: Another lesson at the airline company was that incorporating deterministic, or rule-based, automation into generative AI processes significantly enhanced the accuracy of modernization outcomes.
4. Prep before you start: Onboarding an AI solution to code migration must be built on careful preparation and planning. Thought must be given to research and analysis, which uncovers things like key performance indicators, target systems, and platform determination; baseline estimation (documenting the current state of the codebase before making changes); and prompt development.
5. Find business-IT alignment: A successful migration is a collaborative effort. In-house IT teams and developers should work with business to accurately capture code migration nuances and intricacies. Continuous feedback between business and IT allows for real-time refinements and improvements.
6. Adopt a modular, phased migration approach: Another important aspect is modular design of code components with a phased approach that is adaptable to changing needs as the modernization project scales. This ensures critical systems remain operational and potential downtime is reduced, even while the enterprise remains flexible enough for new adjustments and recalibrations to be made as the migration progresses.
7. Employ managed services to reduce complexity and risk: Managed services also enable faster deployment and allow for focus on core innovation. The following services can be provided by an adequate partner:
   • Code analysis using agents: Code analysis is a key aspect of the code migration process. This exercise can be expedited with the use of automated tools or scripts (agents) and LLMs that can make an estimation of trends in the codebase.
   • Code refinement: Translated code might require a few changes based on the version of the target systems. It could also have to undergo certain conventions as per the requirements of the client or recent industry standards. To decrease the load on the migration team, a capability to refine code as per requirements makes the process hassle-free.
   • Unit test case generation: Once the migration is complete, the code will have to pass through a unit testing and integration testing process, adding more time and requiring more work. A capability can be added to reduce the burden, gauge the efficacy of the translated code, and accommodate for the generation of unit test cases.
   • Integration with DevOps: Once the code has been migrated to the target system(s), it must be checked for its scalability and the platforms on which it can be deployed. DevOps addresses this to ensure that the code migration journey is covered with relative ease.
8. Utilize humans for tasks AI can’t do: Generative AI can accomplish a great deal, but it’s not a silver bullet. For example, for some legacy systems using custom-built software, there might not be enough examples with which to train the AI to generate solutions. In these instances, companies will have to rely on their IT experts to manage modernization patterns. A human in the loop is also needed to check the veracity of optimized code.

Towards competitive differentiation

By following the approach above, businesses can reduce their reliance on aging codebases and a workforce that can’t understand how they work.

Those that get to the other side will find themselves better positioned to innovate, scale, and remain competitive in an ever-evolving market.