Article

Quality Engineering: Building A Strong Foundation For Innovation

By Shalini Chaudhari, Sundar Vallabhan, Arvind Sundar, Sabarinath Keerthivasan, Harry Keir Hughes,

15 Jul, 2021
12 min read

QE brings together testing, quality assurance (QA), and delivery. It ensures an automation-first approach to business success and acts as the voice of the customer in development teams.
When well executed, QE helps align operational resilience, strategic initiatives, and customer sentiment, using cloud as a force-multiplier.
To integrate QA into the software development life cycle, CIO’s should concentrate on quality assessment, problem ownership, built-in app quality, strong user stories and a focus on AI and automation.
A 3-step QE strategy can be devised that 1. Pays attention to test data and test environment 2. Minimizes technical debt 3. Orchestrates tools effectively.
Finally, true QE is underpinned by the 4h’s: holistic skills, hyperautomation, hardware in the loop, and human in the loop.

Slowing down digital transformation was not an option last year, with digital upstarts hungry for market share and the pandemic scrambling supply and demand. The C-suite often focused on increasing operational resilience and used the cloud to rapidly expand their firms’ capabilities. As all this change swirled, executives and boards insisted that their enterprises still remain robust, secure, and trustworthy.¹

Behind the scenes, an A-team of engineers worked hard to ensure solutions were cost effective and scalable, and that they developed quickly. QE was an unlikely hero. In the enterprise metamorphosis of 2020, the division shed its image as “nice to have.” Software testing proved its worth as a critical business component.

QE brings together testing, quality assurance (QA), and delivery. It ensures an automation-first approach to business success and acts as the voice of the customer in development teams.² However, lack of a well-designed QE strategy delays cloud transformation as much as half the time, according to QE practitioners we spoke to. Costs rise, innovation languishes, and firms forgo their share of the $414 billion in profits the cloud promises when implemented effectively, as laid out by recent Infosys research.³

Lack of a well-designed QE strategy significantly delays cloud transformation

Also, when companies get QE wrong, brand reputation suffers. Tesla vehicle orders in China fell by nearly half in May 2021, compared with the previous month, over its handling of customer complaints. Accused of lowering hardware and software quality standards to meet production targets, faith in the brand plummeted when Tesla suggested the complaints were unreasonable. This triggered a wave of negative media coverage.⁴

When well executed, QE helps align operational resilience, gung-ho strategic initiatives, and customer sentiment. But teams must first have deep insight into AI, microservices, and developments in low code-no code (LC-NC) environments. They must also be able to work across numerous platforms, code bases, and software integration-automation technologies. And QE teams must do this with the customer in mind. This is no small feat and one of the reasons the practice now has elevated status.

Tackling QE head-on

When creating new operating models, most successful enterprises are building them around the cloud, which offers a way to quickly differentiate and expand capabilities. That puts greater pressure on the chief information officer (CIO), who must ensure those systems are secure and stable when mammoth projects go live. CIOs must be able to assess system quality in real time and create a durable QE team that fosters business agility, scaled innovation, and a secure ecosystem.

To manage these increasingly complex demands, the CIO needs to integrate QA into the full software life cycle. The following considerations are pertinent:

Quality assessment - The amount by which poor quality affects data leakage, application uptime, regulatory compliance, and customer experience should be measured, along with the number and scale of security breaches.
Ownership - There should be one accountable person on the QE team. Often, various problems get distributed across development and infrastructure leads, the security team, the application management services team, and more. Further, red flags are often raised to someone who doesn’t have authority to rectify the problem, and when the right person is onboarded, they don’t have the right processes, technology, and skills to get the job done.
Built-in quality - Cloud applications must meet appropriate quality standards throughout development - across all increments and elements of the solution. Everyone from software, hardware, operations, security, and compliance should share the goals and principles of built-in quality. This also ensures the enterprise complies with stringent global, regional, and industry security standards. QA becomes an integral part of company culture and enables teams to work with a Lean-Agile mindset, releasing products in short, sharp sprints.
User stories - To enable this sort of Lean startup culture, QE works with various business functions to reduce conflicts and ambiguities in user stories before development begins. Both developers and security experts have a seat at the testing table.
Technology - AI and automation should be validated from the start, with the data produced by test scripts used to optimize machine learning (ML) models. As part of this work, QE teams need to be equipped with customer success metrics and have the ability to work across cloud technologies, Agile, and AI.

Digital trends affect QE strategy

Digital transformation is a complex and shifting domain. As QE teams build QA into systems, they must get a handle on emerging technologies. Table 1 lists six technological trends that will all have significant effects on QE strategies. By understanding these, QE becomes the linchpin for cloud transformation and the foundation for further innovation.

Bringing these trends together is an important initial step. But executives must then determine what an effective QE strategy should look like so it can drive value and deploy nascent technologies and services at scale.

Table 1. How digital trends influence QE strategy

A 3-step QE strategy

A good QE strategy ensures resilience and innovation in digital transformation projects. It solves QA issues holistically and builds a foundation on which to work with emerging QE technologies at scale. The following three-step approach - holistic, with an ecosystem view - has been implemented at scale in partnership with clients on their digital journey.

1. Build the right test data strategy and test environment

Often, QE teams focus primarily on automation, AI-based solutions, and building quality into the DevOps pipeline. However, they don’t always pay enough attention to the test data and environment. Roughly 25%-30% of the defects found after the quality phase are caused by a lack of proper test data or environment mismatches and configurations. If not designed and planned well, data and environment can become the two biggest bottlenecks to a well-functioning QE team.

To avoid these problems, companies should first focus on the test data strategy and not the tool. This strategy will depend on several factors: the need for masked data (due to regulations), the prevention of unauthorized access, the technology landscape (test data provisioning versus virtualization), and external interfaces. Adopting a strong test data strategy through the assessment and discovery process is critical. The strategy should drive the tool selection, not vice versa.

Next, organizations need to develop nimble and easy-to-maintain nonproduction environments. The visibility of releases is affected by efficient demand management, release management, and an ability to understand and resolve environment conflicts. While the cloud and containerization have made it easier to cxonstruct build-use-collapse environments, a strong nonproduction environment management is critical for QE success.

2. Minimize technical debt

Technical debt is the cost of additional rework implied by choosing an easy solution now instead of using a better approach that would take longer. This debt often accrues as a side effect of large, multiyear transformation programs. It’s the QE team’s job to accelerate cycle times and mitigate this avoidable technical debt, paving the way for increased efficiency, reduced costs, and better business outcomes. Teams can often accomplish this with the help of ML.

Testing organizations often build a regression suite with duplicates and redundancies. ML-based clustering can help teams reduce the artifact size while retaining the same coverage.
ML can identify the script affected by code changes, which then triggers automated scripts using the continuous integration-continuous delivery pipeline. This technology also helps reduce testing effort by 20%.
ML analyzes past releases, sprints, and defects and then uses linear regression to predict defects that could occur in future release cycles. The starting and stopping of QA cycles reduces the risk of over- or undertesting.

3. Find the extra orchestration patterns

The problem with QE is not a lack of tools but an inability to orchestrate the tools effectively. Indeed, many modern automation tools fail to address the end-to-end quality view. There are, however, ways that QE teams can overcome this problem.

Robotic process automation (RPA) - This technology should not replace existing automation scripts, but instead, it should integrate and orchestrate automation across technologies and systems. RPA can help identify patterns in an otherwise chaotic mess of tools.
Platform over product - Successful IT organizations understand the need for an always-on platform that assembles all the tools and techniques needed for digital transformation. A custom-developed platform architecture combines a variety of different, best-of-breed technologies and capabilities to deliver an integrated solution. This enables a high degree of agility, scalability, and adaptability.

A well-functioning QE strategy must also be supported by the right ways of working. Development, operations, and business must communicate effectively. By collapsing boundaries among units, work is accomplished much faster and there is a greater focus on the product and customer. Better business outcomes are also discovered by autonomous teams that use human-centered design and work with senior leaders from the product requirements stage all the way to deployment.

QE as a value driver

The quality paradigm initially focused on the 4 r’s of risk, regulatory compliance, rich customer experience, and return on investment (ROI). More recently, as automation, DevOps, and AI have become commonplace, true QE is underpinned by the 4 h’s: holistic skills, hyperautomation, hardware in the loop, and human in the loop. Each is necessary to ensure the cloud is a force multiplier.

Holistic skills

Organizations need full stack testing in legacy and cloud environments, along with a focus on DevSecOps. These skill sets will improve automation and standardization, enable Agile delivery, foster synergies, and provide a transparent view into how much automation currently exists. As software development engineers in test (SDETs) mature, they will move from a focus on technical competency in code bases and automation to delivering strategic outcomes. QE executives need to deeply understand new trends in AI and ML, drive automation at the project level, and understand QE implications at the organizational level. All the while, they must keep innovation at the core.

Hyperautomation

One of the most prolific challenges is the need for speedy ROI in cloud-led digital transformation. Hyperautomation brings together employees, robots, and AI to ensure products and services reach customers quickly and with built-in QA (see Figure 1).

Figure 1. Hyperautomation in the future workforce

Source: Infosys

The symbiotic relationship between the transformed workforce and automation bots will help accelerate delivery cycles. A bot factory’s most important capabilities include:

RPA utilities, including technologies such as Assist Edge, UiPath, BluePrism, and AA.
Model-based testing utilities and a codeless framework to enable end-to-end automation.
AI capabilities that help QE analyze hotspots, synthesize test scripts, automate reporting, review end-user sentiment, reduce product backlogs, and instill change management processes.
AI-assisted self-healing capabilities for environment issue resolution, test data issue resolution, and automated script maintenance.

Hardware in the loop

End-to-end tests for hardware and software are important as new devices hit the market with different versions and various network operating conditions. Elon Musk tweeted in May 2021 about the need for hardware-in-the-loop testing of software, beyond the usual software assurance processes.⁵

Consider Tesla’s connected car ecosystem, which includes devices with edge and cloud capabilities. Testing this setup could be complicated by varying network conditions, user behavior, weather conditions, and geolocation, among many other considerations. It is important to move to a scenario where the individual hardware elements of this end-to-end ecosystem are tested along with the software.

This testing can be done with real devices or with digital twins. The latter is essential for testing complex internet of things and Industry 4.0 implementations. Digital twins can test the signals from thousands of sensors associated with a heavy machinery ecosystem in manufacturing, which is otherwise very difficult to test with real devices.

Human in the loop

The complication with human in the loop is that QE practitioners may feel like they’re going back to where they started, before the cloud and automation. However, this dimension goes beyond having SDETs test in a lab-based environment. In this paradigm, it is important to leverage crowd consensus to help improve the customer experience. With everyone using the cloud in different ways across regions, a persona-based approach to QE is necessary. For example, a company could enlist the public’s help to test software upgrades using a set variety of user personas in different geographic and network conditions.

Figure 2. QE as a value driver in the organization

Source: Infosys

Although quality is valued, it’s not always at the front of the line. Project and product managers tend to prioritize meeting functional requirements, schedules, and budgets. Only late in the testing phase does quality become important.⁶ This is a mistake. Without robust QE early in the software development life cycle, technical debt goes up, proliferation of redundant tools increases costs, and firms fail to realize an end-to-end view of product capabilities and defects. This last fact means that firms can’t confidently say whether a new product will achieve its intended purposes, reducing the impact the product will have as it launches in the market.

Businesses now need an aggressive quality strategy that underpins their digital transformation. This will allow them to launch new, exciting services and products and to both hold and gain markets. For this to work, companies need the right culture. “Culture eats everything for breakfast, not just strategy,” according to one Agile quality practitioner.

When that commitment and culture take hold, businesses can accelerate their journey at an even greater pace. Companies that embed QE in the organization and follow best practices can become more resilient and competitive, and can share in the profits available to those aggressively pursuing their digital future.⁷