Test environment management is bottlenecking DevOps transformations and is driving the need for a new test environment management platform solution. On-demand test environments in which developers and testers do not have to wait for a shared environment are sorely needed.
The evidence is clear. In my book Engineering DevOps I indicated that finding and setting up test environments is a primary source of bottlenecks in DevOps value streams.
In my February 2021 webinar with Plutora, I presented Best Practices for Developing Test Management Environments. I indicated that orchestrating and managing test environments is often a primary source of delays and waste within value streams.
The State of Testing Report 2020 by PractiTest indicates that DevOps practices increased 57%; from 26% in 2017 to 41% in 2020. The report revealed that 46% of testing has shifted to using CI/CD processes, 55% are actively involved in shifting testing left and 36% are testing in production. This is clear evidence that DevOps testing is on the rise.
Wolfgang Platz’s December 2019 article Why Software Testing Remains a Bottleneck indicates that testing is responsible for more delays than any other part of the development process – accounting for 52% of the bottlenecks in the development process. And 53% of respondents to the World Quality Report 2020-21 indicated that test environments are not available when needed, 67% lack the capability to book and manage test environments and 71% lack visibility of available test environments.
An article by Velocity echoes the problems with pre-production environments. “With the rise of microservices and cloud-native applications, environments are becoming more complex. Often you need to wait or compete to get access to develop on a specific microservice, or can spend time waiting to deploy to a shared staging environment only to realize something already changed, and you’re back to square one.”
The Continuous Testing Report – 2020 by Capgemini defines test environment management (TEM) as managing infrastructures, test data and applications to support testing on demand. These environments are one of the greatest impediments to continuous testing. In this report, 36% of respondents said they are spending more than half their time building and managing test environments; 44% of respondents indicate they spend weeks of their time searching, managing and generating test data.
Test environments should be regarded as functions to be spun up or destroyed at will, with virtualization, infrastructure-as-code (IaC) practices, service virtualization, containerization and improved test data management.
Given all the above evidence, it is clear that a new approach to DevOps test environment management is needed.
The following are examples of benefits that DevOps test environment management platforms need to offer:
- Compatible with current tools:
- Automatically map and configure application infrastructure to develop and test multiple services in an environment that runs just like it would in production. This needs to be fully agnostic to cloud and tech stacks, such as Kubernetes and other environments.
- The platform needs to work on applications in parallel without disrupting others, with full access to data and third-party dependencies.
- Automatic configuration and maintenance:
- The platform needs to automatically configure and update any changes within the production infrastructure’s mapping in real-time so ephemeral environments will always be synced with what is in production. Instead of having to spend time making sure any changes are configured, the platform will ensure the right environment that replicates production, without having to wait.
- Data and third-party dependencies:
- The platform needs to load the relevant data and third-party resources that are vital parts of the environment to test, work and debug against, so test environments will, as closely as possible, mimic working in production.
- Team collaboration:
- The platform needs the capability to share ephemeral test environments with other stakeholders so they get visibility and can collaborate for faster feedback cycles. This is also important for collaborating for debugging.
What This Means
DevOps testing is on the rise, and test environments are a major bottleneck for DevOps testing. A new DevOps test environment management platform is needed to reduce the bottlenecks. On-demand test environments in which developers and testers do not have to wait for a shared test environment are needed. Such a solution would improve agility and reduce wasteful orchestration bottlenecks in test, improve quality and risk management through comprehensive and automated testing and enable continuous improvement by providing more intelligent insights and feedback.
