If DevOps is a continuum, it starts with planning and development, and ends with deployment into operations. Much of the focus in DevOps and within the DevOps toolchain has been on application build automation (development) and deployment of apps into operations. But, for DevOps to pay off, there must be a continuous flow and automation from development all the way through to operations, and that means providing automation for the steps in between these two.
What is in the gap between development and production delivery? Testing. Not surprisingly, when Gartner surveyed its clients in 2015, more than 50 percent of them ranked the need for test automation as one of their top priorities for DevOps success. This really illustrates the recognition that DevOps cannot succeed if only one part of the DevOps continuum is automated. Companies that have started to focus on DevOps recognize that they must “Mind the Gap.”
Closing the Testing Gap
Why is automating the various testing steps (regression, performance, stress, GUI, QA, interoperability and security testing) so difficult? There are five key components that must be automated successfully:
- The infrastructure – It starts with one very important characteristic critical to testing that is not automated – the ability to mimic the infrastructure in which the software (or hardware) being tested is expected to run in when it gets to production. Having the ability to give each tester (or set of automated tests) their own personal replica of the target production infrastructure—a “sandbox”—is critical to ensuring the software coming out of test can be deployed into production automatically and continuously. Automating tests is not nearly as difficult as automating the underlying infrastructure in which those tests run.
- The production environment workload – Often overlooked in automating testing is mimicking the workloads that would be common in production. This includes the network traffic, other application workloads and network security profiles. These activities need to be automated within the sandbox to create a more realistic environment for testing.
- Test automation – Of course, the tests themselves also must be automated and there are a variety of tools to assist in this process. As software gets more complex, automation continues to be a challenge, with mobile testing replacing GUI testing as the most challenging type of testing to automate.
- Reporting and result analysis – To provide continuous automation from development through testing to delivery, it is important to see and analyze test results automatically.
- Tool Integration – Finally, automation into and out of testing needs to be enabled. This means that sandboxes and test automation suites need to be API-driven so they can be started by a previous tool in the DevOps tool chain and can also initiate a tool that follows testing in the tool chain.
DevOps projects tend to start in one part of the continuum from development to operations. There is nothing wrong with taking this approach. However, it is also important to “Mind the Gap” and work to expand automation across the continuum continuously. One way to ensure that automation can expand easily is to deploy sandbox technology across the continuum, beginning with development and continuing through configuration and release management. The sandbox ensures that at each step, the production configuration and workload is being replicated.
Sandboxing technology has been used in development in the past, but is relatively new to the entire DevOps process. As more companies move to continuous integration and continuous deployment, however, the need for sandboxes is growing.
The sandbox should provide the following:
- The ability to create an accurate replica of the real production environment that deployment is intended for. This can include the physical, virtual and cloud infrastructure as well as applications, network traffic and load.
- The ability to initiate a sandbox via APIs so that the sandbox can be triggered by DevOps tools as an integral part of a complete end-to-end DevOps automation.
- The ability to automate DevOps processes inside of the sandbox (e.g. automated regression testing), as well as to automate the teardown of the sandbox and trigger the next step in the DevOps cycle when those processes complete.
- Business intelligence about everything that happens inside of sandboxes so that DevOps processes can focus on continuous improvement.
- The ability for many people to have their own sandboxes running simultaneously with isolation in shared labs.
DevOps depends on continuous automation from development, through test and QA, to configuration management and production deployment. Sandboxes provide a common automation environment that can be used at each step of the DevOps cycle to close the gap between development and deployment.
About the Author/Joan Wrabetz
Joan Wrabetz is the CTO for QualiSystems. Most recently, she was theVP and CTO for the Emerging Product Division of EMC. Joan received her MBA at the University of California, Berkley, and MSEE at Stanford, and a BSEE at Yale. She has held teaching positions at the University of St. Thomas, St. Mary’s University and the University of St. Thomas, St. Mary’s University. Joan holds patents in load balancing, distributed systems, and machine learning classification and analytics. Her experience inclues: founder and CEO of Aumni Data, CEO of Tricord Systems, Vice President and General Manager for SAN operations at StorageTek, Founder and CEO of Aggregate Computing, Management and senior technical positions at Control Data Corporation and SRI International, Partner with BlueStream Ventures.
