While development operations are more scalable now than ever, there is one area that continues to constrain the ability of organizations to scale: functional testing.
Most organizations still rely on sequential tests to deliver quality assurance. No matter how rapidly a business can produce great products, its ability to bring them to consumers quickly is limited by its dependency on a slow, tedious testing process that does not adapt well to changing development needs or ever-faster business processes.
That, at least, is the way things traditionally have been. But today there’s a replacement for sequential testing: parallel testing, which lets you leverage the cloud and virtualization to perform more tests at faster speeds than your physical infrastructure or homegrown testing grid could support on its own.
What is Parallel Testing?
First, a definition: Parallel testing is the process of running multiple test cases on multiple combinations of operating systems and browsers at the same time. The process is automated, and it often runs on virtual machines. For the QA professional, no additional effort is required once the appropriate test scripts have been developed. All the testing solution needs is the test suite and a pool of resources.
Granted, new does not always equal better. But it is hard to make an argument against parallel testing, especially if you leverage a robust testing cloud. Here are seven key advantages of parallel testing to consider over sequential testing:
You Can Test for Wider Compatibility
Because parallel testing is faster, it allows you to test for wider compatibility in the same amount of time. If you’re releasing a new app and want to test the user interface (UI) functionality, you don’t have to limit yourself to testing against only the most popular OS and browser combinations or mobile devices—you can test all the combinations that are important to your users.
You Can Radically Reduce Testing Time
Good software testing must be thorough and comprehensive. In a sequential testing environment, it can be (and usually is) a very time-consuming process. Parallel testing can divide invested time (roughly) by the number of test machines being used in parallel, so that overall testing time is a fraction of that required for sequential testing.
For example, if you are running 10 concurrent tests, you can reduce your testing time by a factor of about 10. This can cut days or even weeks from delivery time. This reduction in testing time also allows you to make more efficient use of your engineering team. They are freed from a set of necessary but time-eating and repetitive tasks, allowing them to concentrate on more forward-looking and innovative projects.
Parallel Testing Optimizes Continuous Integration and Delivery
Modern development best practices such as continuous integration and delivery require frequent and rapid functional tests. The execution of pre-scripted tests on a web or mobile app saves considerable time, plus having test data accessible in detailed reports is valuable to development teams who can use this information to quickly identify issues. Parallel testing in the cloud allows you to execute test suites continually as developers submit and integrate code changes throughout the day.
The Cost per Each of Your Tests is Less
Because parallel testing is automated and and can run at high concurrency in the cloud, the cost per test is considerably less. Leasing test time on a cloud testing service generally will be much less expensive than building and maintaining your own test infrastructure.
Even the common practice of using development hand-me-down computers as test machines has its costs and practical limits. For example, as used systems age, they require more maintenance and they become less capable of duplicating real-world deployment conditions. Cloud-based testing grids mean environments have high availability, and they are up-to-date, making parallel tests the better option.
It Supports Testing Science
Parallel testing enables teams to focus on making tests repeatable, measurable and accurate. By simply testing faster, you can test more, and testing more produces more actionable data to find bugs and also improve test cases. Automated parallel testing delivers strong data on test results with key indicators of where tests can be improved. The net result is a test process that is more scientific.
You Can Still Perform Sequential Tests, If and When Needed
Parallel testing simply offers a different framework for testing, which can easily accommodate most or all of your existing test regime. This means that transitioning from sequential to parallel testing does not force you to sacrifice test targets, strategic testing goals or test quality.
If anything, moving to parallel testing increases the range and depth of your test program, allowing you to better meet your test goals and it improves the quality of your testing. When you make the move to parallel testing, you are not abandoning traditional software testing. You are simply applying modern development practices and technology to it.
You Do Not Need to Make the Switch to Parallel Testing All at Once
The great thing about parallel testing is that you can transition to it in steps, starting with the tests that are most easily converted to parallel, then adapting some or all of the remaining tests after your team has had time to adjust to the parallel testing environment. Once you have started to use parallel testing and have integrated it into your development process, it is easy to expand the quantity and scope of parallel tests as the need arises.
Conclusion: A Faster Test Run is a Penny Earned
The faster you run a test, and the more tests you run, the more frequent the releases and the more defects you can find. The math is simple. Parallel testing in the cloud decreases the cost of maintaining test infrastructure, the cost of lengthy testing cycles, the cost of delayed releases and, finally, the branding cost of unhappy customers due to poor application quality.
Even without changing the way you test, a shift to a parallel testing environment will provide an immediate speed enhancement—which means every test that still runs serially is a cost that can be avoided. And every test run in parallel is an opportunity to test your application better and faster at the same time.
About the Author / Lubos Parobek
At Sauce Labs, Lubos Parobek leads strategy and development of Sauce Labs’ web and mobile application testing platform as VP of product. His previous experience includes leadership positions at organizations including Dell KACE, Sybase iAnywhere, AvantGo and 3Com. Parobek holds a Masters in Business Administration from the University of California, Berkeley.
