An Introduction to BoomTown DevOps Processes
BoomTown uses a specific set of DevOps processes in order to deliver websites with the latest real estate listings and housing data as well as a CRM solution and analytics tools for real estate professionals. “Our development, operations, and QA teams automate everything in sight while offloading to cloud providers and other services the tasks that those providers are best suited to,” says Brian Baumgartner, QA Manager, BoomTown.
As only one example, BoomTown recently adopted Amazon Web Services (AWS) for hosting various aspects of its infrastructure. “It enables us to provide high availability and resilient service while allowing our development team to focus on revolutionizing the Real Estate software industry,” says Baumgartner.
Docker containers support regular deployment processes at BoomTown, containerizing the environment setup for a more reliable, orderly deployment experience, whether the firm is pushing code to QA or to production, explains Baumgartner. TeamCity, the increasingly popular and frequently appearing name in build management tools and CI servers courtesy of JetBrains is BoomTown’s go to build platform. New Relic is BoomTown’s performance management solution of choice, enabling live transparency into the production application and server performance that keeps BoomTown citizens (customers) content.
BoomTown Before & After Using Sauce Labs VMs to Scale Testing
Prior to applying Sauce Labs, BoomTown’s QA analyst team processed all software tests exclusively in a manual fashion. Every test run crossed their laps / machines / laptops. “Before a new feature release, an analyst would execute a full set of regression tests against the product or feature, often through several different browsers and versions to verify the new release and its function. This process alone took up to two weeks for large projects,” says Baumgartner.
BoomTown now uses a test automation framework constructed using several popular DevOps tools. The custom built testing automation leverages Ruby, Selenium WebDriver, Cucumber, MySQL, Sinatra, and Angular JS, says Joey Gryder, Senior Software Engineer in Test, BoomTown. The framework offers BoomTown views into historical and real-time test results as well as the capability to run manual tests through its dashboard front end software connection to its backend systems.
BoomTown applies a single web service to assert control and manipulation of test system automation features. A “test runner” library enables QA to run Cucumber features in a parallel execution approach. BoomTown uses MySQL to store test results. “When we combine all this with Sauce Labs’ VMs, we can schedule nightly test runs using our full testing suite on a variety of browser configurations,” says Baumgartner. This has enabled BoomTown to run up to 100 tests on software nightly.
Parallelization
Where time, resource, and person hours’ savings are concerned, the secret sauce in BoomTown’s software testing framework is its use of Sauce Labs VMs to support many automated tests in parallel. This has cut BoomTown’s longest test run time of an hour and a half down to less than 15 minutes.
“With Sauce Labs, it is trivial to spin up any number of parallel VMs, configured against your desired browsers to run through a set of Selenium WebDriver test steps. Without such a solution, BoomTown’s test team would have had to either execute its entire test suite locally through a single machine and browser, or build an in-house testing lab to achieve parallel browser testing,” explains Baumgartner. (Sauce Labs is obviously not the only player with competitors including BrowserStack and TestingBot; let your due diligence be your guide in selecting the best service for your needs.)
BoomTown saw offloading the work as the best option in order to accelerate otherwise time intensive testing tasks, preventing QA from becoming a sort of time-related bottleneck on CD. “We did this simply by increasing our VM count and running more tests in parallel,” says Baumgartner.
Adding VMs and increasing the number of parallel testing instances is an easily scaled approach that continues to speed tests and drive CD even as BoomTown adds tests and lengthens total test runtime.
Using automated parallel tests supported by Sauce Labs VMs frees up QA analysts to be more creative in their testing, finding more and more interesting ways to break the application under testing loads, says Baumgartner; meanwhile the automated testing ‘machine’ verifies the mundane and repetitive tests that the team has already established.
“We recently incorporated automation as a quality gate in our CI process. Before a pull request is accepted, it must pass a set of critical path UI tests that run up in Sauce Labs. This sets a certain level of quality expectations during our daily development process, which saves us time in testing,” explains Gryder.
