How to Automate Testing for Optimal Customer Experience

During a sudden surge in online traffic like the one many companies have seen over the past few months, automated testing can help organizations optimize the customer experience

Imagine you manage an e-commerce office supply company and are dealing with a sudden surge in traffic—with more people working from home, demand for your products has grown significantly. But when a customer attempts to complete a purchase, they encounter a debilitating error such as a broken button or a page that won’t load. Now imagine this happening to hundreds of customers simultaneously, during the most critical time of day for purchases on your site. Imagine the revenue lost, the customers never to return, the complaints on social media—it’s a situation everyone wants to avoid.

But this is a situation many will have to handle at some point. Web pages are large, complex beasts made up of different images, fonts, style sheets, local JavaScript code, asynchronous requests to the backend and more. It’s easy to forget how many scripts hold a website together. And any small change can have a butterfly effect on the overall page performance.

What does the user actually see and experience on the website? And what happens if changes are made in the backend? Does the “Buy” button suddenly disappear? Is everything still working as it should? To understand how an application behaves with users, companies need to understand those users’ experiences when they interact with the application over a browser. All too often this means manually checking the availability and functionality of individual online applications, and then hoping that users do not stumble upon additional problems that may have fallen through the cracks.

Accounting for and identifying these potential problems and regressions is not only an uncomfortable process, but also a lengthy one. Especially in times of increasing traffic and new digital business, as we’ve experienced in these times of WFH and e-commerce, the continuous review of all digital experiences is essential. A customer’s first impression is crucial, and even a long-term customer may go elsewhere if a page doesn’t load or if a button doesn’t work. Options abound on the internet—you can’t risk someone giving up on your business and trying a competitor’s.

Why Automating Browser Tests Is Important

In browser-based applications and websites, browser tests can monitor the user experience and ensure that user actions, such as registering for a new account or adding items to a shopping cart, work smoothly—even after changes in the backend.

Popular browser test solutions have long since reduced the manual effort required to confirm that new features didn’t break already existing functions, but the implementation and maintenance of these tests is very time-consuming. In addition to learning a new test framework for these solutions, teams need to set up a test infrastructure and update faulty tests with every change to the user interface, costing valuable time that could be used for new product development.

The latest generation of fully automated browser tests, on the other hand, allows everyone on the team to record and automate rich browser-based test scenarios in minutes—without any code. The “Tester” opens the corresponding website or application via the test tool and starts the recording. As she interacts with the website or application, the solution automatically records all actions as a series of steps, which can then be edited or changed when reviewing the recording. Assertions can also be added, such as whether a user sees a welcome message when they log in.

If a test is recorded, it is possible to define how often and on which devices it is to be carried out and from which places around the globe the tests are to be played to mimic users’ behavior. The recordings can be played over a variety of websites, including internal, non-publicly accessible applications or pre-production and CI environments. For each test run performed, the end-to-end duration of the test and a visualization of the duration of each step are provided. A screenshot is automatically created for each step and each query so testers can follow exactly what the user sees in each respective step. All the frontend errors and resources happening along the test are also collected for a simplified troubleshooting of failing test runs.

Because websites are constantly modified by developers, once a test is created, machine learning algorithms detect changes to the application and automatically update the tests accordingly. If a browser test fails due to frontend or backend problems, an automated tool provides the context for quick troubleshooting. In addition to screenshots, frontend errors and frontend resources, the behavior of the backend services and infrastructure for this specific step is also provided.

Using Time More Effectively

Automated browser tests have traditionally been notoriously fragile, often breaking with minor changes to the user interface. To eliminate false positives, today’s ML-supported browser tests mimic human decision-making processes and adapt intelligently to changes in the application.

For example, if a button is moved in a web application or the identifier of an element changes, the browser test derives from this how the existing test should be carried out in the updated user interface. If all steps and assertions in the test are successfully completed, the saved identification mechanism and test definition will be updated to reflect the changes on the page.

If you’re always in a reactive mode, putting out fires that could have been avoided, the impact to revenue and customer satisfaction is only one half of the story. The revenue not generated, and customers not acquired, from improvements and new features, is the real problem for a business’ long-term health. Ultimately, browser tests give teams more freedom to focus on developing new features rather than manually reviewing functionality or redefining test frameworks over and over again. And in a distributed, work-from-home, always-online world, new product development is what wins the day.