The Internet of Things (IoT) is becoming a bit more than a fun three letter acronym. And while internet connected devices have been around for a while, turning it into a practice is an important evolution. It allows the establishment of best practices, better integration into standard software development, and thus more robust applications, sometimes too robust. And because software drives everything, and software quality is key. How can you ensure quality in all those connected devices?
Mobile phones have Sauce Labs, Perfecto, Xamerin emulators. They are standard and a great way to test mobile devices. But when you are building the device yourself, unless you spend a large chunk of time building your own physical or software emulation, testing is near impossible. How can you achieve the same level of testing?
As is dependably the case with new or more pervasive technologies: how would you test the product on the day by day work process? What are the new testing concerns? What skills do you have to learn to respond to this staggering growth?
Challenges that a tester can face with our current functional testing process and tools:
- Replicating a fully-connected environment requires too much money and effort—and sometimes it’s simply inconceivable (such as when we need access to not-yet-implemented components being developed in parallel).
- The subsystems, subcomponents, and services that are interconnected are owned by different groups and third party vendors. Inability to access just one dependent subcomponent could impact the testing of the whole application.
- Considerable effort and coordination among different teams is needed to get the right test data populated across all the systems.
- Availability of the device only with limited capacity and at inconvenient times
- The problem of testing for the IoT raises issues of responsibility. Who is at fault if the device in your vehicle starts emitting erroneous data that causes On-Board diagnostic connector to report non-existent events that destabilize the control loops used by different parts of the vehicle, causing a failure that leads to an accident? Is it the vendor of the device or the controller for being unable to cope with extraneous events? Your customers will hate if you use classic test approach.
New solutions needed
The heterogeneous nature of IoT Services demands for strong test capabilities to ensure the performance of the services meets the user’s requirements as well as service level agreements between service providers and consumers.
- A strong test strategy – Think test first
You need to focus on testing methods and practices for effectively executing a testing job. Clear requirements, detailed test plan, unit testing, continuous integration, good communication etc should be a part of your test strategy. Good programming approaches and methodologies will release a high-quality product.
- New platform and test tools
New platforms will be needed for communication and to effectively extract actionable information from vast amounts of raw data, while providing a robust timing and systems framework to support the real-time control. You will need advanced tools, consoles, viewers and simulators or your test effort will be severely hampered.
- Gray Box testing
To design effective test cases, you need to understand the architecture, the OS, third party hardware, apps, firmware, new connectivity protocols and hardware device limitations as fast as possible.
- Real Time Operating System
The number of “things” now available to connect shows a lot of time-dependent behavior. Timing is the most critical factor for making the IoT work. Safety critical systems, from medical devices to house alarms; need faster response time. RTOS being the core of IoT must evolve to deliver the scalability, modularity, connectivity, security, and safety that are demanded by the remotely managed world of machine-to-machine (M2M) networks and IoT.
- Test environments with automated test configuration
Ontology-based testing technique focuses on approaches to automate the test case production process and input data generation.
If the vast majority if you’re functionality is packed into a strong backend then it makes everything easier, because you can easily test backend functionality with traditional testing tools.
DevOps teams and agile workflows can accelerate the process but there is one thing that can hinder even the best DevOps group, and that’s waiting on dependent components. Testing code and framework incorporation can demonstrate unimaginable if your test environment is missing crucial dependencies. The limitations can be overcome by Service Virtualization.
Service Virtualization eliminates Functional Testing Roadblocks
Service Virtualization can speed up things significantly enabling testers to be less dependent on test infrastructures, evading the glitches of interconnectivity. It emulates only the behavior of the specific dependent component. Instead of virtualizing an entire database, you monitor how the application interacts with the database and then replicate the related database behavior.
Service virtualization ought to be used for testing when:
- You can’t test the real-time data because it is out of your scope – for example, a payment service that doesn’t offer a testing environment.
- Your application framework is complex to the point that you can’t test on it without disturbing normal business.
- You shouldn’t open real-time information to a possibly non-secure testing environment.
- You have to test against an alternate framework that is still under development and isn’t yet accessible to test against.
Let us take an example of Home Automation
As you probably know, Nest thermostat uses Nest Sense (an exclusive combination of sensors and algorithms) to notice when you’re away and when you come home. The thermostat can be programmed and remote-controlled by internet-enabled mobile devices enabling homeowners to control and read out the parameters onto a display. In an Internet of Things (IoT) world where the home can be easily connected to the Internet, it could automatically search an online information base and report a fault code based on the personal history or on the environment (an expert system tells you if there’s smoke, which room it’s in, and if you’re in immediate danger so you can quickly figure out the best way to exit your home and notifies your local fire department).
Cloud computing provides a unique opportunity developers to work on sensor technology and expose this rich information through smartphone. With all the latest test strategies, methodologies and approaches the biggest challenge which testers can face is to test their prototypes by physically connecting to the sensors to get the data and test their software and hardware. Early prototype hardware could be unsafe and furthermore no-one wants to use a thermostat while keeping an eye on their untested microcontroller board project that they’ve plugged into the thermostat.
Testing a planned smart house before really constructing it is considered as an impediment towards an effective smart house venture. This is because of the variety of sensors, home appliances and gadgets are interconnected in a smart environment. One way to work toward a solution is to create a system which can simulate a home. To all intents and purposes, under certain conditions, it should behave exactly like a real home would, only in the comfort of a lab rather than in real smart environment.
Service Virtualization generates, deploys, and manages simulated dev/test environments by implementing a multi-purpose smart house simulation system. This simulator provides the ability to design the house plan and different virtual sensors in a two-dimensional model of the virtual house environment. It can connect to any external smart house controlling framework, providing assessment abilities to their system much easier than before. Scenarios can also be defined for testing various possible combinations of device states; so different criteria and variables can be simply evaluated without the need of experimenting with a real environment.
There is no magic answer to how to test the IoT. And I expect this is the next mini-explosion of tooling in the space. With the arrival of more industrial IoT solutions functional testing for IoT and integration tests will be necessary part of the delivery and maintenance processes. And eventually it will just be another component of the standard software delivery pipeline.
