By Jean Ann Harrison
As a mobile software tester, it is important to incorporate various hardware and operating system (OS) conditions to see how software is affected. There are testers who do not consider testing hardware or OS conditions a necessary part of a mobile tester’s repertoire. “You’re wasting your time” or “it’s not our job, we only test the software,” and other such comments have been told to me.
Many of these naysayers are responding based on one type of mobile testing project. These mobile testers are not considering mobile software is a part of an entire system where software, OS, and hardware all work together for the complete user experience. These mobile testers do not consider that hardware and OS conditions have a direct impact on software behavior.
Usually mobile Web applications (apps) do not have a strong dependency on the actual device, whereas mobile hybrid and native apps do. However, there are still some device-dependent mobile Web apps and mobile Web app functions.
Examples of Engagement
Consider specific examples of how hardware and OS conditions affect software behavior. Notifications are something we take for granted when using a device. Does your software app use notifications? Did you consider testing the notification action beyond the positive test case? Did you ask “What if …?” Notifications are functions where the mobile app engages with both the OS and the hardware to display that notification visually, through audio, or both.
Why don’t we explore the example further by looking at set up? Say your mobile hybrid app is designed to send both a visual and audio notification to a device once the app receives data from another source. That data is packaged in a particular protocol and the notification uses both visual and audio indications. What tests would you perform beyond sending data packed per the protocol to witness the device behavior?
Remember, not all devices are equal. So testing on the actual device becomes important for this type of test. Some companies offer virtual access services for various devices where you can rent time to conduct your tests. Examples of mobile device testing providers include Perfecto Mobile, Device Anywhere, and Keynote. Do your research before determining a budget to build your own test lab.
Checking notification functionality involves reviewing whether the software receives a message and reads the protocol to accept the message. The software then generates a notification to the OS to display that notification.
Test Case Ideas
Several test case ideas can help determine a Web app’s behavior based on device-driven scenarios, and mobile testers should consider an end-to-end approach that covers all possible outcomes when it comes to how hardware and OS limitations may affect app behavior.
Timing/response testing determines how soon the software receives the message to display a notification and how long the actual display occurs. Consider the following “what if” scenarios.
Interruption
Interruption testing shows what the app does if the software receives an indication to send a message to the OS to display a notification, and the device receives a phone call before the message is sent to the OS.
It can also indicate what happens if the device receives another notification. Does the notification display?
Battery/Use
Battery/use testing determines what happens to notifications on a low or charging battery. Does the display function change at all? If the notification is supposed to be both audio and visual, are both notifications functioning properly?
Does the notification behavior change if the device is hot from use or a battery charge?
Resource
Resource testing shows an app’s behavior when resources are not optimal. What if the device’s storage is almost full? Does the software’s notification messages generate properly? How does the app handle the lack of resources? What limit does the app need before problems arise?
These example scenarios prove that these questions are all relevant to mobile testers. The notification function itself is primarily a mobile function of the app being tested. A hardware tester would not try to determine how software behaves based on these resources.
End to End
Software affects hardware and vice versa, but when it comes to understanding software behavior, the mobile software tester needs to be a talented test designer. Requirements are rarely written with consideration for hardware and OS limitations.
We don’t know what we don’t know. Mobile testers need to be a bit like Christopher Columbus; if not respected for the complexity of the testing, the test designer is given no time to complete their exploring. Changing various hardware and OS conditions helps flush out more information on the software behavior.
This is why I advocate not relying solely on automation and emulators. There is a need for testing directly on devices and for manual testing that includes the hardware and OS conditions, which can reveal a buried treasure of bugs. SW
Jean Ann Harrison has been in the software testing field for over 14 years including eight years working with testing mobile software on various devices, such as medical devices, city police ticket generators, phones, tablets, and various other proprietary devices. SW
Jul2014, Software Magazine
