By Jean Ann Harrison
Too often, I see project stakeholders and testers alike refer to the task of testing a mobile software project or application as “mobile testing.” But mobile testing is a broad subject with four distinct architecture types.
Mobile testing projects include a mobile native application (app), mobile hybrid app, mobile Web app, and mobile Web site.
Let’s take the ambiguity out of the discussion and distinguish these four mobile testing project types and review distinctive tests based on each type.
Hybrid App
I like Julian Harty’s definition, “a hybrid app is a combination of native code—often written by a person, but may be written by a code-generator or provided as part of an app generated by a code-generator, and a Web view control embedded into that app. So the app includes two technologies, a Web view, where HTML, JavaScript, or CSS are rendered and processed; and native code, which provides the rest of the functionality and UI.”
Processing in a hybrid app uses both the Web server CPU and the device CPU and thus, specific types of tests need to be designed and implemented for the hybrid app testing project. When designing test cases, the tester must consider when the native code is invoked and when the code interacts with the Web view code. Test designers must have architectural knowledge of how the system works, including inter-dependencies within the code. All mobile testing project types must consider timing. However, with hybrid apps, timing tests can become more complex due to the architectural design. Testers need to be aware of when the Web view code is accessed, when native code is accessed, and how hardware and firmware conditions affect the software app. For example, how does the app on the device behave while charging, connecting to a Web database, performing a database search, and then accessing data to be displayed within the app? Add timing as a condition to this test including accessing the app itself, connection to a Web server, performing a search, transmitting the data to the app, and displaying it. What kinds of graphics need to be displayed within each phase of the objective? Are these graphics native code or are they embedded within the CSS, JavaScript, or HTML code? The complexity of both types of technologies interacting for one purpose makes tests unique to the mobile hybrid app.
Native App
Native apps have code contained within the device. All code is downloaded to the device, all processing is done from the device’s CPU, and the app does not contain HTML, JavaScript, or CSS code. The GUI displayed does not utilize a Web browser or connection to the Internet. A native app can interact with the Internet but only for the purpose of sending data from the device to another app. No data processing is done outside of the native app existing on the device. It is a self-contained app, dependent on the device’s firmware and hardware to function. Like the hybrid app, tests surrounding hardware and firmware conditions play a large part in the design of test cases. However, the focus of test cases remains strictly within the device’s own boundaries. Of course there are communication tests, but these tests are limited in comparison with a hybrid app. As such, more tests will cover the interactions between the code and the device itself.
The temperature of the device’s battery plays more importance for the functionality of the native app because processing also generates more heat within a small contained area—the device.
Load testing for the native app is not defined the same way it would be for hybrid, Web site, and Web app testing. Native apps have one user, which is the “load.” However, there are other types of load to consider in the tests. What about the load on the CPU speed while the native app is performing some function? There is load coming from other apps running in the background while your native app is used/in focus/active. Consideration to the resources native to the device is critical when designing native app tests. Native apps do not have the same dependency when it comes to security because interaction outside the device is not part of the functionality. Native apps may send/receive data via wireless transmissions, which should have security tests for data integrity, but if data transmission is not a high priority, security tests will have a lesser focus.
Native app testing requires the tester to consider a variety of device types when building a testing lab. Many tests designed for native apps require direct device testing and cannot be emulated or simulated. Planning a test lab then becomes a significant part to native app projects.
Mobile Web Apps
Mobile Web apps depend on user interaction to fulfill their purpose. User experience, therefore, is different between mobile Web app and mobile Web site software projects. To help understand the difference, look at a restaurant’s mobile Web site and compare with a retail Web site like Amazon.com. Would you apply the same tests to both types of mobile entities? Sure, the restaurant’s site might include a form to fill out; but the Web site does not require the user to fulfill its purpose, which is to display information about the restaurant itself.
Attention to security, data validation, load, stress—on several levels, functionality, network communication, browser compatibility, and carrier tests will have strong focus. Add usability, trainability, and desirability and testing the value and usefulness of the Web apps to a mobile environment creates more complexity than testing a mobile site. Device conditions are not a high priority. The app itself will not change or have problems based on battery consumption. However, the Web app is required to consider device resources and document as such.
Mobile Web Sites
Mobile sites are designed specifically for displaying on a small viewing area. Therefore, there is little if any interaction with the user other than changing what is displayed based on the content of the Web site.
Testing efforts on mobile Web sites include whether it is pleasing to the viewer, easy for the viewer to access considering browser compatibility, secure to view on a user’s device, and performs as expected. Do the images and text display in favorable color and size? Does the content display well on a smaller screen? Does the mobile site load in a reasonable amount of time? Can it load despite heavy traffic either to the Web site or mobile carriers?
Testing Types
Common tests are applied to mobile testing projects and each environment provides specific tests to each. Remember to differentiate which type of mobile testing is the subject at hand and avoid ambiguity. Each mobile project type has a distinct perspective and requires the tester to plan accordingly. SW
Jean Ann Harrison has been in the Software Testing field for over 14 years including eight years working with testing mobile software on various devices including medical devices, city police ticket generators, phones, tablets, and various other proprietary devices.
Feb2014, Software Magazine
