Submitted
By John J. Paliotta
In the early days of software, most manufacturers outsourced development to specialist companies. At the time, product manufacturers were big-factory metal bending companies that focused on the end products rather than the small amount of software embedded within them. Although software was intriguing for the flexibility and cost savings that it provided, it was not a core competency. These companies did not consider themselves to be software companies.
As software evolved to become a critical part of many products, manufacturers realized that designing the hardware and software together resulted in better outcomes, and they began bringing software development in house. This evolution came with its own set of challenges, as organizations struggled to build the internal expertise required to efficiently build software.
40 years later many companies are realizing that they are actually software companies, and that software is critical to differentiating their products from competitors and building their brand.
We’re Not a Testing Company
When organizations brought software development in house, the state of the art was a phased development process that came to be called waterfall development, a linear process consisting of design, coding, and testing, in that order.
Since software testing comes last in this model, guess where the schedule pressure is when coding goes longer than expected, and a release date is looming? An attractive solution to this problem was to outsource testing to low-cost providers, and this is still a very common model today.
Software testing and quality assurance processes continue to be outsourced to subsidiaries and third-party suppliers because these activities are valued less than the design and development activities. Testing is not seen as a core competency, or value-added activity, it is seen as a necessary expense.
While all companies believe in quality, and trumpet that belief in their marketing materials, most do not consider themselves to be software testing companies.
Becoming a Quality Company: Make Software Testing a Core Competency of Businesses
Everyone involved in the software industry would agree that software quality could be better. After all, Cambridge University recently published a report saying that software bug fixing and maintenance now cost the global economy more than $300 billion per year.
Making software testing a core competency will enhance product quality, reduce maintenance costs, and improve customer satisfaction. These are all great outcomes, but where can we start?
Step one is to realize that quality cannot be “bolted on at the end” by sending the completed application to the test group. Everyone on a development team has a role to play in software quality.
Step two is to understand that the goal of testing is to formalize correct behavior in a measurable way. A well-designed test will provide value for years to come and pay for itself many times over.
Measure testing completeness, developing high-quality tests, sharing tests across teams, and making testing automated and continuous are four activities that will improve quality dramatically.
Measure Testing Completeness
Source code coverage analysis is an easy-to-implement way to measure the completeness of existing tests. Measuring code coverage for all types of testing and combining this into a single view provides valuable insight into under-tested and over-tested areas of the application. By understanding gaps in existing tests, users are able to improve testing efficiency and fill these gaps with new tests over time.
It is critical to understand that while code coverage measures testing completeness, high levels of code coverage do not imply high levels of code quality. 100 percent code coverage should not be the goal of testing; it should be the result of complete testing.
Develop High Quality Tests
Just as many groups use coding standards to ensure consistency and portability of source code, testing standards are needed to ensure that good quality tests are developed. All test cases should capture preconditions, test values, and expected results. No special knowledge should be required to run a test or to determine if it passed or not.
Share Tests across Teams
Testing has historically been done in silos with the majority of tests being owned by the software quality assurance (SQA) team. If developers do not have access to these tests, then it is not surprising that bugs are found by SQA late in the release cycle—after new features are integrated and these tests get run for the first time. Sharing tests offers a variety of benefits to the whole team. Bugs will be fixed on the day they are introduced, not weeks later. Code changes that break existing tests will not be integrated team members will focus on edge cases versus nominal path testing. And, SQA will have more time to write new tests.
Test sharing will provide all stakeholders a sense of ownership for application quality.
Make Testing Automated and Continuous
A robust testing infrastructure requires that tests are small and run fast, execution is automated and dependable, and results are easy to understand. This will enable any team member to run any test, at any time, on any version of the code and to easily interpret the results, and most importantly, it must be easy for developers to debug failing tests.
Conclusion
An improvement in quality will only happen when companies realize that testing and quality need to be part of their core competency. Integrating testing into the day-to-day activities of everyone on your development team will give them a sense of ownership for total product quality. Keeping testing separate sends the message that testing is “not my job” or that it is “not valuable,” especially if it is outsourced to a low-cost provider with no measurement of quality improvements.
Aspire to be a quality company, make software testing part of your core competency. SW
John J. Paliotta co-founded Vector Software in 1990. In 1994, he and William McCaffrey built the first version of the VectorCAST product for avionics, military, and space applications. Currently Paliotta serves as the CTO and oversees all engineering and quality assurance activities. He received his AB in Math from Boston College. He writes a weekly blog on software quality.
Dec2015, Software Magazine
