3.22.17
test IO, a leader in crowdsourced software testing, today launched platform-wide enhancements specifically for agile development organizations that have embraced continuous delivery. The enhancements enable customers to verify their apps’ readiness for release — with human testers on real devices — in as quickly as an hour. Crowdtests can now be triggered using an API as part of a continuous delivery pipeline, providing an unprecedented combination of speed and confidence to the software release process.
Continuous delivery is the process by which software development teams produce incremental improvements in short cycles, ultimately releasing code to production reliably at any time. Companies like Box, Etsy, and Hubspot can famously push code daily or more often, creating a competitive advantage. However, as organizations of all sizes move to implement DevOps processes, customers’ experience of product quality can suffer, even with high levels of test automation. By providing immediate human insight about a product’s release readiness, test IO helps customers such as Thumbtack, Europcar, and Thomas Cook remove risk from the software delivery process while delivering updates in record time.
While software teams have long embraced production metrics that track product quality over time, pre-production indicators have proven a harder challenge. Human feedback, in particular, has been difficult for most organizations to obtain quickly and scale predictably. test IO now solves that problem.
Enhancements to test IO’s platform available now to customers include:
• Release Readiness: test IO now benchmarks customers’ quality levels against their own products over time, ensuring that changes to code don’t result in a decline in end-users’ experience of product quality.
• Rapid Testing: test IO customers can now initiate a test and receive results in as soon as an hour, so a developer working on a feature can get functional validation over lunch or teams can “push on green,” knowing that no obvious functional problems have slipped through.
• Test Invocation API: test IO customers can now use a REST API to invoke tests automatically at critical points in their delivery pipeline, such as when code is pushed to a staging server or even to production.
People are uniquely qualified to discover critical issues that automated tests miss and to assess a product’s overall fit-and-finish, but manual testing can be a bottleneck in a rapid release process. test IO’s combination of speed and tester insight solves those problems.
“Many people assume that organizations like Facebook and Google don’t have people testing their software, but that’s not true,” said Philip Soffer, test IO’s CEO. “They’re just large enough that they can push that testing off on their employees and customers and still get results quickly. We’ve enhanced our product and optimized our processes to deliver human insight about software quality to companies large and small without the traditional manual testing bottleneck.”
Continuous Insight
While traditional crowdtesting approaches emphasize the number of bugs discovered as a key performance indicator, this way of thinking is insufficient for organizations that deploy frequently and expect few release-blocking bugs. test IO’s Release Readiness holistically benchmarks your product’s builds over time, providing a traceable KPI to ensure that new code meets your high-quality standards for user experience.
“We release small changes frequently with a high level of test automation, so we don’t expect a large number of issues at the end, and we don’t want marginal bugs to slow us down,” said Mark Schaaf, VP of Engineering at Thumbtack. “But especially on mobile we need to make sure that our users have a flawless experience. With test IO, we’ve replaced a painful manual internal testing process with human oversight that’s quick, transparent, and automatic.”
Release Readiness is benchmarked against testers’ evaluations of hundreds of other products. The result is a remarkably detailed scorecard that predicts your users’ experience of the product’s quality, including a predicted app store rating. These predictions are crucial, as studies have shown that 44 percent of customers will delete an app that doesn’t work, and 34 percent will go to a competitor’s app if they’re not happy with yours.
“For organizations that have invested in DevOps there are almost no impediments to delivering software to customers — except the question ‘Should I actually release now?’” said Soffer. “Our mission is to answer that question by bringing human insight to the software delivery process, wherever and whenever customers need it.”
