Perfecto Mobile Launches Industry's First Global Benchmark for Mobile Testing Coverage

The Mobile Test Coverage Index Provides Organizations with a Valuable Asset for Mobile App Testing Strategies in North America and Europe

BOSTON, May 18, 2015 – Perfecto Mobile, the world’s leader in mobile app quality, today announced the release of its Mobile Test Coverage Index, the industry’s first data-driven benchmark to help organizations determine the appropriate mix of mobile devices, operating systems and form factors to test their apps and responsive websites and ensure optimal app quality for end users.

Today’s mobile market is extremely fragmented and constantly changing, challenging organizations with device mix obstacles, future releases, and the pressure to deliver flawless customer engagement. Currently, most enterprises have no standard approach for testing mobile apps and responsive websites, and the business impact that non-tested or poorly tested apps have is devastating to a brand. To help begin this testing process to ensure continuous quality and develop a comprehensive mobile strategy, enterprises need to define a set of mobile devices to test apps and responsive websites against, a strategy that best represents their target market and end users. This should be part of a larger definition of the user profile, such as the environment that the app is operating in (network conditions, GPS location, etc.). To answer the vital question “what devices should I begin testing on,” the first-ever industry benchmark report provides crucial information and insight into market trends and the correct device mix to ensure coverage.

Perfecto Mobile’s Quarterly Mobile Test Coverage Index aggregates 360,000 hours of exclusive Perfecto Mobile customer usage data, market data, and future device release timelines to create an extensive list of the 32 most relevant devices. It serves as a recommended starting point for organizations beginning the test coverage process.

“The first question both business and technical executives alike ask when building a mobile testing strategy relates to coverage. A relevant device and environment mix offers strong test coverage, getting the organization much closer to the true end user experience,” said Roi Carmel, Senior Vice President of Products and Strategy, Perfecto Mobile. “It is challenging to achieve desired test coverage for mobile end-user profiles in today’s rapidly expanding ecosystem, especially since a mobile device’s shelf-life is around 9 months. Our goal with the quarterly Mobile Test Coverage Index is simple: Help enterprises better understand their customers and automate app testing to deliver a high-quality digital experience.”

The report provides an index for reaching 30 percent, 50 percent and 80 percent test coverage in North America, France, Germany, Italy, Spain and the UK. The device selections address five key sectors: device model, screen size, GA date, hardware system and recommended operating systems. The number of devices in each test bracket may vary from quarter to quarter to reflect the changes in the mobile market. The report will be continuously updated to meet these market changes.

Organizations can use the index as a benchmark to take advantage of the three levels of mobile market coverage and scale as their needs grow. Key takeaways from the research include:

• 30 percent of the market can be covered with 10-16 devices, based on market adoption, market device leaders, reference devices and device characteristics.
• 50 percent market coverage can be covered with 17-25 devices. This level of test coverage yields an increased number of devices from manufacturers such as Sony, LG, Samsung, Apple, HTC and more.
• 80 percent market coverage can be achieved with an index of 26-32 devices. This is the final group of devices to test on that will guarantee the most recommended coverage.
• While Apple pushes its users to upgrade to the latest iOS version, there are earlier iOS 8 versions that should be included in test plans. Additionally, iOS 7 still accounts for 17 percent of total users, and should be included at the same level of iOS 8 test plans.
• Including the most widely adopted Android versions KitKat and Jelly Bean is important to an Android test plan, Lollipop proves to be very important as well. Lollipop, the latest OS from Google, is running on the Google Nexus devices (considered reference devices).

This report analyzed and aggregated a wide range of public, private and proprietary data from 4,000+ device profiles and 360,000+ user hours across U.S. and EU markets, which includes Spain, UK, France, Italy and Germany. 

About Perfecto Mobile

Perfecto Mobile, the world’s leader in mobile app quality, provides a hybrid cloud-based Continuous Quality Lab that enables mobile app development and testing teams to deliver better apps faster. The Continuous Quality Lab supports testing processes earlier and more often in the development cycle, giving way to faster feedback and improved time to market. Users can access an exhaustive selection of real mobile devices connected to live networks around the world and leverage them for testing and monitoring throughout the mobile application development lifecycle – from development, functional and performance testing to monitoring and support. More than 1,500 customers, including many of the Fortune 500 across the banking, insurance, retail, telecommunications and media industries rely on Perfecto Mobile to deliver optimal mobile app functionality and end user experiences, ensuring their brand’s reputation, establishing loyal customers, and continually attracting new users. For more information about Perfecto Mobile visit www.perfecto.io or follow us on Twitter at @PerfectoMobile.