View all web browser and mobile devices available in our cloud-based test lab.
The market is about to see its next seismic shift with the impending release of iOS 16 and Android 13 update. Is your testing strategy ready for these new platforms? It is crucial for your mobile application quality and continuous end-user experience to adapt to these changes.
When a major mobile OS release is coming out, a few things should come to mind for both mobile app developers and testers.
What are the mobile device lab implications?
What are the potential implications for my existing testing scenarios?
What new testing scenarios should I be considering?
What is the timeline for the release and how does it impact my product release schedule?
Below, you will find the mobile vendor distribution across the U.S. market as of June 2022. iOS devices occupy about 60% of the market share, leaving roughly 40% for Android devices led by Samsung, Motorola, Google, Huawei, and LG. This distribution varies across different countries, which means — if you are serving customers globally — you need visibility and analytics across all your supported geographies so you can build the proper test lab.
Mobile Vendors Market Share in North America: Source: (Perfecto Test Coverage Guide)
Before diving into the above considerations, let’s highlight some of the key changes that are coming with Android 13 and iOS 16.
Apple is in an iOS 16 beta phase and the new release that’s expected to go out in September 2022 will be packed with new capabilities. The new features will include UI and UX enhancements, productivity upgrades, and boosts to privacy and security.
With the new iOS 16 and iPadOS 16, users can get richer accessibility support through live captions in FaceTime calls, conversations, audio, and videos. In addition, the new OS version will automatically detect doors and red signs and instruct the user on how to unlock them. The upcoming release will also come with other accessibility enhancements like voice control call hang-ups and more.
With accessibility compliance becoming a core requirement for all digital apps, web, and mobile, teams must build their apps with Accessibility fully enabled throughout the entire user journey. From a testing perspective, automating as many of these scenarios across all iOS devices and screen sizes is key for velocity and continuous compliance with the WCAG and other country specific requirements.
Perfecto supports testing apps against this new iOS 16 version both manually and through automation in its clouds. That way you can start exploring your app for both functional and accessibility testing right away.
iOS 16 brings another change in the support for authentication through Face ID when the device is set to landscape mode. This is a great change from an end-user experience, but it is also a new testing scenario that needs to be covered across all supported iOS 16 devices.
Apple is revamping its entire lock screen and notification abilities to make them more widget-rich and customizable by end users. From a testing perspective, these changes do not present many implications. However, if your app does send push notifications to the end user, it would be wise to validate these notifications across different layouts and devices running the new and older OS version. Find out how it appears and if there are any presentation issues.
Like with any new iOS and iPadOS releases, Apple introduces its new generation of devices and — in most cases — terminates support for older and legacy devices. With the new iOS 16, four new iPhone 14 devices will continue the current iPhone 13 series. The following devices will no longer be eligible for an upgrade and will remain on iOS15.x:
iPhone 7 Plus
iPhone 6S Plus
iPhone SE (2016)
iPad Mini 4
iPad Air 2
iPad Pro 2015
iPad 5th Gen
Of course, this will invariably affect your testing strategy. Engineers and developers who see reasonable usage across the above devices (many of which are still quite popular globally) should continue testing on them with the iOS 15 OS generation. Testing with iOS 15 in these cases will ensure that the new app versions work and perform properly on these devices as well.
Apple previously introduced App Clips as a subset of their apps. They allow users to view apps via browser links or NFC links. Then, if the user enjoys the app, they’re able find the full version and download it from the App Store.
For these types of applications, Apple is now increasing the App Clips size limit by 50%, adding support for live activities within the apps, and providing more precise location suggestions in Spotlight and Siri so users of App Clips can enjoy more context around them.
While Apple is supposed to release its major update towards mid-September, Google is slightly more advanced in its release readiness and is expected to release its GA of the Android 13 update in August 2022.
The new Android update — which will be supported on most of the Pixel devices like the new Pixel 6A Smartphone and other Android devices — will see Google include some nice UI changes like themed app icons for more end-user customized design of the home pages.
Additionally, Google will include the following changes to its major Android platform.
For users that speak multiple languages, Google will now offer a new settings configuration that can allow the users to specify the default language per application. Developers would need to add these new configuration options within the application manifest file. There will also be improved language support for Japanese and non-Latin scripts.
For applications that are being used across geographies and languages, this new capability would need to be tested properly assuming developers make the necessary changes.
Like with iOS 16, Google also enhances its built-in accessibility capabilities with new Audio Description functionality. Users can enable narration track across all apps, which means that you can listen to the narrator talking through the presentation and describing what is happening on the screen during natural pauses in the audio.
As previously mentioned, ensuring full accessibility compliance across both smartphones and tablets is a mandatory requirement that can result in fines for missing some of the required capabilities. Make sure to test your app accessibility against each software iteration your app accessibility, but especially for Android 13. While it is still in beta, make sure that your current app is compliant with the older and newer requirements.
With Android 13, Google will also make security, privacy, performance, and other “cosmetic” changes to its platform. To read more about the entire scope of the release, click here.
Perfecto already provides support for the latest and second public beta of Android 13 in its clouds, so you can start testing immediately.
Noting the key enhancements for both iOS 16 and Android 13, app developers and testers should look at their current testing lab configuration and their entire testing suites (manual, automation, functional, accessibility, performance, etc.) and start making the required adjustments to their tests, devices, and schedule so they are ready for what is coming in the next few months.
With all these key considerations, here are a few tips to keep in mind:
What devices are missing in my lab, what OS versions do I need to keep vs. retire going forward.
Configure my regression cycles (within CI or other triggers) to run against the most relevant platforms (including Android 13 Beta/iOS16 Beta).
What are the potential implications on my existing test scenarios?
Perform sandbox assertions of your entire iOS and Android application testing scenarios against the beta versions of iOS 16 and Android 13 (preferably on a Pixel device). Analyze flakiness and bugs that are platform specific.
Capture test automation framework changes that might require test code refactoring (Appium, Espresso, XCUITest).
What new test scenarios should I be thinking about?
For newly supported features like accessibility changes within the app or other enhancements that can utilize the new features of iOS/Android platforms, allocate time to design and create new test cases.
For new mandatory platform capabilities like landscape Face ID support within iOS 16, build new test cases and include them within your test runs.
What is the timeline for the release and what could the impact be on my product release schedule?
Market events are important as other changes are happening in the marketplace. Teams must account for these disruptions and allocate time within the upcoming sprints to prepare for the new mobile OS releases. Both developers and testers should ensure that their apps are ready for the upcoming GA releases for functionality, performance, security, and accessibility. Without built-in time slots in their schedule, it will not happen.
By now, mobile app DevOps teams should be used to the annual release cadence of Apple and Google. But being used to these events and being prepared are two different things.
It is highly recommended for mobile teams to start looking into the above releases, their scope, and implications on the application functionality and quality and make sure that the testing assets, testing environments, and schedules are all ready for what’s coming.
START TESTING NOW
DevOps Chief Evangelist & Sr. Director at Perforce Software, Perfecto
Eran Kinsbruner is a person overflowing with ideas and inspiration, beyond that, he makes them happen. He is a best-selling author, continuous-testing and DevOps thought-leader, patent-holding inventor (test exclusion automated mechanisms for mobile J2ME testing), international speaker, and blogger.
With a background of over 20 years of experience in development and testing, Eran empowers clients to create products that their customers love, igniting real results for their companies.