View all web browser and mobile devices available in our cloud-based test lab.
Table of Contents:
Using a testing pyramid is an important part of your software testing strategy.
To successfully release an app out into the world, it needs thorough testing. And to successfully test an app, it needs a solid test automation strategy.
You probably already know that you do not automate all test cases. You concentrate your testing efforts on some types of tests more than others. And the platforms you use differ depending on the type of testing.
All of this is mapped out in the test pyramid. Use this blog as a guide to match the proper platforms with the right types of testing within your pipeline.
The testing pyramid is a concept that groups software tests into three different categories. This helps developers and QA professionals ensure higher quality, reduce the time it takes to find the root cause of bugs, and build a more reliable test suite.
The testing pyramid is a concept that strategically groups software tests into different categories.
From top to bottom, the main layers include:
In the world of mobile testing, the pyramid also lends insights into the types of devices your team should use for each testing layer.
The lower portion of the pyramid is more automated, making those tests faster. The top of the pyramid is for slower and more manual test scenarios. The higher you go on the pyramid, the fewer the tests.
In other words, teams should have many unit tests, which are small and isolated, and not so many end-to-end exploratory tests, which are time-consuming and manual. Frontloading tests at the unit level makes fixing bugs quicker and easier, as opposed to defects that are discovered much later in the process.
Now let’s walk through each layer of the pyramid, working from the bottom up.
The base of the pyramid is unit testing. The scope of unit tests is small — and they ensure that isolated units of code work as they should. Unit tests should test one variable and not rely on any external dependencies.
Adopt the Testing Pyramid for a Bulletproof Mobile Testing Strategy get my guide
get my guide
In this layer of the pyramid, tests are executed pre and post-commit. Tests are dev triggered.
We recommend that you use virtual devices — also known as simulators and emulators — for unit tests. Executing these tests on virtual devices is sufficient and provides the necessary feedback and debugging/log information.
Plus, virtual devices offer the benefits of being faster and more cost-efficient for testing.
In the middle of the pyramid, you test across integrations of more than one system. Unlike unit tests, these tests are CI-triggered. In this stage, you have more features to cover and more testing types to conduct.
With integration testing, it is critical to get the right balance of real and virtual devices.
Some types of testing, such as acceptance testing, can be done on virtual devices. However, other types of tests, such as performance testing, security testing, and certain accessibility features, cannot be done on virtual devices, so it is best to use a mix of real and virtual devices at this stage.
You will also want to add additional real device/OS configurations that match the client usage analytics and market trends. You can get device/OS usage data in our Test Coverage Guide.
Using both real and virtual devices in your testing strategy is a best practice. Real and virtual devices complement each other and support your goal of achieving high-quality builds for nightly regressions.
The top layer of the pyramid is for UI and exploratory tests. These tests generally are more complicated and have more dependencies than unit and integration tests. While teams can speed up UI testing with automation, exploratory testing is typically done manually and takes longer.
Keeping in mind the shape of the pyramid, the tests in this section should be far fewer than those at the base.
Once you get to production monitoring at the top of the pyramid, testing must be done on real devices.
End-to-end testing, as well as exploratory testing, requires real devices to more closely mimic the app on a user’s device — because real users report defects on real devices, not on virtual ones.
It is also critical to apply real user simulation to your tests. Testing with conditions such as conflicting apps, call/text interruptions, network throttling, network latency, and more enables you to see how your app will hold up out in the world under varying circumstances.
The testing pyramid strategy doesn’t need to be daunting. Make it easy with Perfecto.
Perfecto’s cloud gives you access to all of the devices, platforms, and browsers that you need to test your app on. You can use the cloud to ensure proper test coverage across platforms. You’ll even be able to apply environmental conditions by using real user simulation.
Plus, with Perfecto, you’ll be able to execute your testing strategy at scale — and deliver true ROI to your organization.
Learn more about implementing the testing pyramid strategy to boost your mobile testing. Or see for yourself how Perfecto can improve coverage across platforms.
improve my mobile testing strategy ▶️ watch a demo of perfecto
DevOps Chief Evangelist & Sr. Director at Perforce Software, Perfecto
Eran Kinsbruner is a person overflowing with ideas and inspiration, beyond that, he makes them happen. He is a best-selling author, continuous-testing and DevOps thought-leader, patent-holding inventor (test exclusion automated mechanisms for mobile J2ME testing), international speaker, and blogger.
With a background of over 20 years of experience in development and testing, Eran empowers clients to create products that their customers love, igniting real results for their companies.