The Test Automation Pyramid & How to Use It

Table of Contents:

• What Is the Testing Pyramid?
• Why Use the Testing Pyramid?
• Level 1 - The Testing Pyramid Base: Unit Testing
• Level 2 - The Middle Stage: Integration Testing
• Level 3 - The Top of the Test Automation Pyramid: UI & Exploratory Tests

Using a testing pyramid is an important part of your software testing strategy. 

To successfully release an app out into the world, it needs thorough testing. And to successfully test an app, it needs a solid test automation strategy.

You probably already know that you do not automate all test cases. You concentrate your testing efforts on some types of tests more than others. And the platforms you use differ depending on the type of testing.

All of this is mapped out in the test pyramid. Use this blog as a guide to match the proper platforms with the right types of testing within your pipeline. 

What Is the Testing Pyramid?

The testing pyramid is a concept that groups software tests into three different categories. This helps developers and QA professionals ensure higher quality, reduce the time it takes to find the root cause of bugs, and build a more reliable test suite. 

The testing pyramid is a concept that strategically groups software tests into different categories.

From top to bottom, the main layers include:

• UI/exploratory tests
• Integration tests
• Unit tests

In the world of mobile testing, the pyramid also lends insights into the types of devices your team should use for each testing layer.

The lower portion of the pyramid is more automated, making those tests faster. The top of the pyramid is for slower and more manual test scenarios. The higher you go on the pyramid, the fewer the tests.

In other words, teams should have many unit tests, which are small and isolated, and not so many end-to-end exploratory tests, which are time-consuming and manual. Frontloading tests at the unit level makes fixing bugs quicker and easier, as opposed to defects that are discovered much later in the process.

Why Use the Testing Pyramid?

The testing pyramid is one of a variety of models testing teams can use to prioritize their testing activities. So why does the testing pyramid continue to stand out?

The testing pyramid is helpful for three main reasons: its logical structure, its emphasis on providing rapid feedback, and its compatibility with agile testing teams. Let us explore each of these reasons in greater depth:

Logical Structure

By running the simpler tests first and reserving the larger, more complicated tests for later in the testing process, the testing pyramid offers a logical, iterative structure for prioritizing which types of testing to focus on and when. 

The pyramid structure is also logical when you consider the number of tests that you will need within each stage of the pyramid. 

For example, teams need many unit tests since their scope is very narrow in terms of how much code they can cover. Yet, unit tests are also very light, so you can run many of them very quickly without any issues. On the other hand, there are usually fewer UI tests because they (understandably) take more time and resources to create, maintain, and run. 

Therefore, the underlying complexity surrounding each type of test within the pyramid lends itself to the pyramid structure.    

Rapid Feedback

By taking tests that should run at both earlier and later stages of software development into account, the testing pyramid also paves the way for testers to get faster feedback on how the application is functioning. 

With a built-in reminder to get feedback as early as the unit test level, teams can both find and resolve bugs more quickly, which is extremely cost-effective in the long run. 

Compatibility With Agile

As organizations are continuously shifting to a more agile way of working, testing teams can also benefit greatly from a testing methodology that aligns with those objectives. The testing pyramid is a great way for testers to become more agile by streamlining the testing process and paving the way to accelerate deliverables in a more effective way.

Start testing with Perfecto for FREE >>>

Level 1 - The Testing Pyramid Base: Unit Testing

Let us walk through each layer of the pyramid, working from the bottom up.

The base of the pyramid is unit testing. The scope of unit tests is small — and they ensure that isolated units of code work as they should. Unit tests should test one variable and not rely on any external dependencies.

Adopt the Testing Pyramid for a Bulletproof Mobile Testing Strategy

 get my guide

In this layer of the pyramid, tests are executed pre and post-commit. Tests are dev-triggered.

Which Devices To Use

We recommend that you use virtual devices — also known as simulators and emulators — for unit tests. Executing these tests on virtual devices is sufficient and provides the necessary feedback and debugging/log information.

Plus, virtual devices offer the benefits of being faster and more cost-efficient for testing.

Level 2 - The Middle Stage: Integration Testing

In the middle of the pyramid, you test across integrations of more than one system. Unlike unit tests, these tests are CI-triggered. In this stage, you have more features to cover and more testing types to conduct.

Which Devices To Use

With integration testing, it is critical to get the right balance of real and virtual devices.

Some types of testing, such as acceptance testing, can be done on virtual devices. However, other types of tests, such as performance testing, security testing, and certain accessibility features, cannot be done on virtual devices, so it is best to use a mix of real and virtual devices at this stage.

You will also want to add additional real device/OS configurations that match the client usage analytics and market trends. You can get device/OS usage data in our Test Coverage Guide.

Using both real and virtual devices in your testing strategy is a best practice. Real and virtual devices complement each other and support your goal of achieving high-quality builds for nightly regressions.

Level 3 - The Top of the Test Automation Pyramid: UI & Exploratory Tests

The top layer of the pyramid is for UI and exploratory tests. These tests generally are more complicated and have more dependencies than unit and integration tests. While teams can speed up UI testing with automation, exploratory testing is typically done manually and takes longer.

Keeping in mind the shape of the pyramid, the tests in this section should be far fewer than those at the base.

Which Devices To Use

Once you get to production monitoring at the top of the pyramid, testing must be done on real devices.

Why?

End-to-end testing, as well as exploratory testing, requires real devices to more closely mimic the app on a user’s device — because real users report defects on real devices, not on virtual ones.

It is also critical to apply real user simulation to your tests. Testing with conditions such as conflicting apps, call/text interruptions, network throttling, network latency, and more enables you to see how your app will hold up out in the world under varying circumstances.

Apply the Testing Pyramid Strategy With Perfecto

Adopting the testing pyramid into your testing process early on allows you to seamlessly shift left as needed, resulting in a more efficient strategy and greater success in the marketplace. 

Perfecto’s cloud gives you access to all of the devices, platforms, and browsers that you need to test your app, and you can use the cloud to ensure proper test coverage across platforms. You’ll even be able to apply environmental conditions by using real user simulation.

Plus, with Perfecto, you’ll be able to execute your testing strategy at scale — and deliver true ROI to your organization.

Learn more about implementing the testing pyramid strategy to boost your mobile testing. Or see for yourself how Perfecto can improve coverage across platforms by signing up for a free 14-day trial. 

start trial                                         ▶️ watch a demo of perfecto

Related Content

• Best Practices for Security Testing for Software
• How to Make a Cross-Platform Test Coverage Strategy
• Cloud Testing Is More Critical Than Ever
• The Complete Guide to Mobile Automation Testing