May 6, 2020

How to Decide Which Types of Test Cases to Automate

Automation

Continuous Testing

Table of Contents:

The Role of Test Automation
3 Types of Test Cases to Automate First
4 Things to Keep in Mind Before Automating

Automation promises it's going to save you time and lower costs while raising quality, something that's seemingly impossible. As we learn and adopt automation, we find out that there are many ways that we can approach it. But only a few ways allow us to be successful long term.

Too often, teams rush to automate everything they can. But that comes back to haunt them. And they run into a slew of problems.

Here, we break down which types of test cases to automate. We help you decide when a test case should be automated. And we discuss what test cases cannot be automated, or should not be automated.

Keep reading to learn more about automating your tests and how to prioritize them.

The Role of Test Automation

Good automation helps testers do their job. It's not there to replace them. It is a tool that enhances testing. Good automation should be baked into your SDLC upfront.

Automation should also be:

Focused
Informative
Adding value
Trustworthy
Repeatable

When Should a Test Case Be Automated?

A test case should be automated if the following criteria for automation testing apply:
The task is going to be repeated.
It’s going to save time.
The requirements, the test, or the task are low risk, stable, and unlikely to change often.
The test is subject to human error.
The test is time consuming.
The test has significant downtime between steps.
The test is repetitive.

These qualifications allow you to set standards for testing across the team and prioritize tests. This increases your chances of releasing on time. Most importantly, this enables you to get strong ROI on your test automation.

Which Types of Test Cases Should Be Automated?

Unit testing should take the top priority, followed by integration testing and functional testing. The types of test cases for which automation testing without coding applies have a lot of complex manual scenarios, especially those that require a lot of data or environment setup.

And we want to start automating the most valuable tests to least valuable tests. A simple scorecard can help you figure out where to start.

Make a Score Card to Prioritize Which Tests to Automate

Make a scorecard broken down features and test scenarios. For each one, you want to apply a set of criteria — your critical success factors — to judge these scenarios by.

You’ll want to ask things like:

Is this something on the critical path or something that's frequently used?
Do we require to actually test this because there's a legal issue?
Is there a lot of data and environmental setup?
Is this something that has a lot of reuse?

Tally up the scores and that will tell you which tests will provide the most value when automated. By creating the scorecard, you prioritize tests. The highest-value ones get automated first.

Utilize Personas in Automation

Another way to prioritize automation is through the use of personas. These help us focus on the most valuable tests. Personas also guide decisions about test functionality and design.

Top 11 Challenges in Automated Testing & What to Do About Them

If we write personas into our automation, it helps prioritize features, focus on target users, and provide a consistent view of them. That way, your overall test coverage is improved and you're focused on that end user behavior.

Apply Insights from Marketing Data

You can also use marketing data to help drive automated tests. Keeping in mind what the customer is doing and how they're interacting with our application is key to our being successful. We use marketing data to understand:

The most popular devices people use.
The most popular browsers people use.
The most common flows that people take through applications.
Points where people jump off of applications because the user experience wasn't great.

This allows us to focus our testing on the people driving our business using the most popular devices.

And we can apply persona-based test conditions that use the geolocation that matches the time zones our customer base is in, using the network profile that matches them.

3 Types of Test Cases to Automate First

As mentioned above, unit testing should take the top priority for automation, followed by integration and functional testing. They should all be baked into your overall objectives for automation.

1. Unit Testing

Unit testing is the fastest method of testing and, therefore, should be the highest priority for your automation. That is because it's easier to debug. These are highly reusable tests. They are low-cost to fix, and there's a whole host of frameworks that you can use to implement this regardless of your programming language.

2. Integration Testing

Integration testing, where we're testing our interfaces or modules, should also take high priority. These tests help us ensure that everything is working as expected. When automated, integration tests will run more quickly for us and be able to give us that feedback.

3. Functional Testing

With functional testing, there are a whole host of tools and frameworks that you can use that will match your development code base. So, you should take an approach that values it as an upfront concern. Running those tests will help identify flaky ones. And we don’t want flaky tests.

Try Perfecto for FREE >>>

4 Things to Keep in Mind Before Automating

When you’re considering which test cases to automate, you need to keep a few things in mind first. This ensures you’re getting ROI from your automation efforts.

1. Test Maintenance

You have to factor in maintenance costs. Every time we write a test, that means we're committing to maintaining that script for the rest of time or until we remove it. So, that should lead into your decision making of whether or not you actually write those scripts.

2. Your Toolchain

Often overlooked are the tools associated with automation. Are you going to purchase tools or go open source? You need to consider the tools, as well as the ancillary tools that support it. And don’t forget about maintenance when making operating system upgrades or language upgrades. Remember, it’s another application to support.

3. Documentation/Implementation

We also have things like documentation, implementation, and onboarding that will take up our time. These things will continue to take up more time as people join and leave the team, as we make changes and start new projects, and more. So, it's important to take in to account all of these costs when building out automation. They should be factored into that return on investment.

4. Organizational Constraints

We additionally have some organizational constraints that are placed upon us from our company. That could be our delivery schedule, our project budget, as well as the technical skills out of the people in our office. So, we should instead try to automate based upon the frequency of use, criticality, and legal risks for our automation scripts in order to get the most value out of them.

Bottom Line

Just because you can automate a test doesn't mean that you should. If you automate a whole slew of tests that are going to require a great deal of upkeep, then you are investing additional time and money that you may not have.

Instead, you should focus on adopting a risk-based approach, such that you're only automating the most valuable tests. Take great care in automating those most valuable features, and making sure that they're automated correctly for long-term sustainability.

Try Perfecto for Seamless Test Automation

Automated testing is key to successful DevOps. But you have to make sure you’re doing it correctly and with the right tools in order to reap the benefits.

Perfecto makes test automation, maintenance, and execution seamless. With Perfecto, teams can automate advanced test scenarios on thousands of devices and browsers in the cloud, from anywhere in the world.

See how Perfecto can help you build a cutting-edge test automation process with maximum coverage at scale. Get your free trial today.

start trial

The 2023 State of Test Automation