Blog
May 6, 2022
Automation promises it's going to save you time and lower costs while raising quality, something that's seemingly impossible. As we learn and adopt automation, we find out that there are many ways that we can approach it. But only a few ways allow us to be successful long term.
Too often, teams rush to automate everything they can. But that comes back to haunt them. And they run into a slew of problems.
Here, we break down which types of test cases to automate. We help you decide when a test case should be automated. And we discuss what test cases cannot be automated, or should not be automated.
Keep reading to learn more about automating your tests and how to prioritize them.
What Is Test Automation?
Back to topTest automation refers to the process of using software tools and scripts to know what to automate in software testing. This includes creating automated test cases that can be run automatically, rather than manually.
The Role of Test Automation
Good automation helps testers do their job. It's not there to replace them. It is a tool that enhances testing. Good automation should be baked into your SDLC upfront.
Related Reading: A beginner's guide to building a foolproof Test automation strategy
Automation should also be:
- Focused
- Informative
- Adding value
- Trustworthy
- Repeatable
When Should a Test Case Be Automated?
A test case should be automated if the following criteria for automation testing apply:
- The task is going to be repeated.
- It’s going to save time.
- The requirements, the test, or the task are low risk, stable, and unlikely to change often.
- The test is subject to human error.
- The test is time consuming.
- The test has significant downtime between steps.
- The test is repetitive.
These qualifications allow you to set standards for testing across the team and prioritize tests. This increases your chances of releasing on time. Most importantly, this enables you to get strong ROI on your test automation.
Which Types of Test Cases Should Be Automated?
Unit testing should take the top priority, followed by integration testing and functional testing. The types of test cases for which automation testing without coding applies have a lot of complex manual scenarios, especially those that require a lot of data or environment setup.
And we want to start automating the most valuable tests to least valuable tests. A simple scorecard can help you figure out where to start.
Make a Score Card to Prioritize Which Tests to Automate
Make a scorecard broken down features and test scenarios. For each one, you want to apply a set of criteria — your critical success factors — to judge these scenarios by.
Related Reading: Explore our Annual State of Test Automation Report
You’ll want to ask things like:
- Is this something on the critical path or something that's frequently used?
- Do we require to actually test this because there's a legal issue?
- Is there a lot of data and environmental setup?
- Is this something that has a lot of reuse?
Tally up the scores and that will tell you which tests will provide the most value when automated. By creating the scorecard, you prioritize tests. The highest-value ones get automated first.
Utilize Personas in Automation
Another way to prioritize automation is through the use of personas. These help us focus on the most valuable tests. Personas also guide decisions about test functionality and design.
Top 11 Challenges in Automated Testing & What to Do About Them
If we write personas into our automation, it helps prioritize features, focus on target users, and provide a consistent view of them. That way, your overall test coverage is improved and you're focused on that end user behavior.
Apply Insights from Marketing Data
You can also use marketing data to help drive automated tests. Keeping in mind what the customer is doing and how they're interacting with our application is key to our being successful. We use marketing data to understand:
- The most popular devices people use.
- The most popular browsers people use.
- The most common flows that people take through applications.
- Points where people jump off of applications because the user experience wasn't great.
This allows us to focus our testing on the people driving our business using the most popular devices.
Related Reading: Mobile & Web Test Coverage Guide
And we can apply persona-based test conditions that use the geolocation that matches the time zones our customer base is in, using the network profile that matches them.
Back to top3 Types of Test Cases to Automate First
As mentioned above, unit testing should take the top priority for automation, followed by integration and functional testing. They should all be baked into your overall objectives for automation.
1. Unit Testing
Unit testing is the fastest method of testing and, therefore, should be the highest priority for your automation. That is because it's easier to debug. These are highly reusable tests. They are low-cost to fix, and there's a whole host of frameworks that you can use to implement this regardless of your programming language.
2. Integration Testing
Integration testing, where we're testing our interfaces or modules, should also take high priority. These tests help us ensure that everything is working as expected. When automated, integration tests will run more quickly for us and be able to give us that feedback.
3. Functional Testing
With functional testing, there are a whole host of tools and frameworks that you can use that will match your development code base. So, you should take an approach that values it as an upfront concern. Running those tests will help identify flaky ones. And we don’t want flaky tests.
Back to top4 Things to Keep in Mind Before Automating
When you’re considering which test cases to automate, you need to keep a few things in mind first. This ensures you’re getting ROI from your automation efforts.
1. Test Maintenance
You have to factor in maintenance costs. Every time we write a test, that means we're committing to maintaining that script for the rest of time or until we remove it. So, that should lead into your decision making of whether or not you actually write those scripts.
Related Reading: The Complete Guide to Mobile Automation Testing
2. Your Toolchain
Often overlooked are the tools associated with automation. Are you going to purchase tools or go open source? You need to consider the tools, as well as the ancillary tools that support it. And don’t forget about maintenance when making operating system upgrades or language upgrades. Remember, it’s another application to support.
Related Reading: The Buyer's Guide to Web & Mobile Test Automation Tools
3. Documentation/Implementation
We also have things like documentation, implementation, and onboarding that will take up our time. These things will continue to take up more time as people join and leave the team, as we make changes and start new projects, and more. So, it's important to take in to account all of these costs when building out automation. They should be factored into that return on investment.
4. Organizational Constraints
We additionally have some organizational constraints that are placed upon us from our company. That could be our delivery schedule, our project budget, as well as the technical skills out of the people in our office. So, we should instead try to automate based upon the frequency of use, criticality, and legal risks for our automation scripts in order to get the most value out of them.
Back to topBottom Line
Just because you can automate a test doesn't mean that you should. If you automate a whole slew of tests that are going to require a great deal of upkeep, then you are investing additional time and money that you may not have.
Instead, you should focus on adopting a risk-based approach, such that you're only automating the most valuable tests. Take great care in automating those most valuable features, and making sure that they're automated correctly for long-term sustainability.
Try Perfecto for Seamless Test Automation
Automated testing is key to successful DevOps. But you have to make sure you’re doing it correctly and with the right tools in order to reap the benefits.
Perfecto makes test automation, maintenance, and execution seamless. With Perfecto, teams can automate advanced test scenarios on thousands of devices and browsers in the cloud, from anywhere in the world.
See how Perfecto can help you build a cutting-edge test automation process with maximum coverage at scale. Get your free trial today.