View all web browser and mobile devices available in our cloud-based test lab.
Automation promises it's going to save you time and lower costs while raising quality, something that's seemingly impossible. As we learn and adopt automation, we find out that there are many ways that we can approach it. But only a few ways allow us to be successful long term.
Too often, teams rush to automate everything they can. But that comes back to haunt them. And they run into a slew of problems.
Here, we break down which types of test cases to automate. We help you decide when a test case should be automated. And we discuss what test cases cannot be automated, or should not be automated.
Keep reading to learn more about automating your tests and how to prioritize them.
Good automation helps testers do their job. It's not there to replace them. It is a tool that enhances testing. Good automation should be baked into your SDLC upfront.
Related Reading: Test Automation Strategy For Beginners
Automation should also be:
A test case should be automated if the following criteria for automation testing apply:The task is going to be repeated. It’s going to save time.The requirements, the test, or the task are low risk, stable, and unlikely to change often.The test is subject to human error.The test is time consuming.The test has significant downtime between steps.The test is repetitive.
A test case should be automated if the following criteria for automation testing apply:
These qualifications allow you to set standards for testing across the team and prioritize tests. This increases your chances of releasing on time. Most importantly, this enables you to get strong ROI on your test automation.
Unit testing should take the top priority, followed by integration testing and functional testing. The types of test cases for which automation testing without coding applies have a lot of complex manual scenarios, especially those that require a lot of data or environment setup.
And we want to start automating the most valuable tests to least valuable tests. A simple scorecard can help you figure out where to start.
Make a scorecard broken down features and test scenarios. For each one, you want to apply a set of criteria — your critical success factors — to judge these scenarios by.
Related Reading: The 2021 State of Test Automation Report
You’ll want to ask things like:
Tally up the scores and that will tell you which tests will provide the most value when automated. By creating the scorecard, you prioritize tests. The highest value ones get automated first.
Another way to prioritize automation is through the use of personas. These help us focus on the most valuable tests. Personas also guide decisions about test functionality and design.
Top 11 Challenges in Automated Testing & What to Do About Them
If we write personas into our automation, it helps prioritize features, focus on target users, and provide a consistent view of them. That way, your overall test coverage is improved and you're focused on that end user behavior.
You can also use marketing data to help drive automated tests. Keeping in mind what the customer is doing and how they're interacting with our application is key to us being successful. We use marketing data to understand:
This allows us to focus our testing on the people that drive our business by using the most popular devices.
Related Reading: Mobile & Web Test Coverage Index
And we can apply persona-based test conditions that use the geolocation that matches the time zones our customer base is in, using the network profile that matches them.
As mentioned above, unit testing should take the top priority for automation, followed by integration and functional testing. They should all be baked in to your overall objectives for automation.
Unit testing is the fastest method of testing and, therefore, should be the highest priority for your automation. That is because it's easier to debug. These are highly reusable tests. They are low cost to fix, and there's a whole host of frameworks that you can use to implement this regardless of your programming language.
Integration testing, where we're testing our interfaces or modules, should also take high priority. These tests help us ensure that everything is working as expected. When automated, integration tests will run more quickly for us and be able to give us that feedback.
With functional testing, there are a whole host of tools and frameworks that you can use that will match your development code base. So, you should take an approach that values it as an upfront concern. Running those tests will help identify flaky ones. And we don’t want flaky tests.
When you’re considering which test cases to automate, you need to keep a few things in mind first. This ensures you’re getting ROI from your automation efforts.
You have to factor in maintenance costs. Every time we write a test, that means we're committing to maintaining that script for the rest of time or until we remove it. So, that should lead into your decision making of whether or not you actually write those scripts.
Related Reading: The Complete Guide to Mobile Automation Testing
Often overlooked are the tools associated with automation. Are you going to purchase tools or go open source? You need to consider the tools, as well as the ancillary tools that support it. And don’t forget about maintenance when making operating system upgrades or language upgrades. Remember, it’s another application to support.
Related Reading: The Buyer's Guide to Web & Mobile Test Automation Tools
We also have things like documentation, implementation, and onboarding that will take up our time. These things will continue to take up more time as people join and leave the team, as we make changes and start new projects, and more. So, it's important to take in to account all of these costs when building out automation. They should be factored into that return on investment.
We additionally have some organizational constraints that are placed upon us from our company. That could be our delivery schedule, our project budget, as well as the technical skills out of the people in our office. So, we should instead try to automate based upon the frequency of use, criticality, and legal risks for our automation scripts in order to get the most value out of them.
Just because you can automate a test doesn't mean that you should. If you automate a whole slew of tests that are going to require a great deal of upkeep, then you are investing additional time and money that you may not have.
Instead, you should focus on adopting a risk-based approach, such that you're only automating the most valuable tests. Take great care in automating those most valuable features, and making sure that they're automated correctly for long-term sustainability.
Automated testing is key to successful DevOps. But you have to make sure you’re doing it correctly and with the right tools in order to reap the benefits.
Perfecto makes test automation, maintenance, and execution seamless. With Perfecto, teams can automate advanced test scenarios on thousands of devices and browsers in the cloud, from anywhere in the world.
See how Perfecto can help you build a cutting-edge test automation process with maximum coverage at scale. Get your free trial today.
Test Automation Architect, Red Green Refactor
Thomas is presently serving as a Test Automation Architect. He leads a team of testers, ops engineers, and production support analysts in the adoption of DevOps practices. Previously, he led the enterprise automation support of 73 applications at Huntington National Bank that encompassed testing, metrics & reporting, and data management. Thomas has a background in Physics & Biophysics, with over a decade spent in research science studying fluorescence spectroscopy and microscopy before joining IT.