View all web browser and mobile devices available in our cloud-based test lab.
Ready or not, digital platforms were thrust onto center stage in 2020, raising the stakes for high-performing web and mobile apps and making the ability to test remotely a requirement.
Where did we land? What’s challenging teams today? What are they focusing on? To find out, we surveyed over 700 leading digital enterprises across industries. Here’s what we found, and where we think testing is going this year.
This year, the majority of testing was done by internal QA supporting development.
When companies rely on testing resources within development:
When companies rely on internal QA supporting dev, they are more likely to…
Teams who outsource their testing are more likely to struggle with test stability.
Teams with both internal QA team and testing resource are the most likely to shift testing left.
Agile and DevOps maturity is driving the shift of testers into dev feature teams. This is to reduce the time to create and run tests and also to respond faster within the cycle.
Across the board, teams find the lack of resources for creating test automation to be the biggest challenge they face, followed closely by proper coverage for test automation scenarios.
Top challenges vary based on role, too.
Teams automating fewer than 10% of their test cases are:
On the other end of the spectrum, teams who have shifted left or are automating over 50% of their test cases:
To better tackle test automation creation, teams need to match test automation tools with the skillsets of team members. This allows even more team members to partake in the test automation creation process. It also allows teams to improve their test scenario coverage.
For extended test coverage of platforms, teams can make two improvements:
Related Reading: 5 Things to Consider When We Talk About Test Coverage
Successful organizations are able to fit their test automation within the build process and drive faster feedback. Our research finds that ~70% are still struggling to accomplish that objective. And that requires better cross-team collaboration and alignment.
To mature DevOps, most checkpoints must be checked automatically. The more human intervention there is, the slower the process will be. An example of a best practice can be something like this. Instead of test engineers waiting for a new build to start their test automation, developers can grant an early access build that has the right “tag” attached to it to allow test automation activities to begin.
Related Reading: SDLC Tools For Testing: The Right One For Each Phase
Only 9% of those surveyed release daily, which is down from 14 % last year. The challenges of remote testing throughout 2020 appear to be slowing teams down.
The keys to releasing more often depends on the entire software development process from coding, building, integrating, testing, and deploying the code to production. Whenever tasks are broken, or not automated, the entire process slows down and delays the release.
While in this research we focus on test automation, more than just that needs to be automated. This needs to be a DevOps team priority and not a single team’s goal. Making the connection between the different players in the pipeline comes through process and technology, and this requires choices based on priorities to be made by management.
Related Reading: Top 5 Things Slowing Down Your Testing Life Cycle
Teams want to shift their testing left — but manual testing is an obstacle to achieving that. Manual testing will always be a reality to some extent, as it’s recommended to mix manual and exploratory testing as part of each software iteration. However, the more tests that can be automated and shifted left, the better.
Almost 10% of survey respondents are still manually testing all of their tests. Another 20% are just getting started with automation. And 47% of teams automate fewer than 50% of their tests.
Teams struggling with manual testing are…
Related Reading: How to Transition From Manual Testing to Automated Testing in 7 Steps
Over 60% of respondents said native mobile and mobile web apps are very important to their testing strategy. That’s almost a 10% increase from 2020 — clearly mobile is top of mind.
This comes as no surprise —81% of US adults own a smartphone. And mobile outranks desktop and tablets for market share.
The increased importance of mobile and mobile web apps leads to more complex test automation. Mobile apps have become more advanced and mobile phone releases/functionality are constantly moving targets.
Related Reading: The Complete Guide to Mobile Automation Testing
Scriptless testing is top of mind in 2021. Over 40% of respondents have an interest in using scriptless/intelligent test automation solutions this year. Scriptless testing is also the top trend respondents are looking to invest in throughout 2021.
Other emerging areas of interest include Cypress, Flutter, 5G, and PWAs.
And 26% of teams report using commercial test creation solutions. And almost half of companies are looking to invest in testing tools in 2021.
Teams using commercial lab vendors are….
Commercial lab vendors help connect open source test frameworks to mobile and web platforms. Lab vendors facilitate a stable test automation and execution environment. This promotes greater test coverage for automation. Teams also encounter less “noise” that is associated with DIY solutions.
Many commercial tools like Perfecto, also provide additional productivity features like test reporting, test creation, scalability, app debugging, and more.
Related Reading: Buyer's Guide to Web & Mobile Test Automation Tools
48% of those surveyed are looking to invest in automation solutions within the next year.
Scoping Out Test Automation Tools? Download our Buyer's Guide to Web & Mobile Test Automation Tools.
2020 might have been the most challenging year for software testing and development to date.
Almost overnight, teams had to find ways to develop, test, and innovate from remote — sometimes with less budget, and with different tools and processes.
As we look to 2021 and beyond, some of the changes from 2020 will still remain a reality for teams as part of lessons learned, risk mitigation, and process improvements.
In 2020, organizations moved almost entirely to digital as their only way to engage with clients. This isn’t going away — it’s only made digital channels more dominant for business.
With that change, organizations put greater emphasis on web and mobile technologies from an innovation and testing perspective.
Serving global users through web and mobile channels requires rigorous testing, bigger coverage considerations, and continuous testing and monitoring of these services — for both functionality and performance (responsiveness and availability).
In 2021, teams will increase their investment in mobile and web applications through progressive web apps (PWAs) and newer and more engaging mobile apps like AppClips/APKs to provide better user experiences and retain connections with customers.
Here, cloud testing solutions will play a major role in serving as the unified testing solution or CFTA (continuous functional test automation).
With more testing done on digital platforms, teams show a greater increase in test automation investment. We have seen in 2020 not only test suite sizes growing for mobile and web apps, but also the adoption of additional technologies that can serve different personas within the teams.
Frameworks such as Cypress and Flutter, together with more BDD adoption, have seen tremendous usage as developers and testers take more and more testing responsibilities. This is not going to change going forward as digital winners will continue with such investments.
Additionally, the growing emphasis on digital testing will place more focus on accessibility test automation across platforms. This type of testing has become top of mind for digital businesses serving all types of users — with and without any disabilities — as well as to eliminate manual testing activities for accessibility that happen too late in the process.
With the emphasis on shift left and DevOps, and in light of 2020, we have seen a dramatic change in the way testing teams are structured.
With over 42% of survey respondents identifying as QA practitioners residing within feature teams/squads, it is clear that the transition from COE into smaller and focused testing teams is happening across the board.
As products become more complex, having testers in proximity to developers in smaller groups makes E2E software delivery much more efficient.
In 2021 and beyond, we will still be seeing hybrid models of centralized testing teams that serve big organizations, but the trend will be to shift the testing practice into smaller, dedicated testing teams within feature teams.
To succeed in scaling test automation and ensuring digital quality, organizations who started looking into more intelligent testing processes and tools in 2020 will continue doing so in 2021.
This spans test reporting and analysis abilities to cope with growing numbers of test executions, and more maintainable test creation through scriptless and self-healing tests. With the help of AI and ML within intelligent testing solutions, teams will be able to uplift their daily activities to obtain more value from testing, expedite their feedback loops, and deliver products with fewer escaped defects.
Perfecto by Perforce enables exceptional digital experiences and helps you strengthen every interaction with a quality-first approach for web and mobile apps through a cloud-based test platform, comprised of real devices and real end-user conditions, giving you the truest test environment available.
Our customers, including 50 percent of the Fortune 500 companies across banking, insurance, retail, telecommunications, and media rely on Perfecto to deliver optimal mobile app functionality and end-user experiences, ensuring their brand’s reputation, establishing loyal customers, and continually attracting new users. For more information about Perfecto, visit perfecto.io.