View all web browser and mobile devices available in our cloud-based test lab.
Artificial intelligence (AI) and machine learning (ML) are transforming multiple sectors of the economy, and impacting several aspects of our daily lives.
Workplaces such as those in finance, healthcare, retail, education, and technology are leveraging AI to automate tasks, reduce costs, and make data-driven decisions. In our homes, AI is powering television and movie recommendations, personal digital assistants, security cameras, and home automation.
This blog will focus on how the coming together of artificial intelligence (AI) and software testing have made a huge impact on test automation. Let us discuss this developing trend in more depth.
AI software testing uses automated software testing tools to leverage AI (machine learning) to produce better results. By incorporating AI into automated testing tools, teams will be able to overcome many of the traditional challenges of automated software testing.
As AI continues to permeate our world, it is becoming more and more critical to validate that these types of systems are functional, safe, secure, performant, available, and resilient. In other words, AI needs testing. Unfortunately, we haven’t seen many advances in the area of testing AI-based systems.
Read Perfecto's Annual State of Test Automation Report
On the bright side, researchers and practitioners are recognizing the potential for AI and ML to bridge the gap between human and machine-driven testing capabilities. As a result, several organizations are developing AI-powered automated testing tools.
Watch: Advancing the State of The Art in AI and Testing
Since 2014, there has been a spike in the number of vendors offering AI-driven test automation services. The majority of these tool vendors are start-up companies targeting system-level testing of mobile applications, and the subject is generating some much-needed buzz in the industry.
Hence, it seems that just like AI needs testing, testing needs AI.
Although some aspects of AI for Software Testing (AIST) are still not receiving enough attention, in the last decade we have seen it emerge as a discipline centered at the intersection of three areas:
The intersection of AI and software testing
The current state of the practice uses autonomous and intelligent agents, referred to as “test bots,” to automate activities such as application discovery, modeling, test generation, and failure detection. A combination of machine learning techniques are used to implement test bots. These include but are not limited to decision tree learning, neural networks, and reinforcement learning.
Try Testing With Perfecto
Machine learning allows the test bots to be robust and act under conditions of uncertainty unlike traditional test automation tools and frameworks. Examples of AI-driven automation testing approaches that have formed over the last decade include:
Related Reading: Automated Visual Testing for Mobile and Web Applications
Element location using AI has been incorporated into the open-source mobile testing framework Appium. The project is a collaboration between Jason Arbon, CEO of Test.ai, and Appium’s creator Jonathan Lipps. It uses AI to identify web elements in mobile applications.
Leveraging AI for web element location makes the tests quicker to create and less brittle. Furthermore, the testing community can improve the AI by adding new training data, alternative training methods, more rigorous relevance testing, or new labels.
Related Reading: How to Use Page Object Model in Selenium
The training data is open and available on Kaggle, thereby allowing individuals or organizations to fork the data, clean it up, add it to their proprietary test frameworks, or compete to improve the classifiers.
Related Reading: Test Automation Strategy For Beginners
Tariq King and his team over at Ultimate Software developed and released an open-source framework called AGENT — AI Generation and Exploration iN Test.
AGENT is an ML-based, multi-agent prototype for web application testing. The prototype consists of a collection of intelligent agents that work in conjunction with abstract test generation and web UI classification subsystems. The agents incorporate long short-term memory units trained by human testers.
For any given subsequence of steps through the application, the agents predict sequences of the next possible set of actions and assertions that a human tester might make and automatically execute them on the system under test.
One of the trends that started this past decade, which is expected to continue, is the use of AI to enhance existing tools and frameworks that target specific testing problems.
Examples at this time include functional testing of web and mobile applications, visual testing of user interfaces, and UI element location and auto-correcting element selectors. Beyond this we should expect to see AI starting to replace entire technology stacks for automated testing.
Related Reading: ML & AI in DevOps: Experts Predict 7 Big Changes
At all testing levels, AI automation testing will take over tasks that require decisions that a human could make in less than a second. Initially, higher-order testing tasks may still require human input or intervention. These are the tasks that require a bit more thought such as test generation, usability testing, security testing, and edge cases.
However, over time as technology progresses, and as the machines are trained on the actions of these higher-order tasks, AI is likely to take over those activities as well and tackle problems that require deeper context. The image below depicts this progression of the AI testing singularity as we move into the next decade.
The AI testing singularity: a projection of the next decade of software testing
So what can testers do to prepare for a future in which AI tests software? The initial step is to determine whether you are interested in designing and developing AI-driven systems, or being an end user of these tools.
Related Reading: 5 Types of ML & AI Testing Tools
An engineer who is building these types of systems needs to have a good understanding of AI and machine learning. Both MIT and Stanford University have freely available introductory courses on machine learning.
Tariq King and Jason Arbon have open-sourced the materials for an introductory course on AI and ML skills for the testing world, including the slides and tools used during the exercises. It is unlikely that end users of AI-driven testing tools will need to have deep knowledge of machine learning.
Related Reading: Why Test Failure Analysis Is Key to Faster Delivery
Vendors are likely to provide easy-to-use interfaces and APIs for using AI-driven testing features and customizing pre-trained AI models.
This trend is already being followed with the advent of AItools and platforms like Google’s AutoML Vision, H2O.ai, and DataRobot.
AI is already altering the landscape of testing. And while we don’t know exactly what the future of software testing holds, we can prepare for it by stabilizing and scaling test automation to mature our processes.
Perfecto can help you take your test automation to the next phase, no matter where you are at.
Whether you’re just beginning on automation or you’ve reached continuous testing, Perfecto has the end-to-end testing support to help you accelerate. Try it for free today.
Founder and CEO, Selftest IO
Tariq King is the founder and CEO of Selftest IO, a company on a mission to develop the next generation of systems and services with intrinsic self-testing properties. Tariq has over 15 years' experience in software testing research and practice and has held positions as a test architect, engineering manager, director, and head of quality. Tariq holds Ph.D. and M.S. degrees in computer science from Florida International University (FIU). His areas of research are software testing, artificial intelligence, autonomic and cloud computing, model-driven engineering, and computer science education. He has published over 40 research articles in peer-reviewed IEEE and ACM journals, conferences, and workshops, and has been an international keynote speaker at leading software conferences in industry and academia. He is the cofounder with Jason Arbon of the Artificial Intelligence for Software Testing Association (AISTA). Contact Tariq via LinkedIn or Twitter.