View all web browser and mobile devices available in our cloud-based test lab.
Enterprises are under constant pressure to improve software delivery cycles while maintaining — or even improving — quality. The only feasible way to iterate more quickly while ensuring quality is to implement continuous testing as part of the overall development strategy.
Continuous testing (CT) allows organizations to gather fast feedback per each code change (commit) as part of the continuous integration (CI) workflow. When implemented properly, CT can help organizations attain true DevOps status. The only real downside associated with CT is the vast amount of test results data that is generated with a high number of commits - and subsequent volume of test cases - throughout code cycles.
Testing is a critical component in building a successful DevOps organization. The more tests executions you have across a large array of platforms (mobile devices, tablets, desktop browsers), the more test results data is created. Analyzing, understanding, and filtering the results data quickly is critical in order to prevent bottlenecking the DevOps pipeline.
In a DevOps environment, the window to review and qualify test results has shrunk dramatically from days to hours (and even minutes in some cases). What’s no longer acceptable is a situation where a regression testing suite is running for 2-3 days and requires a similar amount of time to analyze results.Our research at Perfecto tells us that organizations spend between 50-72 hours per regression cycle analyzing test results, filtering out noise and assessing failures which may impact their software releases. It is essential that teams have a better solution for test reporting in order to analyze data, triage issues, and act upon failures with the best possible insights. Unlike desktop browser testing, mobile devices create more complex issues as failures can be caused by network connectivity, device locks, improper object use, popups and more.
When dealing with large test results datasets in a CI/DevOps environment, there are a few essential tools that teams need:
To excel in software delivery for mobile, native, or responsive web apps, teams not only need better automation as part of their processes, test flows, and CI/CD workflows, but also a test results analytics platform to manage all of their test results data in a way such that the teams can evaluate the data, act upon it, and deliver iterations and features with confidence - and quickly!
Since DevOps involves a collection of team members from all parts of the SDLC process, the central platform needs to meet the needs of ALL team members.
That’s why executives are keenly interested in high-level, quality dashboards offering CI health (pipeline) status, while developers and testers care most about test artifacts, noise reduction, and single test reports.
To learn more about next-gen reporting and analytics for DevOps, visit our Smart Reporting and Analytics page. Or, give it a try to see for yourself.
Get My Free Trial
DevOps Chief Evangelist & Sr. Director at Perforce Software, Perfecto
Eran Kinsbruner is a person overflowing with ideas and inspiration, beyond that, he makes them happen. He is a best-selling author, continuous-testing and DevOps thought-leader, patent-holding inventor (test exclusion automated mechanisms for mobile J2ME testing), international speaker, and blogger.
With a background of over 20 years of experience in development and testing, Eran empowers clients to create products that their customers love, igniting real results for their companies.