View all web browser and mobile devices available in our cloud-based test lab.
CI/CD in DevOps has changed significantly over the past decade. Here, we recap what CI/CD in DevOps is, including the current state and future.
CI/CD is a process in DevOps where you automate steps and work continuously to integrate, test, and deploy code.
Today, most organizations have transformed to an Agile/DevOps business model. This allows them to respond much faster to production issues, change of customer/business requirements, and more.
As identified by the 2019 Global Developer Report by GitLab, organizations have shifted to continuous deployment. This is done on-demand once or several times per day, or every few days.
Software development has evolved to address situations that we used to see a decade ago. Extreme fire drills and catastrophes reported by clients caused major disruptions to the business.
Today, organizations deliver software and value faster and in smaller chunks. If something breaks, the impact is less of an emergency and the fix is faster.
Code deployment frequency survey from from Global Developer Report (2019)
There are many DevOps challenges today. In particular...
Quality is a major issue for organizations that are trying to embrace CI/CD, according to the recent World Quality Report. Test automation is lagging behind. However — and due to the nature of DevOps — the failures are in many cases less painful.
Developers must get pinpointed test reports containing only real defects. Test reports need to exclude all types of noise and false negatives. This will allow teams to address the risk of critical defects slipping into production, and to reduce the turnaround time to resolve them.
As seen in the below visual, most test failures from test automation are due to noise. This noise is associated with coding practices, test environments, labs, hosted platforms, backend services, etc., leaving a small percentage per lines of code for severe defects.
Noise reduction within test automation characteristics
In addition to noise reduction, measuring test value is also critical to improving DevOps. Measuring test value can help teams assess their maturation of test automation. A decade ago, measuring test ROI was a main priority for management. One decade later, it’s all about measuring the test value.
The reason for the change is the cadence of testing and the constant changes that are made to test code. With so many changes and test executions, it’s about getting continuous value from investment in test automation tools, development, and executions.
The below visual offers some well-adopted metrics that adhere to the Agile Testing Manifesto around the value of testing as a cross-team responsibility.
Metrics for test automation value
Finally, around failing fast and fixing fast, the past decade has shown a dramatic adoption of operations and monitoring tools. These include Grafana, Nagios, Kibana, and Prometheus.
Such tools enable ops teams to visualize, analyze, and identify production and efficiency issues. Either used alone or together in parallel, these tools have become dominant in the transition to DevOps. The tools collect live metrics and data from production, visualize them for the right teams, and allow continuous monitoring and improvements.
Automation across the board is key to CI/CD in DevOps. And of every stage in the CI/CD pipeline, testing is the most critical to automate.
Perfecto can help you take your test automation to the next level. See for yourself the power of the world’s only end-to-end continuous testing platform. Try it for free today.
DevOps Chief Evangelist & Sr. Director at Perforce Software, Perfecto
Eran Kinsbruner is a person overflowing with ideas and inspiration, beyond that, he makes them happen. He is a best-selling author, continuous-testing and DevOps thought-leader, patent-holding inventor (test exclusion automated mechanisms for mobile J2ME testing), international speaker, and blogger.
With a background of over 20 years of experience in development and testing, Eran empowers clients to create products that their customers love, igniting real results for their companies.