5 Reasons Why Shift Left Testing Is Not Happening
Shift left testing is the strategy of testing earlier in the cycle. And while it is lauded for its many benefits to DevOps teams, in reality it has not been widely adopted. Why not?
To better understand the current state of automated testing and see where shift left testing is at, Perfecto partnered with Gatepoint research to survey leading digital enterprises.
We also spoke with testing and DevOps experts to understand why shift left testing is not taking hold and what it will take to implement.
Keep reading to learn more.
Shift Left Testing Benefits
Shift left testing offers a number of tangible benefits for teams.
- By testing earlier in the cycle, teams can find costly defects sooner. This saves time and money.
- Shift-left testing enables teams to fail fast and fix fast.
- By combining coding and testing together, the shift left approach reduces code instabilities. And this increases developers’ efficiency.
- The shift left approach enables developers to quickly test code via continuous integration (CI) and test automation. This allows teams to mature their SDLC toward continuous testing and the CI/CD pipeline.
- The shift left testing approach allows teams to better structure the entire testing scope for their projects.
- Shift left testing in DevOps results in a higher quality product. This increases customer satisfaction and improves business results.
Shift Left Testing By the Numbers
The benefits the shift left testing approach are clear. But our 2020 State of Test Automation report found that few teams are successfully implementing it.
Few Actually Test in Dev
Given the benefits of shift left testing, we wanted to know how testing is structured within organizations. For the vast majority of companies, testing is done by a mixture of both QA and dev. 81% of companies surveyed rely on QA for testing, either supporting dev or working within dev. Only 19% of companies surveyed rely on development teams for testing.
For teams that rely on development for testing, open source test automation frameworks and tools are most commonly used. And 56% are automating less than a quarter of their test cases.
On the other hand, companies relying on QA for testing are more likely to use commercial tools and automate more of their tests. However, these teams struggle with test instability/flakiness and false negatives. While these teams face skillset challenges for creating test scripts, they are investing in commercial tools to ensure their success.
Most Companies Test Only After a New Build
Too many teams are waiting to test until after a new dev build is ready. Organizations need to test throughout the process rather than at the end.
The majority of companies surveyed test after a new development is ready or after a new build is ready. Only about 40% of companies test upon each code change or at the start of each software sprint.
To mature DevOps, most checkpoints must be checked automatically. The more human intervention there is, the slower the process will be. Automating such checkpoints does not mean these gates are being put at risk. Rather, it means teams found the right ways to automate the flow and approve it.
An example of a best practice can be something like this. Instead of test engineers waiting for a new build to start their test automation, developers can grant an early access build that has the right “tag” attached to it to allow test automation activities to begin.
Why Shift Left Testing Is Not Happening
The numbers tell us that shift left testing is not happening. But why? Hear five experts weigh in on their theories.
1. Lack of Leadership Support & Alignment
“In theory, shift-left makes total sense. Security bugs can be extremely costly when they make it to production,” says Kristin Davis of 42Crunch. Despite this, testing is not shifting left. Why? Davis offers two reasons: lack of alignment with leadership and current processes.
Lack of Company Leadership Support
“If R&D is incentivized solely on delivering the functionality and meeting the deadlines, any additional steps and solutions that are not aligned with that will meet resistance. Of course everyone would agree that security is important, but if the team's sole mandate is to ship that business-critical app by the target launch date, everything else will be pushed to the side.”
Lack of Alignment With Existing Processes
“When R&D is not involved in tool selection, the chances are that the tools and processes may not fit them well. Developers can be very religious about their tools, whether it’s developer environments (IDEs) or continuous integration/continuous delivery (CI/CD) pipelines. If they are asked to use something ad-hoc that doesn’t integrate, they will view it as a total productivity killer.”
2. Lack of Testing Resources
One reason testing is not shifting left is due to workload capacity.
“Teams need to free up resources to actually do the shift-left testing while still continuing with existing testing. If your resources are already seconded to projects and testing and then a new project wants them to do testing early, it doubles the workload,” says Neil Price-Jones, President of NVP Software Solutions.
Additionally, skepticism whether shift left testing holds true benefits could be slowing down its adoption.
“People are not convinced, myself included, that shifting testing left will cover all the cases that need to be done. I have yet to get a convincing argument that just doing this testing at the beginning is sufficient to remove the risk.”
3. The Culture
Culture is another impediment to shift left testing.
“The biggest reason why testing isn't shifting left as expected is a mix of culture and fear. Usually when companies start moving toward that, there's too much testing on the part of the tester. And the reason for that is there are a lot of bugs affecting quality, and the developers are working furiously to fix them,” says Michael Fritzius, President of Arch DevOps.
“Shifting left takes some of the testing effort off the testers but places it on an already overworked dev team. The fear of changing how the work is done, and expecting that the style of work changes to account for it, is met with resistance.”
4. Lack of Tester Engagement
Another reason behind the lack of shift left testing could be the lack of testing represented in early phase discussions.
“The main reason shift left testing is not happening is product owners are not engaging the testers in the early phases of design and development. They do not give much importance to testing before the application hits production,” says Naveen Kumar Namachivayam, Performance Test Engineer at QA Insights.
“During the initial important project discussions, oftentimes testers are not invited. Especially, product owners never involve nonfunctional testers in their regular stand up meetings. They think test automation solves most of the testing effort. It is not possible to automate everything. If the testers are involved during the inception phase, shift-left testing delivers fruitful outputs.”
5. The Scope of Testing Doesn’t Fit in the Build Cycle
“One reason that shift left testing is not taking hold is that the scope of testing is not well defined,” says Perfecto Chief Evangelist Eran Kinsbruner. “If it is too large without proper justification, there’s no room to integrate it into the build cycle to shift left.”
For each build to be tested, Kinsbruner recommends the scope of the test suites includes:
- Key functional testing scenarios.
- Unit testing that is specific to what was changed.
- Security, performance, and relevant exploratory testing in parallel.
It’s also important to ensure your test suites remain valuable over time. Teams need to audit and maintain their suites between builds. This ensures that the testing scope is relevant over time.
A few things still need to happen for the shift left approach to take hold. Shifting left needs to align with leadership’s vision. Teams need to be ready to show tangible results. And testing must be represented in the earliest phases of the project.
Learn more about current testing trends in the community. For more information on the current state of testing, download our report, The 2020 State of Test Automation.
Or, try testing with Perfecto today. Perfecto allows you to…
- Scale testing on multiple platforms.
- Sync test automation with your CI/CD pipeline.
- Find and fix bugs faster.
Start your free trial today.