Blog
June 26, 2024
This blog was originally published in 2020 and has since been updated for clarity and accuracy.
Many organizations are beginning to adopt shift left testing into their development model, though a high percentage of companies still fail to do so, despite the proven benefits of shift left testing.
In this blog, we cover shift left testing, why it is not happening at a larger scale, and how you can embrace shift left testing for your team.
What Is Shift Left Testing?
Shift left refers to the approach of testing earlier in the software development lifecycle. Since testing earlier means bugs are less costly and time-consuming to fix, which improves overall app quality, shift left testing is encouraged as a best practice.
Yet while it is lauded for its many benefits to DevOps teams, shift left testing is still a struggle for many organizations (and enterprises, in particular).
To better understand the current state of automated testing and see the progress of shift left testing, Perfecto partnered with Gatepoint research to survey leading digital enterprises.
We also spoke with testing and DevOps experts to understand why shift left testing is not taking hold and what it will take to implement.
Keep reading to learn more.
Back to topShift Left Testing Benefits
Shift left testing offers a number of tangible benefits for teams.
- By testing earlier in the cycle, teams can find costly defects sooner. This saves time and money.
- Shift left testing enables teams to fail fast and fix fast.
- By combining coding and testing together, the shift left approach reduces code instabilities. And this increases developers’ efficiency.
- The shift left approach enables developers to quickly test code via continuous integration (CI) and test automation. This allows teams to mature their SDLC toward continuous testing.
- The shift left testing approach allows teams to better structure the entire testing scope for their projects.
- Shift left testing in DevOps results in a higher quality product. This increases customer satisfaction and improves business results.
Curious about the current state of shift left testing? Watch this panel webinar with QA industry oracles Michael Bolton, Eran Kinsbruner, and James Bach. It is a frank discussion on timely and timeless topics in the world of testing.
Back to top
What Is the Current Shift Left Testing Approach?
The benefits the shift left testing approach are clear. But our State of Test Automation report found that few teams are successfully implementing it.
Few Actually Test in Dev
Given the benefits of shift left testing, we wanted to know how testing is structured within organizations. For the vast majority of companies, testing is done by a mixture of both QA and dev. 81% of companies surveyed rely on QA for testing, either supporting dev or working within dev. Only 19% of companies surveyed rely on development teams for testing.
For teams that rely on development for testing, open source test automation frameworks and tools are most commonly used. And 56% are automating less than a quarter of their test cases.
On the other hand, companies relying on QA for testing are more likely to use commercial tools and automate more of their tests. However, these teams struggle with test instability/flakiness and false negatives. While these teams face skillset challenges for creating test scripts, they are investing in commercial tools to ensure their success.
Most Companies Test Only After a New Build
Too many teams are waiting to test until after a new dev build is ready. Organizations need to test throughout the process rather than at the end.
The majority of companies surveyed test after a new development is ready or after a new build is ready. Only about 40% of companies test upon each code change or at the start of each software sprint.
To mature DevOps, most checkpoints must be checked automatically. The more human intervention there is, the slower the process will be. Automating such checkpoints does not mean these gates are being put at risk. Rather, it means teams found the right ways to automate the flow and approve it.
An example of a best practice can be something like this. Instead of test engineers waiting for a new build to start their test automation, developers can grant an early access build that has the right “tag” attached to it to allow test automation activities to begin.
Back to top5 Reasons Why Shift Left Testing Is Not Happening
The numbers tell us that shift left testing is not happening. But why? Hear five experts weigh in on their theories.
1. Lack of Leadership Support & Alignment
“In theory, shift-left makes total sense. Security bugs can be extremely costly when they make it to production,” says Kristin Davis of 42Crunch. Despite this, testing is not shifting left. Why? Davis offers two reasons: lack of alignment with leadership and current processes.
Lack of Company Leadership Support
“If R&D is incentivized solely on delivering the functionality and meeting the deadlines, any additional steps and solutions that are not aligned with that will meet resistance. Of course everyone would agree that security is important, but if the team's sole mandate is to ship that business-critical app by the target launch date, everything else will be pushed to the side.”
Lack of Alignment With Existing Processes
“When R&D is not involved in tool selection, the chances are that the tools and processes may not fit them well. Developers can be very religious about their tools, whether it’s developer environments (IDEs) or continuous integration/continuous delivery (CI/CD) pipelines. If they are asked to use something ad-hoc that doesn’t integrate, they will view it as a total productivity killer.”
2. Lack of Testing Resources
One reason testing is not shifting left is due to workload capacity.
“Teams need to free up resources to actually do the shift-left testing while still continuing with existing testing. If your resources are already seconded to projects and testing and then a new project wants them to do testing early, it doubles the workload,” says Neil Price-Jones, President of NVP Software Solutions.
Additionally, skepticism whether shift left testing holds true benefits could be slowing down its adoption.
“People are not convinced, myself included, that shifting testing left will cover all the cases that need to be done. I have yet to get a convincing argument that just doing this testing at the beginning is sufficient to remove the risk.”
3. The Culture
Culture is another impediment to shift left testing.
“The biggest reason why testing isn't shifting left as expected is a mix of culture and fear. Usually when companies start moving toward that, there's too much testing on the part of the tester. And the reason for that is there are a lot of bugs affecting quality, and the developers are working furiously to fix them,” says Michael Fritzius, President of Arch DevOps.
“Shifting left takes some of the testing effort off the testers but places it on an already overworked dev team. The fear of changing how the work is done, and expecting that the style of work changes to account for it, is met with resistance.”
4. Lack of Tester Engagement
Another reason behind the lack of shift left testing could be the lack of testing represented in early phase discussions.
“The main reason shift left testing is not happening is product owners are not engaging the testers in the early phases of design and development. They do not give much importance to testing before the application hits production,” says Naveen Kumar Namachivayam, Performance Test Engineer at QA Insights.
“During the initial important project discussions, oftentimes testers are not invited. Especially, product owners never involve nonfunctional testers in their regular stand up meetings. They think test automation solves most of the testing effort. It is not possible to automate everything. If the testers are involved during the inception phase, shift-left testing delivers fruitful outputs.”
5. The Scope of Testing Doesn’t Fit in the Build Cycle
“One reason that shift left testing is not taking hold is that the scope of testing is not well defined,” says Perfecto Chief Evangelist Eran Kinsbruner. “If it is too large without proper justification, there’s no room to integrate it into the build cycle to shift left.”
For each build to be tested, Kinsbruner recommends the scope of the test suites includes:
- Key functional testing scenarios.
- Unit testing that is specific to what was changed.
- Security, performance, and relevant exploratory testing in parallel.
It’s also important to ensure your test suites remain valuable over time. Teams need to audit and maintain their suites between builds. This ensures that the testing scope is relevant over time.
Back to topHow Do You Implement Shift Left?
A few things still need to happen for the shift left approach to take hold. Shifting left needs to align with leadership’s vision. Teams need to be ready to show tangible results. And testing must be represented in the earliest phases of the project.
Learn more about current testing trends in the community. For more information on the current state of testing, download our State of Continuous Testing report.
Back to topEnable Shift Left Testing With Perfecto & BlazeMeter
Shift left testing is crucial to maintaining a successful business. Adopting shift left testing will ensure that your company achieves a higher quality product early on, and will pay enormous dividends by saving time, money, and effort down the line.
Perfecto and BlazeMeter, now under the same umbrella of Perforce Software, pairs together to enable shift left testing at scale, empowering early detection of bugs and providing the tools needed to adapt prior to launch, allowing you to release high-quality apps with ease.
Perfecto and BlazeMeter offers the following benefits:
- Create, execute, and analyze all testing types: API, Functional, Non-Functional, Exploratory, Unit, all at maximum scale.
- Get a clear view of your entire mobile experience. Test on thousands of devices, OSs, and browsers across the globe and see how your mobile app performs in real-world conditions.
- Test apps for web & mobile as if you were a real user, down to the very swipe/click and even with image injection, biometric testing, location, accessibility testing, network virtualization, and more.
Learn more about this powerful pairing by signing up for a custom demo today.