How Ten24 Scaled QA & Reduced Regression Testing Time by 73%
After 8 months with Perfecto Scriptless:
2K test executions
$400K cost avoidance
1,700 hours saved
- Reduced regression testing time from 3-5 days to 1-2 days within bimonthly sprints.
- Minimized technical debt significantly with Perfecto Scriptless AI.
- Improved quality of tests with Perfecto Scriptless intuitive UI.
Perfecto Scriptless has taken our efficiency and overall product quality to a whole new level. We would have never tested as much as we do now as quickly as we do now without it.
Kevin Jason, Director of Development at ten24
The Need for Codeless Automation
For many eCommerce companies, choosing the right platform to operate their business can often feel like a daunting task. To address this, ten24 created Slatwall, a comprehensive eCommerce software platform, so companies can run their eCommerce businesses more easily and in a more streamlined manner. Customers ranging from Sotheby’s to Scientific American Magazine use Slatwall to respond to new changes in the eCommerce landscape without changing how they operate.
To reduce the time spent on delivering or releasing updates to their customers’ platforms, ten24 searched for a functional automated testing tool that would help them test more quickly and efficiently. With a small QA department that was only doing manual testing at the time, ten24 was looking to free up time and resources for their testers as well as speed up their regression testing and end-to-end testing processes. This would also allow the QA team to focus more closely on the manual tests that actually require human involvement.
When searching for a solution, ten24 opted for a codeless test automation tool after deciding that they needed the following from their automated testing platform:
- User-friendly. The QA team at ten24 is comprised of business testers who do not have the technical background to work with a code-based test automation tool. Because of this, ten24 prioritized a tool with an intuitive UI that allowed for simpler test creation and execution.
- Scalable. The ten24 team started their test automation journey with Selenium but found it tough to scale and maintain over time. They did not have the resources to bring in engineers to manage Selenium testing, nor did they feel that it would be the best way to allocate company resources. In order to scale, ten24 needed a long-term solution that would grow with the QA department over time.
How ten24 Got Started
Since ten24 deploys eCommerce platforms for multiple companies, they started their test automation journey by focusing on select end-to-end testing and regression testing projects. They chose these projects as part of a larger strategy to start with tests that they felt would benefit the most from automation.
While any new tool requires a bit of a learning curve, ten24 found that Perfecto Scriptless was very easy to learn and implement into their environment. Perfecto’s user-friendly platform made onboarding new members of the ten24 QA team both simple and efficient.
ten24 also felt secure in their ability to rely on the Perfecto team for support. They felt that Perfecto was always available to address questions about the platform, as well as offer best practices for moving forward more efficiently.
Accelerating Testing With Visual Modeling
One of the first projects that ten24 chose to automate included a regression testing suite that they run every 2 weeks. After automating this suite with Perfecto Scriptless, ten24 reduced their time spent on their bimonthly regression testing, from 3-5 days of manual regression testing to just 1-2 days. This not only saved them hours of regression testing time, but shortened the entire project by allowing the QA team to test manually with the automated tests running in the background. In other projects, ten24 was able to minimize their testing time even more, from half a day of testing to just fifteen minutes.
ten24 determined that they were able to build and execute their regression tests both quickly and efficiently due to Perfecto Scriptless’s visual test modeling capabilities. The ten24 team found the drag-and-drop feature to be particularly helpful because it made test creation a much simpler and more intuitive process. By creating a model (as opposed to a test recording), ten24 was free to reuse test steps easily for future test scenarios. Any changes made to these steps would then be applied to all other scenarios that used those test steps. This reduced test creation time as well as test maintenance time dramatically.
*Note: TestCraft is now Perfecto Scriptless.
Perfecto Scriptless’s drag-and-drop feature also created a positive ripple effect on future projects. By allowing them to create new test suites quickly and easily, ten24 recorded that they could create new test suites 1.5 times more quickly. For example, if it took ten24 five days to complete their first project from the initial strategy to full completion, it would take four days to complete the next project and three days to complete the third. By taking advantage of reusable test steps feature, ten24’s was able to benefit from long-term time savings that were sustainable.
Reusing test steps in Perfecto Scriptless is great. It made our project turnaround time consistently faster.
Jada Flournoy, QA Lead at ten24
Catching Bugs, Eliminating Human Error
The scheduling feature and simple test creation capabilities in Perfecto Scriptless gave ten24 the flexibility to run a wider range of tests more frequently, which enabled them to catch bugs more quickly and more easily. This allowed ten24 to find issues before their clients did and avoid these bugs later on in the regression testing cycle, saving them the future costs of resolving bugs at a later stage of development.
Using Perfecto Scriptless also helped ten24 ensure that their tests were executed thoroughly, regardless of their frequency. They recognized that with the number of times they wanted to test their core end-to-end processes, manual testing would not cut it. Humans would either skip tests or make mistakes due to frequent repetition. With Perfecto Scriptless, ten24 was able to eliminate the “human error” that comes with manual testing, and reassured ten24 that their tests would run the same way every time.
Perfecto Scriptless completely took away the human error component of testing. No matter how many times we went through the test or scheduled it to run, we knew that we could rely on Perfecto’s results.
Jada Flournoy, QA Lead at ten24
Another way Perfecto Scriptless helped eliminate human error when testing was by providing a different point of view about tests. Not only would Perfecto Scriptless catch bugs earlier in the testing process, but the platform would also point out errors that the QA team wouldn’t necessarily consider or find on their own. This different perspective helped QA look at tests differently, which helped them later on when documenting how to build test suites for subsequent projects.
Reducing Costs & Technical Debt
The ability to catch bugs quickly also helped reduce ten24’s technical debt tremendously. Since ten24 built the vast majority of their tests using reusable components, the issues that they resolved often applied to multiple test suites. Perfecto Scriptless’s AI-based self-healing mechanism also helped reduce technical debt by ensuring that ten24 was creating resilient test scenarios that could scale.
Ten24 was not only able to speed up their end-to-end testing and regression testing, but they did this without having to hire any additional personnel to conduct automated tests or focus on maintaining a code-based tool like Selenium. The ability to create and execute test suites quickly while maintaining their existing QA team made a major financial impact, saving ten24 roughly $400,000 in personnel costs annually. These cost savings allowed ten24 the flexibility to better allocate their resources for future company needs.
Improving Test Automation Strategy
While ten24 reaped the various test automation benefits that Perfecto Scriptless had to offer, they also found that it had a positive impact on their overall testing strategy. From the beginning of their test automation journey, ten24 had a very clear plan about which tests they wanted to automate and implement. Yet they also understood that while you can have a great plan, the plan can change when going through the actual tests themselves and running through a wide range of different scenarios.
ten24 found that Perfecto Scriptless helped bridge the gap between their original testing plans and what actually happened in real-time. As a scriptless automation platform, Perfecto Scriptless made it very easy for ten24’s QA team to find holes in their tests. This helped them regroup and build future scenarios that would account for these previously overlooked areas.
You can have a great plan when it comes to your automated testing, but things can change when you’re actually going through the process. Perfecto Scriptless helps us bridge the gap.
Kevin Jason, Director of Development at ten24
Defining a comprehensive test automation strategy, coupled with the right tooling, creates a great foundation for achieving your testing goals. With Perfecto Scriptless, ten24 took their testing to the next level, which gave them the time they needed to keep their clients on the cutting edge of eCommerce.
Test Smarter With Perfecto Scriptless
Watch a demo or start a trial to learn more about codeless UI test automation with Perfecto Scriptless.