October 4, 2022

The Value of Using Virtual Devices for Testing

Mobile Application Testing

The need for virtual devices for testing is a widely discussed topic in the world of automation, and there are many differing opinions as to whether virtual devices — also known as iOS simulators and Android emulators — are truly necessary. 

For example, in BrowserStack’s discussion of emulators and simulators, they note that despite their support of virtual devices for those that want it, their first choice and ultimate recommendation is to test only on real devices: 

“Bear in mind that emulators and simulators are only fit for use in the initial stages of testing. Website emulators, browser simulators, mobile phone simulators, online virtual devices, and the like are not reliable enough to run tests on for public release of apps and sites. The best practice, without exception, is to test on real devices." 

While we at Perfecto agree that simulators and emulators are best to use during earlier stages of testing, we also believe that virtual devices have value on their own. When used correctly, virtual devices have the potential to help teams achieve better test automation.  

This blog will discuss the value of using virtual devices for testing in greater detail and offer our best practices for incorporating them effectively into your testing. 

Why Use Virtual Devices for Testing 

Testing teams use virtual devices for testing to achieve two major goals: increase testing speeds while lowering total cost of ownership.  

Virtual devices are known for their speed, especially when compared to real devices. By using virtual devices during earlier stages of software development, teams can get faster feedback on issues that may arise. For one enterprise financial services company, using virtual devices helped them cut their feedback loop during their nightly regression cycle from 1500 hours when testing sequentially to 7.5 hours.  

This fast feedback allows them to resolve issues earlier in development, making this type of debugging much less costly than when bugs are caught at a later development stage.  

Additionally, the technology behind simulators and emulators have evolved significantly over time. While they certainly cannot address some of the more advanced mobile testing challenges, virtual devices are still very impressive in their own right and can automate at least 50-60% of your testing scenarios. That number should not be taken lightly. 

Another reason why teams use virtual devices for testing is to expand the responsibility of testing to the developers. As organizations are becoming more agile, testers and developers are working more closely together, as opposed to in a silo. When both testers and developers can be involved in testing, organizations can ensure that they are doing comprehensive testing throughout the software development lifecycle. 

How to Use Virtual Devices for Testing 

When determining how to use virtual devices for testing, it is helpful to visualize the testing pyramid, which groups tests strategically into three different categories: unit, integration, and UI tests.  

Pyramid of virtual devices for testing

Related Reading: Learn more about high-velocity testing with a mix of real and virtual devices. >> 

Tests in the lower portion of the pyramid, i.e. unit and integration tests, are faster because they are more automated, while UI tests at the upper portion of the pyramid are slower and more manual.  

Virtual devices are great to use for both unit and certain types of integration tests, which are earlier in the software testing life cycle. During unit testing, where developers can test isolated units of code per each commit, simulators and emulators can provide essential feedback that developers need.  

When it comes to integration testing, virtual devices can be used along with real devices. While real devices are great for the performance testing, security, and accessibility testing aspects of integration testing, virtual devices are better for acceptance and unit tests. In fact, virtual devices can run these types of tests more efficiently and quickly than real devices.  

While you could technically use real devices for these types of tests, it is not the recommended choice. By using virtual devices for testing in these areas, and using real devices at later stages in the testing cycle, teams can quickly benefit from higher speeds and reduced costs, specifically relating to: 

  • The software development cycle.
  • Fixing bugs.
  • Bugs creeping into production.
Graph of virtual devices for testing

 

Bottom Line 

While real devices are great for mobile testing, they are not always better. Virtual devices hold a unique place in the world of testing and should not be treated as a second-rate option.  

Perfecto strongly believes that by using virtual devices for testing as part of a robust mobile testing strategy, organizations can deliver applications with a powerful combination of speed and quality. With Perfecto’s support of virtual devices, and our latest offering of virtual devices as part of our free trial, teams can experience for themselves how within one platform, they can test their mobile apps both thoroughly and cost-effectively. 

Experience Virtual Devices in Action 

Try Perfecto for free today or get a custom demo to discuss your mobile testing strategy with one of our experts. 

Start Trial 

Get Custom Demo