cloud computing illustration concept
May 23, 2017

Running XCUITest in Parallel with Quantum on Perfecto Cloud

Mobile Application Testing

If you're an iOS developer, you're probably familiar with Apple XCTest and XCUITest automation framework, which came with Xcode to allow you to easily develop UI tests that reflect the users’ interface with the application.

Since Xcode 8, Apple has deprecated UIAutomation in favor of the XCUITest framework.  XCTest has been part of Xcode for a few years now, but it is finally catching up and more developers are getting on the bandwagon. XCTest and XCUITest provide feature-rich capabilities for iOS developers and test automation engineers that aim to implement different levels of tests using Xcode features and supported programming languages, like Objective-C and Swift.

One clear benefit of using XCTest/XCUITest is that this framework bundles the tests on the client side (the device under test), therefore the tests run faster.  Additional advantages of this method are reduction of network traffic and greater control over the classes and controller while other open-source frameworks like Appium has multiple layers that need to be dealt with.

Of course XCUITest cases can only be executed on a single device at a time, so if you want to run these tests in parallel, you will not be able to do this out of the box.

In this post, I’ll reveal a way that will allow you to overcome that obstacle!

We will be able to set up XCUITest to run on multiple devices and in parallel with the help of an open-source Quantum Framework on the Perfecto cloud devices.

Before I explain how easily you can do this, let me explain the nature of the Quantum automation Framework (QAF).  QAF is an open-source test automation framework for web and mobile test authoring, execution and analysis. The framework enables its users to easily set up their environment to access the Perfecto Lab in the cloud, with user authentication and driver management. It allows you to create test scripts in BDD that are Gherkin-based or written in Java, and it will generatePerfecto DigitalZoom™ reports at the end of the test execution for quick drill down into the root cause of the failure.

ThisQAF Starter-Kit is available on GitHub.

The following steps describe a high-level how-to that, once followed, can enable XCUITest scripts to run on multiple devices in parallel using testing in a Quantum project.

First Step is to Create IPA Test Files Using Xcode for Testing

  1.     There are plenty of online instructions on how to set up and generate XCUITest files.
  2.     In this example, I am using a project called Document_Manager, and once XCUITest script is created, I build the ipa files and go through the code signing and profile provisioning. The two generated files are below.



  1.     Once the ipa files are built, next thing is to upload these files onto your Perfecto-hosted cloud repository.

Second Step is Preparing the Quantum Starter-Kit Project

  1.     Download and install QAF start project.
  2.     In my example, I am using IntelliJ Community Edition as my IDE for Quantum Project, and I have installed PerfectoContinuous Quality Lab plugin for my IDE tool.
  3.     Once installation is completed, you will first need to make edits to the file under resources folder, and change the following to your user credential and hosted-cloud domain name:
  1.     Next, you will need to create a custom steps file, and place it under the steps folder (src/main/java/com.quantum/steps)
  •  Right-click on steps folder -> New -> File, and then name it and save it as

  •  Then add the following code as shown below:

    • Make sure you are referencing to the test ipa files that you had created from the above instructions
  1.     Now, we need to create a scenario file and place it under scenario folder (src/main/resources/scenarios)
  • Right-click on scenarios folder -> New -> File, and then name it and save it as xcuitest.feature

  • Then add the following code as shown below:

Coded 2
  1.     The final file we need is a TestNG configuration file to allow us to run multiple devices in parallel. Therefore, we need to create this file and place it under config folder (src/main/resources/config)
  • Right-click on config folder -> New -> File, and then name it and save it as testng_XCUITestDemo.xml (or whatever name make sense)

  •  Alternatively, you can copy testng_appium.xml file and then make edits to your devices needs.

  •  I will be setting up to run three different iPhone devices in parallel. The selection of device is set to be any available iPhone S-type phone, however, if you want to select a specific device then you need to change the parameter capabilities from “model” to “deviceName” and set it with the device UDID number.

    Capture 66
  • In my example, I will be using the wildcard for iPhone.*  phones, and each of my test suites are unique. (i.e., iPhone Simple Document Manager XCUITest Test1)

    More Code 3

The Final Step is Running XCUITest on Multiple Devices in Parallel

1.Right-click on testng_EspressoDemo.xml file -> and select “Run…” from menu item.

Codee 6
  1. Viewing the all devices running XCUITest script
  • Click on Perfecto Dashboard pane to view devices testing in progress.
Code 55

In summary, if you’re using Xcode today to build and execute XCUITest scripts on a single device, and want to increase your testing coverage as well as your overall velocity, and move to parallel execution, it is recommended to follow the above instructions. Start by using the Quantum starter-kit project, which includes all the open-source components and vast capabilities required to accomplish these goals.

Good luck!