Laptop computer illustration
April 12, 2017

Test Automation for Windows 10 Surface


About Windows 10 Surface

Windows 10 Surface machines are becoming popular laptops, so many enterprises are developing Windows applications for this platform. The default browser on these machines is Microsoft Edge, which has several form factors and can behave as a laptop or a tablet. In short, there’s a lot that’s new and different on these machines that can go wrong. Naturally, test automation is the appropriate direction for ensuring high quality of applications and websites on this platform.

Test automation on Windows 10 machines is supported by WinAppDriver. This framework is an area of ongoing investment by Microsoft and the open source community to increase automation capabilities, and by the way, it applies to Mac machines as well. However, it’s not as capable as the Appium web driver offered on, for example, iOS and Android devices. And as mentioned in a previous post, even Appium/Selenium do not cover some scenarios, specifically those pertaining to visual screen analysis.

Maximizing Test Automation By Leveraging Native Objects and Visual Analysis

In this post, I will describe a method to combine native object and visual automation for Surface applications. Two test cases will be included in the project (see code): testing the stock Calculator app using native Windows objects (based on this example), and testing a simple web flow in the Edge browser using visual automation.

The base: a simple Maven/TestNG project with the following definitions

Test NG

Note, I included the WinAppID in the TestNG XML file. The way to get it is described here. contains the functions to instantiate an Appium or Selenium driver. It’s called from @BeforeTest in, which is the body of the test.

In, once the driver is initiated in @BeforeTest, we run the testEdge(driver) method to run the Edge test case or testCalculator(driver) for the native test.

testEdge launches the Edge browser, enters a search string into the Bing search box, and validates the results. Since I’m an Apple fan and am suffering through this Windows discussion anyway, I was searching for an “iPad Pro” and waiting to see the word “Apple” show up among the results. I also wanted to measure the time it takes to get the results with a UX timer.


For test stability, I then close the browser and manage the popup “Close all”.

Capture 2

A few comments:

  • All the visual functions are handled in java
  • I intentionally used all visual functions. This is not a best practice and was done strictly for educational purposes. As a best practice, use native objects where possible to reliably navigate through the app. Visual analysis should be used where you cannot access the objects or for validating accurate rendering or measure responsiveness.
  • It would be easy to change in java to run the testCalculator test:

private static void testCalculator (RemoteWebDriver driver) {

Capture 3

As another best practice, externalize the object names and expected results from the code, and store them in an object repository for easy maintenance and reusability of your scripts.

Test Automation Applicability

Test frameworks need to support both native object automation as well as visual automation. Sometimes, to compensate for the Appium driver limitations, one needs to use visual commands. Here’s a suggested implementation:

The script itself never changes. For example, when using Gherkin for BDD:


Let’s say the search box is not easily accessible as a native object, what I would do is:

  • Set separate object identifiers
    1. For native objects, using an XPath or object ID
    2. For visual analysis, I would look for the words “web address” on the screen


Where to Next
  • At the code layer, I would implement the commands based on the type of driver in use:
    • For Surface/Windows 10, I would use
  • For iOS/Android, I would enter the text FindElementByID(..).sendKeys()

In other words, the script doesn’t change, only the underlying implementation changes.

Automation Best Practice

Complete test automation must include both native object automation and visual automation. It can be further augmented using in-depth visual rendering validation with tools like Applitools, as mentioned in the past. Hopefully this post provides insight and some best practices on how to go about it. For more information, please click here.