Appium & Flutter for Testing Mobile Apps: The Tradeoffs
December 23, 2021

Appium & Flutter for Testing Mobile Apps: The Tradeoffs

Mobile Application Testing

From BMW to Nubank, applications built in Flutter continue to grow in popularity. As more organizations use Flutter for their mobile apps, finding the best way to test them is increasingly important.

While some opt to test within the native Flutter framework, other teams prefer to test with the Appium-Flutter driver. Here, we delve into the tradeoffs of using Flutter and Appium for testing Flutter applications, as well as the best time to use each framework.

Types of Automated Flutter Tests

There are three types of tests that go into testing a Flutter application properly:

  • Unit testing. These tests validate the behavior of a single method, function, or class.
  • Widget testing. This verifies that the widget’s UI looks and behaves as expected when responding to certain interactions.
  • Integration testing. These tests verify application performance and ensure that all the services and widgets under test work together properly.

While unit testing and widget testing typically fall under the purview of the development team, testers are often the ones responsible for integration testing.

Testers must write these integration tests and determine that they run properly on each device of their choosing. These devices can be locally hosted or cloud-based. Unlike unit and widget testing, integration tests are the only ones that test the application in its entirety.

When discussing when to use Appium and Flutter to test Flutter applications, this comparison will focus on integration tests.

Option #1: Native Flutter Framework

Testers can run integration tests natively in the Flutter framework using Dart, an easy-to-use non-programming language that is like Java. Integration tests in Flutter follow these steps:

  • Create an app to test.
  • Add the flutter_driver dependency.
  • Create the test files.
  • Instrument the app.
  • Write the integration tests.
  • Run the integration test.
Sample Flutter integration test.
Sample Flutter integration test.

The major difference between Flutter and other frameworks like Espresso and XCUITest is that the Flutter script under test applies to both Android and iOS apps (since Flutter enables developers to build both types of application with one single codebase).

Additionally, native Flutter tests can easily handle the new features that come with each release. For example, Flutter’s latest 2.5 release in September 2021 offers support for Android Full Screen edge-to-edge mode, ScrollUnder support within the Material Design, and the ability to do static code analysis of the Dart code.

Another Flutter feature that is especially relevant to testing is the Flutter inspector. Inspector helps testers visualize and explore Flutter widget trees, which helps them get the element properties needed for writing integration tests. This is a major advantage for testing Flutter apps within the native Flutter framework.

Yet, it is important to keep in mind that the Flutter framework is limited in its testing scope to the Flutter application under test. Therefore, testing events that happen outside the context of the application on a mobile device are not supported.

Option #2: Appium_Flutter Driver

The second way to test Flutter applications is with the Appium_Flutter driver, an open-source solution that uses Webdriver.io.

From a scripting perspective, testing Flutter apps do not require any changes in writing Appium tests. By configuring a few settings to expose object properties correctly, you can also test Flutter applications on iOS and Android devices within the Perfecto cloud.

Executing Flutter tests with Appium as part of the Perfecto platform offers many capabilities that are not possible within Flutter on its own, such as biometric authentication, accessibility support, and two-factor authentication. Appium is also unique in that it can account for system-level controls within tests, such as notifications and changing device settings.

Option #3: Appium 2.0 Beta

The last method for testing Flutter applications is on the horizon, as Appium 2.0 is still in beta mode. With this new version, Appium 2.0 will split the drivers by different technologies, including Flutter.

You can start experimenting with Appium 2.0 and Flutter by installing the beta version through:

npm install –g [email protected]

Next, you just need to install the Flutter driver from the Appium 2.0 beta list of supported drivers, which you can see below:

Appium 2.0 beta with Flutter and other drivers

 

Installing the Flutter driver that is already baked into Appium will make testing Flutter applications with Appium more robust, advanced, and connected.

Appium-Flutter Testing & Flutter Testing Tradeoffs: When to Use Each Option

With this understanding of the three different ways to test Flutter applications, below you will see a comparison between running Flutter tests with Appium and Flutter or with the native Flutter framework.

Appium-Flutter Testing vs. Flutter Testing comparison chart

 

While this is a comparison, doing Flutter native testing and Appium-Flutter testing does not have to be mutually exclusive. Rather, there are opportunities for a single testing and development team to enjoy the benefits of both options.

The way to enjoy both Flutter and Appium-Flutter testing is by separating the application itself from the other important functionalities that surround the application. Testing natively within Flutter may be a better option for testing the actual application, due to the Flutter inspector and other important features.

On the other hand, Appium-Flutter testing can help expand the coverage of the Flutter application by offering capabilities and support not found within Flutter itself. Within Perfecto, Appium-Flutter testing is also a well-maintained option that will ensure testing that is both stable and scalable.

The following video shows a short demo of Flutter testing within the Perfecto Cloud:

 

Bottom Line

There are multiple ways to execute Flutter tests, from the Appium_Flutter driver to natively within Flutter itself. Understanding the pros and cons of each testing method is critical to building a comprehensive test strategy for your Flutter applications that makes the most of each testing option.

Watch Appium and Flutter Application Testing in Action

See a demo of testing Flutter apps with Appium and Perfecto.

Get Demo