Appium's Pros, Cons & What The (Testing Framework) Future Might Look Like
Since Perfecto is sponsoring and attending the annual Appium 2018 conference today, it’s a great time to take a look at some pros and cons, how it stacks up against the competing tools and what might be around the corner.
Benefits of Appium
What's great about Appium:
- It has a strong active open source community
- Appium is by far the leading open-source test framework for cross-platform(mobile) native test automation (iOS, Android)
- Appium is consistently backed by a large, very dynamic community, with steady support, commits, etc.
- It has strong support for end-to-end testing in multiple programming languages
- Appium provides support for multiple development languages through Remote WebDriver language bindings (Java, JavaScript, Perl, Python, C#)
- Appium can cover black box end-to-end test flows including outside-the-app scenarios (e.g. initiating a call, sending a text message)
Challenges with Appium
Some areas where Appium could be better:
- Setting up Appium locally can be a challenge: Teams are required to download, install and configure their environment which means having a local Android and iOS device available and connected.
- Working with app objects for both iOS and Android isn’t easy, and falls into the top challenges (below) coming from practitioners. Since the Appium framework relies on iOS XCUITest and Espresso (Android’s UI automation framework), users needs to be familiar with the object structure, and know how to use the Appium object spy correctly.
- Slow test execution. Tests can be slow due to the remote webdriver dependency, network issues and processing commands.
- Test framework stability – Stability issues can occur when executing through CI or at scale and in parallel.
- Test automation coverage and keeping up with latest mobile OS - Being able to fully cover gestural inputs, environment conditions, device settings and more, as well as support immediately latest Beta and GA versions of iOS and Android. Community supported solutions frequently move slower than handset vendor/mobile OS innovations.
Comparison of Mobile App Testing Frameworks
Despite Appium's leadership today, DevOps teams are also adopting Espresso and XCUITest.
Since there is no perfect solution with regards to testing frameworks, the best solution for your needs might be best met by mixing various test frameworks across the DevOps pipeline.
Here's a comparison of the leading testing frameworks:
Where is Mobile Application Testing Headed?
My testing framework intuition tells me that:
- Appium will share more of the testing framework market with Espresso and XCUITest.
- Functional testing using Espresso/XCUITest will become part of commit- triggered built-testing.
- Full end-to-end testing using Appium will be leveraged during full regression testing.
- Appium stability will improve and execution times will shrink – it will get better & faster!
- Hybrid test execution will become supported. Appium scripts will be able to trigger embedded Espresso/XCITests.
Conclusion
Appium is great! It’s got:
- A strong open source community
- Outstanding support for a number of programming languages
- The ability to handle end-to-end test flows
Appium is weak:
- In test performance and stability
- In keeping up with the latest OS features (e.g. gestural)
- In setup time
XCUITest and Espresso are also leading the market with strong user bases and helps fill in the gaps where Appium falls short - so keep an eye out for those other tools!!
It will be interesting to hear the upcoming Appium roadmap at the Appium 2018 Conference. Sign up for our live webinar to hear our take on the current and future state of testing frameworks and how Appium might fit into your DevOps toolchain.
Ready to Give Perfecto a Try?
It's open and fully integrates with all your favorite automation testing tools, such as Appium or Selenium. See for yourself what the world's only end-to-end continuous testing platform can do for you. Get started today for free.
