Cypress vs. Selenium
October 4, 2019

Cypress vs. Selenium: What’s the Right Cross-Browser Testing Solution for You?

Continuous Testing
Automation

Selenium is without a doubt the de-facto test automation framework for cross-browser testing — and it has been for many years.

But now, another framework is gaining traction. With dozens of variations of frameworks that were built on its protocol, WebDriver, momentum in the market is growing for Cypress.io.

In this post, I will break down Cypress vs. Selenium and explore some of the objectives that can help teams decide which option is right for them — or if using both is better.

Introduction to Selenium and Cypress

To get started with the comparison, let’s define what these two solutions are and what they offer practitioners.

Selenium

Selenium is a test automation tool that enables developers to automate web browser testing. The Selenium WebDriver protocol enables sending commands in various development languages like Java, Java Script, C#, Python, and others from the test environment (IDEs) to a selected desktop browser (Chrome, Firefox, Edge, Safari).

Each of the browsers has its own WebDriver that is a dependency for the test to be able to communicate and perform the required actions (click, swipe, assert, etc.).

As a leading solution, Selenium serves as a foundation to common test frameworks like Protractor, WebDriverIO, and others, as well as mobile app testing frameworks like Appium. Looking at the market trends around adoption and downloads, it is obvious that Selenium is a key enabler for browser test automation. We can also see the growth in adoption of Cypress that has passed WebDriverIO in the number of downloads.

Getting Started With Selenium

Getting started with Selenium test automation is quite easy. It requires a local or cloud-based setup of a Selenium grid, Selenium WebDriver for the browsers you wish to test against, and development skills in the various WebDriver supported languages.

To get started with local Selenium, navigate to the Selenium home page, download the relevant web drivers, and set up the IDE environment with relevant development languages.

A simple local Selenium script would look like this:

package com.perfecto.sampleproject;

            import java.util.concurrent.TimeUnit;

            


            import org.openqa.selenium.WebDriver;

            import org.openqa.selenium.chrome.ChromeDriver;

            import org.testng.annotations.Test;

            


            


            public class LocalSelenium {

            


                        @Test

                        public void main() {

                                    //Note: Download chromeDriver for windows and update the below if running from Windows.

                                    System.setProperty("webdriver.chrome.driver",System.getProperty("user.dir") + "//libs//chromedriver");

                                    //A sample chrome driver script to access perfecto website and verify the title

                                    WebDriver driver = new ChromeDriver();

                                    driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

                                    driver.manage().timeouts().pageLoadTimeout(15, TimeUnit.SECONDS);

                                    driver.get("https://www.perfecto.io");

                                    String aTitle = driver.getTitle();

                                    System.out.println(aTitle);

                                    //compare the actual title with the expected title

                                    if (aTitle.equals("Web & Mobile App Testing | Continuous Testing | Perfecto"))

                                    {

                                                System.out.println( "Test Passed") ;

                                    }

                                    else {

                                                System.out.println( "Test Failed" );

                                    }

                                    driver.close();

                                    driver.quit();

                        }

            


            }

A sample Selenium project code in Java would look like this:

Package com.perfecto.sampleproject;

            import java.net.MalformedURLException;

            import java.net.URL;

            import java.util.concurrent.TimeUnit;

            


            import org.openqa.selenium.Platform;

            import org.openqa.selenium.remote.DesiredCapabilities;

            import org.openqa.selenium.remote.RemoteWebDriver;

            import org.testng.annotations.Test;

            


            import com.perfecto.reportium.client.ReportiumClient;

            import com.perfecto.reportium.client.ReportiumClientFactory;

            import com.perfecto.reportium.model.Job;

            import com.perfecto.reportium.model.PerfectoExecutionContext;

            import com.perfecto.reportium.model.Project;

            import com.perfecto.reportium.test.TestContext;

            import com.perfecto.reportium.test.result.TestResult;

            import com.perfecto.reportium.test.result.TestResultFactory;

            


            


            public class PerfectoSelenium {

            


                        @Test

                        public void main() throws MalformedURLException {

                                    //Update cloudName variable with your perfecto cloud name

                                    String cloudName = System.getProperty("cloudName");

                                    //Update securityToken variable with your perfecto security token.

                                    String securityToken = System.getProperty("securityToken");

                                    String browserName = "mobileOS";

                                    DesiredCapabilities capabilities = new DesiredCapabilities(browserName, "", Platform.ANY);

                                    capabilities.setCapability("securityToken", securityToken);

                                    capabilities.setCapability("platformName", "Android");

            


                                    RemoteWebDriver driver = new RemoteWebDriver(new URL("https://" + cloudName + ".perfectomobile.com/nexperience/perfectomobile/wd/hub"), capabilities);

                                    driver.manage().timeouts().implicitlyWait(15, TimeUnit.SECONDS);

                                    driver.manage().timeouts().pageLoadTimeout(15, TimeUnit.SECONDS);

                                    // Reporting client. For more details, see https://developers.perfectomobile.com/display/PD/Java

                                    PerfectoExecutionContext perfectoExecutionContext;

                                    if(System.getProperty("reportium-job-name") != null) {

                                                perfectoExecutionContext = new PerfectoExecutionContext.PerfectoExecutionContextBuilder()

                                                                        .withProject(new Project("My Project", "1.0"))

                                                                        .withJob(new Job(System.getProperty("reportium-job-name") , Integer.parseInt(System.getProperty("reportium-job-number"))))

                                                                        .withContextTags("tag1")

                                                                        .withWebDriver(driver)

                                                                        .build();

                                    } else {

                                                perfectoExecutionContext = new PerfectoExecutionContext.PerfectoExecutionContextBuilder()

                                                                        .withProject(new Project("My Project", "1.0"))

                                                                        .withContextTags("tag1")

                                                                        .withWebDriver(driver)

                                                                        .build();

                                    }

                                    ReportiumClient reportiumClient = new ReportiumClientFactory().createPerfectoReportiumClient(perfectoExecutionContext);

                                    try {

                                                reportiumClient.testStart("Perfecto mobile web test", new TestContext("tag2", "tag3"));

                                                reportiumClient.stepStart("browser navigate to perfecto");

                                                            driver.get("https://www.perfecto.io");

                                                reportiumClient.stepEnd();

                                                reportiumClient.stepStart("Verify title");

                                                            String aTitle = driver.getTitle();

                                                            System.out.println(aTitle);

                                                            //compare the actual title with the expected title

                                                            if (!aTitle.equals("Web & Mobile App Testing | Continuous Testing | Perfecto"))

                                                                        throw new RuntimeException("Title is mismatched");

                                                reportiumClient.stepEnd();

            


                                                //STOP TEST

                                                TestResult testResult = TestResultFactory.createSuccess();

                                                reportiumClient.testStop(testResult);

            


                                    } catch (Exception e) {

                                                TestResult testResult = TestResultFactory.createFailure(e);

                                                reportiumClient.testStop(testResult);

                                                e.printStackTrace();

                                    } finally {

                                                driver.close();

                                                driver.quit();

                                                // Retrieve the URL to the DigitalZoom Report

                                                String reportURL = reportiumClient.getReportUrl();

                                                System.out.println(reportURL);

                                    }

                        }

            }

The above script will simply validate on the Perfecto.io website the page title and report back results to the reporting dashboard. The exact script above could be also executed against a local desktop machine through the ChromeDriver or others that test developers would set up.

Cypress.io

Cypress is a JavaScript test automation solution for web applications. It enables teams to create web test automation scripts. This solution aims to enable frontend developers and test automation engineers to write web tests in the de-facto web language that is JavaScript.

Cypress also supports the Mocha test framework so the core technologies in which you would develop your web test automation are Java Script on top of Mocha.

Getting Started with Cypress.io

To get started with Cypress, you would need to download the NPM package through “npm install cypress” as well as “npm install mocha” or download the actual Cypress desktop application from the cypress.io website.

The solution comes with a nice set of JavaScript examples. Once you launch Cypress (./node_modules/bin/cypress open) you can trigger them to run with an embedded Chrome browser that the tools come built with. The fact that the solution already bundles a browser ready to be used makes the ramping up and execution even easier.

Cypress, while built on top of MochaJS, comes with a large set of APIs to leverage browser-specific commands, like setting viewport, clear cookies, basic browsing commands, and more.

An example of a test written in Cypress would look something like this:

The cy.visit will simply navigate with the Chrome browser to the URL. cy.get will perform the required action on a specific element on the page like click, perform assertions, or type text.

context('Waiting', () => {

  beforeEach(() => {

    cy.visit('https://example.cypress.io/commands/waiting')

  })

  // BE CAREFUL of adding unnecessary wait times.

  // https://on.cypress.io/best-practices#Unnecessary-Waiting


  // https://on.cypress.io/wait

  it('cy.wait() - wait for a specific amount of time', () => {

    cy.get('.wait-input1').type('Wait 1000ms after typing')

    cy.wait(1000)

    cy.get('.wait-input2').type('Wait 1000ms after typing')

    cy.wait(1000)

    cy.get('.wait-input3').type('Wait 1000ms after typing')

    cy.wait(1000)

  })


  it('cy.wait() - wait for a specific route', () => {

    cy.server()


    // Listen to GET to comments/1

    cy.route('GET', 'comments/*').as('getComment')


    // we have code that gets a comment when

    // the button is clicked in scripts.js

    cy.get('.network-btn').click()


    // wait for GET comments/1

    cy.wait('@getComment').its('status').should('eq', 200)

  })


})

Pros and Cons: Cypress vs. Selenium

While there is no right or wrong with these two frameworks, it is important to distinguish them and understand some of the benefits and in some cases the disadvantages each framework has.

Test Framework

Supported Dev Languages

Supported Browsers

Supported Test Frameworks

Setup and Execution

Integrations

Breadth of Testing Options

Maturity, Documentation, Support

Selenium WebDriver

Java, C#, Java Script, Python, Ruby, Objective-C

Chrome, Safari, Firefox, Edge, IE

Mocha JS, Jest, Other super set on top of Selenium (Protractor, WebDriverIO, etc.

Download relevant driver, set up a grid, network and location impacts execution speed

Plenty of integrations (CI, CD, reporting, cisual testing, cloud vendors)

End-to-end, security, unit,

Robust community, multiple bindings, best practices

Cypress.io

JavaScript

Chrome, Electron

Mocha JS

Comes with bundled Chrome browser, no complex environment setup

Limited integrations

End-to-end

Good documentation and code sample, growing community

The two solutions provide different capabilities as in the above table. There are other elements, like the simplicity of writing scripts, reporting, and dashboards, but as a whole and in every tool selection, it all depends on the team’s objectives, skillset, scope to test, and other product-specific considerations.

Cypress is a great growing tool. It is fast to ramp up with and provides a good execution environment that is baked in. It is fully JavaScript/MochaJS oriented with specific new APIs to ease the scripting.

On the other hand, Selenium is a mature framework covering multiple browsers with different development languages. And it works well within a grid to scale testing. The element of test flakiness is debatable between the two tools.

Some would argue that Cypress produces a more robust and reliable test scripts, while Selenium experts can provide good practices to overcome such problems.

Bottom Line

Teams should start exploring Cypress to see if it can complement their existing Selenium scripts and grow their overall test coverage and stability.

If you already have a good working Selenium suite that is stable and covers enough functionality, there is no real urgency to switch tools. If you are starting a new project, perhaps having a simple POC with Cypress can prove to be a good future solution for you.

Perfecto makes testing with Selenium and Cypress even better with end-to-end continuous testing support. Try it today.

Try Perfecto