Cross-browser Testing With Cypress.io

Two leading tools in the field of software testing are Cypress.io and Applitools. Cypress.io is an open-source framework best known for its tools that help developers write integration and End-to-End (E2E) testing. Applitools, on the other hand, integrates with Cypress.io, giving testers, and developers, a chance to upgrade their test scripts into Visual ones.

While Cypress.io lacks support for cross-browser testing, Applitools supports it via their Applitools Ultrafast Grid.

You can combine the best of both worlds by using Cypress.io to write your integration and E2E tests and letting Applitools Ultrafast Grid handle cross-browser testing for you.

My goal, in this article, is to take you through a step-by-step guide that will include:

1. Writing a Cypress.io E2E test.
2. Running the E2E test locally using Google Chrome.
3. Converting the E2E test into a Visual test using Applitools Eyes Cypress SDK.
4. Running the Visual test across multiple browsers via the Applitools Ultrafast Grid.

Let’s start!

Welcome to Cypress.io

Cypress.io gives you a complete end-to-end testing experience by providing you with the right tools to set up your testing environment locally, write E2E tests, run the tests, and record them. It has been written from the ground-up without any dependency on other tools like Selenium.

To learn and use Cypress.io, I suggest watching the course Introduction to Cypress by Gil Tayar as he covers all the necessary information you need to get started using Cypress.io.

Show me an E2E test

I won’t keep you waiting any longer. Here’s a sample E2E test I wrote in Cypress.io:

The sample E2E test script above starts by:

1. Navigating to the root page of the application.
2. Selecting two product items displayed on the page.
3. Verifying the two items selected are the ones displayed in the comparison section at the bottom of the page.

In the demo section below, I will delve into more detail on installing Cypress.io, setting it up for your projects, writing E2E tests, and much more. Stay tuned!

Oh, Cypress.io only works with Google Chrome!?!

Writing automated E2E tests with Cypress.io is fun and easy. Cypress.io has been gaining momentum in the software testing field due to its solid architecture, well-designed API and rich documentation just to name a few.

Cypress.io allows testers and developers to write E2E tests that run locally on the Google Chrome browser only. There has been debate, and requests from developers in general, to have this extend to other browsers as well. The Cypress.io team has listened, and are in the process of doing so.

You can follow the discussion on this GitHub Issue, Proposal: Support for Cross Browser Testing, for more information, and to stay up-to-date with the roadmap to support this. However, for now, you can only run your E2E tests locally with Google Chrome.

Is there a solution? Yes!

You can use Applitools’s Ultrafast Grid system to run Cypress.io E2E tests against multiple browsers and guarantee a cross-browser testing result.

Locally, you will be running your tests on Google Chrome while Applitools Ultrafast Grid will handle setting up the cross-browser environment to run your tests against multiple browsers.

Let’s see how it works!

Applitools to complement Cypress.io

Applitools offers a test framework to automate Visual testing of your application. It focuses on the visual aspects of your app and plays a major role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to seamlessly communicate, and interact with Applitools.

The Applitools Ultrafast Grid runs your Visual tests quickly and supports a variety of configuration settings, such as multiple browsers, viewports, and devices. The Applitools Ultrafast Grid is split into two components: server-side and client-side. Here’s a diagram of how it works:

Applitools Ultrafast Grid – Setup Code

The Ultrafast Grid client is a Node.js library, used internally by both Applitools, Storybook and Cypress SDKs. Here’s more on the Applitools Ultrafast Grid client.

That code snippet instantiates a new Grid client and returns an object. In this case, we are only interested in the openEyes() method returned.

Applitools Ultrafast Grid – Snapshot Capture

When called, the openEyes() method starts a new test run and sends the Grid backend server details about it. These details include the application name, batch or test name, the browsers used, and different variations of browser settings to be adapted while running the tests. This method returns a Promise to an object with the following functions: checkWindow() and close().

This method call instructs the Ultrafast Grid backend server to prepare a testing environment for a new test run that requires running the test over two different browsers. The first browser emulates an iPhone X, while the second specifies the desktop viewport to run the test.

To start collecting snapshots, the test issues a call to checkWindow() function. The Ultrafast Grid client handles this call and collects all the needed resources to store the snapshot locally.

At the end of the test, you issue a call to close() method to close and finish the test run.

Applitools Ultrafast Grid – Grid Tests

At this moment, all the DOM Snapshots collected locally during the test run are sent to the Ultrafast Grid backend server. Based on the test run requirements, sent previously with the openEyes() method, the Ultrafast Grid:

• Launches a number of browsers with different viewports and device emulators.
• Runs all of the DOM snapshots in parallel.
• Collects image screenshots.
• Sends all the screenshots to the Applitools AI server to do all the testing analysis and to return the results.

We will focus on the power of the Applitools Ultrafast Grid, to set up a testing environment in the cloud that allows running Visual tests against multiple browsers and resolutions.

Before I go further into writing and running Visual/E2E tests with Applitools Ultrafast Grid, let’s have a look at how Applitools integrates itself into Cypress.io.

Applitools/Cypress.io Integration

Applitools integrates with Cypress.io, and offers a general-purpose library that can be embedded within your project, whether it be the Angular, Vue, or React app.

The Applitools Eyes Cypress SDK is a simple plugin to Cypress.io. Once installed, it adds a few commands to the main cy object. Specifically, it adds three main methods: cy.eyesOpen to start the test, cy.eyesCheckWindow to take screenshots (for each test step) and cy.eyesClose to close the test. Here’s a diagram:

Now that you understand how the Applitools/Cypress.io integration works, let’s switch gears and start coding an E2E test.

Cross-browser testing step by step guide

Let’s get our hands dirty and build some E2E tests. In this section you will learn how to:

• Install Cypress.
• Write an E2E test.
• Run the test.
• Install Applitools Eyes Cypress SDK.
• Write a Visual test.
• Run that test.
• Finally, do an analysis on the test results.

The E2E test I will be using for demonstration runs against one of the Applitools customers, https://www.salesforce.com. The test does the following:

• Navigates the browser to SalesForce contact form located at: https://www.salesforce.com/uk/form/contact/contactme/
• Fills the form fields with valid data.
• Locates the Submit button and clicks it.
• Finally, the test script verifies that the form has been successfully submitted by making sure the page displays a success message coming from the server.

Let’s start!

Create a new project folder

Start by creating a new folder by running these commands:

```

mkdir projects

cd projects

mkdir cross-browser-testing

cd cross-browser-testing

```

Now that you are inside the cross-browser-testing folder, run the following command to initialize NPM:

```

npm init -y

```

The command initializes your project with a package.json file. Now you can start installing NPM packages to use inside your project.

Install Cypress.io

Add Cypress package to the project by running this command:

```

npm install cypress --save-dev

```

The Cypress npm package adds a set of test files to help you write your own automated tests.

Open the Cypress Test Runner (Electron app) by issuing:

```

npx cypress open

```

This command opens the Cypress Electron app and also creates a cypress.json file and a cypress folder in your app’s root directory. The cypress folder is where we will be writing our tests.

To simplify our E2E tests, open the cypress.json file located at the root of your project and paste the following content:

```

{

"baseUrl": "https://www.salesforce.com/uk"

}

```

Now that Cypress.io is running properly, let’s add our first E2E test.

Write your first E2E test

Inside the cypress\integration folder, create the salesforce-contactus.spec.js file, and paste the following:

The single E2E test focuses on testing the contact us form by filling its fields on the SalesForce website.

The spec file is self-explanatory. There’s just one thing to note here in relation to the selectors used inside cy.get() methods. The best practice is to avoid using IDs and CSS classes to select elements from the DOM. They will make your tests brittle because they’re subject to change. A better approach is to use data-* attributes, or actual component names.

In a controlled environment, I would stick to using data attributes. For now, we will keep on using the ID and Class CSS selectors for querying for DOM elements.

Run the test

The E2E test is ready, so let’s run it by issuing the following command:

```

npx cypress open

```

Cypress.io opens an instance of the Google Chrome browser to run your tests. Now, to run the same test against other browsers, we have to convert this E2E test into a Visual test and run it with Applitools Ultrafast Grid.

Let’s see how it works.

Install Applitools Eyes Cypress SDK

Let’s add the Applitools Cypress NPM package to the project.

Add the Applitools Eyes Cypress SDK package to the project by issuing the following command:

```

npm install @applitools/eyes-cypress --save-dev

```

Configure the Applitools Eyes Cypress SDK by issuing the command:

```

npx eyes-setup

```

Now that Applitools Eyes Cypress SDK is configured, we can start using it in our tests.

The Applitools Eyes Cypress SDK page has thorough documentation on how to use and customize the Applitools-Cypress integration, and how to configure the SDK to run tests against the Ultrafast Grid.

For this article, we will focus on how we can instruct the Visual Grid to run our test against multiple browsers and resolutions.

In general, there are three ways to specify test configuration:

1. Arguments to cy.eyesOpen().
2. Environment variables.
3. The applitools.config.js file.

You can read more about each method under the Advanced Configuration section.

I will make use of the first method, and supply a list of browsers and screen resolutions as arguments to the cy.eyesOpen() function.

Configure Applitools API Key

Before you can locally run Applitools Eyes Cypress SDK, make sure you get an Applitools API Key and store it on your machine as an environment variable.

To set the APPLITOOLS_API_KEY environment variable, you can use the export command on Mac, or set command on Windows as follows:

Write your first Visual test

Now that you have the E2E test running, let’s convert this E2E test into a Visual test using Applitools Eyes Cypress SDK.

Revisit the cypress\integration\salesforce-contactus.spec.js file, and replace its content with:

The code is self-documented.

Inside the beforeEach() test function, you start a new Visual test by calling the cy.eyesOpen({}) function that accepts a configuration object as input. You can specify the app name, batch name, browsers you want the Ultrafast Grid to use in order to run your tests, and many other configuration options available to you, on Applitools Eyes Cypress SDK page.

Close the current Applitools test by calling the cy.eyesClose() function inside the afterEach() test function.

Take a snapshot (You may repeat this step wherever you want to take a snapshot) by calling the cy.eyesCheckWindow() function. This function can take a snapshot name in case you want to distinguish snapshots from each other.

The test takes three snapshots at three different stages:

• The blank SalesForce Contact Us page.
• Once the SalesForce Contact Us form filled.
• On the success confirmation page after submitting the Contact Us form.

Let’s run the test, and verify the results together.

Run the test

The Visual test is ready. Let’s run it by issuing the command:

“`

npx cypress open

“`

The Eyes Cypress SDK are all logged.

Let’s review the test results on the Applitools Test Manager.

Verify Test Results

Let’s visit the Applitools Test Manager, and verify the results.

On the left-side panel, there is a single test run named “fill the contact us form”. On the right-side panel, there are three test results- one for each “browser”. Remember, we’ve configured the Eyes SDK to run this Visual test across three different browsers/screen-resolutions:

For each browser specified, the Grid spins a new instance of the browser and runs the test. This explains why we have three test results.

Applitools Ultrafast Grid takes the burden off your shoulders and establishes a cross-browser testing infrastructure in the cloud.

Each and every test result holds three snapshots. Let’s explore the snapshots for a single browser test result:

The first snapshot represents the form under test shown with empty fields.

The second snapshot represents the form under test shown with all fields filled.

The third snapshot represents the message shown to the user upon filling the “Contact us” form.

Using Applitools SDKs and Ultrafast Grid, we were able to run a single E2E test, written in Cypress.io, in a cross-browser/device testing environment!

Conclusion

Applitools, being a Visual Testing framework, fills the gap of E2E testing frameworks like Cypress.io by offering cross-browser testing. Its UltrafastGrid can set up a cross-browser testing environment in the cloud and can run your tests there.

Other Related Content

• How Cross-Browser Testing Is Evolving
• Angular and Cypress UI Testing with Applitools
• How to visually test VueJS apps using Cypress.io and Applitools [step-by-step tutorial]
• React, Cypress and Applitools for Functional and Visual Testing
• Learn Cypress Testing on Test Automation University
• More blog posts by Bilal Haidar
• Request an Applitools demo
• Sign up for a free Applitools account
• Check out the Applitools tutorials.