In the fast-paced and competitive landscape of software development, ensuring the quality of applications is of utmost importance. Functional testing plays a vital role in verifying the robustness and reliability of software products. With the increasing complexity of applications with a long list of use cases and the need for faster release cycles, organizations are challenged to conduct thorough functional testing across different platforms, devices, and screen resolutions. 

This path to a better quality of software products is where Applitools, a leading provider of functional testing solutions, becomes a must-have tool with its innovative offering, the Execution Cloud.

Applitools’ Execution Cloud is a game-changing platform that revolutionizes functional testing practices. By harnessing the power of cloud computing, the Execution Cloud eliminates the need for resource-heavy local infrastructure, providing organizations with enhanced efficiency, scalability, and reliability in their testing efforts. The cloud-based architecture integrates with existing testing frameworks and tools, empowering development teams to execute tests across various environments effortlessly.

This article explores how the Execution Cloud and its self-healing capabilities can be used to run our functional test coverage. We demonstrate this cloud platform’s features, like auto-fixing selectors caused by a change in the production code. 

Why Execution Cloud

As discussed, the Applitools Execution Cloud is a great tool to enhance any team’s quality pipeline.

One of the main features of this cloud platform is that it can “self-heal” our tests using AI. For example, if, during refactoring or debugging, one of the web elements had its selectors changed and we forgot to update related test coverage, the Execution Cloud would automatically fix our tests. This cloud platform would use one of the previous runs to deduce another relevant selector and let our tests continue running. 

This self-healing capability of the Execution Cloud allows us to focus on actual production issues without getting distracted by outdated tests. 

Functional Testing and Execution Cloud

It’s fair to say that Applitools has been one of the leading innovators and pioneers in visual testing with its Eyes platform. However, with the Execution Cloud in place, Applitools offers its users broader, more scalable test capabilities. This cloud platform lets us focus on all types of functional testing, including non-Visual testing.

One of the best features of the Execution Cloud is that it’s effortless to integrate into any test case with just one line. There is also no requirement to use the Applitools Eyes framework. In other words, we can run any functional test without creating screenshots for visual validation while utilizing the self-healing capability of the Execution Cloud.

Adam Carmi, Applitools CTO, demos the Applitools Execution Cloud and explores how self-healing works under the hood in this on-demand session.

Writing Test Suite

As we mentioned earlier, the Execution Cloud can be integrated with most test cases we already have in place! The only consideration is at the time of writing this post, the current version of the Execution Cloud only supports Selenium WebDriver across all languages (Java, JavaScript, Python, C#, and Ruby), WebdriverIO, and any other WebDriver-based framework. However, more test frameworks will be supported in the near future.

Fortunately, Selenium is a highly used testing framework, giving us plenty of room to demonstrate the power of the Execution Cloud and functional testing.

Setting Up Demo App

Our demo application will be a documentation site built using the Vercel Documentation template. It’s a simple app that uses Next.js, a React framework created by Vercel, a cloud platform that lets us deploy web apps quickly and easily.

To note, all the code for our version of the application is available here.

First, we need to clone the demo app’s repository: 

git clone git@github.com:dmitryvinn/docs-demo-app.git

We will need Node.js of version 10.13 to work with this demo app, which can be installed by following the steps here.

After we set up Node.js, we should open a terminal and run the following command to install the necessary dependencies:

npm install

The next step is to navigate into the project’s directory and start the app locally:

cd docs-demo-app

npm run dev

Now our demo app is accessible at ‘http://localhost:3000/’ and ready to be tested.

Docs Demo App 

Deploying Demo App

While the Execution Cloud allows us to run the tests against a local deployment, we will simulate the production use case by running our demo app on Vercel. The steps for deploying a basic app are very well outlined here, so we won’t spend time reviewing them. 

After we deploy our demo app, it will appear as running on the Vercel Dashboard:

Demo App Deployed on Vercel

Now, we can write our tests for a production URL of our demo application available at `https://docs-demo-app.vercel.app/`.

Setting Up Test Automation

Execution Cloud offers great flexibility when it comes to working with our tests. Rather than re-writing our test suites to run against this self-healing cloud platform, we simply need to update a few lines of code in the setup part of our tests, and we can use the Execution Cloud. 

For our article, our test case will validate navigating to a specific page and pressing a counter button. 

To make our work even more effortless, Applitools offers a great set of quickstart examples that were recently updated to support the Execution Cloud. We will start with one of these samples using JavaScript with Selenium WebDriver and Jest as our baseline.

We can use any Integrated Development Environment (IDE) to write tests like IntelliJ IDEA or Visual Studio Code. Since we use JavaScript as our programming language, we will rely on NPM for the build system and our test runner.

Our tests will use Jest as its primary testing framework, so we must add a particular configuration file called `jest.config.js`. We can copy-paste a basic setup from here, but in its shortest form, the required configurations are the following.

module.exports = {

    clearMocks: true,

    coverageProvider: "v8",

  };

Our tests will require a `package.json` file which should include Jest, Selenium WebDriver, and Applitools packages. Our dependencies’ part of the `package.json` file should eventually look like the one below:

"dependencies": {

      "@applitools/eyes-selenium": "^4.66.0",

      "jest": "^29.5.0",

      "selenium-webdriver": "^4.9.2"

    },

After we install the above dependencies, we are ready to write and execute our tests.

Writing the Tests

Since we are running a purely functional Applitools test with its Eyes disabled (meaning we do not have a visual component), we will need to initialize the test and have a proper wrap-up for it.

In `beforeAll()`, we can set our test batching and naming along with configuring an Applitools API key.

To enable Execution Cloud for our tests, we need to ensure that we activate this cloud platform on the account level. After that’s done, in our tests’ setup, we will need to initialize the WebDriver using the following code:

let url = await Eyes.getExecutionCloudUrl();

driver = new Builder().usingServer(url).withCapabilities(capabilities).build();

For our test case, we will open a demo app, navigate to another page, press a counter button, and validate that the click incremented the value of clicks by one.

describe('Documentation Demo App', () => {

…

    test('should navigate to another page and increment its counter', async () => {

       // Arrange - go to the home page

       await driver.get('https://docs-demo-app.vercel.app/');

       // Act - go to another page and click a counter button

        await driver.findElement(By.xpath("//*[text() = 'Another Page']")).click();

        await driver.findElement(By.className('button-counter')).click();

      // Assert - validate that the counter was clicked

        const finalClickCount = await driver.findElement(By.className('button-counter')).getText();

        await expect(finalClickCount).toContain('Clicked 1 times');

    }

…

Another critical aspect of running our test is that it’s a non-Eyes test. Since we are not taking screenshots, we need to tell the Execution Cloud when a test begins and ends. 

To start the test, we should add the following snippet inside the `beforeEach()` that will name the test and assign it to a proper test batch:

await driver.executeScript(

            'applitools:startTest',

            {

                'testName': expect.getState().currentTestName,

                'appName': APP_NAME,

                'batch': { "id": batch.getId() }

            }

        )

Lastly, we need to tell our automation when the test is done and what were its results. We will add the following code that sets the status of our test in the `afterEach()` hook:

await driver.executeScript('applitools:endTest', 

       { 'status': testStatus })

Now, our test is ready to be run on the Execution Cloud.

Running test

To run our test, we need to set the Applitools API key. We can do it in a terminal or have it set as a global variable:

export APPLITOOLS_API_KEY=[API_KEY]

In the above command, we need to replace [API_KEY] with the API key for our account. The key can be found in the Applitools Dashboard, as shown in this FAQ article.

Now, we need to navigate to the location where our tests are located and run the following npm test command in the terminal:

npm test

It will trigger the test suite that can be seen on the Applitools Dashboard:

Applitools Dashboard with Execution Cloud enabled

Execution Cloud in Action

It’s a well-known fact that apps go through a lifecycle. They get created, get bugs, change, and ultimately shut down. This ever-changing lifecycle of any app is what causes our tests to break. Whether it’s due to a bug or an accidental regression, it’s widespread for a test to fail after a change in an app.

Let’s say a developer working on a counter button component changes its class name to `button-count` from the original `button-counter`. There could be many reasons this change could happen, but nevertheless, these modifications to the production code are extremely common. 

What’s even more common is that the developer who made the change might forget or not find all the tests using the original class name, `button-counter`, to validate this component. As a result, these outdated tests would start failing, distracting us from investigating real production issues, which could significantly impact our users.

Execution Cloud and its self-healing capabilities were built specifically to address this problem. This cloud platform would be able to “self-heal” our tests that were previously running against a class name `button-counter`, and rather than failing these tests, the Execution Cloud would find another selector that hasn’t changed. With this highly scalable solution, our test coverage would remain the same and let us focus on correcting issues that are actually causing a regression in production.

Although we are running non-Eyes tests, the Applitools Dashboard still allows us to see several valuable materials, like a video recording of our test or to export WebDriver commands! 

Want to see more? Request a free trial of Applitools Execution Cloud.

Conclusion

Whether you are a small startup that prioritizes quick iterations, or a large organization that focuses on scale, Applitools Execution Cloud is a perfect choice for any scenario. It offers a reliable way for tests to become what they should be – the first line of defense in ensuring the best customer experience for our users.

With the self-healing capabilities of the Execution Cloud, we get to focus on real production issues that actively affect our customers. With this cloud platform, we are moving towards a space where tests don’t become something we accept as constantly failing or a detriment to our developer velocity. Instead, we treat our test coverage as a trusted companion that raises problems before our users do. 

With these functionalities, Applitools and its Execution Cloud quickly become a must-have for any developer workflow that can supercharge the productivity and efficiency of every engineering team.

The post Functional Testing’s New Friend: Applitools Execution Cloud appeared first on Automated Visual Testing | Applitools.

Introducing TestZeus for More Effective Test Automation for Salesforce

TestZeus is an open-source UI automation framework that provides an efficient way to find elements on Salesforce apps. It relies on what’s called AutoLocators strategy that uses the Salesforce Developer API to find UI elements on the webpage. 

In order for us to properly demonstrate how TestZeus works, we first need to set up our Salesforce environment.

Setting up Salesforce Environment

Salesforce is an entire ecosystem created around its Customer Relationship Management (CRM) system. And as fun as testing in production could be, we should make a development environment with Salesforce.

Luckily, Salesforce has a highly engaging community of developers and many learning resources. One of the community members has covered in great detail how to setup what’s called a Salesforce Developer Edition instance. In this article, we will only focus on the core parts of the process (for more information, you can read the community forum).

After you navigate to the signup page, you will need to fill out some basic information about your account:

Signup Page for Salesforce Developer Edition

Immediately after you sign up, you will receive a confirmation email that will look something like this:

Confirmation Email Post Instance Creation

Apart from pressing “Verify Account” button, you will need to make sure to capture your instance’s URL (HOME_URL as we will call it in the test properties) and username (USERNAME in the test properties):

Instance URL and Username Properties

When you first attempt to log into the Salesforce instance, it will prompt you to create a password which we will later use as the PASSWORD property in the tests.

New Password Creation Flow

Using Salesforce Developer API

TestZeus requires us to use the Salesforce Developers API. For this article, we will need a security token for your account and a consumer key/secret combination.

Without diving into too many details, a security token is another way to authenticate the Salesforce Developer API. At the same time, a consumer key/secret combination lets Salesforce control the scope of your API calls.

Getting a security token

The security token is one of the components required for API authentication. To get it for the first time, you need to “Reset Security Token” through the Salesforce UI. This video goes through details on how to do it step-by-step. Ultimately, you will receive an email containing a security token that we will use as SECURITY_TOKEN properties in our tests.

Security Token for Salesforce Developer API

Acquiring a consumer key/secret combination

The last piece of information we need to authenticate your tests successfully is what’s called a Consumer Key and Secret. We will need to create a Connected app that will, in turn, provide the authentication details. You can think of this step as a way to control the permission scope of the API usage.
This video walks through the steps to create the Connected app and retrieve its authentication details. In short, we need first need to create a New Connected App:

Salesforce App Manager

Then fill in all the required fields, enable OAuth settings and choose the necessary scope (as shown in the video):

Creating a Connected App

Following the app’s creation, you must click “Manage Consumer Details” to retrieve the authentication information:

Connected App Details

This new window will show us the exact data we need: a Consumer Key and a Consumer Secret:

Consumer Details

In our test, we will later use this information as CONSUMER_KEY and CONSUMER_SECRET properties.

Functional Testing with TestZeus

Now with all the necessary information and demo environment set up, we can proceed with writing our tests. For this article, we put together a small project that you can see here. We will first walk through this code to show how to use a TestZeus framework for testing a Salesforce application and, later, add a visual testing component using Applitools.

Preparing Test Scenario

For our article, we will do a primary case of validating UI for a test account we create beforehand.

In your Salesforce instance, navigate to Accounts and click “New account”:

Accounts Tab

Then we will create an Account named “Test Account,” and our test scenario is ready to be used.

Creating New Account

Setting Up TestZeus

First, we need to clone our demo project:

git clone git@github.com:dmitryvinn/applitools-testzeus-demo.git

Our demo project uses Java programming language and Maven as its build system. Hence, we encourage you to use an IDE like Intellij IDEA to simplify the development experience.

After you open the project with your IDE of choice, you must use Maven to build the project and download all necessary dependencies. One of the critical dependencies we are relying on for this part of the article is TestZeus dependency:

<dependency>
            <groupId>com.testzeus</groupId>
            <artifactId>Test_Zeus</artifactId>
            <version>1.0.2</version>
        </dependency>

The next step is to fill out all the required properties we will use in our tests in the config.properties file. So far, we have covered all but the APPLITOOL_KEY property, which we will discuss later.

Writing Our First TestZeus Test

In this article, we explore a simple use case of validating an account we created beforehand.

For brevity purposes, we will skip going into the general Selenium Webdriver code and focus only on specific details around TestZeus.

First, we need to authenticate our test, so we can use the Salesforce Developer API to work with data objects, retrieve specific selectors, and more.

In BaseTest.java, we have the following method we invoke in our setup() to handle authentication:

private void authenticateUser() {
    HTTPClientWrapper.SFLogin_API(HOME_URL, TOKEN_GRANT_URL, CONSUMER_KEY, CONSUMER_SECRET, USERNAME, PASSWORD + SECURITY_TOKEN);
}

Then, we need to create a new test named AccountsTest.java where we will have our #findTestAccount() test case.

We will begin our test scenario by Arranging it, or in other words, by setting up our test case.

We will first initiate an SFPageBase with the current webdriver instance and then navigate to the home URL:

final SFPageBase salesforcePage = new SFPageBase(driver);

salesforcePage.openHomepage(HOME_URL);
salesforcePage.maximize();

Now, we will proceed to log into our demo environment:

driver.findElement(By.id("username")).sendKeys(USERNAME);
driver.findElement(By.id("password")).sendKeys(PASSWORD);

final WebElement loginButton = driver.findElement(By.id("Login"));
salesforcePage.safeClick(loginButton);

In the next step, we will transition into the Act portion of the test case, starting with test-specific actions.

We will use TestZeus’ SFBasePage to navigate to the Accounts tab and open our “Test Account”:

  salesforcePage.appLauncher("Account");

final WebElement testAccountItem = driver.findElement(By.xpath(String.format("//a[@title='%s']", TEST_ACCOUNT_NAME)));
 salesforcePage.safeClick(testAccountItem);

Lastly, in the Assert stage of the test case, we will validate one of the fields shown on the UI:

driver.findElement(By.xpath("//p[text()='Account Owner']"));

As we can see, TestZeus helps simplify waiting for UI elements, navigation, and working with Salesforce Objects.

Now with this highly efficient way of locating UI elements using TestZeus, we can take our test automation to the next level with Applitools and visual testing.

Taking Next Steps: Functional Testing for Salesforce apps with Applitools and TestZeus

Functional testing is a type of testing that focuses on the core functionalities of an application while following predetermined steps. In other words, it’s a way to evaluate a scenario with two potential outcomes, failure or success, using the same input.

While there are many functional testing levels like unit or integration testing, end-to-end tests give us the highest confidence level in customer use cases. It’s also one of the most expensive types of testing since it takes a lot of effort to write assertions for every single element of the app and then keep maintaining this workflow. But not with Applitools and Visual AI analysis.

With the Visual AI in Applitools Eyes, rather than writing assertions focused on just one object, Applitools lets us capture the entire web page or mobile app screen, then send it to the Applitools Visual AI for analysis.

Simplifying Functional Testing with Applitools

This visual testing allows us to see our app from our customers’ points of view. And that’s where the Applitools Visual AI comes in! This AI-based visual testing tool can enhance the existing end-to-end test coverage to ensure our app’s user experience conforms to the expected design.

Using AI algorithms, Applitools allows developers, testers, and product/business teams to effectively compare visual elements, with corresponding baselines, across various screens to find visible defects in your test environments. This way, you can prevent functional and visual defects from escaping to production and affecting your customers negatively.

Applitools has integrations with numerous testing platforms like Cypress, WebdriverIO, Selenium, and many others. This article will showcase Applitools with TestZeus to add visual test coverage to our Salesforce app.

Setting up Applitools

Applitools has fantastic documentation on how to get started with the project. For this article, we will first need to register with Applitools and retrieve an APPLITOOLS_API_KEY for our config.properties.

We will also add the following dependency to our Maven build:

      <dependency>
            <groupId>com.applitools</groupId>
            <artifactId>eyes-selenium-java5</artifactId>
            <version>5.39.0</version>
        </dependency>

Now, we are all set to write our test cases with Applitools.

Visual Testing with Applitools and TestZeus

By using this great starter project, we can quickly enhance our existing test suite with visual test automation using Applitools.

First, we will create a new test class named AccountsApplitoolsTest.java. Then, we will need to add some additional setup code to enable Applitools and its Ultrafast Grid, so we can scale our tests in the future.

    @BeforeAll
    public static void setUpConfigAndRunner() {
        runner = new VisualGridRunner(new RunnerOptions().testConcurrency(1));

        config = new Configuration();
        config.setApiKey(APPLITOOLS_API_KEY);

        final BatchInfo batch = new BatchInfo("Salesforce Accounts Page Tests with TestZeus");
        config.setBatch(batch);

        config.addBrowser(1024, 768, BrowserType.CHROME);
    }

In the above code snippet, we set our tests to be run one at a time (the limit for free Applitools accounts), specified the Applitools API Key, and chose Chrome as our test browser.

The next step is to “open the eyes” which is an API terminology for starting visual testing for our test suite:

    @BeforeEach
    public void openBrowserAndEyes(TestInfo testInfo) {
        eyes = new Eyes(runner);
        eyes.setConfiguration(config);

        eyes.open(
                driver,
                "TestZeus Demo",
                testInfo.getDisplayName());
    }

Now, similarly to the previous example with TestZeus, we will have a new test named findTestAccount(). One of the best things about using Applitools is that we don’t have to make any additional changes to our test scenario! So everything from Arrange to Act steps will remain the same:

    @Test
    public void findTestAccount() throws Exception {

        // Arrange
        final SFPageBase salesforcePage = new SFPageBase(driver);

        salesforcePage.openHomepage(HOME_URL);
        salesforcePage.maximize();

        driver.findElement(By.id("username")).sendKeys(USERNAME);
        driver.findElement(By.id("password")).sendKeys(PASSWORD);

        final WebElement loginButton = driver.findElement(By.id("Login"));
        salesforcePage.safeClick(loginButton);

        // Act
        salesforcePage.appLauncher("Account");

        final WebElement testAccountItem = driver.findElement(By.xpath(String.format("//a[@title='%s']", TEST_ACCOUNT_NAME)));
        salesforcePage.safeClick(testAccountItem);

The only change we will need to make is in the Assert part of the test case. Here, we will have code to validate that the page loaded correctly and that its layout matches between test runs. We could also validate a snapshot of the entire page with Applitools, but Salesforce apps are known to have varying load times. So the layout match level helps to avoid false positives:

        eyes.check(Target.window().fully().withName("Test Account").layout());

We are almost done! To make sure that our Applitools tests are closed correctly after their run, we will need to add this @AfterEach method:

    public void cleanUpTest() {
        eyes.closeAsync();
    }

With this, our first test with Applitools and TestZeus for the Salesforce app is complete. Let’s visit our Applitools Dashboard to review the results:

Visual Test Results on Applitools

The post Lightning Fast Testing for Salesforce with Applitools and TestZeus appeared first on Automated Visual Testing | Applitools.

Is Remix too new to be visually tested? Let’s find out with Applitools and Cypress.

In this blog post, we answer a single question: how to best visually test a Remix-based app? 

We walk through Remix and build a demo app to best showcase the framework. Then, we take a deep dive into visual testing with Applitools and Cypress. We close on scaling our test coverage with the Ultrafast Test Cloud to perform cross-browser validation of the app.

So let our exciting journey begin in pursuit of learning how to visually test the Remix-based app.

What is Remix?

Web development is an ever-growing space with almost as many ways to build web apps as there are stars in the sky. And it ultimately translates into just how many different User Interface (UI) frameworks and libraries there are. One such library is React, which most people in the web app space have heard about, or even used to build a website or two. 

For those unfamiliar with React, it’s a declarative, component-based library that developers can use to build web apps across different platforms. While React is a great way to develop robust and responsive UIs, many moving pieces still happen behind the scenes. Things like data loading, routing, and more complex work like Server-Side Rendering are what a new framework called Remix can handle for React apps.

Remix is a full-stack web framework that optimizes data loading and routing, making pages load faster and improving overall User Experience (UX). The days are long past when our customers would wait minutes while a website reloads, while moving from one page to another, or expecting an update on their feed. Features like Server-Side Rendering, effective routing, and data loading have become the must for getting our users the experience they want and need. The Remix framework is an excellent open-source solution for delivering these features to our audience and improving their UX.

What Does Remix Mean For UI Testing?

Our end-users shouldn’t care what framework we used to build a website. What matters to our users is that the app works and lets them achieve their goals as fast as possible. In the same way, the testing principles always remain the same, so UI testing shouldn’t be impacted by the frameworks used to create an app. The basics of how we test stay the same although some testing aspects could change. For example, in the case of an Angular app, we might need to adjust how we wait for the site to fully load by using a specialized test framework like Protractor.

Most tests follow a straightforward pattern of Arrange, Act, and Assert. Whether you are writing a unit test, an integration test, or an end-to-end test, everything follows this cycle of setting up the data, running through a set of actions and validating the end state.

When writing these end-to-end tests, we need to put ourselves in the shoes of our users. What matters most in this type of testing is replicating a set of core use-cases that our end-users go through. It could be logging into an app, writing a new post, or navigating to a new page. That’s why UI test automation frameworks like Applitools and Cypress are fantastic for testing – they are largely agnostic of the platform they are testing. With these tools in hand, we can quickly check Remix-based apps the same way we would test any other web application.

What about Remix and Visual Testing?

The main goal of testing is to confirm the app’s behavior that our users see and go through. This reason is why simply loading UI elements and validating inner text or styling is not enough. Our customers are not interested in HTML or CSS. What they care about is what they can see and interact with on our site, not the code behind it. It’s not enough for a robust coverage of the complex UI that modern web apps have. We can close this gap with visual testing. 

Visual testing allows us to see our app from our customers’ point of view. And that’s where the Applitools Eyes SDK comes in! This visual testing tool can enhance the existing end-to-end test coverage to ensure our app is pixel-perfect.

To simplify, what Applitools does for us is that it allows developers to effectively compare visual elements across various screens to find visible defects. Applitools can record our UI elements in their platform and then monitor any visual regressions that our customers might encounter. More specifically, this testing framework exposes the visible differences between baseline snapshots and future snapshots.

Applitools has integrations with numerous testing platforms like Cypress, WebdriverIO, Selenium, and many others. For this article, we will showcase Applitools with Cypress to add visual test coverage to our Remix app.

Introducing Remix Demo App

We can’t talk about a framework like Remix without seeing it in practice. That’s why we put together a demo app to best showcase Remix and later test it with Applitools and Cypress.

We based this app on the Remix Developer Blog app that highlights the core functionalities of Remix: data loading, actions, redirects, and more. We shared this demo app and all the tests we cover in this article in this repository so that our readers can follow along.

Running Demo App 

Before diving into writing tests, we must ensure that our Remix demo application is running.

To start, we need to clone a project from this repository:

git clone https://github.com/dmitryvinn/remix-demo-app-applitools

Then, we navigate into the project’s root directory and install all dependencies:

cd remix-demo-app-applitools
npm install

After we install the necessary dependencies, our app is ready to start:

npm run dev

After we launch the app, it should be available at http://localhost:3000/, unless the port is already taken. With our Remix demo app fully functional, we can transition into testing Remix with Applitools and Cypress.

Visual Testing of Remix App with Applitools and Cypress

There is this great quote from a famous American economist, Richard Thaler: “If you want people to do something, make it easy.” That’s what Applitools and Cypress did by making testing easy for developers, so people don’t see it as a chore anymore.

To run our visual test automation using Applitools, we first need to set up Cypress, which will play the role of test runner. We can think about Cypress as a car’s body, whereas Applitools is an engine that powers the vehicle and ultimately gets us to our destination: a well-tested Remix web app.

Setting up Cypress

Cypress is an open-source JavaScript end-to-end testing framework developers can use to write fast, reliable, and maintainable tests. But rather than reinventing the wheel and talking about the basics of Cypress, we invite our readers to learn more about using this automation framework on the official site, or from this course at Test Automation University.

To install Cypress, we only need to run a single command:

npm install cypress

Then, we need to initialize the cypress folder to write our tests. The easiest way to do it is by running the following:

npx cypress open

This command will open Cypress Studio, which we will cover later in the article, but for now we can safely close it. We also recommend deleting sample test suites that Cypress created for us under cypress/integration.

Note: If npx is missing on the local machine, follow these steps on how to update the Node package manager, or run ./node_modules/.bin/cypress open instead.

Setting up Applitools

Installing the Applitools Eyes SDK with Cypress is a very smooth process. In our case, because we already had Cypress installed, we only need to run the following:

npm install @applitools/eyes-cypress --save-dev

To run Applitools tests, we need to get the Applitools API key, so our test automation can use the Eyes platform, including recording the UI elements, validating any changes on the screen, and more. This page outlines how to get this APPLITOOLS_API_KEY from the platform.

After getting the API key, we have two options on how to add the key to our tests suite: using a CLI or an Applitools configuration file. Later in this post, we explore how to scale Applitools tests, and the configuration file will play a significant role in that effort. Hence, we continue by creating applitools.config.js in our root directory.

Our configuration file will begin with the most basic setup of running a single test thread (testConcurrency) for one browser (browser field). We also need to add our APPLITOOLS_API_KEY under the `apiKey’ field that will look something like this:

module.exports = {
  testConcurrency: 1,
  apiKey: "DONT_SHARE_OUR_APPLITOOLS_API_KEY",
  browser: [
    // Add browsers with different viewports
    { width: 800, height: 600, name: "chrome" },
  ],
  // set batch name to the configuration
  batchName: "Remix Demo App",
};

Now, we are ready to move onto the next stage of writing our visual tests with Applitools and Cypress.

Writing Tests with Applitools and Cypress

One of the best things about Applitools is that it nicely integrates with our existing tests with straightforward API.

For this example, we visually test a simple form on the Actions page of our Remix app.

To begin writing our tests, we need to create a new file named actions-page.spec.js in the cypress/integration folder:

Since we rely on Cypress as our test runner, we will continue using its API for writing the tests. For the basic Actions page tests where we validate that the page renders visually correctly, we start with this code snippet:

describe("Actions page form", () => {
  it("Visually confirms action form renders", () => {
    // Arrange
    // ...

    // Act
    // ..

    // Assert
    // ..

    // Cleanup
    // ..
  });
});

We continue following the same pattern of Arrange-Act-Assert, but now we also want to ensure that we close all the resources we used while performing the visual testing. To begin our test case, we need to visit the Action page:

describe("Actions page form", () => {
  it("Visually confirms action form renders", () => {
    // Arrange
    cy.visit("http://localhost:3000/demos/actions");

    // Act
    // ..

    // Assert
    // ..

    // Cleanup
    // ..
  });
});

Now, we can begin the visual validation by using the Applitools Eyes framework. We need to “open our eyes,” so-to-speak by calling cy.eyesOpen(). It initializes our test runner for Applitools to capture critical visual elements just like we would with our own eyes:

describe("Actions page form", () => {
  it("Visually confirms action form renders", () => {
    // Arrange
    cy.visit("http://localhost:3000/demos/actions");

    // Act
    cy.eyesOpen({
      appName: "Remix Demo App",
      testName: "Validate Action Form",
    });

    // Assert
    // ..

    // Cleanup
    // ..
  });
});

Note: Technically speaking, cy.eyesOpen() should be a part of the Arrange step of writing the test, but for educational purposes, we are moving it under the Act portion of the test case.

Now, to move to the validation phase, we need Applitools to take a screenshot and match it against the existing version of the same UI elements. These screenshots are saved on our Applitools account, and unless we are running the test case for the first time, the Applitools framework will match these UI elements against the version that we previously saved:

describe("Actions page form", () => {
  it("Visually confirms action form renders", () => {
    // Arrange
    cy.visit("http://localhost:3000/demos/actions");

    // Act
    cy.eyesOpen({
      appName: "Remi Demo App",
      testName: "Validate Action Form",
    });

    // Assert
    cy.eyesCheckWindow("Action Page");

    // Cleanup
    // ..
  });
});

Lastly, we need to close our test runner for Applitools by calling cy.closeEyes(). With this step, we now have a complete Applitools test case for our Actions page:

describe("Actions page form", () => {
  it("Visually confirms action form renders", () => {
    // Arrange
    cy.visit("http://localhost:3000/demos/actions");

    // Act
    cy.eyesOpen({
      appName: "Remi Demo App",
      testName: "Validate Action Form",
    });

    // Assert
    cy.eyesCheckWindow("Action Page");

    // Cleanup
    cy.eyesClose();
  });
});

Note: Although we added a cleanup-stage with cy.eyesClose() in the test case itself, we highly recommend moving this method outside of the it() function into the afterEach() that will run for every test, avoiding code duplication.

Running Applitools Tests

After the hard work of planning and then writing our test suite, we can finally start running our tests. And it couldn’t be easier than with Applitools and Cypress! 

We have two options of either executing our tests by using Cypress CLI or Cypress Studio.

Cypress Studio is a great option when we first write our tests because we can walk through every case, stop the process at any point, or replay any failures. These reasons are why we should use Cypress Studio to demonstrate best how these tests function.

We begin running our cases by invoking the following from the project’s root directory:

npm run cypress-open

This operation opens Cypress Studio, where we can select what test suite to run:

To validate the result, we need to visit our Applitools dashboard:

To make it interesting, we can cause this test to fail by changing the text on the Actions page. We could change the heading to say “Failed Actions!” instead of the original “Actions!” and re-run our test. 

This change will cause our original test case to fail because it will catch a difference in the UI (in our case, it’s because of the intentional renaming of the heading). This error message is what we will see in the Cypress Studio:

To further deal with this failure, we need to visit the Applitools dashboard:

As we can see, the latest test run is shown as Unresolved, and we might need to resolve the failure. To see what the difference in the newest test run is, we only need to click on the image in question:

A great thing about Applitools is that their visual AI algorithm is so advanced that it can test our application on different levels to detect content changes as well as layout or color updates. What’s especially important is that Applitools’ algorithm prevents false positives with built-in functionalities like ignoring content changes for apps with dynamic content. 

In our case, the test correctly shows that the heading changed, and it’s now up to us to either accept the new UI or reject it and call this failure a legitimate bug. Applitools makes it easy to choose the correct course of action as we only need to press thumbs up to accept the test result or thumbs down to decline it.

Accepting or Rejecting Test Run in Applitools Dashboard

In our case, the test case failed due to a visual bug that we introduced by “unintentionally” updating the heading. 

After finishing our work in the Applitools Dashboard, we can bring the test results back to the developers and file a bug on whoever made the UI change.

But are we done? What about testing our web app on different browsers and devices? Fortunately, Applitools has a solution to quickly scale the tests automation and add cross-browser coverage.

Scaling Visual Tests Across Browsers

Testing an application against one browser is great, but what about all others? We have checked our Remix app on Chrome, but we didn’t see how the app performs on Firefox, Microsoft Edge, and so on. We haven’t even started looking into mobile platforms and our web app on Android or iOS. Introducing this additional test coverage can get out of hand quickly, but not with Applitools and their Ultrafast Test Cloud. It’s just one configuration change away!

With this cloud solution from Applitools, we can test our app across different browsers without any additional code. We only have to update our Applitools configuration file, applitools.config.js.

Below is an example of how to add coverage for desktop browsers like Chrome, Firefox, Safari and E11, plus two extra test cases for different models of mobile phones:

module.exports = {
  testConcurrency: 1,
  apiKey: "DONT_SHARE_YOUR_APPLITOOLS_API_KEY",
  browser: [
    // Add browsers with different viewports
    { width: 800, height: 600, name: "chrome" },
    { width: 700, height: 500, name: "firefox" },
    { width: 1600, height: 1200, name: "ie11" },
    { width: 800, height: 600, name: "safari" },
    // Add mobile emulation devices in Portrait or Landscape mode
    { deviceName: "iPhone X", screenOrientation: "landscape" },
    { deviceName: "Pixel 2", screenOrientation: "portrait" },
  ],
  // set batch name to the configuration
  batchName: "Remix Demo App",
};

It’s important to note that when specifying the configuration for different browsers, we need to define their width and height, with an additional property for screenOrientation to cover non-desktop devices. These settings are critical for testing responsive apps because many modern websites visually differ depending on the devices our customers use.

After updating the configuration file, we need to re-run our test suite with npm test. Fortunately, with the Applitools Ultrafast Test Cloud, it only takes a few seconds to finish running our tests on all browsers, so we can visit our Applitools Dashboard to view the results right away:

As we can see, with only a few lines in the configuration file, we scaled our visual tests across multiple devices and browsers. We save ourselves time and money whenever we can get extra test coverage without explicitly writing new cases. Maintaining test automation that we write is one of the most resource-consuming steps of the Software Development Life Cycle. With solutions like Applitools Ultrafast Test Cloud, we can write fewer tests while increasing our test coverage for the entire app.

Verdict: Can Remix Apps be Visually Tested with Applitools and Cypress?

Hopefully, this article showed that the answer is yes; we can successfully visually test Remix-based apps with Applitools and Cypress! 

Remix is a fantastic framework to take User Experience to the next level, and we invite you to learn more about it during the webinar by Kent C. Dodds “Building Excellent User Experiences with Remix”.

For more information about Applitools, visit their website, blog and YouTube channel. They also provide free courses through Test Automation University that can help take anyone’s testing skills to the next level.

The post How to Visually Test a Remix App with Applitools and Cypress appeared first on Automated Visual Testing | Applitools.