Curious about the latest updates in Storybook.js, including the upcoming Storybook 7? In this post, which will be continuously updated, we sum up the latest Storybook news.

The highly anticipated Storybook 7.0 is currently in alpha, and there is a lot to get excited about. Let’s take a look at everything we know so far.

What’s Coming in Storybook 7

Storybook 7 promises significant changes. In fact, the Storybook team describes it as “a full rework of Storybook’s core with fast build and next-generation interaction testing.” Interaction testing was first included in the most recent Storybook 6.5 release, but we can expect further development there as Storybook 7 develops.

A Storybook 7 Design Preview

Storybook’s developers have just revealed a “sneak peek” at the design and layout in Storybook 7 [update 8/18: these changes are now available in the latest alpha]. Here’s a few of the changes that were highlighted:

• 3.5% more screen space for the Canvas (where components are developed in isolation).
• A “Reload” tool to reload a selected story (component) without refreshing the whole browser.
• Better access for integrators to the same design patterns used to develop Storybook.
• 196 icons that can be used in all projects, each redrawn from scratch. This is 20 more icons than the previous set.
• Form components like Toggle and Slider conform to the new design language.
• Pre-bundling of Storybook to deliver faster start times and avoid dependency issues (dependencies were also audited to reduce bundle size).

Storybook 7 Release Date

We don’t yet know the exact release date for Storybook 7, but we can guess based on their recent development history.

Storybook Version 6 was originally released in August 2020. Since then, there have been 5 major updates, most recently ending in version 6.5 which was released in May 2022. On the journey to the Storybook 6 release, the development team hit the following milestones:

• 47 alphas between January 2020 and April 2020
• 47 betas between April 2020 and July 2020
• 30 release candidates between July 2020 and August 2020

Storybook 7 began its first alpha in June 2022, and as of this writing is on alpha-26. If it follows the same trajectory is Storybook 6, we can estimate that it will enter beta in September, RC in December and official release in January 2023. Of course, time will tell, and we’ll update this post when any concrete information becomes available.

How Visual Testing Helps Component Testing

Component testing is form of software testing that focuses on software components in isolation. Component testing takes each rendered state (or Storybook story) and tests it.

Visual testing of components allows teams to find bugs earlier – and without writing any additional test code. It works across a variety of browsers and viewports at speeds almost as fast as unit testing.

You can learn more about how you can save time by using Applitools and our AI-powered visual testing with Storybook here:

• Visual Testing with Storybook
• Testing Storybook Components in Any Browser – Without Writing Any New Tests!
• How to Add Applitools Eyes to Storybook

Stay Tuned

We’ll be sure to keep this page updated with the latest on what’s new in Storybook 7, so check back often. And of course, we’re working hard to ensure our own Applitools SDKs for Storybook React, Storybook Angular and Storybook Vue are always compatible with the latest Storybook features.

Last Updated: August 26th, 2022

The post What’s New in Storybook 7? appeared first on Automated Visual Testing | Applitools.

Summer is a time for new things and a time for play. We’re excited to announce that the Applitools Storybook SDK now supports Play Functions, giving modern frontend teams even more power when it comes to testing their component systems before production. Play Functions enable rich functionality in your Storybook designs, enabling you to interact with components and test scenarios that otherwise required user intervention. This means interacting and testing interactions such as form fills or date pickers in your component system is now possible! This capability was made available in Storybook version 6.4 and now is available in Applitools Storybook SDK version 3.28.

Applitools Storybook SDK can now consume these interactions through Play Functions and apply Visual AI to help your team spot any visual regression or defect in a component. For stories that have the play function, Applitools will automatically take a screenshot after the play function is finished.

To learn more about this specific feature, you can read our Storybook readme on NPM or the official Storybook Play Article.

Learn how to automatically do ultrafast cross-browser testing for Storybook components without needing to write any new test automation code in Testing Storybook Components in Any Browser by Andrew Knight.

Happy testing!

The post Storybook Play Functions Now Supported in Applitools appeared first on Automated Visual Testing | Applitools.

We’re approaching the end of May, which means we’re just a handful of weeks the midpoint of 2022 already. If you’re like me, you’re wondering where the year has gone. Maybe it has to do with life in the northeastern US where I live, where we’ve really just had our first week of warm weather. Didn’t winter just end?

As always, the year is flying by, and it can be hard to keep up with all the great videos or events you might have wanted to watch or attend. To help you out, we’ve rounded up some of our most popular test automation videos of 2022 so far. These are all top-notch workshops or webinars with test automation experts sharing their knowledge and their stories – you’ll definitely want to check them out.

Cross Browser Test Automation

Cross-browser testing is a well-known challenge to test automation practitioners. Luckily, Andy Knight, AKA the Automation Panda, is here to walk you through a modern approach to getting it done. Whether you use Cypress, Playwright, or are testing Storybook components, we have something for you.

Modern Cross Browser Testing with Cypress

For more, see this blog post: How to Run Cross Browser Tests with Cypress on All Browsers (plus bonus post specifically covering the live Q&A from this workshop).

Modern Cross Browser Testing in JavaScript Using Playwright

For more, see this blog post: Running Lightning-Fast Cross-Browser Playwright Tests Against any Browser.

Modern Cross Browser Testing for Storybook Components

For more, see this blog post: Testing Storybook Components in Any Browser – Without Writing Any New Tests!

Test Automation with GitHub or Chrome DevTools

GitHub and Chrome DevTools are both incredibly popular with the developer and testing communities – odds are if you’re reading this you use one or both on a regular basis. We recently spoke with developer advocates Rizel Scarlett of GitHub and Jecelyn Yeen of Google as they explained how you can leverage these extremely popular tools to become a better tester and improve your own testing experience. Click through for more info about each video and get watching.

Make Testing Easy with GitHub

For more, see this blog post: Using GitHub Copilot to Automate Tests.

Automating Tests with Chrome DevTools Recorder

For more, see this blog post: Creating Your First Test With Google Chrome DevTools Recorder.

Test Automation Stories from Our Customers

When it comes to implementing and innovating around test automation, you’re never alone, even though it doesn’t always feel that way. Countless others are struggling with the same challenges that you are and coming up with solutions. Sometimes all it takes is hearing how someone else solved a similar problem to spark an idea or gain a better understanding of how to solve your own.

Accelerating Visual Testing

Nina Westenbrink, Software Engineer at a leading European telecom, talks about how the visual time to test the company’s design system was decreased and simplified, offering helpful tips and tricks along the way. Nina also speaks about her career as a woman in testing and how to empower women and overcome biases in software engineering.

Continuously Testing UX for Enterprise Platforms

Govind Ramachandran, Head of Testing and Quality Assurance for Asia Technology Services at Manulife Asia, discusses challenges around UI/UX testing for enterprise-wide digital programs. Check out his blueprint for continuous testing of the customer experience using Figma and Applitools.

This is just a taste of our favorite videos that we’ve shared with the community from 2022. What were yours? You can check out our full video library here, and let us know your own favorites @Applitools.

The post Our Best Test Automation Videos of 2022 (So Far) appeared first on Automated Visual Testing | Applitools.

Learn how to automatically do ultrafast cross-browser testing for Storybook components without needing to write any new test automation code.

Let’s face it: modern web apps are complex. If a team wants to provide a seamless user experience on a deadline, they need to squeeze the most out of the development resources they have. Component libraries help tremendously. Developers can build individual components for small things like buttons and big things like headers to be used anywhere in the frontend with a consistent look and feel.

Storybook is one of the most popular tools for building web components. It works with all the popular frameworks, like React, Angular, and Vue. With Storybook, you can view tweaks to components as you develop their “stories.” It’s awesome! However, manually inspecting components only works small-scale when you, as the developer, are actively working on any given component. How can a team test their Storybook components at scale? And how does that fit into a broader web app testing strategy?

What if I told you that you could automatically do cross-browser testing for Storybook components without needing to define any new tests or write any new automation code? And what if I told you that it could fit seamlessly into your existing development workflow? You can do this with the power of Applitools and your favorite CI tool! Let’s see how.

Adding Visual Component Testing to Your Strategy

Historically, web app testing strategies divide functional testing into three main levels:

1. Unit testing
2. Integration testing for APIs
3. End-to-end testing for UIs and APIs

These three levels make up the classic Testing Pyramid. Each level of testing mitigates a unique type of risk. Unit tests pinpoint problems in code, integration tests catch problems where entities meet, and end-to-end tests exercise behaviors like a user.

The rise of frontend component libraries raises an interesting question: Where do components belong among these levels? Components are essentially units of the UI. In that sense, they should be tested individually as “UI units” to catch problems before they become widespread across multiple app views. One buggy component could unexpectedly break several pages. However, to test them properly, they should be rendered in a browser as if they were “live.” They might even call APIs indirectly. Thus, arguably, component testing should be sandwiched between traditional integration and end-to-end testing.

Wait, another level of testing? Nobody has time for that! It’s hard enough to test adequate coverage at the three other levels, let alone automate those tests. Believe me, I understand the frustration. Unfortunately, component libraries bring new risks that ought to be mitigated.

Thankfully, Applitools provides a way to visually test all the components in a Storybook library with the Applitools Eyes SDK for Storybook. All you need to do is install the @applitools/eyes-storybook package into your web app project, configure a few settings, and run a short command to launch the tests. Applitools Eyes will turn each story for each component into a visual test case. On the first run, it will capture a visual snapshot for each story as a “baseline” image. Then, subsequent runs will capture “checkpoint” snapshots and use Visual AI to detect any changes. You don’t need to write any new test code – tests become a side effect of creating new components and stories!

In this sense, visual component testing with Applitools is like autonomous testing. Test generation and execution is completely automated, and humans review the results. Since testing can be done autonomously, component testing is easy to add to an existing testing strategy. It mitigates lots of risk for low effort. Since it covers components very well, it can also reduce the number of tests at other layers. Remember, the goal of a testing strategy is not to cover all the things but rather to use available resources to mitigate as much risk as possible. Covering a whole component library with an autonomous test run frees up folks to focus on other areas.

Adding Applitools Eyes to Your Web App

Let’s walk through how to set up visual component tests for a Storybook library. You can follow the steps below to add visual component tests to any web app that has a Storybook library. Give it a try on one of your own apps, or use my example React app that I’ll use as an example below. You’ll also need Node.js installed as a prerequisite.

To get started, you’ll need an Applitools account to run visual tests. If you don’t already have an Applitools account, you can register for free using your email or GitHub account. That will let you run visual tests with basic features.

Once you get your account, store your API key as an environment variable. On macOS or Linux, use this command:

export APPLITOOLS_API_KEY=<your-api-key>

On Windows:

set APPLITOOLS_API_KEY=<your-api-key>

Next, you need to add the eyes-storybook package to your project. To install this package into a new project, run:

npm install --save-dev @applitools/eyes-storybook

Finally, you’ll need to add a little configuration for the visual tests. Add a file named applitools.config.js to the project’s root directory, and add the following contents:

module.exports = {
    concurrency: 1,
    batchName: "Visually Testing Storybook Components"
}

The concurrency setting defines how many visual snapshot comparisons the Applitools Ultrafast Test Cloud will perform in parallel. (With a free account, you are limited to 1.) The batchName setting defines a name for the batch of tests that will appear in the Applitools dashboard. You can learn about these settings and more under Advanced Configuration in the docs.

That’s it! Now, we’re ready to run some tests. Launch them with this command:

npx eyes-storybook

Note: If your components use static assets like image files, then you will need to append the -s option with the path to the directory for static files. In my example React app, this would be -s public.

The command line will print progress as it tests each story. Once testing is done, you can see all the results in the Applitools dashboard:

Run the tests a second time for checkpoint comparisons:

If you change any of your components, then tests should identify the changes and report them as “Unresolved.” You can then visually compare differences side-by-side in the Applitools dashboard. Applitools Eyes will highlight the differences for you. Below is the result when I changed a button’s color in my React app:

You can give the changes a thumbs-up if they are “right” or a thumbs-down if they are due to a regression. Applitools makes it easy to pinpoint changes. It also provides auto-maintenance features to minimize the number of times you need to accept or reject changes.

Adding Cross-Browser Tests for All Components

When Applitools performs visual testing, it captures snapshots from tests running on your local machine, but it does everything else in the Ultrafast Test Cloud. It rerenders those snapshots – which contain everything on the page – against different browser configurations and uses Visual AI to detect any changes relative to baselines.

If no browsers are specified for Storybook components, Applitools will run visual component tests against Google Chrome running on Linux. However, you can explicitly tell Applitools to run your tests against any browser or mobile device.

You might not think you need to do cross-browser testing for components at first. They’re just small “UI units,” right? Well, however big or small, different browsers render components differently. For example, a button may have rectangular edges instead of round ones. Bigger components are more susceptible to cross-browser inconsistencies. Think about a navbar with responsive rendering based on viewport size. Cross-browser testing is just as applicable for components as it is for full pages.

Configuring cross-browser testing for Storybook components is easy. All you need to do is add a list of browser configs to your applitools.config.js file like this:

module.exports = {
  concurrency: 1,
  batchName: "Visually Testing Storybook Components",
  browser: [
    // Desktop
    {width: 800, height: 600, name: 'chrome'},
    {width: 700, height: 500, name: 'firefox'},
    {width: 1600, height: 1200, name: 'ie11'},
    {width: 1024, height: 768, name: 'edgechromium'},
    {width: 800, height: 600, name: 'safari'},
    // Mobile
    {deviceName: 'iPhone X', screenOrientation: 'portrait'},
    {deviceName: 'Pixel 2', screenOrientation: 'portrait'},
    {deviceName: 'Galaxy S5', screenOrientation: 'portrait'},
    {deviceName: 'Nexus 10', screenOrientation: 'portrait'},
    {deviceName: 'iPad Pro', screenOrientation: 'landscape'},
  ]
}

This declaration includes ten unique browser configurations: five desktop browsers with different viewport sizes, and five mobile devices with both portrait and landscape orientations. Every story will run against every specified browser. If you run the test suite again, there will be ten times as many results!

As shown above, my batch included 90 unique test instances. Even though that’s a high number of tests, Applitools Ultrafast Test Cloud ran them in only 32 seconds! That really is ultrafast for UI tests.

Running Visual Component Tests Autonomously

Applitools Eyes makes it easy to run visual component tests, but to become truly autonomous, these tests should be triggered automatically as part of regular development workflows. Any time someone makes a change to these components, tests should run, and the team should receive feedback.

We can configure Continuous Integration (CI) tools like Jenkins, CircleCI, and others for this purpose. Personally, I like to use GitHub Actions because they work right within your GitHub repository. Here’s a GitHub Action I created to run visual component tests against my example app every time a change is pushed or a pull request is opened for the main branch:

name: Run Visual Component Tests

on:
  push:
  pull_request:
    branches:
      - main

jobs:
  test:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout
        uses: actions/checkout@v2
      
      - name: Set up Node.js
        uses: actions/setup-node@v2

      - name: Install dependencies
        run: npm install

      - name: Run visual component tests
        run: npx eyes-storybook -s public
        env:
          APPLITOOLS_API_KEY: ${{ secrets.APPLITOOLS_API_KEY }}

The only extra configuration needed was to add my Applitools API key as a repository secret. 

Maximizing Your Testing Value

Components are just one layer of complex modern web apps. A robust testing strategy should include adequate testing at all levels. Thankfully, visual testing with Applitools can take care of the component layer with minimal effort. Unit tests can cover how the code works, such as a component’s play method. Integration tests can cover API requests, and end-to-end tests can cover user-centric behaviors. Tests at all these levels together provide great protection for your app. Don’t neglect any one of them!

The post Testing Storybook Components in Any Browser – Without Writing Any New Tests! appeared first on Automated Visual Testing | Applitools.

Component libraries and design systems are important development and design tools that allow teams to focus on building consistent and high quality experiences. Storybook is a tool that helps us with those experiences, but as your library and system grows, it becomes more difficult to maintain that level of high quality.

With the Applitools Eyes Storybook SDK, we’re able to provide visual testing coverage for all stories in your Storybook. The cool thing is this works for whatever UI framework you choose! Whether you’re using Storybook with React, Vue, Angular, Svelte, or even React Native, Applitools is able to support any framework Storybook supports.

Let’s dig in and find out what exactly Storybook is, where it helps, and how Applitools can easily provide coverage for any number of stories that a Storybook library supports.

What is Storybook?

Storybook is a JavaScript-based tool that helps teams build component libraries, design systems, and beyond.

It does this by providing a library-like interface, that shows different components and pages in isolation allowing developers, designers, testers, and users of the library to consume and work in each “story” in a focused environment.

Each Storybook story is composed of a component or set of components that represent one piece of what someone would interact with or visualize in an application.

Testing and validating UI at the component level

While most stories don’t show an entire page built with these components, each of them are fit together in various ways to build the actual interfaces that are used by visitors of that app.

This gives the development team or users of the library an easy and flexible way to make sure each component works as expected from the start, where they can then have confidence to use it however they’d like in the application.

But the tricky part is making sure we’re providing proper coverage of each of the components in a way that will help us build that confidence along with finding a scalable solution as the library grows.

Solutions exist to provide code-based testing, where we might validate the string-based output of a component or test the components virtually mounted, but they’re not actually capturing what the users of the components are actually seeing.

Visual testing helps us solve that. We can capture what the user is seeing right in the browser, and use that to make sure our components and stories are working exactly like they should, giving us that needed confidence.

AI-Powered Visual Testing and Storybook

With the Applitools Eyes Storybook SDK, we’re able to easily provide coverage for any number of stories that our Storybook library supports.

After installing @applitools/eyes-storybook, all you need to do is simply run the following command:

npx eyes-storybook

Once run, Applitools will find all stories and start to capture each story much like other SDKs available for Eyes.

Because Storybook is ultimately web-based, where it renders any UI framework it supports right in a browser, the Eyes SDK is able to capture a DOM snapshot of that rendered output.

That means, the Eyes Storybook SDK can support any framework that Storybook can support!

The SDK uses Puppeteer, similar to the Eyes Puppeteer SDK, where each story is loaded in a new Puppeteer tab.

It then collects a DOM snapshot which is used to render each page cross-browser in the Applitools cloud, where you’ll then get the same AI-powered Visual Testing coverage that you would with any other Applitools integration.

This helps so that we can visually identify the issues with our application, based on what people are actually seeing. Saving us time when trying to solve where the issue occurred or if it’s a false positive in the first place.

Getting started with Applitools Eyes and Storybook

If you want to get started with integrating Applitools Eyes with your Storybook library, we have a variety of resources including a walkthrough for adding Applitools Eyes to Storybook along with dedicated tutorials for React, React using CSF, Vue, and Angular.

You can also learn more about the importance of design systems with my free webinar Bringing Quality Design Systems to Life with Storybook & Applitools.

The post How to Maintain High Quality Design Systems with Storybook by Leveraging Visual AI appeared first on Automated Visual Testing | Applitools.

Creating component and design systems are an important way to make sure you’re delivering consistent experiences to the people visiting your website or application. Storybook makes this easy with its JavaScript API, but how can we make sure we’re tracking the quality of those stories with automated testing?

Just looking for an example? Check out the available Applitools tutorials for React, Vue, and Angular on GitHub.com.

What is Storybook?

Storybook is a JavaScript-powered web tool that gives teams the ability to build component and design systems for their websites and applications.

By adding Storybook to your project, you can develop individual components of your UI in a focused, self-documenting environment, that helps everyone from design to development to product release carefully craft an experience.

It supports a variety of the development community’s most popular frameworks including React, Vue, and Angular. It also has a few documented ways to add testing to those components, but that typically involves meticulously writing tests that could be fragile and not necessarily representing what people are using in their applications.

That’s where Applitools and automated visual testing come in!

How does Storybook work with Applitools?

Pretty much out of the box, Applitools will give you full automated visual testing coverage of all of the components of your Storybook library.

By installing the Applitools Eyes Storybook SDK and running a simple command, Applitools will automatically pull in all of those stories, take a snapshot, and check for regressions in the Applitools cloud.

What are we going to build?

We’re going to start off with an existing React project that already has Storybook installed.

For more information about installing Storybook, visit the Storybook docs at: https://storybook.js.org/docs/react/get-started/install

In this walkthrough, we’ll be using a Storybook React application, but both Storybook and Applitools support more frameworks like Vue and Angular!

But with our project ready to go, we’ll first install the Applitools Eyes Storybook SDK using npm.

Once installed, we’ll show an example of how you can use Applitools Eyes to add Visual Testing coverage to your Storybook.

Step 1: Installing the Applitools Eyes Storybook SDK

To get started with the Applitools Eyes Storybook SDK on a project, first install the SDK from npm:

npm install @applitools/eyes-storybook --save-dev

This will pull down the SDK from npm and add it as a package to your project.

Step 2: Running Applitools Eyes for automated Storybook visual testing

Now that our SDK is installed, we actually don’t need to do anything else to get set up, we can dive right into running our tests!

To run our tests, make sure you first head over to the applitools.com dashboard and find your API key, then run the following in your terminal:

APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook

After that command finishes, you can see that by default, Applitools will go through each story, capture a snapshot, and check for visual differences.

Because this was our first time running our tests, all of our snapshots are new.

If we head over into our Applitools dashboard, we can even see these tests!

We can see that inside Applitools, all of our tests are new and green, which is a good thing. These will serve as the baseline for all of our tests.

Now to see the magic of how Applitools works, we can try breaking one of our components.

Inside Header.js,on line 30, let’s change our user condition to:

{!user ? (

We’re essentially switching around the logic, where if this was happening on production, we would want this bug caught!

Let’s run our tests again:

APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook

We can see this time, we have differences in our tests.

If we go into the Applitools dashboard, not only can we see that we now have Unresolved tests, we can open up one of those tests, and see the difference.

Applitools Eyes was able to see exactly where that issue was and highlight the change, so we can then see that there is a bug in our application!

Inside of the dashboard, we can click the “thumbs down” button, which tells Applitools we reject that change, meaning it failed.

But once we revert our application change and run our tests again, we can see that our tests are now passing again, because they’re working exactly how they should!

What else can you do with Applitools and Storybook?

Ignoring regions

One of the powerful features of Applitools is the ability to ignore specific regions of your visual tests by passing in selectors.

By using the ignoreRegions property inside of your Story’s eyes configuration, you can specify any selector you’d like to ignore in that Story.

Learn more about ignoring regions on npmjs.com

Specifying region types

In addition to simply ignoring regions, you can also set a custom match level for specific regions, such as if you wanted to define a region to include dynamic content, so that Applitools can use AI to understand if the content changed or if the layout changed.

Learn more about configuring regions for content, layout, and more on npmjs.com.

Get Started Today

Want to try out Applitools Eyes with Storybook for yourself? You can get started today at the link below – the account is free forever.

The post How to Add Visual Testingto Storybook appeared first on Automated Visual Testing | Applitools.

Recently I’ve put together a bunch of tutorials: an overview of Applitools, how to test Storybook with Angular, how to quickly troubleshoot bugs in React apps, and how to test Vue apps with Cypress.

If you’ve gone through these, you’ll notice that the visual tests complete in seconds — faster than Nic Cage can jack your ride.

How so fast?

The answer is simple: Applitools Ultrafast Grid. Each time you run a visual test in Applitools, Ultrafast Grid renders that page across a broad range of browsers and configurations. In parallel. In seconds.

Applitools Visual AI then grabs a screenshot of each page, compares it to its baseline screenshot, and determines visual differences and the root cause unpinning them. In less than a minute, your cross-browser testing is done and your developers have what they need to fix any visual bugs.

It’s our very own ludicrous mode (a.k.a. how a Tesla can go from zero to 60 mph in a little over two seconds).

But more than a cool technical trick, Ultrafast Grid can help you keep your app development projects on schedule.

In this article, I will demonstrate the Ultrafast Grid system with test runs and a variety of configuration settings, such as multiple browsers, viewports, and devices, to show Applitools’ efficiency in delivering results.

First off, make sure you have the following installed and configured on your machine:

• Angular
• Storybook
• Applitools

Before we delve into writing code, let’s take a behind-the-scenes look at how Applitools Ultrafast Grid works.

Life before Ultrafast Grid

Before the arrival of the Ultrafast Grid, visual tests were run in this specific order:

• A test run checked the pages one by one.
• A screenshot was taken for each page.
• The screenshot was uploaded to the backend server, where it was analyzed (that is, compared to previous baselines).
• Based on the results, the test would either fail or keep running, checking other pages.

So far, so good!

But problems arise when hundreds of screenshots are uploaded to the backend server to be analyzed. Varying parameters ranging from different viewports, different browsers, and different device emulators can also slow things down.

To help solve this, Applitools designed the Ultrafast Grid system to run tests quickly.

Life with Ultrafast Grid

Applitools Ultrafast Grid is split into two components: server-side and client-side. Here’s a diagram of how it works:

The Ultrafast Grid client is a Node JS library, used internally by both Applitools Storybook and Cypress SDKs. Here’s more on the Ultrafast Grid client.

That code snippet instantiates a new Ultrafast Grid client and returns an object. In this case, we are only interested in the openEyes() method returned.

When called, the openEyes() method starts a new test run and sends the Ultrafast Grid backend server details about it– including, but not limited to the application name, batch or test name, the browsers used, and different variations of browser settings to be adapted while running the tests. This method returns a Promise to an object with the following functions: checkWindow() and close().

This method call instructs the Ultrafast Grid backend server to prepare a testing environment for a new test run that requires running the test over two different browsers. The first browser emulates an iPhone X, while the second specifies the desktop viewport to run the test.

To start collecting snapshots, the test issues a call to checkWindow() function. The Ultrafast Grid client handles this call and collects all the needed resources to store the snapshot locally.

At the end of the test, you issue a call to close() method to close and finish the test run.

At this moment, all the DOM Snapshots collected locally during the test run are sent to the Ultrafast Grid backend server. Based on the test run requirements, sent previously with the openEyes() method, the Ultrafast Grid:

• Launches a number of browsers with different viewports and device emulators.
• Runs all of the DOM snapshots in parallel.
• Collects image screenshots.
• Sends all the screenshots to the Applitools AI server to do all the testing analysis and returns the results.

Finally, it’s worth mentioning that Ultrafast Grid sits behind a family of Applitools SDKs including the following:

• Applitools for Selenium IDE

• Java 3 Selenium SDK
• JS4 Selenium SDK
• C# Selenium SDK
• Python Selenium SDK
• Ruby Selenium SDK
• WebdriverIO SDK
• Cypress
• Storybook

Now that you understand how the Ultrafast Grid system works so efficiently, let’s throw a spanner in the works by running tests with a variety of browser configuration settings. Then, we’ll review the results on the Applitools Test Manager.

Source code

For this article, I’ve chosen to write a few Storybook stories and use Applitools for the custom Angular ContentEditable component. I’ve described and used this in my previous article on Mixing Storybook with Angular with a sprinkle of Applitools.

There have been some changes to the Applitools Eyes SDK for Storybook npm package since my previous article was published. I’d recommend removing the old package and installing the new one by following these steps:

Step 1: Remove the old @applitools/eyes.storybook NPM package by issuing the following command:

npm remove @applitools/eyes.storybook

copy code to clipboard

Step 2: Install the new @applitools/eyes-storybook NPM package by issuing the following command:

npm i @applitools/eyes-storybook

copy code to clipboard

That’s it!

Applitools Ultrafast Grid Demo

Before writing our tests, let’s run the application and make sure it’s working fine.

Step 1: Clone the application repo.

git clone https://github.com/thisdot/storybook-angular-applitools.git

copy code to clipboard

Step 2: Install all app dependencies by issuing the following command:

Go to the folder /storybook-angular-applitools then run this command:

npm install

copy code to clipboard

Step 3: Run the app by issuing the following command:

ng serve

copy code to clipboard

Now that we are sure the application is up and running switch back to the Terminal and hit Ctrl+c twice. This terminates the running application.

Step 4: Run the Storybook stories by issuing the following command:

npm run storybook

copy code to clipboard

Step 5: Run the Applitools Eyes SDK for Storybook by issuing the following command:

npx eyes-storybook

copy code to clipboard

Make sure you get an Applitool API Key and store it on your machine.

To set the APPLITOOLS_API_KEY environment variable locally on your machine you can use the export command on Mac OS or set command on Windows as follows.

For a complete tutorial on how to install and run Applitools Storybook SDK for Angular, check out this article: Storybook Angular Tutorial.

Back to the command above, this command simply opens the Storybook stories, runs them one by one, and then sends all the DOM snapshots to Applitools Ultrafast Grid, which then renders them.

If you expand the first test run, you get the following results:

Now that the app, Storybook stories, and Applitools are running locally, let’s start playing with our visual tests.

Visual UI tests with custom viewports

Keeping in mind that Applitools with Storybook doesn’t need any kind of configuration, the important test configurations are auto-inferred by Applitools. However, if you want to play around with the test configuration, you have two options to specify:

• Environment variables
• The applitools.config.js file

For this demonstration, I will make use of the applitools.config.js file. For a full list of test configuration, check the Advanced Configuration section on the @applitools/eyes-storybook repo on Github.

When you’re using Applitools with Storybook, there’s no need to add any specific Applitools commands or set test configuration to enable visual testing.

Applitools Eyes SDK runs the Storybook stories, takes a snapshot of each, and sends the DOM snapshots to the Ultrafast Grid backend server. It saves the test run with the default test configuration.

Step 1: Add a new JavaScript file to the root of your application. Give it a name of applitools.config.js.

Step 2: Add the appName and batchName configuration settings as follows:

Step 3: Run the test case on two different browsers with two different viewports by adding the following:

Consequently, Ultrafast Grid launches a set of instances for each of the browsers and runs the tests simultaneously. The first time these tests are run, we create baseline images to compare against during subsequent test runs.

Step 4: Save and run the tests again by issuing the following command:

npx eyes-storybook

copy code to clipboard

Step 5: Check the test run in Applitools Test Manager.

I’ve switched the Test Manager view to the Batch Summary View.

Notice how Applitools Ultrafast Grid ran your tests twice, as per the test configuration file above. The test configuration above specified two different browsers with two different viewports. This means that the Ultrafast Grid runs every test twice– once per browser configuration set.

Running this test yields an execution time of just four seconds! To be clear, this is what we spent rendering baseline images, once the DOM snapshots were uploaded. Even so, that’s really fast.

It’s the duty of the Ultrafast Grid to analyze the test configuration file attached to the test and spawn, in parallel, a number of browser instances to handle the test run efficiently.

Visual UI tests with device emulation

Let’s run another test. This time, instead of specifying the browser viewports, we will specify a device emulation for the browser. Device emulation uses a Chrome browser, with the viewport dimensions, and user agent string, specific to that device.

Step 1: Replace the content of the applitools.config.js file with the following configuration:

In this test configuration, we are specifying to the Ultrafast Grid to run this test over a Chrome browser with a device emulation of an iPhone X using a landscape screen orientation.

Ultrafast Grid supports only the Chrome browser for running tests over a device emulation, so you can skip the name of the browser. I’m explicitly specifying the browser name to make it very clear that this is a browser emulation and not a real device.

For a complete list of device emulators supported by the Chrome browser, take a look at this page: Chromium Device Emulators.

Step 2: Save and run the tests by issuing the following command:

npx eyes-storybook

copy code to clipboard

Step 3: Check the test run in Applitools Test Manager.

I’ve switched the Test Manager view to the Batch Summary View.

As per the new test configuration file, the Ultrafast Grid runs your tests on a Chrome browser with a device emulation for iPhone X. Hence, the browser name shown is Safari 11.0 and landscape with a viewport of 812×375.

Notice that running this test yields an execution time of just two seconds! Again, this is once the DOM snapshot has been uploaded. But still…

Finally, in Applitools, baseline snapshots are stored as per test configuration settings. Different test configuration settings generate different baseline snapshots. You can’t compare apples to oranges, right?

That’s why, if you click on the snapshot above (after switching back to Batch steps view) and show both the baseline and the current test run, you won’t get any baseline snapshot–because it’s the first time we’re running the test with these configuration settings.

100 Visual UI tests

Let’s go crazy and run 100 Visual/UI tests with a combination of test configurations.

Step 1: Replace the content of the applitools.config.js file with the following configuration:

In this test configuration, we are running 100 tests with 50 unique view ports on two major browsers: Firefox and Google Chrome.

Step 2: Save and run the tests by issuing the following command:

npx eyes-storybook

copy code to clipboard

During this time, the Eyes Storybook SDK will read all Storybook stories in the application, run them one by one using the Storybook engine, generate a DOM snapshot for each test, and finally upload the results to the Eyes server. At this moment, the Eyes server will run each and every DOM snapshot, render an image snapshot, and store it as a baseline snapshot.

The Eyes SDK/Server took only 21 seconds to complete all the above tasks, create baselines, and report the results on Applitools Test Manager Dashboard, for the entire batch of 100 test runs for a single Storybook story.

Step 3: Check the test run in Applitools Test Manager.

I’ve switched the Test Manager view to the Batch Summary View.

Let’s see the power of the Eyes server when running a regression testing cycle.

Step 4: Let’s introduce a visual change on the Storybook story and run the Eyes SDK once again.

Replace the content of the /src/stories/index.stories.js file with the following:

view rawregression-testing-cycle hosted with by GitHub

Now the ContentEditableComponent should render with a Blue background.

Step 5: Run the tests again by issuing the following command:

npx eyes-storybook

copy code to clipboard

Once again, the Eyes SDK/Server took only 21 seconds to upload the DOM snapshot, run the entire batch of visual UI tests, and display the results on the Applitools Test Manager Dashboard.

Notice the error message generated saying that “A total of 100 differences were found”.

Because the change we introduced affected the entire batch of test runs, therefore, all the current tests failed in comparison with the baseline snapshots, due to the visual change introduced earlier.

Step 6: Check the test run in Applitools Test Manager.

The Test Manager highlights the differences between the baseline snapshot and the current test run and displays a mismatch icon near the snapshots.

At this moment, you can either accept the new changes and make them the new baseline or reject them and keep the old baseline.

Once again, it takes the power of the Ultrafast Grid to analyze the test configuration file and set up the testing environment as needed.

See for yourself

Here’s a step-by-step tutorial video of how I did all the steps above. You can see how it took just 20 seconds for the Applitools Storybook SDK to upload all the DOM snapshots, then use Applitools Visual AI to look for visual regressions:

Conclusion

By now, you have some insight into how Applitools Ultrafast Grid functions behind the scenes when running your visual tests.

Want to get run your own tests ludicrously fast? Applitools Ultrafast grid is GA for Storybook, Cypress, and Selenium IDE. If you use Selenium WebDriver with Java, JavaScript, C#, Python, or Ruby, request to join our early access program.

Happy Testing!

Photo by Sander Jeurissen on Unsplash

The post How I ran 100 UI tests in just 20 seconds [Revisited] appeared first on Automated Visual Testing | Applitools.

How do you test a design system? You got here because you either have a design system or know you need one. But, the key is knowing how to test its behavior.

Marie Drake, Principal Test Automation Engineer at News UK, presented her webinar, “Roadmap To Testing A Design System”, where she discussed this topic in some detail.

Marie is many things. In addition to her work at News UK, she is a Cypress Ambassador and organizer of the Cypress UK community group. If you want to know more about using Cypress, she’s a great speaker. In addition, she blogs about testing and tech at her own blog, mariedrake.com.

This post summarizes her webinar and highlights some of the key points.

Who Is News UK?

News UK is the UK subsidiary of News Corp, the large global publishing and media company. Marie’s team supports the sites that develop and deliver online versions of The Sun, The Times, and The Sunday Times. They also run the Wireless media site. Marie supports the various development teams that deliver news and information that change regularly.

Why A Design System At News UK?

Think of a design system as building blocks. A design system provides a repository for design components used to construct your web application. Or, more precisely, applications. By using a design system, you can eliminate redundant work across different parts of your web application.

Marie gave the example of “share bars” at News UK. Share bars let you share content to social networks like Twitter, Facebook, Instagram, and WhatsApp. You likely have seen share bars on blogs or media pages. Inside News UK, design groups had coded their own share bars. They found19 different share bars in use across different parts of the News UK business.

The implication of lots of redundant code written by different people involves the cost of maintenance. Sure, having 19 different teams write 19 different parts of the app sounds like great division of labor. But, when you get 19 different sound bars – how do you maintain them? How do you choose one for your next part of your web business? What happens when you decide to resize the share bars across your site?

Fast Coding Does Not Equal Agility

Marie showed an even more problematic example of a web business, Fabulous, that wanted to change their brand color from #E665BF to #EA6596. When engineers looked at the potentially impacted code and the areas requiring post-change validation, they estimated the change would take six months. Half a year for a color change?

The coding effort at Fabulous involved two code bases. First, they had the website that needed to be updated. Second, they had the tests that needed to be updated to match the new site. A large part of the test change – even with no functional or other visual change – just required code inspection to ensure that the desired branding color change had been applied as expected across the entire site.

Marie’s seven-plus years in software quality led her to understand that raw coding speed rarely correlated with agility. Here, “agility” means something different from “agile.” In software, agility comes from the ability to make quick changes and have confidence on both their impact on intended behavior and avoiding unexpected changes. While many software developers can write code quickly, few write thoughtfully in ways that make code maintainable – especially across the entire site.

Benefits of the News UK Design System

In describing the design system deployed by News UK, Marie quickly pointed out its benefits.

• Cost efficient. Once you set up a design system, you have standardized building blocks for building your site. If you can use the design system to customize, your teams can consume the building blocks instead of rewriting from scratch. And, you reduce software maintenance costs.
• Reusable. A good design system allows you to re-use code.
• Speed To Market. As mentioned in the section on agility, the design system reduces the amount of code you need to write from scratch. It also reduces the amount of code you will manually change as you make updates.
• Scalable. A good design system lets multiple users access the system – making the developers much more efficient.
• Standard Way Of Working. With a design system, you standardize the process of writing code. You can help new people get up to speed on existing code and simplify the code maintenance process.
• Consistency. In the end, you can look to the design system to ensure consistent behavior (visual, functional) from your applications.

Marie showed a loop of the design system at News UK. The components get developed and maintained in Storybook. Developers can grab elements and add them into applications being built. The playground feature in Storybook makes it easy for developers and designers to play with Storybook components to mock up the functioning web application before it gets built.

As Marie pointed out, consistency in the components simplifies both development and testing.

Testing A Design System – Requirements

If a design system should make code easier to create and maintain, how do you test a design system?

Marie started outlining the testing requirements developed by News UK.

• Test different components easily. Expect the system to mature and develop over time. Some components will be entirely visual, and some may include audio. Make sure all this works.
• Test cross-browser. News UK needed this capability as they knew their content got consumed on mobile devices and a range of platforms.
• Visual Tests – Write visual tests with less maintenance in mind to reduce impact on testing workflow and speed the process of testing small changes that touch lots of components.
• Deliver a high-performing build pipeline – build plus test concludes within 10 minutes
• Integrate design review earlier in the process to improve collaboration, find misunderstandings and differences between design and development early in the process.
• Test for accessibility on both the component and site level for all users.
• Catch functional issues early.
• Have all tests written before deploying a feature. There are 2 full-time QA engineers on the Product Platforms team, so they need to share QA responsibility with developers.

Testing A Design System – Strategy

From here, Marie outlined the strategy to run tests of the design system.

• First, unit testing. Developers must write unit tests for each component and component modification.
• Second, snapshot testing. Capture snapshots and validate the status of component groups.
• Third, component testing. All components need to be validated for functionality, visual behavior, and accessibility.
• Fourth, front-end testing.  Make sure the site behaves correctly with updated components. Validate for functionality, visual behavior, and accessibility.
• Fifth – cross-browser tests. Ensure there are no unexpected differences on customer platforms.

Testing A Design System – Challenges

Marie described some of the challenges with different test approaches.

Purely functional tests can include lots of code. Marie’s pseudocode shows this problem. The more comprehensive your functional tests, the more code that exists in those tests. Assertion code – the code used to inspect the DOM for visual elements – becomes a burden for your team going forward.

Visual testing serves a strategic function, except that most visual testing approaches suffer from inaccuracy. Marie showed an example of a “spot-the-differences” game, which highlighted the challenges of a manual visual test. Then, she showed pixel differences, which she found become problematic on cross-browser tests. From a user’s perspective, the pages looked fine. The pixel differences highlighted differences that, after time-consuming inspection, her team judged as inconsequential pixel variations.

Another visual testing inaccuracy Marie described involved visual testing of dynamic data. On news sites, content changes frequently as news stories get updated. When the data changes, does the visual test fail?

Marie and her team had chosen to use available open-source tools for visual testing. Marie showed some of the visual errors that got through her testing system. These had passed functional tests but weren’t caught visually.  

So, Marie and her team discovered that their existing tests let visual bugs through. They knew they needed to solve their visual testing problem.

Choosing New Tools

Marie’s team looked at three potential solutions to their visual testing problem: Screener, Applitools and Happo.  After putting all three through their paces, the team settled on Applitools for accuracy. Being way more accurate helped Marie write up the use case for News UK to purchase a commercial tool instead of adopting an open-source solution.

The team also looked at UI testing tools. They looked at Puppeteer, Selenium, and Cypress for driving web application behavior. As a team, they chose Cypress. They could have used any of these tools with Applitools. Marie’s team chose Cypress because its developer-friendly user experience made it easy for developers to write and maintain tests.  

The final test suite included:

• Jest for managing test flow
• Cypress for running tests
• Applitools for visual tests
• Google Lighthouse and Cypress AXE for accessibility testing
• WAVE for testing Doc accessibility
• Safari Voiceover for sightless test validation

Using Applitools

Next, Marie shared the approach her team used for deploying Applitools.

Prior to using any part of Applitools, the team needed to deploy an API key. This key, found on the Applitools console, permits users the access the Applitools API. Once read into the test environment, the key grants the tests access to the Applitools service.

The team needed to add the Eyes code to Storybook for component tests and to Cypress for the site-level tests.

Component Tests

Next, Marie demonstrated the code for validating the Storybook components. The tests involved cycling through Storybook and having each component captured by Applitools. Individual component tests either matches in Applitools, or showed differences. The test team would walk through the inspected differences to either approve the changes and update the baseline image with the new capture, or rejected the change and send the component back to the developers.

Cypress Tests

Similar to the component tests, the Cypress tests integrated Applitools into captures of the built site using the new components. Again, Applitols compared each capture against the existing baseline to find differences.

For Marie’s team, one great part about using Applitools involved the built-in cross-browser testing using the Applitools Ultrafast Grid. Simply by specifying a file of target browsers and viewport sizes, Applitools could automatically capture images for the targets separately and compare each against its baseline.

Auto Maintenance

Marie talked about one of the great features in Applitools – Auto Maintenance. When Applitools discovers a visual difference, it looks for similar differences on other pages captured during a test run. When an Applitools user finds a visual difference and approves it, Auto Maintenance lists the other captures for which the identical difference exists. The Applitools user can then batch-approve the identical changes found elsewhere. A single user, in a single step, can approve site-wide menu, logo, color, and other common changes all at once.

Handling Dynamic Changes

Another benefit of Applitools involves pages with dynamic data. In addition to the example of news items updating regularly, Marie showed an example of the new Internet radio service offered by News UK. The player page can sometimes show different progress in a progress bar during different captures, depending on data being read when taking a screen capture.

Applitools has a layout mode that ensures that all the items exist in a layout, including the relative location of the items, but layout mode ignores content changes within the layout.

Accessibility Tests

Next, Marie talked about accessibility tests.

Marie demonstrated component accessibility testing with Cypress AXE. She showed that, once integrated with Cypress, AXE can cycle through components. Unfortunately, AXE and other automated tests uncover only about 20% of accessibility tests.

Lighthouse and other tests get run manually to validate the rest of visual accessiblity.

She also showed the Safari screen reader accessibility testing.

Workflow Integration

Marie then described workflow, and how the workflow integration mattered to the Product Platforms team.

She made the team’s first point – quality is everyone’s responsibility. For the product platforms team, the two quality engineers serve as leads on approaches and best practices. Developers must deliver quality in the design system.

To accentuate this point, she explained that the team had developed pull request guidelines. Check-ins and pull requests required documentation and testing checklist of unit, component, and page-level tests. Everyone agreed to this level of work for a pull request.

Next, Marie showed the workflow for a pull request. Each pull request at the component level required a visual validation of the component before merging. She explained how Applitools could maintain separate baselines for each branch and manage checkpoints independently. Then, she showed the full develop workflow build pipeline.

Finally, she showed how Github integration linked visual testing fit into the entire Circle CI build. She also showed how the buld process linked to Slack, so that the team could be notified if the build or testing encountered problems. The build, including all the tests, needed to complete within 10 minutes.

Overall Feedback

Marie provided her team’s general feedback about using Applitools. They concluded that they required Applitools for visual validation of the component-level and site-level tests. Developers appreciated how easily they could use Applitools with Cypress, and how they could run 60 component tests in under 5 minutes across a range of browsers. The design team also uses Applitools to validate the design, and they found the learning curve was fast for figuring out the visual elements.

As users, they did have feedback for improvement to share with the Applitools product team. One of the most interesting came from the design team, who wondered if they could use UI design tools (Sketch, Figma, Abstract, etc.) to seed the initial baseline for an application.

Beyond Applitools, the accessibility testing has helped ensure that News UK can deliver visual and audio accessibility for their users.

Conclusion

Marie Drake made a strong case for using a design system whenever there are multiple design and development groups working independently on a common web application. The design system eliminates redundancy and helps speed the rate of change whenever groups want to roll out application enhancements.

She also made a strong case for building testing into every phase of the design system, from component-level unit, functional, visual, and accessibility tests all the way to page-level tests of the same. For testing speed, testing accuracy, ease of test maintenance, and cross-browser tests, Marie made a strong case for using Applitools as the visual test solution for the News UK design system.

For More Information

• How Do You Simplify End-To-End Test Maintenance?
• Measure Your Test Automation Maturity
• Adding Log Analysis to QA Will Improve Your Processes
• How Do You Catch More Bugs In Your End-To-End Tests?
• How Can 2 Hours Now Save You 1,000 Hours In The Next Year?
• Request an Applitools Demo
• Sign up for a free Applitools account.

The post How Do You Test A Design System? appeared first on Automated Visual Testing | Applitools.

Cypress, among other things, validates the structure of your DOM. It verifies that a button is visible or that it has the correct text. But what about the look and styling of our app? How can we test that our application looks good visually? We can use Cypress to verify that it has the correct CSS properties but then our code would look very long and complex. It will be a guarantee that engineers will avoid maintaining this test ?

Visual Testing

This is why we need visual testing as part of our test strategy. With visual testing, we are validating what the user sees on different browsers/viewports/mobile devices. However, it’s very time consuming when you do it manually.

Imagine if someone told you that you have to test the below image manually.

There are 30 differences. You can probably find them after quite some time. But then this provides a really slow feedback loop. The obvious solution would be of course to automate this!

Now, automated visual testing is not new. There are so many tools out there which can help you with visual testing. These tools use pixel-by-pixel comparison to compare the two images. The idea is you have a baseline image which is your source of truth. Every time there is a code change and our automated tests are run, a test image will be taken. The visual testing tool will then compare this test image with the baseline image and then check the differences. At the end, it will report to us whether our test passed or failed.

Problem with pixel-by-pixel visual testing

The problem with pixel-by-pixel visual testing though is it’s very sensitive even with small changes. Even if you have a 1px difference, your test will fail even though on the human eye, the two images look exactly similar.

You also get the issue that if you try to run these tests on your build pipelines, you might see a lot of pixel differences especially if the base image is generated locally such as the image above. Looking at this test image, if you ignore the mismatch image in the middle and compare the two images on left and the right, some of the changes that were reported look ok.  But because these images were taken on different machine setups , the tool has reported a lot of pixel differences. You can use Docker to solve this and generate the base image using the same configuration as the test image, but from personal experience, I still get flakiness with using pixel-by-pixel comparison tools.

Also, what if you have dynamic data? In the test image above, we have a change of data here but the overall layout is similar. You can probably set the mismatch threshold to be slightly higher so your tests will fail only if they reach the threshold that you defined. The problem with this though is that you might miss actual visual bugs.

Cross Browser Visual Validation

Most of the existing open source tools for visual testing only runs on 1 or 2 browsers.  For example, one of the tools that we were using before, called BackstopJS,  which is a popular visual testing framework, only runs visual tests on Chrome headlessly.  AyeSpy, which is a tool that was actually created by one of the teams here at News UK, is another visual testing tool, which hooks into your Selenium Grid to run your visual tests on different browsers. But still, it’s a bit limited. If you are using the Selenium Docker images, they only have images for Chrome and Firefox. What if you want to run your visual tests on Safari or Internet Explorer?  You can definitely verify this yourself, but again as mentioned, it’s time consuming.

How can we solve these different visual testing issues?

Applitools

This is where Applitools comes in. Unlike existing visual tools, Applitools uses AI comparison algorithms so images are compared just like a human would. It was founded in 2013 and pretty much integrates with almost all testing frameworks out there. You name it – Selenium, Cypress, Appium, WebdriverIO, Espresso, XCUITest, even Storybook! With Applitools, you can validate visual differences on a wide range of browsers and viewports. By using different comparison algorithms (exact, strict, layout and content), you have different options to compare images and can cater different scenarios such as dynamic data or animations.

Rather than taking a screenshot, Applitools will extract a snapshot of your DOM. This is one of the reasons why visual tests are fast to run in Applitools. Once the DOM snapshots have been uploaded to the Applitools Cloud, the Applitools Ultrafast Grid, which offers users a way to render screens in multiple browsers and viewports simultaneously to generate the screenshots and do the AI powered image comparison.

Getting Started

To get you started, you need to install the following package to your project. This is specific to Cypress so you would need to install the correct package depending on your test framework of choice.

npm i --D @applitools/eyes-cypress

Once this has been installed, you need to configure the plugin and the easiest way to do this is to run the following command on your terminal

npx eyes-setup

This will automatically add the necessary imports needed to get Applitools working in your Cypress project.

Writing our first test with just Cypress

Let’s start doing some coding and add some validations on a sample react image app that I created a while back. It is a simple image gallery which uses unsplash API for the backend. An example github repo which contains the different code examples can be found here.

Our Cypress test can look like the following code snippet. Keep in mind, this only asserts the application to some extent. I can add more code to verify that it has the correct CSS attributes but I don’t want to make the code too lengthy.

Writing our first test with Cypress and Applitools

Now, let’s look at how we can write the test using Cypress and Applitools.

Applitools provides the following commands to Cypress as a minimum setup. `cy.eyesOpen` initiates the Cypress eyes SDK and we pass some arguments such as our app name, batch name and the browser size (defaults to Chrome). The command `cy.eyesCheckWindow` will take a DOM snapshot so every call to this command means a DOM snapshot will be generated. You can call this every time you do an action such as visiting your page under test or clicking buttons, and dropdown menus. Finally, once you are finished, you just call `cy.eyesClose`. To know more about the Cypress eyes SDK, please visit their documentation here to find more information.

In order to run this in Applitools, you need to export an API key which is detailed on this article. Once you have the API key, on your terminal you need to run:

export APPLITOOLS_API_KEY=<your_key>

npx cypress open

Once the test is finished, if you go to the Applitools dashboard, you should see your test being run. The first time you run it, there will be no baseline image then when you reran the test and everything looks good, you should see the following.

Handling Dynamic Data

Since we are using the unsplash API, we don’t have control as to what data gets returned. When we refresh the app, we might get different results. As an example, the request that I am making to unsplash will get the popular photos on a given day. If I reran my test again tomorrow, then the images will be different like the one shown below.

The good thing is we can apply a layout region so the actual data will be ignored or we can also set the match level as Layout in our test which we can preview on the dashboard. If the layout of our image gallery has changed, Applitools will report it as an issue.

Making code changes to our application

Now, let’s create some changes in our application (code references found here) and introduce the following:

• Update the background colour of the search bar (bug)
• Add a new footer component (new feature)
• Remove camera icon on the subtitle (bug)

If we run the test where we only use Cypress, how many of these changes do you think your test will catch? Will it catch that there is a new footer component? How about updating the background colour of the search bar? How about the missing header icon? Probably not because we didn’t write any assertions for it. Now, let’s rerun the test written in Cypress and Applitools.

Looking at the image above, it caught all the changes and we didn’t had to update our test since all the maintenance is done on the Applitools side. Any issues can be raised directly on the dashboard and you can also configure it to integrate to your JIRA projects.

Cross Browser Visual Validation

To run the same test on different browsers, you just need to specify the browser options on your Applitools configuration. I’ve refactored the code a bit and created a file called `applitools.config.js` and moved some of the setup we added initially in our `cy.eyesOpen` in this class.

Just simply reran your test and check the results in the Applitools dashboard.

Final Thoughts

This is just an introductory post on how you can use Applitools so if you want to know more about its other features, check out the following resources:

• Applitools Master Class Webinar
• Applying Match Levels
• Applying Match Regions
• Test Automation University – different courses on using Applitools with other frameworks

While open source pixel by pixel comparison tools can help you get started with visual testing, using Applitools can modernize the way you do testing. As always, do a thorough analysis of a tool to see if it will meet your testing needs ?

The post Modern Cross Browser Testing with Cypress and Applitools appeared first on Automated Visual Testing | Applitools.

What is a design system? Who would use it, and for what benefit?

In May 2020, Applitools had the pleasure of hosting Tyler Krupicka from Intuit for an hour-long webinar discussing design systems and testability.

Tyler works at Intuit, a 9,400 employee company headquartered in Mountain View, California, that specializes in accounting and tax preparation software. At Intuit, Tyler works on the “Player/Design Systems” team, where he focuses on design systems.

Tyler has been working on web development at Intuit since 2015. These days, he works on UI components, tools, and testing. He primarily uses TypeScript, React and Vue. He also focuses on accessibility, and he serves as a “Level 2” Accessibility Champion inside Intuit.As part of Tyler’s job, he ensures that design systems are testable. 

What Is A Design System

Tyler started asking this question. Some people think about large design systems:

• Material Design (Google’s design system)
• Human Interface Guidelines (Apple’s design system)
• Bootstrap (set of UI interface components and CSS that are popular on the web)

People look at these and think, “Is this overkill?” After all, Google uses Material Design for Android. How many people design an entire operating system. So, Tyler dove in to answer his question.

First, Tyler sees design systems as really “Design + Code.” One part of a design system are the “design” – the common components and patterns that get used throughout products. The other part of the design system, Code, involves chunks of functionality interacting with design developers can reuse across a product.  For example, CSS Frameworks, or React, or in some other way that lets teams reuse the same code.

Design

“Design” turns into a consistent voice for your product. Those are things like typography, colors, and spacing, as well as UI components (forms, charts, site navigation), Graphics and Animation standards. These help standardize the visualization – the “look” – of the application. One can imagine an application developed by a large team that lacked a standard design – it would be disorienting to customers who expect consistency from software. A design system lets  you take base pieces, document them well, and reuse them.

Code

“Code” lets you build and document components individually, test them once, and have a way to set expectations on whether or not this application will work.  For instance, you can build a component in the app for one persona and reuse that component multiple times inside the app without worrying that you must retest it. Reuse makes code standard and creates standards for interacting with design. If you ever change design elements, the standard code reduces the effort to validate the new design with existing code. 

Combining design and code into a design system results in increased developer productivity and a consistent voice for the application. And using a design system simplifies the management of an application.

Design Systems at Intuit: TurboTax Example

Next, Tyler dove into the use of design systems for the Intuit TurboTax product. 

Tyler explained how TurboTax works. By providing users with interview-style questions, TurboTax helps millions of people, primarily in the United States and Canada, file their income tax returns. Intuit offers both web and mobile versions of TurboTax.

Because TurboTax uses interview questions to create conditional flows through the product, there are thousands of screens users can encounter based on their unique needs. Screens can range in complexity from a single question to a complete form. 

Building the TurboTax Design System

To create the design, Intuit develops about 50 UI components needed to run the experience. Headers, radio buttons, tiles, multi-select, data entry, action buttons – all become part of the basic UI. 

Intuit also builds a set of about 500 “mocks”, or mock-ups, of what common screens look like for testing. These help show the UI components in common layouts for testing purposes.

By using a design system, Intuit can make changes to the application and ensure consistency for all these screens. 

Tyler gave the example of a decision to deliver a uniform increase in contrast in TurboTax for accessibility. The contrast changes required updates to typography weight, button color, and link color. This work would impact over 90% of all the screens. How could Intuit deliver these changes to the application and ensure they occurred consistently across the application? The team did not have the staff needed to manually check every possible screen. Instead, they made the design changes to the UI components and validated the mocks.

Testing A Design System

Next, Tyler dove into a discussion of testing a design system. 

Approach

Since the design system represents all the functionality and visual representation of the application, Tyler recommends going broad with testing across all the elements, including:

• Functionality
• Visual
• Performance
• Accessibility

From a functional perspective, unit tests serve as key building blocks for code. Integration tests ensure that the code works wiht the back end. Cross-vrowser tests help ensure that the code works on a bunch of different browsers and mobile devices. 

Visual testing validates the impact of UI component changes on the overall UI. If buttons change size or Intuit uses a new font size, does the page continue to render correctly? Do unexpected breaks happen on pages? Do space reductions make pages look too constrained?

Performance testing involves checking the impact of code changes on page loading times.  Specifically, do new packages or package upgrades change the amount of data transferred between the server and the browser – slowing the user experience? 

Finally, Tyler tests accessibility to make sure that any customer who wants to use the product can do so. Whether a user has a screen reader or navigates solely by keyboard, ensure that the user can get around.

Getting Organized

With testing goals articulated, Tyler introduced Storybook as the tool for creating documentation and initial testing. If you use projects like React, Vue and Angular that allow you to build user interfaces in small components, the Storybook project lets you build interactions for those components and document their behavior.  

Effectively, you demonstrate a  use case for your components and show how they all interact together on a page.  Each use case is one story – hence Storybook is a collection of these stories. 

Storybook allows for a range of tests, including accessibility tests. For instance, Tyler uses Storybook to test keyboard navigation and focus. He also tests for things like hover behaviors – where components appear while the cursor hovers over another element, which may be difficult to test with automation. Testing with Storybook lets the testing team run tests early across the range of design system dimensions.

Dog Fooding

To deliver successful products, Tyler’s team must support the construction and use of these UI components. So, they actually write code to try out the components – just to see how they work. Putting together checkboxes or radio buttons, or drop-down lists helps show how the code interacts with components and creates mocks for testing the design system.

Tyler showed the example of a page that looked like static boxes. Each box expanded when clicked to show hidden detail and slid the rest of the page down. His team dog foods this kind of behavior.

All of these activities help create a body of testable code, so that a design change in the design system can be mocked up and validated without impacting production code.

Testing Walkthrough

Tyler then walked us through a simple change that can have a signficant implications – resizing a button. 

Without a design system, you might have to make a button change like this ad-hoc across your application. As a result, you risk changing the page-to-page behavior and look-and-feel. Using a design system lets you make the change globally, and then focus on the implications.

What kinds of implications? A larger button might displace elements on a page. The button may not function as expected with existing elements on a given page. All these changes need to be tested.

Unit Tests

First, Tyler looked at unit tests. Does the button click? Can the button work with an icon associated with it? Will the button work with an transparent background? Does the button work with assigned ARIA (Accessible Rich Internet Applications) attributes? Testing puts these Storybook components through their paces.  

Intuit have a pretty standard way of running through these tests.  Tyler talked about using JEST and WebdriverIO and Selenium to automate this testing across multiple browsers. They have lots of users on multiple web browsers, as well as Mac users with Safari an iOS, so they pay special attention to those target platforms. 

Accessibility Tests

Next, Tyler talked about accessibility testing. His team uses AXE to perform static validation for ARIA standards on application pages, including contrast checking.  Tyler explained how AXE helps perform contrast checking. For instance, if a button is one color, does the background let the button stand out? AXE can do the contrast checking by comparing the two colors. Or, will the text be readable against its background – or is it too small or too thin? Again – AXE can make these calculations.

Tyler also talked about HTML customizations. For example, you want to use a special radio button design that isn’t included in the existing HTML. You build this behavior as a custom implementation – and then you have to validate this behavior across the browsers your customers use. Tyler pointed out that companies make these decisions to help brand their websites for customers. And, these custom behaviors must be validated for functionality and accessible behavior across multiple platforms.  At the same time, since you are replacing functionality built into HTML, keyboard navigation still needs to behave as expected. So, you need to test those behaviors. 

At Intuit, the team tests these behaviors manually – and with tools designed to help. By using popular screenreaders like Jaws, Voiceover, and NVDA, Tyler’s team ensures that these screen readers don’t get thrown off by customizations. Also through manual testing, the team checks to ensure that keyboard navigation results in expected page responses – for example, they make sure that focus states behave as expected. And, they also ensure that the page behaves correctly when a customer has selected “reduced motion”.

Performance Tests

From a design perspective, the design system simplifies app delivery. By using standard libraries and building standard behaviors, the design system creates this great look-and-feel. At the same time, developers need to consider the impact on performance. For instance, a specific package might add great mobile behavior – at the cost of an additional 1-2 seconds of page load time. Users likely won’t value the behavior over slower page load time.  

You can choose a great library that gives you great functionality – but at what cost? How does the size of your Javascript package impact your page downloads? Often, this problem creeps up on you.  You use an early version of the library and everything works well. But, when you update your application, your library update suddenly doubles in size. This happens quite often. New versions of libraries include new features – which add to the library load times. And, you have to rethink the software you’re using.

Tyler talked about how Intuit handles these kinds of challenges. If Intuit uses code from an open source project, they might contribute back to the project to make a library with reduced size. Or, they might use a different open-source package.  

Visual Testing

Each time a new component gets introduced or modified, Intuit runs tests through Applitools Ultrafast Grid. Intuit has been running Grid since its first beta tests. With Ultrafast Grid, Intuit can set checkpoints and take screenshots across a range of browsers and devices. They can set multiple screen sizes (from mobile to Full HD and  higher). Their goal – ensure that responsive pages behave as expected, and understand the impact of design changes on existing pages. Ultrafast grid gives them the capability to easily compare changes across different versions of an application.

Tyler talked about using Ultrafast Grid to test against Chrome, Edge, and Safari. He talked about setting different viewport sizes to check responsive page behavior against expectations. 

Component and Mock Level

Tyler explained how Intuit tests components and mocks. 

In Storybook, Intuit keeps the latest components, as well as page mocks for each component in use. When a change occurs in the design system, Intuit can do two kinds of checks:

1. They can check the components to see if the component changes have violated any specifications
2. They check the mocks, to determine how the component changes affect page rendering.

So, for instance, Intuit can see if a collection of multiple new components violate spacing guidelines or other design requirements on pages. Applitools Ultrafast Grid helps Intuit do this evaluation.

Also using the mocks, Intuit can also see if a change violates accessibility guidelines using AXE.  

Accessing Intuit Tools for Design Systems

Tyler discussed the two tools Intuit uses to ensure success in their design systems.

The first, the Intuit DS-CLI, makes it easy to implement Storybook for your designs. Just like Create-React-App lets you build a web app easily, DS-CLI contains all the code you need to utilize Storybook for component and mock management. This includes preconfigured Storybook, component templates with best practices, great Typescript support, and bundle size tracking.  Tyler mentioned that DS-CLI is free and open source. 

One great part about DS-CLI is size tracking.  Each component can be tracked over time for package size. You can see when your bundles are growing – and why.

With the second, Intuit Proof, you can automate the development of test running in Storybook. Proof is a tapable testrunner for Storybook. Proof helps you automate tests for components and mocks by helping you know which ones have tests. In addition, Proof has built-in hooks for Applitools and Axe. So, instead of fumbling about inside Storybook to see what has been tested and what remains, you know what tests exist, which still need to be written, and helps you track their status. 

Proof is also free and open source.

Conclusion

Tyler’s presentation made it clear that a design system can make your life much easier, but you need to invest in tools and methodology to help ensure consistency – the benefit of a design system. 

You need a component evaluation system. You need mockups of your pages to test. And you need a testing environment.  Intuit does all this inside Storybook.

Next, you need a testing strategy to get components evaluated and integrated into your pages. Invest in unit testing and test runners. Especially if you care about being perceived as open to all users – invest in accessibility tools. And, finally, know that rendering matters – invest in visual evaluation solutions like Applitools.

Finally, you need some pieces of infrastructure to make your design system work. Intuit shares theirs open source.

If you make these investments up front, a design system can serve your development and test strategy to help you easily develop incremental improvements across your application without fear of code changes. 

Read More

• Impact of Visual AI on Test Automation Report
• A/B Testing: Validating Multiple Variations
• Modern Dev Teams Are More Efficient Than Ever — Quality Engineering Should Be Too.
• The Next Generation of Cross Browser Testing is Ultrafast
• Sign up for a free Applitools account
• Request an Applitools Demo

The post Design Systems and Testability appeared first on Automated Visual Testing | Applitools.

At one point in time, the server-side code was responsible for generating and serving every page in our application. Nowadays, Single Page Apps (SPA) are taking a bigger slice of the pie, as more clients are leaning towards building their applications as an SPA.

Developers spend most of their time debugging a SPA inside the browser, by debugging the JavaScript files, CSS styles and the DOM, hoping to find the root cause of any visual bug appearing in their application.

Applitools realizes the complexity of visually debugging an application and introduced the Root Cause Analysis (RCA) module on their Test Manager Dashboard to help quickly identify the root cause behind visual UI changes.

In this article, I will demonstrate the Applitools RCA feature by walking you through a complete step by step guide to visually test a Vue.js app with Storybook and Applitools.

If you’re not already familiar, Applitools is an automated visual regression testing framework. It focuses on the visual aspects of your app and plays a major role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of development and testing frameworks, such as Cypress.io, Storybook and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools, in order to send both baseline and test screenshots.

So, get started with visual UI testing. Your visual testing project will depend on React, Storybook and Applitools. You will also need the node.js tools. So, firstly, make sure you know the following software tools and install on your machine:

• Git
• Node.js and npm
• Vue.js
• Storybook
• Applitools

Before we immerse ourselves in writing code, let’s look at how Applitools and Storybook work together.

How Applitools Works With Storybook

Whether you use Vue.js, React or Angular, Storybook doesn’t offer any extension points for third-party frameworks, such as Applitools, to integrate with. The Applitools team worked around this limitation by providing the Applitools Eyes Storybook SDK.

Once installed, this SDK collects all stories in the application, runs them all via the Storybook engine, generates DOM snapshots for each story and uploads these snapshots to the Applitools Eyes server. Here’s a visual of this flow:

The Applitools server runs the tests generates image snapshots. Applitools compares each snapshot to a baseline snapshot (if one exists) and displays the results on the Test Manager Dashboard.

This kind of integration provides a smooth experience in terms of writing Storybook stories. Nothing changes in the way you write your Vue.js apps.

Automated Root Cause Analysis

The concept of Root Cause Analysis stems from the world of management, where it’s defined as a method of problem solving, used for identifying the root causes of faults or problems.

So far, Applitools has been focusing on visual testing by comparing the baseline and the current test run snapshot through their AI engine. It finds the visual differences in order to map them graphically. Previously, there was no way to search your codebase for a reason behind any visual testing bug. As you may know, searching through code is nightmarish, time-consuming and tedious!

This is where Applitools steps in with RCA, showing you the exact DOM and CSS changes between the baseline snapshot and the current test run snapshot.

Why is this so important for both developers and QA testers? Let’s take a look.

Step By Step Guide

For this demonstration of visual testing a Vue.js application with Storybook and Applitools, I’ve chosen the Bootstrap Vue open-source library hosted on GitHub under this repo. This library, with over 40 plugins and more than 75 custom components, provides one of the most comprehensive implementations of Bootstrap v4 components and grid system for Vue.js. With extensive and automated WAI-ARIA accessibility markup.

The plan is to start by installing Storybook onto the Bootstrap Vue project. Write and run a few Storybook stories, install the Applitools Eyes SDK for Storybook and finally demonstrate the RCA feature.

Now, let’s get on with it and start coding …

Step 1: Clone the Bootstrap Vue GitHub repo locally on your machine by running this command:

git clone git@github.com:bootstrap-vue/bootstrap-vue.git

Step 2: Install all NPM package dependencies by running this command:

yarn

Step 3: Install the Storybook NPM package as a dev-dependency by issuing this command:

yarn add @storybook/vue -D

Step 4: Install all peer dependencies required by Storybook to function properly by issuing this command:

yarn add vue-loader babel-loader babel-preset-vue -D

Step 5: Install css-related NPM packages in order to load CSS styles inside your Storybook stories by issuing this command:

yarn add css-loader style-loader -D

Step 6: Add the following NPM script to your package.json file in order to start the Storybook:

{

  "scripts": {

    "storybook": "start-storybook"

  }

}

Step 7: Create the Storybook config.js file inside a .storybook folder located at the root of the solution folder as follows:

import { configure } from '@storybook/vue';

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/);

  req.keys().forEach(filename => req(filename));

}

configure(loadStories, module);

Step 8: Create a scss-loader.scss file inside the .storybook folder to include all the styles required by Bootstrap Vue library components to load properly inside our Storybook stories:

@import ‘../node_modules/bootstrap/scss/bootstrap.scss’

@import ‘../src/index.scss’

Then import this file inside the config.js file as follows:

import { configure } from '@storybook/vue';

import ‘!style-loader!css-loader!sass-loader!./scss-loader.scss’

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/);

  req.keys().forEach(filename => req(filename));

}

Step 9: Create the navbar.stories.js file inside the stories folder, located at the root of the solution folder, to hold our Storybook stories for the NavBar component:

import { storiesOf } from '@storybook/vue'

const navbarStories = storiesOf('BNavBar', module)

navbarStories.add('Full Navbar', () => ({

  template: `<div>

  <b-navbar toggleable="lg" type="dark" variant="info">

    <b-navbar-brand href="#">NavBar</b-navbar-brand>

    <b-navbar-toggle target="nav-collapse"></b-navbar-toggle>

    <b-collapse id="nav-collapse" is-nav>

      <b-navbar-nav>

        <b-nav-item href="#"><span>Link</span></b-nav-item>

        <b-nav-item href="#" disabled>Disabled</b-nav-item>

      </b-navbar-nav>

      <b-navbar-nav class="ml-auto">

        <b-nav-form>

          <b-form-input size="sm" class="mr-sm-2" placeholder="Search"></b-form-input>

          <b-button size="sm" class="my-2 my-sm-0" type="submit">Search</b-button>

        </b-nav-form>

        <b-nav-item-dropdown text="Lang" right>

          <b-dropdown-item href="#">EN</b-dropdown-item>

          <b-dropdown-item href="#">ES</b-dropdown-item>

          <b-dropdown-item href="#">RU</b-dropdown-item>

          <b-dropdown-item href="#">FA</b-dropdown-item>

        </b-nav-item-dropdown>

        <b-nav-item-dropdown right>

          <template slot="button-content"><em>User</em></template>

          <b-dropdown-item href="#">Profile</b-dropdown-item>

          <b-dropdown-item href="#">Sign Out</b-dropdown-item>

        </b-nav-item-dropdown>

      </b-navbar-nav>

    </b-collapse>

  </b-navbar>

</div>`

}))

The story groups all NavBar stories under the name BNavBar and assigns the first story the name of Full Navbar.

There are two ways to tell Storybook stories about your Vue.js components. Either define all used components in a components object property on the story itself or define the components globally inside .storybook/config.js file so that they are available to all running stories in your solution.

Let’s follow the latter option and define all components needed by NavBar story inside .storybook/config.js file using the Vue.component() helper method:

import { configure } from '@storybook/vue'

import Vue from 'vue'

import '!style-loader!css-loader!sass-loader!./scss-loader.scss'

import {

  BNavbar,

  BNavbarNav,

  BNavbarBrand,

  BNavbarToggle,

  BButton,

  BCollapse,

  BNavItem,

  BFormInput,

  BNavForm,

  BNavItemDropdown,

  BDropdownItem

} from '../src'

Vue.component('b-navbar', BNavbar)

Vue.component('b-navbar-nav', BNavbarNav)

Vue.component('b-navbar-brand', BNavbarBrand)

Vue.component('b-navbar-toggle', BNavbarToggle)

Vue.component('b-button', BButton)

Vue.component('b-collapse', BCollapse)

Vue.component('b-nav-item', BNavItem)

Vue.component('b-form-input', BFormInput)

Vue.component('b-nav-form', BNavForm)

Vue.component('b-nav-item-dropdown', BNavItemDropdown)

Vue.component('b-dropdown-item', BDropdownItem)

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/)

  req.keys().forEach(filename => req(filename))

}

configure(loadStories, module)

Step 10: Now the story is ready, let’s run it with Storybook using this command:

yarn storybook

The Storybook engine runs successfully and renders our Navbar story.

Now let’s install the Applitools Eyes SDK for Storybook.

Step 11: Install the Applitools Eyes SDK for Storybook NPM package as a dev-dependency by issuing this command:

yarn add @applitools/eyes-storybook -D

In order to authenticate via the Applitools server, you need to supply the Eyes-Storybook SDK with the API key you got from Applitools. Read more about how to obtain the API key here.

To do this, set the environment variable APPLITOOLS_API_KEY to the API key before running your tests. For example, on Linux/Mac:

export APPLITOOLS_API_KEY=<your_key>

And on Windows:

set APPLITOOLS_API_KEY=<your_key>

Step 12: Let’s run the first cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Once the tests are done running, let’s check back with the Applitools Test Manager Dashboard and verify the results.

The left side of the Dashboard lists all batch test runs that you have previously performed. Currently, there is a single test run with a status of Passed. This is the first test run and so Applitools engine automatically passes this test and all snapshots thereafter as baseline snapshots.

Click on the single batch test to view the snapshots available. For each and every Storybook story, a snapshot is generated by Applitools.

Expand the BNavBar: Full Navbar test run and click the snapshot to view it in a bigger scale:

As you can tell, the test run snapshot is simply an image snapshot for the story that runs inside the Storybook playground application.

At this moment of time, Applitools stores all the generated snapshots as baseline snapshots. The future test runs would generate a new set of snapshots and Applitools would compare them to the baseline ones and report the results accordingly.

You have noticed that Applitools Eyes SDK for Storybook is prepackaged with some default settings allowing you to do visual testing for your stories without having to specify any test configuration settings. However, that doesn’t mean that Applitools doesn’t offer you the means to customize your tests by specifying test configuration settings either through environment variables or storing your settings inside the applitools.config.js configuration file.

Let’s switch gears and explore a unique gem of Applitools that is the Regression UI testing. The traditional regression testing verifies recent source code changes or additions and that the functionality of the software has not changed or been affected in any way.

Applitools allows you to perform visual UI Regression testing. You can run multiple cycles of test runs and all visual changes collected from one to another, are detected by Applitools servers and logged into the Dashboard.

Step 13: Let’s introduce a visual change in the Navbar component by including the following CSS selector inside .storybook/scss-loader.scss file and run our tests again.

@import "../node_modules/bootstrap/scss/bootstrap.scss";

@import "../src/index.scss";

a {

  background-color: yellow;

}

The change causes all the hyperlinks on the Navbar component to render with a background color of yellow.

Step 14: Let’s run the second cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Notice the logs on the terminal windows after running the command:

The test failed and that’s exactly what we expect. Applitools detected the change we introduced to the background color of the hyperlinks and reported it.

Let’s switch back to the Applitools Test Manager Dashboard to check the results.

The new test run is marked as Unresolved. Expand on the Navbar test run to view the visual differences highlighted to you by Applitools:

Make sure both snapshots are selected so that you can easily view the differences. If not, locate the following menu and choose the option Both:

The snapshot above highlights all the hyperlinks, indicating that those hyperlinks have changed.

Step 15: Click on the RCA tool icon (highlighted in the red rectangle) to get a full analysis of the source of this visual bug.

Next, you select the visual difference you want RCA to assess.

I’ve selected to click on the Lang hyperlink on the Navbar. Notice how the RCA tool detects a css style change on the <a> tag represented by the Yellow background.

The RCA tool runs and shows the root cause of this visual difference, which is a change in the CSS Rules.

Step 16: The RCA tool can also detect Attribute changes. Let’s change the tag of the Search box to have a size of lg as follows and run the test once more.

<b-form-input size="lg" class="mr-sm-2" placeholder="Search"></b-form-input>

The RCA tool detects a change in the attributes of the Search box.

It also shows the change of CSS rules as follows:

Step 17: Another category RCA detects is the DOM changes category. Let’s wrap the Search button text with a paragraph tag as follows and run the test again:

<b-button size="sm" class="my-2 my-sm-0" type="submit"><p>Search</p></b-button>

The RCA tool detects a change in the text inside the Search button. Instead of rendering the text as is, the new test snapshot renders the text inside a <p> tag.

Any test run with status Unresolved requires someone’s attention and manual intervention to accept or reject the change. In this case, you may accept the new change(s) and thus update the baseline snapshots.

On the other hand, if the change detected is undesirable, then this implies a bug in the source code and you can reject this new test run to keep the original baseline snapshot without any changes.

In Conclusion

Applitools RCA is both powerful and convenient. It can pinpoint the exact DOM/CSS change. If you did not intend the change, you can identify the code the responsible code quickly and make appropriate modifications to pass your tests.

RCA supports a set of DOM/CSS categories of change, including:

• Textual changes
• CSS Property changes
• Attributes changes
• Bounding Box changes
• Tag changes (for example, when you replace a button with a hyperlink)

Happy Testing!

Video Demonstration

More about Applitools

• Request a demo of Applitools.
• Sign up for a free Applitools account.
• Check out our tutorials.

The post Vue.js Development with Storybook and Applitools appeared first on Automated Visual Testing | Applitools.

ReactJS provides a library of reusable components for building JavaScript applications. When combined with Storybook and Applitools Eyes, you can validate the visual appearance of your React components in a controlled testbed. With Applitools Ultrafast Grid, you can run your visual validations across a range of browsers and viewport sizes across multiple screen combinations simultaneously. Together, React, Storybook, and Applitools make you much more productive by developing once in React, testing once in Storybook, and rapidly validating everywhere in Applitools.

React Development and Test Best Practices

When developing a React UI component library, you are likely aware that a well-known best practice splits the components into self-contained js/jsx files. Splitting benefits you, whether you are the developer developing the components or another developer consuming those components in your apps.

Once you develop using standalone components,  you need to consider how to test your components inside a browser during the development phase. You have a few options here. One of them is to develop a demo or showcase application. You can use the showcase app to render components and make sure they run correctly inside the browser. However, you end up coding something you likely won’t use again – and that seems like a waste of resources.

The best option is to make use of a third-party playground framework such as Storybook that can kick off a new web server for you and run all the components under development. Also, Storybook gives you the option to generate a static documentation playground website that you can host online to showcase your components live.

As you develop components, you revise components, change their code, and test them to verify nothing broke with the new changes. Some of your changes will be functional, and you can create new functional tests for those components. However, the visual changes are problematic. Manual validation of all of them is cumbersome, time-consuming, and inaccurate. That’s where visual UI testing frameworks come in handy.

So, let’s talk Applitools. Not only does Applitools provide you with the means to do Visual UI testing, but you can also use for Visual UI Regression testing. That is heavenly in terms of Visual UI testing.

In this article, I’ll walk you through a complete, step by step guide to visually test a React app with Storybook and Applitools.

What is Applitools?

If you’re not already familiar, Applitools is an automated visual regression testing framework. It focuses on the visual aspects of your app and plays a significant role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of development and testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools, to capture both baseline and test screenshots.

So, let’s get started with some visual UI testing. Your visual testing project will depend on React, Storybook, and Applitools. You will also need node.js tools. So, firstly, make sure you’re familiar with the following and have them installed on your machine:

• Git
• Node.js and npm
• React
• Storybook
• Applitools

Before we immerse ourselves in writing code, let’s look at how Applitools and Storybook work together.

How Applitools Works With Storybook

No matter whether you use Vue.js, React, or Angular, Storybook doesn’t offer any extension points for third-party frameworks, such as Applitools, to integrate with. The Applitools team worked around this limitation by providing the Applitools Eyes Storybook SDK.

The behavior of this SDK is that once you run it locally on your machine, it will start by collecting all stories in the application, run them all via the Storybook engine, generate DOM snapshots for each and every story and upload these snapshots to the Applitools Eyes server. Here’s a visual of this flow:

Tests run on the server to generate actual image snapshots. Each snapshot is compared to a baseline snapshot (if one exists), and the results are displayed on the Test Manager Dashboard.

This kind of integration provides a smooth experience in terms of writing Storybook stories. Nothing changes in the way you write your React apps.

Now, let’s get on with it and start coding …

Step By Step Guide

For this demonstration of visual testing a React application with Storybook and Applitools, I’ve chosen the React95 open-source library hosted on GitHub under this repo. This library is both unique and fun at the same time. It creates a rich set of components having the theme of Windows 95. This library already uses Storybook and by adding Applitools to the mix, should be much easier and less time-consuming.

Let’s follow the steps to run the React95 Storybook stories and use Applitools to generate a few visual UI test runs.

Steps 1-3 – Start with a React Library and Install Storybook

Step 1: Clone the React95 GitHub repo locally on your machine by running this command:

git clone git@github.com:React95/React95.git

Step 2: Install all NPM package dependencies by running this command:

npm install

Step 3: As you now know, this library already uses Storybook. Hence, the scripts section of the package.json file already contains the script to run Storybook. Let’s give it a shot and run Storybook stories by issuing this command:

npm run storybook

As a side note, in case you are installing Storybook from scratch, here’s a mini-guide to help you get started adding Storybook to your application.

Run the following command to install Storybook into your application:

npx -p @storybook/cli sb init

The Storybook NPM package {{ link to the NPM package here }} has a smart engine behind it that automatically detects the current development framework and installs the correct version of Storybook.

Then to test the Storybook installation is successful, you may run this command:

npm run storybook

For more information on getting started with Storybook and React, visit this online document at https://www.learnstorybook.com/react/en/get-started.

Step 4 – Install Applitools

Step 4: Now that Storybook is running let’s install Applitools. Applitools offers the Applitools Eyes SDK for Storybook NPM package. Let’s add this package to our application by running this command:

npm i @applitools/eyes-storybook

In order to authenticate via the Applitools server, you need to supply the Eyes-Storybook SDK with the API key you got from Applitools. Read more about how to obtain the API key here.

To do this, set the environment variable APPLITOOLS_API_KEY to the API key before running your tests. For example, on Linux/Mac:

export APPLITOOLS_API_KEY=<your_key>

And on Windows:

set APPLITOOLS_API_KEY=<your_key>

Step 5 – Validate using Applitools

Step 5: Let’s run the first cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Once the tests are done running, let’s switch back to the Applitools Test Manager Dashboard to check and verify the results.

The left side of the Dashboard lists all batch test runs that you have previously performed. In the above screen capture, the single test run shows the status Passed. In handling this first run, the Applitools engine automatically passes this test and all snapshots thereafter as baseline snapshots.

Click on the single batch test to view the snapshots available. For each and every Storybook story, a snapshot is generated by Applitools.

Let’s take, for example, the Modal story test run. Locate the Modal story test run, click on the snapshot and expand the Window to view the snapper on a bigger scale. The expand button is available at the top-right corner of the Window as shown:

As you can tell, the test run snapshot captures an image snapshot for the story that runs inside the Storybook playground application.

At this moment of time, Applitools stores all the generated snapshots as baseline snapshots. The future test runs would generate a new set of snapshots and Applitools would compare them to the baseline ones and report the results accordingly.

Let’s switch gears and explore a unique gem of Applitools that is the Regression UI testing. Traditional regression testing validates that recent source code changes, deletions, or additions result in behavior that continues to align with expectations.

Applitools allows you to perform visual UI Regression testing. By using it, you can run multiple cycles of test runs and all visual changes collected from one to another, are detected by Applitools servers and logged into the Dashboard.

Steps 6-7 – Create And Validate A Change

Step 6: Let’s introduce a visual change in the Modal component and run our tests again.

Navigate to /components/Modal/Modal.jsx file and change the background-color of the Title Bar to yellow:

const TitleBar = styled.div`

  height: 18px;

  margin-bottom: 2px;

  background-color: yellow;

  color: white;

  padding: 2px 2px 0;

  display: flex;

`;

The change causes the title bar of the Modal window to change color to yellow.

Step 7: Let’s run the second cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Notice the logs on the terminal windows after running the command?

Modal: default [1024x768] - Failed 1 of 1

A total of 1 difference was found.

One of the tests failed and that’s exactly what we expect. Applitools detected the change we introduced to the background color of the Title bar and reported that accordingly.

Let’s switch back to the Applitools Test Manager Dashboard to check the results.

Applitools marks this new test run as Unresolved. Expand on the Modal test run to view the visual differences highlighted to you by Applitools:

Make sure to select the option to view both snapshots so that you can easily view the differences. To do so, locate the following menu and choose the option Both as shown in the figure:

Any test run that results in status Unresolved requires someone’s manual intervention to validate or reject the change. In this case, you may accept the new change(s) and thus update the baseline snapshots.

With an undesirable change, you infer a bug in the source code. You can reject this new test run and keep the original baseline snapshot without any changes.

See The Video Demo

Check out all the steps above in this step-by-step video of how to test React and Storybook:

Want To Know More?

Read about Angular and Storybook with Applitools.

How to troubleshoot and fix React bugs fast [step-by-step tutorial]

Find out more about Applitools. Request a demo, sign up for a free account, or check out our tutorials.

The post Visual testing for React and Storybook appeared first on Automated Visual Testing | Applitools.