Creating component and design systems are an important way to make sure you’re delivering consistent experiences to the people visiting your website or application. Storybook makes this easy with its JavaScript API, but how can we make sure we’re tracking the quality of those stories with automated testing?

Just looking for an example? Check out the available Applitools tutorials for React, Vue, and Angular on GitHub.com.

What is Storybook?

Storybook is a JavaScript-powered web tool that gives teams the ability to build component and design systems for their websites and applications.

By adding Storybook to your project, you can develop individual components of your UI in a focused, self-documenting environment, that helps everyone from design to development to product release carefully craft an experience.

It supports a variety of the development community’s most popular frameworks including React, Vue, and Angular. It also has a few documented ways to add testing to those components, but that typically involves meticulously writing tests that could be fragile and not necessarily representing what people are using in their applications.

That’s where Applitools and automated visual testing come in!

How does Storybook work with Applitools?

Pretty much out of the box, Applitools will give you full automated visual testing coverage of all of the components of your Storybook library.

By installing the Applitools Eyes Storybook SDK and running a simple command, Applitools will automatically pull in all of those stories, take a snapshot, and check for regressions in the Applitools cloud.

What are we going to build?

We’re going to start off with an existing React project that already has Storybook installed.

For more information about installing Storybook, visit the Storybook docs at: https://storybook.js.org/docs/react/get-started/install

In this walkthrough, we’ll be using a Storybook React application, but both Storybook and Applitools support more frameworks like Vue and Angular!

But with our project ready to go, we’ll first install the Applitools Eyes Storybook SDK using npm.

Once installed, we’ll show an example of how you can use Applitools Eyes to add Visual Testing coverage to your Storybook.

Step 1: Installing the Applitools Eyes Storybook SDK

To get started with the Applitools Eyes Storybook SDK on a project, first install the SDK from npm:

npm install @applitools/eyes-storybook --save-dev

This will pull down the SDK from npm and add it as a package to your project.

Step 2: Running Applitools Eyes for automated Storybook visual testing

Now that our SDK is installed, we actually don’t need to do anything else to get set up, we can dive right into running our tests!

To run our tests, make sure you first head over to the applitools.com dashboard and find your API key, then run the following in your terminal:

APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook

After that command finishes, you can see that by default, Applitools will go through each story, capture a snapshot, and check for visual differences.

Because this was our first time running our tests, all of our snapshots are new.

If we head over into our Applitools dashboard, we can even see these tests!

We can see that inside Applitools, all of our tests are new and green, which is a good thing. These will serve as the baseline for all of our tests.

Now to see the magic of how Applitools works, we can try breaking one of our components.

Inside Header.js,on line 30, let’s change our user condition to:

{!user ? (

We’re essentially switching around the logic, where if this was happening on production, we would want this bug caught!

Let’s run our tests again:

APPLITOOLS_API_KEY="[Your Key]" npx eyes-storybook

We can see this time, we have differences in our tests.

If we go into the Applitools dashboard, not only can we see that we now have Unresolved tests, we can open up one of those tests, and see the difference.

Applitools Eyes was able to see exactly where that issue was and highlight the change, so we can then see that there is a bug in our application!

Inside of the dashboard, we can click the “thumbs down” button, which tells Applitools we reject that change, meaning it failed.

But once we revert our application change and run our tests again, we can see that our tests are now passing again, because they’re working exactly how they should!

What else can you do with Applitools and Storybook?

Ignoring regions

One of the powerful features of Applitools is the ability to ignore specific regions of your visual tests by passing in selectors.

By using the ignoreRegions property inside of your Story’s eyes configuration, you can specify any selector you’d like to ignore in that Story.

Learn more about ignoring regions on npmjs.com

Specifying region types

In addition to simply ignoring regions, you can also set a custom match level for specific regions, such as if you wanted to define a region to include dynamic content, so that Applitools can use AI to understand if the content changed or if the layout changed.

Learn more about configuring regions for content, layout, and more on npmjs.com.

Get Started Today

Want to try out Applitools Eyes with Storybook for yourself? You can get started today at the link below – the account is free forever.

The post How to Add Visual Testingto Storybook appeared first on Automated Visual Testing | Applitools.

What is a design system? Who would use it, and for what benefit?

In May 2020, Applitools had the pleasure of hosting Tyler Krupicka from Intuit for an hour-long webinar discussing design systems and testability.

Tyler works at Intuit, a 9,400 employee company headquartered in Mountain View, California, that specializes in accounting and tax preparation software. At Intuit, Tyler works on the “Player/Design Systems” team, where he focuses on design systems.

Tyler has been working on web development at Intuit since 2015. These days, he works on UI components, tools, and testing. He primarily uses TypeScript, React and Vue. He also focuses on accessibility, and he serves as a “Level 2” Accessibility Champion inside Intuit.As part of Tyler’s job, he ensures that design systems are testable. 

What Is A Design System

Tyler started asking this question. Some people think about large design systems:

• Material Design (Google’s design system)
• Human Interface Guidelines (Apple’s design system)
• Bootstrap (set of UI interface components and CSS that are popular on the web)

People look at these and think, “Is this overkill?” After all, Google uses Material Design for Android. How many people design an entire operating system. So, Tyler dove in to answer his question.

First, Tyler sees design systems as really “Design + Code.” One part of a design system are the “design” – the common components and patterns that get used throughout products. The other part of the design system, Code, involves chunks of functionality interacting with design developers can reuse across a product.  For example, CSS Frameworks, or React, or in some other way that lets teams reuse the same code.

Design

“Design” turns into a consistent voice for your product. Those are things like typography, colors, and spacing, as well as UI components (forms, charts, site navigation), Graphics and Animation standards. These help standardize the visualization – the “look” – of the application. One can imagine an application developed by a large team that lacked a standard design – it would be disorienting to customers who expect consistency from software. A design system lets  you take base pieces, document them well, and reuse them.

Code

“Code” lets you build and document components individually, test them once, and have a way to set expectations on whether or not this application will work.  For instance, you can build a component in the app for one persona and reuse that component multiple times inside the app without worrying that you must retest it. Reuse makes code standard and creates standards for interacting with design. If you ever change design elements, the standard code reduces the effort to validate the new design with existing code. 

Combining design and code into a design system results in increased developer productivity and a consistent voice for the application. And using a design system simplifies the management of an application.

Design Systems at Intuit: TurboTax Example

Next, Tyler dove into the use of design systems for the Intuit TurboTax product. 

Tyler explained how TurboTax works. By providing users with interview-style questions, TurboTax helps millions of people, primarily in the United States and Canada, file their income tax returns. Intuit offers both web and mobile versions of TurboTax.

Because TurboTax uses interview questions to create conditional flows through the product, there are thousands of screens users can encounter based on their unique needs. Screens can range in complexity from a single question to a complete form. 

Building the TurboTax Design System

To create the design, Intuit develops about 50 UI components needed to run the experience. Headers, radio buttons, tiles, multi-select, data entry, action buttons – all become part of the basic UI. 

Intuit also builds a set of about 500 “mocks”, or mock-ups, of what common screens look like for testing. These help show the UI components in common layouts for testing purposes.

By using a design system, Intuit can make changes to the application and ensure consistency for all these screens. 

Tyler gave the example of a decision to deliver a uniform increase in contrast in TurboTax for accessibility. The contrast changes required updates to typography weight, button color, and link color. This work would impact over 90% of all the screens. How could Intuit deliver these changes to the application and ensure they occurred consistently across the application? The team did not have the staff needed to manually check every possible screen. Instead, they made the design changes to the UI components and validated the mocks.

Testing A Design System

Next, Tyler dove into a discussion of testing a design system. 

Approach

Since the design system represents all the functionality and visual representation of the application, Tyler recommends going broad with testing across all the elements, including:

• Functionality
• Visual
• Performance
• Accessibility

From a functional perspective, unit tests serve as key building blocks for code. Integration tests ensure that the code works wiht the back end. Cross-vrowser tests help ensure that the code works on a bunch of different browsers and mobile devices. 

Visual testing validates the impact of UI component changes on the overall UI. If buttons change size or Intuit uses a new font size, does the page continue to render correctly? Do unexpected breaks happen on pages? Do space reductions make pages look too constrained?

Performance testing involves checking the impact of code changes on page loading times.  Specifically, do new packages or package upgrades change the amount of data transferred between the server and the browser – slowing the user experience? 

Finally, Tyler tests accessibility to make sure that any customer who wants to use the product can do so. Whether a user has a screen reader or navigates solely by keyboard, ensure that the user can get around.

Getting Organized

With testing goals articulated, Tyler introduced Storybook as the tool for creating documentation and initial testing. If you use projects like React, Vue and Angular that allow you to build user interfaces in small components, the Storybook project lets you build interactions for those components and document their behavior.  

Effectively, you demonstrate a  use case for your components and show how they all interact together on a page.  Each use case is one story – hence Storybook is a collection of these stories. 

Storybook allows for a range of tests, including accessibility tests. For instance, Tyler uses Storybook to test keyboard navigation and focus. He also tests for things like hover behaviors – where components appear while the cursor hovers over another element, which may be difficult to test with automation. Testing with Storybook lets the testing team run tests early across the range of design system dimensions.

Dog Fooding

To deliver successful products, Tyler’s team must support the construction and use of these UI components. So, they actually write code to try out the components – just to see how they work. Putting together checkboxes or radio buttons, or drop-down lists helps show how the code interacts with components and creates mocks for testing the design system.

Tyler showed the example of a page that looked like static boxes. Each box expanded when clicked to show hidden detail and slid the rest of the page down. His team dog foods this kind of behavior.

All of these activities help create a body of testable code, so that a design change in the design system can be mocked up and validated without impacting production code.

Testing Walkthrough

Tyler then walked us through a simple change that can have a signficant implications – resizing a button. 

Without a design system, you might have to make a button change like this ad-hoc across your application. As a result, you risk changing the page-to-page behavior and look-and-feel. Using a design system lets you make the change globally, and then focus on the implications.

What kinds of implications? A larger button might displace elements on a page. The button may not function as expected with existing elements on a given page. All these changes need to be tested.

Unit Tests

First, Tyler looked at unit tests. Does the button click? Can the button work with an icon associated with it? Will the button work with an transparent background? Does the button work with assigned ARIA (Accessible Rich Internet Applications) attributes? Testing puts these Storybook components through their paces.  

Intuit have a pretty standard way of running through these tests.  Tyler talked about using JEST and WebdriverIO and Selenium to automate this testing across multiple browsers. They have lots of users on multiple web browsers, as well as Mac users with Safari an iOS, so they pay special attention to those target platforms. 

Accessibility Tests

Next, Tyler talked about accessibility testing. His team uses AXE to perform static validation for ARIA standards on application pages, including contrast checking.  Tyler explained how AXE helps perform contrast checking. For instance, if a button is one color, does the background let the button stand out? AXE can do the contrast checking by comparing the two colors. Or, will the text be readable against its background – or is it too small or too thin? Again – AXE can make these calculations.

Tyler also talked about HTML customizations. For example, you want to use a special radio button design that isn’t included in the existing HTML. You build this behavior as a custom implementation – and then you have to validate this behavior across the browsers your customers use. Tyler pointed out that companies make these decisions to help brand their websites for customers. And, these custom behaviors must be validated for functionality and accessible behavior across multiple platforms.  At the same time, since you are replacing functionality built into HTML, keyboard navigation still needs to behave as expected. So, you need to test those behaviors. 

At Intuit, the team tests these behaviors manually – and with tools designed to help. By using popular screenreaders like Jaws, Voiceover, and NVDA, Tyler’s team ensures that these screen readers don’t get thrown off by customizations. Also through manual testing, the team checks to ensure that keyboard navigation results in expected page responses – for example, they make sure that focus states behave as expected. And, they also ensure that the page behaves correctly when a customer has selected “reduced motion”.

Performance Tests

From a design perspective, the design system simplifies app delivery. By using standard libraries and building standard behaviors, the design system creates this great look-and-feel. At the same time, developers need to consider the impact on performance. For instance, a specific package might add great mobile behavior – at the cost of an additional 1-2 seconds of page load time. Users likely won’t value the behavior over slower page load time.  

You can choose a great library that gives you great functionality – but at what cost? How does the size of your Javascript package impact your page downloads? Often, this problem creeps up on you.  You use an early version of the library and everything works well. But, when you update your application, your library update suddenly doubles in size. This happens quite often. New versions of libraries include new features – which add to the library load times. And, you have to rethink the software you’re using.

Tyler talked about how Intuit handles these kinds of challenges. If Intuit uses code from an open source project, they might contribute back to the project to make a library with reduced size. Or, they might use a different open-source package.  

Visual Testing

Each time a new component gets introduced or modified, Intuit runs tests through Applitools Ultrafast Grid. Intuit has been running Grid since its first beta tests. With Ultrafast Grid, Intuit can set checkpoints and take screenshots across a range of browsers and devices. They can set multiple screen sizes (from mobile to Full HD and  higher). Their goal – ensure that responsive pages behave as expected, and understand the impact of design changes on existing pages. Ultrafast grid gives them the capability to easily compare changes across different versions of an application.

Tyler talked about using Ultrafast Grid to test against Chrome, Edge, and Safari. He talked about setting different viewport sizes to check responsive page behavior against expectations. 

Component and Mock Level

Tyler explained how Intuit tests components and mocks. 

In Storybook, Intuit keeps the latest components, as well as page mocks for each component in use. When a change occurs in the design system, Intuit can do two kinds of checks:

1. They can check the components to see if the component changes have violated any specifications
2. They check the mocks, to determine how the component changes affect page rendering.

So, for instance, Intuit can see if a collection of multiple new components violate spacing guidelines or other design requirements on pages. Applitools Ultrafast Grid helps Intuit do this evaluation.

Also using the mocks, Intuit can also see if a change violates accessibility guidelines using AXE.  

Accessing Intuit Tools for Design Systems

Tyler discussed the two tools Intuit uses to ensure success in their design systems.

The first, the Intuit DS-CLI, makes it easy to implement Storybook for your designs. Just like Create-React-App lets you build a web app easily, DS-CLI contains all the code you need to utilize Storybook for component and mock management. This includes preconfigured Storybook, component templates with best practices, great Typescript support, and bundle size tracking.  Tyler mentioned that DS-CLI is free and open source. 

One great part about DS-CLI is size tracking.  Each component can be tracked over time for package size. You can see when your bundles are growing – and why.

With the second, Intuit Proof, you can automate the development of test running in Storybook. Proof is a tapable testrunner for Storybook. Proof helps you automate tests for components and mocks by helping you know which ones have tests. In addition, Proof has built-in hooks for Applitools and Axe. So, instead of fumbling about inside Storybook to see what has been tested and what remains, you know what tests exist, which still need to be written, and helps you track their status. 

Proof is also free and open source.

Conclusion

Tyler’s presentation made it clear that a design system can make your life much easier, but you need to invest in tools and methodology to help ensure consistency – the benefit of a design system. 

You need a component evaluation system. You need mockups of your pages to test. And you need a testing environment.  Intuit does all this inside Storybook.

Next, you need a testing strategy to get components evaluated and integrated into your pages. Invest in unit testing and test runners. Especially if you care about being perceived as open to all users – invest in accessibility tools. And, finally, know that rendering matters – invest in visual evaluation solutions like Applitools.

Finally, you need some pieces of infrastructure to make your design system work. Intuit shares theirs open source.

If you make these investments up front, a design system can serve your development and test strategy to help you easily develop incremental improvements across your application without fear of code changes. 

Read More

• Impact of Visual AI on Test Automation Report
• A/B Testing: Validating Multiple Variations
• Modern Dev Teams Are More Efficient Than Ever — Quality Engineering Should Be Too.
• The Next Generation of Cross Browser Testing is Ultrafast
• Sign up for a free Applitools account
• Request an Applitools Demo

The post Design Systems and Testability appeared first on Automated Visual Testing | Applitools.

ReactJS provides a library of reusable components for building JavaScript applications. When combined with Storybook and Applitools Eyes, you can validate the visual appearance of your React components in a controlled testbed. With Applitools Ultrafast Grid, you can run your visual validations across a range of browsers and viewport sizes across multiple screen combinations simultaneously. Together, React, Storybook, and Applitools make you much more productive by developing once in React, testing once in Storybook, and rapidly validating everywhere in Applitools.

React Development and Test Best Practices

When developing a React UI component library, you are likely aware that a well-known best practice splits the components into self-contained js/jsx files. Splitting benefits you, whether you are the developer developing the components or another developer consuming those components in your apps.

Once you develop using standalone components,  you need to consider how to test your components inside a browser during the development phase. You have a few options here. One of them is to develop a demo or showcase application. You can use the showcase app to render components and make sure they run correctly inside the browser. However, you end up coding something you likely won’t use again – and that seems like a waste of resources.

The best option is to make use of a third-party playground framework such as Storybook that can kick off a new web server for you and run all the components under development. Also, Storybook gives you the option to generate a static documentation playground website that you can host online to showcase your components live.

As you develop components, you revise components, change their code, and test them to verify nothing broke with the new changes. Some of your changes will be functional, and you can create new functional tests for those components. However, the visual changes are problematic. Manual validation of all of them is cumbersome, time-consuming, and inaccurate. That’s where visual UI testing frameworks come in handy.

So, let’s talk Applitools. Not only does Applitools provide you with the means to do Visual UI testing, but you can also use for Visual UI Regression testing. That is heavenly in terms of Visual UI testing.

In this article, I’ll walk you through a complete, step by step guide to visually test a React app with Storybook and Applitools.

What is Applitools?

If you’re not already familiar, Applitools is an automated visual regression testing framework. It focuses on the visual aspects of your app and plays a significant role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of development and testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools, to capture both baseline and test screenshots.

So, let’s get started with some visual UI testing. Your visual testing project will depend on React, Storybook, and Applitools. You will also need node.js tools. So, firstly, make sure you’re familiar with the following and have them installed on your machine:

• Git
• Node.js and npm
• React
• Storybook
• Applitools

Before we immerse ourselves in writing code, let’s look at how Applitools and Storybook work together.

How Applitools Works With Storybook

No matter whether you use Vue.js, React, or Angular, Storybook doesn’t offer any extension points for third-party frameworks, such as Applitools, to integrate with. The Applitools team worked around this limitation by providing the Applitools Eyes Storybook SDK.

The behavior of this SDK is that once you run it locally on your machine, it will start by collecting all stories in the application, run them all via the Storybook engine, generate DOM snapshots for each and every story and upload these snapshots to the Applitools Eyes server. Here’s a visual of this flow:

Tests run on the server to generate actual image snapshots. Each snapshot is compared to a baseline snapshot (if one exists), and the results are displayed on the Test Manager Dashboard.

This kind of integration provides a smooth experience in terms of writing Storybook stories. Nothing changes in the way you write your React apps.

Now, let’s get on with it and start coding …

Step By Step Guide

For this demonstration of visual testing a React application with Storybook and Applitools, I’ve chosen the React95 open-source library hosted on GitHub under this repo. This library is both unique and fun at the same time. It creates a rich set of components having the theme of Windows 95. This library already uses Storybook and by adding Applitools to the mix, should be much easier and less time-consuming.

Let’s follow the steps to run the React95 Storybook stories and use Applitools to generate a few visual UI test runs.

Steps 1-3 – Start with a React Library and Install Storybook

Step 1: Clone the React95 GitHub repo locally on your machine by running this command:

git clone git@github.com:React95/React95.git

Step 2: Install all NPM package dependencies by running this command:

npm install

Step 3: As you now know, this library already uses Storybook. Hence, the scripts section of the package.json file already contains the script to run Storybook. Let’s give it a shot and run Storybook stories by issuing this command:

npm run storybook

As a side note, in case you are installing Storybook from scratch, here’s a mini-guide to help you get started adding Storybook to your application.

Run the following command to install Storybook into your application:

npx -p @storybook/cli sb init

The Storybook NPM package {{ link to the NPM package here }} has a smart engine behind it that automatically detects the current development framework and installs the correct version of Storybook.

Then to test the Storybook installation is successful, you may run this command:

npm run storybook

For more information on getting started with Storybook and React, visit this online document at https://www.learnstorybook.com/react/en/get-started.

Step 4 – Install Applitools

Step 4: Now that Storybook is running let’s install Applitools. Applitools offers the Applitools Eyes SDK for Storybook NPM package. Let’s add this package to our application by running this command:

npm i @applitools/eyes-storybook

In order to authenticate via the Applitools server, you need to supply the Eyes-Storybook SDK with the API key you got from Applitools. Read more about how to obtain the API key here.

To do this, set the environment variable APPLITOOLS_API_KEY to the API key before running your tests. For example, on Linux/Mac:

export APPLITOOLS_API_KEY=<your_key>

And on Windows:

set APPLITOOLS_API_KEY=<your_key>

Step 5 – Validate using Applitools

Step 5: Let’s run the first cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Once the tests are done running, let’s switch back to the Applitools Test Manager Dashboard to check and verify the results.

The left side of the Dashboard lists all batch test runs that you have previously performed. In the above screen capture, the single test run shows the status Passed. In handling this first run, the Applitools engine automatically passes this test and all snapshots thereafter as baseline snapshots.

Click on the single batch test to view the snapshots available. For each and every Storybook story, a snapshot is generated by Applitools.

Let’s take, for example, the Modal story test run. Locate the Modal story test run, click on the snapshot and expand the Window to view the snapper on a bigger scale. The expand button is available at the top-right corner of the Window as shown:

As you can tell, the test run snapshot captures an image snapshot for the story that runs inside the Storybook playground application.

At this moment of time, Applitools stores all the generated snapshots as baseline snapshots. The future test runs would generate a new set of snapshots and Applitools would compare them to the baseline ones and report the results accordingly.

Let’s switch gears and explore a unique gem of Applitools that is the Regression UI testing. Traditional regression testing validates that recent source code changes, deletions, or additions result in behavior that continues to align with expectations.

Applitools allows you to perform visual UI Regression testing. By using it, you can run multiple cycles of test runs and all visual changes collected from one to another, are detected by Applitools servers and logged into the Dashboard.

Steps 6-7 – Create And Validate A Change

Step 6: Let’s introduce a visual change in the Modal component and run our tests again.

Navigate to /components/Modal/Modal.jsx file and change the background-color of the Title Bar to yellow:

const TitleBar = styled.div`

  height: 18px;

  margin-bottom: 2px;

  background-color: yellow;

  color: white;

  padding: 2px 2px 0;

  display: flex;

`;

The change causes the title bar of the Modal window to change color to yellow.

Step 7: Let’s run the second cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Notice the logs on the terminal windows after running the command?

Modal: default [1024x768] - Failed 1 of 1

A total of 1 difference was found.

One of the tests failed and that’s exactly what we expect. Applitools detected the change we introduced to the background color of the Title bar and reported that accordingly.

Let’s switch back to the Applitools Test Manager Dashboard to check the results.

Applitools marks this new test run as Unresolved. Expand on the Modal test run to view the visual differences highlighted to you by Applitools:

Make sure to select the option to view both snapshots so that you can easily view the differences. To do so, locate the following menu and choose the option Both as shown in the figure:

Any test run that results in status Unresolved requires someone’s manual intervention to validate or reject the change. In this case, you may accept the new change(s) and thus update the baseline snapshots.

With an undesirable change, you infer a bug in the source code. You can reject this new test run and keep the original baseline snapshot without any changes.

See The Video Demo

Check out all the steps above in this step-by-step video of how to test React and Storybook:

Want To Know More?

Read about Angular and Storybook with Applitools.

How to troubleshoot and fix React bugs fast [step-by-step tutorial]

Find out more about Applitools. Request a demo, sign up for a free account, or check out our tutorials.

The post Visual testing for React and Storybook appeared first on Automated Visual Testing | Applitools.

ReactJS provides a great library for app JavaScript development. Cypress is a great app functional UI test framework. Add Applitools, and you gain productivity in visual testing that allows you to deliver visual testing your app across a  range of browsers and operating systems. Put them together, and React plus Cypress plus Applitools is a great combination to develop, functionally test, and visually test your application.

You are reading this article because you either use ReactJS today or have considered using the React library for building front-end components using JavaScript. And, while I assume you are using Cypress.io for functional testing, you could just as easily be using a different toolset and are looking at Cypress asking, “What’s different?”

And, of course, the big reason you are reading this article is to answer the question, “What tools out there give me the productivity gains for visual testing cross-platform that I get when I develop across multiple browser-OS combinations with React and Cypress?” You want to know about Applitools.

Applitools for Visual Test Productivity

Applitools offers a test framework to automate visual testing of your application. It focuses on the visual aspects of your app and plays a major role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools.

In my previous post, I showcased the beauty of the Applitools Ultrafast Grid system and demonstrated how Applitool integrates fluidly with Storybook inside an Angular app. With all the attention on React visual UI testing, I thought it is appropriate to share step by step tutorial on this topic. Specifically, React UI testing using Applitools and Cypress.io.

First, ensure that these are properly installed on your machine:

• React JS
• Cypress.io
• Applitools

Before we delve into writing code, let’s take a behind-the-scenes look at how Applitools and Cypress.io integrate together.

Applitools / Cypress.io Integration

Applitools integrates with Cypress.io and offers a general-purpose library that can be embedded within your project, whether it be the Angular, Vue, or React app.

The Applitools Eyes Cypress SDK is a simple plugin to Cypress. Once installed, it adds a few commands to the main cy object. Specifically, it adds three main methods: cy.eyesOpen to start the test, cy.eyesCheckWindow to take screenshots (for each test step), and cy.eyesClose to close the test. Here’s a diagram:

Now that you understand how the Applitools/Cypress.io integration works, let’s switch gears and start coding a few visual tests for a React app using Applitools and Cypress.io.

Source code

For this article, I’ve chosen to write a few E2E Cypress.io tests against an open source project written in React JS. The Product Compare app allows the user to select a couple of products and compare them side by side.

Demo Steps 1-5: Get Started

I will take you on a series of steps to install Applitools and Cypress.io on an existing React JS app, add a Cypress.io test, and finally, verify the results on the Applitools Test Manager.

Step 1: Clone the repository locally by issuing the following git command:

git clone git@github.com:Rhymond/product-compare-react.git

Step 2: Install all the npm dependencies by issuing the following command:

npm install

Step 3: Run the app:

npm run start

And you should see something like this:

Voila. It works!

Step 4: Add Cypress package to the project:

npm install --save-dev cypress

The Cypress npm package adds a set of test files to help you write your own automated tests.

Step 5: Open the Cypress Dashboard by issuing the following command:

node_modules/.bin/cypress open

This command opens the Cypress Dashboard and also creates a cypress.json file and a cypress folder in your app’s root directory. The cypress folder is where we will be writing our tests.

Demo Steps 6-8: Add Cypress Tests

Step 6: Add a new, shorter script inside the package.json file to open Cypress Dashboard without having to type that long command you used in the previous step:

"cypress": "cypress open"

In addition, inside the cypress.json file in the root of the application folder, add the following:

{

     "baseUrl": "http://localhost:3000"

}

Now that Cypress is running properly, let’s add our first E2E test.

Step 7: Inside the cypress\integration folder, create the compare.spec.js file and paste the following:

View the code on Gist.

The single E2E test focuses on testing the functionality of adding two products to compare them side by side. There are many tests that can be written, but for our sake, we will play around with a single E2E test to demonstrate the integration of Applitools with Cypress.io.

The spec file is self-explanatory. There’s just one thing to note here in relation to the selectors used inside cy.get() methods. The best practice is to avoid using IDs and CSS classes to select elements from the DOM. They will make your tests brittle because they’re subject to change. A better approach is to use data-* attributes or actual component names.

Now, switch back to the Compare\index.js and Product\index.js files and go through all the data-* attributes I’ve added to facilitate writing this E2E test.

Step 8: Run the single E2E test by issuing the following command:

npm run cypress

Great! Our E2E test successfully passes, verifying that the app functions well and can select multiple products to compare them without any issues.

E2E tests are sufficient in testing the overall Web app. However, wouldn’t it be beneficial and more effective to complement the E2E tests with visual ones? This adds the value of detecting any visual change or difference upon running E2E regression testing.

As you already know, Applitools integrates seamlessly with Cypress.io, so let’s add the Applitools Cypress NPM package to the project.

Demo Steps 9-12: Install and integrate Applitools Eyes

Step 9: Add the Applitools Eyes Cypress SDK package to the project.

npm install @applitools/eyes.cypress --save-dev

Step 10: Configure the Applitools Eyes Cypress SDK by issuing the following command:

npx eyes-setup

Now that Applitools Eyes Cypress SDK is configured, we can start using it in our tests.

Let the visual testing begin!

Step 11: Revisit the cypress\integration\compare.spec.js file and replace its content with the following:

View the code on Gist.

The code is self-documented.

To integrate Applitools Eyes Cypress SDK into a spec file, you’ll follow the Workflow below:

Start a new test

cy.eyesOpen({

   appName: '...',

   testName: '...',

   browser: { ... },

});

Take a snapshot (You may repeat this step wherever you want to take a snapshot)

cy.eyesCheckWindow('...');

End the test

cy.eyesClose();

Notice the Applitools assertions at lines 1, 7, 12, and 17? This type of logging informs you about the steps Applitools is taking while Cypress runs your tests.

Step 12: Check the test run in Applitools Test Manager:

Clicking on the test name (in this case, Select 2 products to compare) yields the two snapshots that the Eyes SDK has taken during the running of the test.

The first snapshot is labeled first product selected. The image shows the Cherry product selected.

The second snapshot is labeled second product selected and shows the Orange product selected.

Since this is the first run of the test, Applitools will save these as the baseline.

We will simulate a visual difference by introducing an underline to all prices displayed on the page and let Applitools detect this. Now run the test again and see the results.

Demo Steps 13-15: Introduce Changes and Validate

Step 13: Open the src/components/Product/styles.css file and change the following CSS class as follows:

.product .stats-container .product_price {

   float: right;

   color: #48cfad;

   font-size: 22px;

   font-weight: 800;

   text-decoration: underline;

}

Let’s run the test.

Step 14: Issue the command to run the E2E tests again. The test case fails, and Applitools detects a visual change for the product image overlays.

Notice how the Applitools Test Manager recorded the second run of the test case and highlighted the visual difference between the two snapshots?

Step 15: Click on the first snapshot and compare to the baseline.

In case you can’t see both snapshots side by side, locate the View menu and select Show both.

In addition to the capacity to visually test our React JS apps, Applitools offers more functions and features to give you the upper hand in testing your web apps quickly and locate the visual differences easily.

To access the source code of this article, feel free to grab a copy of it here: product-compare-react.

Conclusion

You now have some insight on how Applitools speeds up your visual testing experience when you’re developing apps with React JS and Cypress.

In upcoming articles, I will explore Applitools further, unveil more features, and give you the power to carry on with E2E and Visual UI Testing easily.

Happy Testing!

Some Of My Recent Posts

Here are some of the previous Applitools topics I’ve discussed:

1. Applitools – The automated visual regression testing framework
2. Mixing Storybook with Angular with a sprinkle of Applitools
3. Troubleshoot and fix React bugs fast
4. Visually test VueJS apps using Cypress.io and Applitools
5. How I ran 100 UI tests in just 20 seconds

For More Information From Applitools

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post React, Cypress.io, and Applitools for Functional and Visual Testing appeared first on Automated Visual Testing | Applitools.

I’ve been playing around with Applitools for quite some time now, and I’d like to share what I’ve found to help you quickly troubleshoot React bugs and fix them fast.

My most recent articles in this series showed you how to integrate Applitools with Angular and Storybook. In case you missed these, you can find them here:

• Applitools – The automated visual regression testing framework

• Mixing Storybook with Angular with a sprinkle of Applitools

If you’re not already familiar, Applitools is an automated visual regression testing framework. It focuses on the visual aspects of your app — as opposed to functional testing — and plays a major role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of testing frameworks, such as Cypress.io, Storybook, and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools, to send both baseline and test screenshots.

This article delves into a great new feature offered by Applitools: Automated Root Cause Analysis (RCA). The demo section will use a ReactJS app, together with Cypress.io, to demonstrate the RCA feature and its application.

Disclaimer: The main goal of this article is simply to introduce you to the Applitools RCA feature. If you are looking for an in-depth tutorial on the topics mentioned in this article, I recommend the following links:

• React JS
• Cypress.io
• Applitools
• Applitools RCA

Automated Root Cause Analysis

The concept of Root Cause Analysis stems from the world of management, where it’s defined as a method of problem-solving used for identifying the root causes of faults or problems.

So far, Applitools has been focusing on visual testing by comparing the baseline and the current test run snapshot through their AI engine. It finds the visual differences in order to map them graphically. There was previously no way to search your codebase for a reason behind any visual testing bug. As you may know, searching through code is nightmarish, time-consuming, and tedious!

This is where Applitools steps in with RCA, showing you the exact DOM and CSS changes between the baseline snapshot and the current test run snapshot.

Why is this so important for both developers and QA testers? Let’s take a look.

Demo

I’ll demonstrate the RCA feature using the Calculator project, previously published as part of the example projects on the React JS website.

Next, we’ll follow these steps to run the Calculator and add a few Cypress test cases.

Step 1: Clone the repository locally by issuing the following git command:

git clone git@github.com:ahfarmer/calculator.git

Step 2: Install all the npm dependencies by issuing the following command:

npm install

Step 3: Run the app:

npm run start

And you should see something like this:

Voila. It works!

Step 4: Add Cypress package to the project:

npm install --save-dev cypress

The Cypress npm package adds a set of test files to help you with writing your own automated tests.

Step 5: Run the Cypress tests available now in the project by issuing the npx Cypress CLI command:

npx cypress run

Now that Cypress is running properly, let’s add the Applitools Eyes SDK for Cypress package.

Step 6: Add the Applitools Eyes Cypress SDK package to the project:

npm install @applitools/eyes.cypress --save-dev

The Applitools Eyes Cypress SDK is a simple Cypress plugin. Once installed, it adds a few commands to the main cy object.

More specifically, it adds three main methods: cy.eyesOpen to start the test, cy.eyesCheckWindow to take screenshots (for each test step), and cy.eyesClose to close the test.

Let’s write our first Cypress test case to simulate adding two numbers and validating the result of the addition operation.

Step 7: Inside the cypress\integration folder, create the addition.spec.js file and paste the following:

View the code on Gist.

The spec file is self-explanatory. However, there’s one thing to note here. In terms of the selectors used inside cy.get() methods, the Calculator app never uses any ID or Class to distinguish buttons.

I used a nice feature offered by Google Chrome DevTools to copy the selector of the buttons I am interested in.

To do so yourself, simply follow these steps:

1. Right-click on the button.
2. Select “Inspect.” Chrome’s DevTools opens on the Elements tab with the element highlighted in blue.
3. Right-click the highlighted element.
4. Select “Copy” > “Copy selector.”

Next, let’s run the spec file to make sure our test runs successfully.

Step 8: Run the spec file:

npx cypress run --spec="cypress\integration\operations\addition.spec.js"

Now that our spec file runs successfully, let’s make use of the Applitools Eyes Cypress SDK commands to capture a few snapshots.

Step 9: Add the Applitools Eyes Cypress SDK commands to the addition.spec.js file:

View the code on Gist.

The code is self-documented.

Now, to integrate Applitools Eyes Cypress SDK into a spec file, you follow the Workflow below:

Start a new test

cy.eyesOpen({
    appName: '...',
    testName: '...',
    browser: { ... },
});

Take a snapshot (You may repeat this step wherever you want to take a snapshot)

cy.eyesCheckWindow('...');

End the test

cy.eyesClose();

Step 10: Run the spec file:

npx cypress run --spec="cypress\integration\operations\addition.spec.js"

Step 11: Check the test run in Applitools Test Manager.

Notice the Add two numbers test run on the grid. Clicking this test run opens the three snapshots that were taken when we ran the Cypress spec file.

The first snapshot is labeled “Number 8 clicked.”The image shows the number 8 in the display of the calculator.

The second snapshot is labeled “Number 7 clicked.” This one shows the number 7 in the display.

Finally, the third snapshot is labeled “Display value of 15.” The image shows the number 15 in the display of the calculator.

Since this is the first test run, Applitools will save these as the baseline images.

Next, we will simulate a visual difference by changing a CSS selector and letting Applitools detect this change. Then, we’ll run the test again and see the results.

Step 12: Let’s assume that the CSS selectors were changed and caused a change in the background of the Number 8 button. Add the following CSS selector to Button.css file as follows:

div.component-button-panel > div:nth-child(2) > div:nth-child(2) > button {
  background-color: #eee;
}

Ready? Let’s run the spec file once again.

Step 13: Run the spec file. The test case fails, as expected, and Applitools detects a background color change for the Number 8 button.

npx cypress run --spec="cypress\integration\operations\addition.spec.js"

Notice how the Applitools Test Manager recorded the second run of the test case and highlighted the visual difference in the three snapshots.

Step 14: Click on the first snapshot and compare to the baseline.

Applitools Test Manager displays both the baseline snapshot and the current test case run. Also, the number 8 button is highlighted to signala visual difference.

Step 15: Click on the RCA tool icon (highlighted in the red rectangle) to get a full analysis of the source of this visual bug.

Next, you will be asked to select the visual difference you want the RCA to assess.

The RCA tool runs and shows the root cause of this visual difference, which is a change in the CSS Rules.

The RCA tool can also detect DOM changes. Simply use the RCA Pointer Tool to select two buttons: 7 on the baseline (left side) and 5 on the current test run snapshot (right side). The results are in the image below.

In the Text content, the RCA shows the difference between the two buttons. It also gives details about the bounding box for each button and the position.

Let’s change other DOM attributes and run the test again.

Step 16: Next, locate the src/component/Button.js file and add a class=”btn” to the HTML button:

<div className={className.join(" ").trim()}>
   <button class="btn" onClick={this.handleClick}>{this.props.name}
   </button>
</div>

Now, let’s run the test and check the results.

Step 17: Run the spec file. The test case fails, and Applitools detects a change in attributes for the buttons used to form the Calculator.

npx cypress run --spec="cypress\integration\operations\addition.spec.js"

Another category RCA detects is the Attributes category. It shows that in the current test run, each button has a new class attribute set.

Applitools RCA is both powerful and convenient: It can pinpoint out the exact DOM/CSS change. Then, all you have to do is locate the codebase and do whatever is required to pass your tests.

Remember that you can always accept the current test run and update the baseline, or reject the test run and keep the existing baseline snapshot.

RCA supports a set of DOM/CSS categories of change, including:

• Textual changes
• CSS Property changes
• Attributes changes
• Bounding Box changes
• Tag changes (When an entire Tag gets changed from one test run to another. For instance, when a button is replaced by a hyperlink.)

Conclusion

The advent of Applitools RCA is a game-changer! Not only can you use Applitools for validating visual differences in your web app, but you can also make use of RCA to go in-depth and find the exact source of a visual difference bug with ease and minimal effort.

Happy Testing!

How much time will you save with Root Cause Analysis? Let us know in the comments.

The post How to troubleshoot and fix React bugs fast [step-by-step tutorial] appeared first on Automated Visual Testing | Applitools.

Watch this webinar with Kent C. Dodds about frontend testing, and prepare to re-think everything you think you know about shipping apps with confidence. We want to make sure that when we ship new code, our users can use the application. The best way we’ve found to do that is to write automated tests that run before we deploy a new version of the app.

But there’s a catch with frontend testing: if our tests aren’t doing exactly what the user will do, then how do we really know that things will work when users interact with our apps?

Expert Kent C. Dodds makes us rethink everything we know about frontend testing: what it means to ship applications with confidence, and what it takes to get there. With improved tools and practices, we can be more certain that what we’re shipping to our users will work the way it’s intended. Watch his full session right here:  

Kent’s slide-deck:

Slides: https://kcd.im/confident-react

Full webinar recording:

General Resources – Recommended by Kent:

• Kent’s Twitter (he does not don’t use LinkedIn): https://twitter.com/kentcdodds
• The Biggest Online Testing Workshop (you want to get this!): https://testingjavascript.com
• Kent’s Ask Me Anything: https://kcd.im/ama
• Kent’s Newsletter: https://kcd.im/news
• Kent’s 3 Minute (week)daily Podcast: https://kcd.im/3-mins
• My YouTube channel: https://kcd.im/youtube
• What We Can Learn About Testing From The Wheel – Kent C. Dodds (Ignite Fluent 2016): https://kcd.im/wheel
• Kent’s (week)daily livestream playlist: https://kcd.im/devtips
• Blog (LOTS of testing content here): https://kcd.im/blog

Kent’s Recommended Reading List & Referenced Blog Posts:

• Testing Implementation Details: https://kcd.im/imp-deets
• Write tests. Not too many. Mostly integration (there’s also a recording of a talk version of the blog post): https://kcd.im/write-tests
• Why I Never Use Shallow Rendering: https://kcd.im/shallow

Kent’s Referenced Tools:

• Jest: https://jestjs.io
• Cypress: https://cypress.io
• dom-testing-library: https://testing-library.com
• react-testing-library: https://testing-library.com/react
• cypress-testing-library: https://testing-library.com/cypress

Additional Reading [written by Sr. Architect Gil Tayar]:

• Visual UI Testing at the Speed of Unit Testing, with new SDKs for Cypress and Storybook [on-demand webinar]
• How to Visually Test React Components Using Our Storybook SDK [how-to post]
• Cypress vs Selenium WebDriver: Better, or Just Different? [use-case post]
• Making Your Website/Application Accessible: a Guide for Front End Developers [how-to post]
• Comparing JavaScript Browser Automation Frameworks: Selenium Versus Webdriver.io Versus Puppeteer [use-case post]

About Kent C. Dodds:

Kent C. Dodds works at PayPal as a full stack JavaScript engineer. He represents PayPal on the TC39. He’s actively involved in the open source community.

Kent is the creator of TestingJavaScript.com and he’s an instructor on egghead.io and Frontend Masters. He’s also a Google Developer Expert.

Kent is happily married and the father of four kids. He likes his family, code, JavaScript, and React.

You can follow Kent on Twitter (@kentcdodds).

The post Confident React – Kent C. Dodds on Frontend Visual Testing appeared first on Automated Visual Testing | Applitools.

On September 9, 1947, Grace Hopper recorded the first computer bug ever in the Harvard Mark II computer’s logbook. The bug in question? Believe it or not, an actual bug – a moth – flew into the relay contacts in the computer and got stuck. Hopper duly taped the moth into the logbook. Then she added the explanation: “First actual case of bug being found.” (This might be the most famous moth in history.)

If only things were this simple today. As software continuously grows in complexity, so does the process of testing and debugging. Nowadays, the lifecycle of a bug in software can be lengthy, costly, and frustrating.

Finding and fixing bugs early on in the application development stage is cheaper and easier than doing so during QA, or worse, in production. This is because, as any developer knows, debugging is often a case of finding a needle in a haystack. The smaller the haystack (the code you’re looking through) the easier it is to find the needle (or bug). That haystack is going to be smaller when you’re looking at the code written by one developer, as opposed to several developers or more.

This dynamic is what’s driving the trend of ‘Shift-Left’ to do more testing – not just unit, but functional and visual – during the development phase of the app lifecycle. Recognizing the importance of this trend, we want to help frontend developers and have developed new visual testing capabilities as part of Applitools Eyes.

Expanding Left to Developers

When you adopt Shift Left, it impacts your software development lifecycle. Developers now need to do more testing as they code, to discover bugs as soon as they create them.

Shift-Left testing paired with an increasing velocity of releasing software to meet business demand, all while maintaining high quality, presents huge challenges for R&D teams. If you’re a developer, you have to do more. Write more code, do more testing, manage the merge, etc. All to deliver more features in less time and with better quality. Because of these challenges, we are seeing new tools being adopted by developers such as the Cypress test framework and Storybook UI component development framework, and more.

So we expanded our AI powered visual testing platform to integrate with Cypress and Storybook. Now you have the tools to do visual UI testing as you code — at the speeds you need for Agile development, continuous integration, and continuous delivery (CI-CD).

Ultrafast Cross-browser Visual Testing with Cypress

The Applitools Cypress SDK lets you instantly run visual UI tests on top of your functional tests by running browsers in parallel, in the cloud, to generate screenshots for multiple browsers and viewports. Our visual AI engine then compares these screenshots to find significant differences and instantly detect visual bugs.

If you’re making the move to Cypress, we made this process run ultra-fast so that you won’t experience any slowdown when adding Applitools Visual AI Testing into your work stream. We do this by using containers to test all options simultaneously, in parallel. We call this feature our Applitools Ultrafast Grid. It takes the DOM snapshot and renders the various screens on different browsers and various viewport sizes corresponding to multiple mobile devices, desktops, etc.  

For the first time, you can use Cypress to effectively test to see if you have any cross-browser bugs. And, you can now do responsive testing to see if any visual bugs occur on different screen sizes.

This increased speed lets you cover more tests cases, increase test coverage, and catch more bugs before they hit production.

Check out Applitools’ new Cypress SDK and sign up for a free Applitools Eyes account at (https://applitools.com/cypress)

Instantly Detect UI Component Bugs with Visual Testing

The Applitools Storybook SDK visually tests UI web components, without any test code and works with React, Angular or Vue.

The way it works is that we scan the Storybook library to inventory all components. Then we upload a DOM Snapshot for each component’s visual state to our Applitools Ultrafast Grid.

Once these DOM Snapshots are on our ultrafast grid, we run visual validations in parallel. Validations of all visualization states of all components, on all specified viewport sizes, on both Chrome and Firefox, happen at once.

All this lets you perform ultra-fast codeless validation of your Storybook component libraries in seconds. You can think of this as visual testing at the speed of unit testing. You can learn more in this blog post on visually testing React components in Storybook and this webinar on our Cypress and Storybook SDKs.

Check out Applitools’ new Storybook SDK and sign up for a free Applitools Eyes account at (https://applitools.com/storybook).

The Final Word: Expanding Left, Not Shifting Left!

I want to emphasize that, even though we’ve talked a lot about developers and Cypress in this post, we still remain as committed as ever to QA teams, and the frameworks they use, including Selenium.

Selenium remains an incredibly important part of the test automation world, and we continue to remain a sponsor of both the Selenium project itself and SeleniumConf.  In the coming weeks and months, you’ll see us release new functionality that backs this point up.

While we are delivering new visual testing tools to developers, we remain committed to serving test automation engineers and QA analysts. That’s why we are talking about expand left, rather than using the more common term shift-left.

If you have any questions or want to try the application visual testing approach, please reach out or sign up for a free Applitools account today!

How will you start visual UI testing with Cypress or Storybook? 

The post How to Implement Shift Left for your Visual Testing appeared first on Automated Visual Testing | Applitools.

Listen to Gil Tayar’s webinar on the new Applitools SDKs for Cypress and Storybook, which enable developers to test the visual appearance of their apps across all responsive web platforms, including React, Vue, and Angular.  

Developers want their tests to run fast. In an increasingly agile world, waiting ten minutes or more for test results is a huge no-no. A new generation of browser automation tools recognize that need-for-speed and enable frontend developers to quickly automate their browser tests.

But those tests also need to check the application’s visual elements. Does the login page look okay? Does it look good in Firefox? What about mobile browsers, in 768px width? And does it still look good in 455px with a device pixel ratio of 2? These are just a few of the questions developers ask when building responsive web applications.

To check the visual quality of your application, across all browsers and in all those responsive widths, would necessitate a humongous grid of browsers, and an unreasonable amount of time. Far more than the two to five minutes usually available for a developer’s tests.

Applitools’ new SDKs for Cypress and Storybook enable you to do just that: write a set of visual regression tests that run through your pages and components, and have the pages and components render in Applitools Ultrafast Grid, in parallel, on a large set of browsers and widths, and return the result in less than a minute.

One minute. That’s quicker than most functional test suites, and approaching the time required for unit testing.

Listen to Applitools Architect Gil Tayar, as he discusses the new generation of visual UI testing tools. Tools that move the burden of the visual work to cloud, and enable you to check what was till now impossible to check locally: the visual appearance of your application across multiple responsive platforms.

Source code, slides, and video

You can find companion code for Gil’s talk on this GitHub repo.

We also have detailed, step-by-step tutorials on how to use Applitools with Cypress, Storybook + React, Storybook + Vue, and Storybook + Angular.

Here’s Gil’s slide deck:

And here’s the full webinar recording:

Check out these resources, and if you have any questions, please contact us or set up a demo.

How are you going to start visually testing with Cypress or Storybook?

The post Visual UI Testing at the Speed of Unit Testing, with new SDKs for Cypress and Storybook appeared first on Automated Visual Testing | Applitools.

React is, first and foremost, a library for creating reusable components. But until React Storybook came along, reusability of components between applications was very limited. While a library of common components could be created, it was very difficult to use, as there was no useful way of documenting it.

Yes, documenting the common components via a Readme file is possible, even useful. But a component is, in its essence, something visual. A Readme file just doesn’t cut it, neither for a developer and definitely not for a designer that wants to browse a library of shared components and choose the one that fits their purpose.

This limited the usefulness of React’s component reusability.

And then React Storybook came along. React Storybook has changed the way many companies approach React by allowing them to create a library of components that can be visually browsed so as to be able to pick and choose the component they want:

Finally, a real library of shared components can be created and reused across the company. And examples abound, some of them open-source: from Airbnb’s set of date components to Wix’s styled components.

But a library of widely used company-wide components brings its own set of problems: how does one maintain and test the set of components in a reliable way? Components that are internal to one application are easily maintained — they are part of the QA process of that application, whether manual or automated. But how does one change the code of a shared component library in a reliable manner? Any bug introduced into the library will immediately percolate to all the applications in the company. And in these agile days of continuous delivery, this bug will quickly reach production without anybody noticing.

Moreover, everyone in the company assumes that these shared components are religiously tested, but this is not always the matter. And even when they are tested, it is usually their functional aspect that is tested, while testing how they look is largely ignored.

Testing a Shared Library

So how does one go about writing tests for a library of shared components? Let’s look at one of the Shared Components libraries out there — Airbnb’s React Dates. I have forked it for the purpose of this article, so that we can look look at a specific version of that library, frozen in time.

Let’s git clone this fork:

$ git clone https://github.com/applitools/eyes-storybook-example.git
$ cd eyes-storybook-example

To run the storybook there, just run npm run storybook, and open your browser to http://localhost:6006.

To run the tests, run the following:

$ npm install
$ npm test

This runs the tests, which generate the following output:

Wow! That’s over 800 tests. I have to say that this component library is very well tested. And how do I know that it is very well tested? It’s not just because there are a lot of tests, but also because I randomly went into some source file, changed something, and ran the tests again.

Let’s try it. Change line 75 in file: src/components/CalendarDay.jsx to:

if (!onDayMouseEnter) onDayMouseEnter(day, e);

(I just added the if to the line). Now let’s run the tests again using npm test:

Perfect! The tests failed, as they should. Really great engineering from the Airbnb team.

(If you’re following with your own repository, please undo the change you just made to the file.)

Functional Testing Cannot Catch Visual Problems

Let’s try again. Revert the previous change, so the tests will pass again. Now Edit line 1 in the file src/svg/arrow-right.svg, by changing the first M694 to M94, as follows: 

Now let’s run the tests again using npm test:

Oops. They’re passing. We’ve changed the source code for the components, and yet the tests pass.

Why did this happen? Because we changed only the visual aspect of the components, without changing any of the functional aspect of it. And Airbnb does only functional testing.

This is not to denigrate the wonderful engineers at Airbnb who have given us this wonderful libraries (and other open-source contributions). This library is tested in a way that should make Airbnb proud. And I am sure that manual testing of the components is done from time to time to ensure that the components are OK.

(If you’re following with your own repository, please undo the change you just made to the file.)

Visual Testing of a Shared Library

So how does one go about testing the visual aspects of the components in a shared library? Well, if you’re using ReactJS Storybook, you’re in luck (and if you’re building a shared component library, you should be using React Storybook). You are already using React Storybook to visually display the components and all their variants, so testing them automatically is just a simple step (are you listening, Airbnb?).

We’ll be using Applitools Eyes, which has a suite of SDKs that enable Visual UI Testing inside a variety of frameworks. In this case, we’ll use the Applitools Eyes SDK for React Storybook (yes, I know, it’s a mouthfu).

(Full disclosure— I am a developer advocate and architect for Applitools. But I love the product, so I have no problem promoting it!)

To install the SDK in the project, run:

npm install @applitools/eyes-storybook --save-dev

This will install the Eyes SDK for Storybook (locally for this project). If you’re following with your own repository, please undo the changes you made to the files. or otherwise you will get failures. To run the test, just run:

npx eyes-storybook

You will get this error: 

Yup. To use the Eyes SDK, you need an Applitools API Key, which you can get by registering at the Applitools site, here. Once you have that, login and go to the Eyes admin tool, here. You will find the API Key using the following menu:

To make Eyes Storybook use the API key, set the environment key APPLITOOLS_API_KEY to the key you have:

// Mac, Linux:
$ export APPLITOOLS_API_KEY=your-api-key

// Window
C:> set APPLITOOLS_API_KEY=your-api-key

Now run it again:

npx eyes-storybook

Eyes storybook will run Chrome headless, open your storybook, take a screenshot of all your components and variants, and send them to the Applitools service. When the test ends, as this is the first test, it will establish that these screenshots are the baseline against which all other tests will be checked against. You can check the results with the link it adds to the end of the tests, or just go to http://eyes.applitools.com:

Failing the Test

So let’s test if the change we made to the SVG will fail the test. Re-edit line 6 in the file src/svg/arrow-right.svg, by changing the first M694 to M94, thus:

Now let’s run the test again using ./node_modules/.bin/eyes-storybook.

Success! Or rather failure! The test recognized that some of the component screenshots differed from the baseline ones we created earlier. We can see the result of the diffs in the Applitools Eyes management UI:

We can either select these changes as the new baseline (which in this case we don’t want to!) or fix the bug and run the test again, in which case it will be green.

And that is the beauty of Storybook. Because it enables you to create a central repository of components, it feeds three birds with one bird feed (I’m sorry, I can’t kill three birds, even if it is with one stone…):

• Creating and documenting a shared ReactJS components library, with little effort.
• Testing that the functionality of the components works (albeit with lots of effort)
• Visually testing the components with literally zero code.

Conclusion

React Storybook is a game-changer for frontend development— it enables the creation and maintenance of a shared component library for the whole company.

And the best thing about React Storybook is that if you’ve already invested the effort to set it up, you get a suite of tests that check the visual look of your components… for free!

Applitools has visual regression testing tutorials for React Storybook, as well as for Storybook components built with Vue and Angular. As well as tutorials for over 40 other frontend technologies. Check out these examples, and if you have any questions, please contact us or set up a demo.

What Storybooks are you going to start visually testing?

— About the Author — 

30 years of experience have not dulled the fascination Gil Tayar has with software development. From the olden days of DOS, to the contemporary world of Software Testing, Gil was, is, and always will be, a software developer. He has in the past co-founded WebCollage, survived the bubble collapse of 2000, and worked on various big cloudy projects at Wix.

His current passion is figuring out how to test software, a passion which he has turned into his main job as Evangelist and Senior Architect at Applitools. He has religiously tested all his software, from the early days as a junior software developer to the current days at Applitools, where he develops tests for software that tests software, which is almost one meta layer too many for him.

In his private life, he is a dad to two lovely kids (and a cat), an avid reader of Science Fiction, (he counts Samuel Delany, Robert Silverberg, and Robert Heinlein as favorites) and a passionate film buff. (Stanley Kubrick, Lars Von Trier, David Cronenberg, anybody?)

Unfortunately for him, he hasn’t really answered the big question of his life – he still doesn’t know whether static languages or dynamic languages are best.

The post How to Visually Test React Components Using Our Storybook SDK appeared first on Automated Visual Testing | Applitools.

Guest post by Jared Sulzdorf, Director of Product Development at LexBlog

There is an art to managing websites that many do not appreciate.

The browser wars of the late 90s carried over into the early 2000s, and while web standards have helped align how these browsers render front-end code, there are still dozens of versions of the five major browsers (Edge has joined the crew!), all with their own understanding of how the web should look and act. Add to that the fact most digital properties ship with many templates, each with their own way of displaying content, and those templates in turn may run on dozens or hundreds of websites, and you’re left wondering if this internet thing is really worth it.

At LexBlog, where I hang my digital hat as the lead of the product development team, we manage well over one thousand websites. From the front to the back-end, it’s our responsibility to ensure things are running smoothly for our clients who range from solo lawyers to some of the largest law firms in the world. With that in mind, it’s no surprise that expectations are high, and our team works to meet those standards during every update.

While our product development team makes great use of staging environments, functional testing, performance monitoring/server error logs, and unit tests to catch issues through the course of the development cycle, it still isn’t enough. The reality of developing on the web is that you can’t unit test how CSS and HTML will render in a browser and humans can only look at so many screenshots before losing focus.

Prior to finding Applitools, LexBlog used Fake (http://fakeapp.com/), a web automation software, to visit our sites in a staging environment after a front-end update and take a screenshot. One of our support teammates would then leaf through these screenshots in an effort to find unexpected changes and report back when they were done. Unfortunately, this approach was just not scaling, and our team was running into a myriad of issues:

• Deployments would wait for weeks while manual testing was performed
• Manual tests would invariably miss errors and when the code was deployed, hotfixes would quickly follow
• The overhead in managing the humans and systems necessary for performing manual tests was untenable
• All of the above led to low morale

The first major breakthrough was finding Selenium to replace Fake. This allowed the product team to better manage the writing and reviewing of tests. However, now that the ball was fully in our court, there was still the problem of reviewing all these screenshots.

Being technically inclined, we looked for programmatic solutions to comparing two screenshots (one taken before an update, the other after an update) and finding the diff. A myriad of tools like PhantomJS (http://phantomjs.org/) or BackstopJS (https://garris.github.io/BackstopJS/) were researched and thrown aside until we found Applitools.

Applitools had everything we were looking for:

• An SDK that supported languages our team was familiar with
• UI for reviewing screenshot comparisons
• A cloud-based portal allowing multiple teammates to log in and view the results of the same test

After spending some time investing in a host of desktop scripts written in Python, we noticed immediate improvements. Deployments for front-end changes went out the door faster and with less errors and our team was able to focus on higher-purpose work, leading to happier clients.

Over the years, we’ve consistently updated our approach to visual regression testing to the point where we can now introduce large changesets in code that renders on the front-end and catch nearly every regression in staging environments before deploying to production. Applitools, in combination with Selenium, Node, React, and the WordPress REST API, has allowed us to create a fully-fledged visual regression testing application that saves our team countless hours and headaches all while ensuring happy clients. It really is a match made in heaven.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post How Applitools changed visual UI testing at LexBlog appeared first on Automated Visual Testing | Applitools.