Get all the latest test automation videos you need in one place.

It’s summertime (at least where I am in the US), and this year has been a hot one. Summer is a great season to take a step back, to reflect, and hopefully to relax. The testing world moves so quickly sometimes, and while we’re all doing our jobs it can be hard to find the time to just pause, take a deep breath, and look around you at everything that’s new and growing.

Here at Applitools, we want to help you out with that. While you’ve hopefully been enjoying the nice weather, you may not have had a chance to see every video or event that you might have wanted to, or you may have missed some new developments you’d be interested in. So we’ve rounded up a few of our best test automation videos of the summer so far in one place.

All speakers are brilliant testing experts and we’re excited to share their talks with you – you’ll definitely want to check them all out below.

ICYMI: A few months back we also rounded up our top videos from the first half of 2022.

The State of UI/UX Testing: 2022 Results

Earlier this year, Applitools set out to conduct an industrywide survey on the state of testing in the UI/UX space. We surveyed over 800 testers, developers, designers, and digital thought leaders on the state of testing user interfaces and experiences in modern frontend development. Recently, our own Dan Giordano held a webinar to go over the results in detail. Take a look below – and don’t forget to download your free copy of the report.

Front-End Test Fest 2022

Front-End Test Fest 2022 was an amazing event, featuring leading speakers and testing experts sharing their knowledge on a wide range of topics. If you missed it, a great way to get started is with the thought-provoking opening keynote for the event given by Andrew Knight, AKA the Automation Panda. In this talk, titled The State of the Union for Front End Testing, Andrew explores seven major trends in front end testing to help unlock the best approaches, tools and frameworks you can use.

For more on Front-End Test Fest 2022 and to see all the talks, you can read this dedicated recap post or just head straight to our video library for the event.

Cypress Versus Playwright: Let the Code Speak

There are a lot of opinions out there on the best framework for test automation – why not let the code decide? In the latest installment in our popular versus series, Andrew Knight backs Playwright and goes head to head with Cypress expert Filip Hric. Round for round, Filip and Andy implement small coding challenges in JavaScript, and the live audience voted on the best solution. Who won the battle? You’ll have to watch to find out.

Just kidding, actually – at Applitools we want to make gaining testing knowledge easy, so why would we limit you to just one way of finding the answer? Filip Hric summarizes the code battle (including the final score) in a great recap blog post right here.

Can’t get enough of Cypress vs Playwright? Us either. That’s why we’re hosting a rematch to give these two heavyweights another chance to go head to head. Register today for to be a part of the Cypress vs Playwright Rematch Event on September 8th!

Coded vs. Codeless Testing Tools—And the Space In Between

There are a lot of testing debates out there, and coded vs codeless testing tools is one of the big ones. How can you know which is better, and when to use one or the other? Watch this panel discussion to see leading automation experts discuss the current landscape of coded and codeless tools. Learn what’s trending, common pitfalls with each approach, how a hybrid approach could work, and more.

Your panel for this event includes our own Anand Bagmar and Andrew Knight, along with Mush Honda, Chief Quality Architect and Coty Resenblath, CTO, both from Katalon.

Autonomous Testing, Test Cloud Infrastructure, and Emerging Trends in Software Testing

Looking to get a handle on the where testing is heading in the future? Hear from our Co-Founder and CEO, Gil Sever, as he sits down for a Q&A with QA Financial to discuss the future of testing. Learn about the ways autonomous testing is transforming the market, advancements in the cloud and AI, and the ups and downs of where testing could go in the next few years. Gil also shares insights he’s learned from our latest State of UI/UX Testing survey.

Test Automation Stories from Our Customers

We know that every day you and countless others are innovating in the test automation space, encountering challenges and discovering – or inventing – impressive solutions. Our hope is that hearing how others have solved a similar problem will help you understand that you’re not alone in facing these obstacles, and that their stories will give you a better understanding of your own challenges and spark new ways of thinking.

Automating Manufacturing Quality Control with Visual AI

We all know about web and mobile regression testing, but did you know that Visual AI is solving problems in the manufacturing space as well? Jerome Rieul, Test Automation Architect, explains how a major Swiss luxury brand uses uses Visual AI to detect changes in CAD drawings and surface issues before they hit production lines. A great example of an out-of-the-box application of technology leading to fantastic results.

Simplifying Test Automation with Codeless Tools and Visual AI

Test automation can be hard, and many shops struggle to do it effectively. One way to lower the learning curve is to take advantage of a codeless test automation tool – and that doesn’t mean you have to forego advanced and time-saving capabilities like Visual AI. In this webinar Applitools’ Nikhil Nigam shares how Visual AI can integrate seamlessly with codeless tools like Selenium IDE, Katalon Studio, and Tosca to supercharge verifications and meet industrial-grade needs. (And for more on codeless testing tools, don’t forget to watch our lively panel discussion!)

How EVERFI Moved from No Automation to Continuous Test Generation in 9 Months

Starting up test automation from scratch can be a daunting challenge – but it’s one that countless testing teams across the world have faced before you. In this informative talk, Greg Sypolt, VP of Quality Engineering, and Sneha Viswalingam, Director of Quality Engineering, both from EVERFI, share their journey. Learn about the tools they used, how they approached the project, and the time and productivity savings they achieved.

More to Come!

This is just a selection of our favorite test automation videos that we’ve shared with the community this summer. We’re continuously sharing more too – keep an eye on our upcoming events page to see what we have in store next.

What were your favorite videos? Check out our full video library here, and you can let us know your own favorites @Applitools.

The post Test Automation Video Summer Roundup: May-August 2022 appeared first on Automated Visual Testing | Applitools.

User experience is becoming more and more critical to the success of applications every year, making the world of front-end testing a fast-moving and exciting place. Along with our partners at Netlify, we were excited to host our second annual Front-End Test Fest last month. The event featured expert speakers from global brands, lively Q&A sessions as well as games and fun. It was hosted by Angie Jones, Head of Developer Relations at Block, and Cassidy Williams, Head of Developer Experience and Education at Remote.

We’ve rounded up all the videos in one place, so feel free to dive right in, or keep reading for a recap of the event.

Front-End Test Fest 2022

The event began with opening remarks from hosts Angie and Cassidy, before Andrew Knight (AKA the Automation Panda) took the stage to present the opening keynote.

OPENING KEYNOTE: The State of the Union for Front End Testing – Andrew Knight

Web development moves fast, and new techniques are constantly emerging with promises of better performance and simpler maintenance. The JAM stack, server-side rendering frameworks, and APIs for all the things are on the rise. How can we most effectively test our web apps when they follow these new patterns? In this talk, Andrew Knight, Developer Advocate at Applitools and Automation Panda, explores seven major trends in front end testing to help unlock the best approaches, tools and frameworks you can use. Andrew considers the value of new techniques like component testing and visual testing, and touches on the future with a glance at autonomous testing. Check out this exciting “state of the union” on current trends in front end testing!

How to Avoid Common Mistakes When Writing Your Cypress Tests – Filip Hiric

Filip Hric, QA Lead at Slido and Cypress expert, knows the pain of a failed, flaky test. Cypress is a great tool, but it’s only as effective as the tests you design with it. In this talk, Filip shows you a few powerful tools that are built into Cypress that can help you navigate the rough terrain of flaky tests so you can start designing tests that are evergreen and stable. He discusses common mistakes, the proper use of commands and plenty more. Check out his great presentation.

Coffee Break!

One of the nice things about the conference was the “coffee breaks” between the sessions, that gave attendees a chance to “mingle” with some of the speakers. Gleb Bahmutov (Senior Director of Engineering, Mercari US), Tiffany Le-Nguyen, (Sr. Frontend Engineer, Netlify) and Nick Taylor (Staff Software Engineer, Netlify) were available at the morning break.

Shifting Accessibility Testing to the Left – Marie Drake

Why are there still so many websites that are not accessible? Many companies still view accessibility testing as an afterthought or something that will be difficult to implement. In this talk, Marie Drake, Quality Engineering Manager at Zoopla, busts some of these myths around accessibility testing and explains how you can shift it to the left. She also covers the many benefits of improving accessibility, and how to integrate accessibility testing into development and testing workflows. Watch the talk right here.

Expert Panel: Trending Tools and Frameworks – What’s Hype and What’s Not

Cutting through the marketing noise to understand what’s hype and what’s real can be a challenge. In this panel, you’ll hear from leaders Skyler Brungardt, Sr. Manager of Digital Experience & Commerce at Gap, Dan Giordano, Sr. Director of Product Marketing at Applitools, and Tiffany Le-Nguyen, Sr. Frontend Engineer, and Nick Taylor, Staff Software Engineer, both from Netlify.

Take a look at this panel for an exciting conversation based on hands-on experience with the latest tools and frameworks being used at leading companies. The panel was expertly moderated by Joe Colantonio, the founder of TestGuild.

Lunch Break: Making Music with (Live) Code – Dan Gorelick

This was a fun one. For the lunch break, independent developer and artist Dan Gorelick gave a live musical performance using only code. Using the coding framework TidalCycles, Dan briefly introduces us to the technology and how it works before improvising a great live show. Check this one out for a bit of musical theory, conversation around how to combine code with musical ideas, and just for some fun music.

Slice and Dice Your End-to-End Tests – Gleb Bahmutov

In this presentation, Gleb Bahmutov, Senior Director of Engineering at Mercari US, shares some practical tips and tricks for reliable end-to-end testing at serious scale. Using parallelization and a bit of custom tinkering, his CI system has his team running 500+ end-to-end front-end tests – and running them quickly. Want to get developers and other stakeholders quick, high-quality feedback from your E2E tests? Watch this talk.

Increasing Automated Test Flow in a Monorepo – David Lindley, Ben Hudson

David Lindley and Ben Hudson, Software Engineering Manager and Senior Software Engineer at Vodafone UK, respectively, talk about the challenges and benefits involved in one of their global projects – a shared React component library situation in a monorepo. The library has 220+ shared components and developers across the globe contribute to it, so ensuring quality is critical! David and Ben share how they overcame issues that came up as they progressed, including how they used dependency checking to make sure they test only what is needed, speeding up test flow. See it here.

Coffee Break!

Another opportunity to schmooze with the speakers! Andrew Knight (Developer Advocate and Automation Panda, Applitools), Adam Murray (Developer Experience Team Lead, Cypress.io) and Ben Hudson (Sr. Software Engineer, Vodafone UK) were hanging out to chat in this afternoon break.

Getting Started with Component Testing in Cypress – Adam Murray

In the latest Cypress 10 release, the Cypress team has added something new – official component testing (previously, it was an experimental feature). In this hands-on talk, Adam Murray, Developer Experience Team Lead at Cypress, helps you get going with it. He starts at the very beginning, showing you how to install Cypress 10, configure it in a new application, and add your first component tests. Adam goes on to cover more advanced topics like cy.intercept() as well. Take a look.

CLOSING KEYNOTE: Looking Forward – Trends That Will Shape Modern Web Development – Jason Lengstorf

The web ecosystem is changing quickly, and it can feel hard to keep up with all the changes. In this fascinating closing keynote, Jason Lengstorf, VP of Developer Experience at Netlify, delivers some comfort – things are perhaps not as “new and different” as they seem. Jason looks forwards and back at the landscape of the web, discussing not only where we’re going but how it all builds on historical trends. Check it out here.

After the closing keynote, Angie and Cassidy delivered the closing remarks as the event concluded.

Thank You!

We want to extend a huge thanks to everyone involved in making this event such a huge success. The list could go on, but we’ll start with Angie and Cassidy for being rock star hosts, our brilliant speakers for sharing their wisdom and experience, and of course to every attendee for adding your voices and presence. Thank you – we could never have delivered an event this great without all of you.

If you liked these videos, you might also like our videos from our previous virtual events too – all free to watch. Happy testing!

The post Front-End Test Fest 2022 Recap appeared first on Automated Visual Testing | Applitools.

Last month, Applitools and Cypress hosted the Front-End Test Fest, a free event that brought together leading experts in test automation for a full day of learning and discussion around front-end testing. It was a great opportunity to hear about the latest in the industry and get to hear some really innovative and interesting stories.

We’ve got all the videos ready for you here, so feel free to jump right in below, but let’s recap the event below.

Opening Keynote: Applitools & Cypress: State of the Union – Angie Jones and Amir Rustamzadeh

This talk opened with Amir Rustamzadeh, Director of Developer Experience at Cypress, getting us all familiar with the latest and greatest in the testing tool. We all already know that Cypress is an excellent tool that is highly interactive and visual, but Amir took us on a tour of two new features that look pretty powerful.

These new features were Test Retries and Component Testing. Test Retries allows you to easily retry a test multiple times, helping you to catch and defeat test flake by highlighting how frequently a test passes with some handy analytics in the Cypress Dashboard. A related feature, Test Burn-In, allows you to do the same thing with brand new tests as they’re introduced. As for Component Testing, Amir noted that while this is typically done in a virtual DOM that you can’t debug, Cypress now has a beta where you can use the real browser DOM to mount a component and test it in isolation. Much better!

Angie Jones, Senior Director of Developer Relations at Applitools, then helped us understand the dangers of all-too-common visual bugs. Angie walked us through how Applitools Eyes can give your code super powers to find visual bugs, thanks to Visual AI. This talk covered visual component testing, visual testing of dynamic content, accessibility and localization testing, as well as cross-browser/viewport testing using the Ultrafast Grid. Check it out for a great overview of how to improve your visual testing.

The Allure of Adding Tests to Azure DevOps – Bushra Alam

Azure DevOps is a powerful tool, and if you’re curious about it, this talk will help get you started with it. Busra Alam, Software Quality Analyst, begins by covering the basics about what Azure, DevOps, and of course Azure DevOps means.

Azure Pipelines, part of Azure DevOps, is a tool to build, test and deploy your apps. By running tests in the pipeline, we can discover bugs early and deliver faster feedback with quicker time to market and overall better code quality. Bushra takes us through a live demo that shows how to create a pipeline, run a test and check the results – all automated through Azure, and quickly. She went on to share some advanced tips for running tests in parallel and utilizing release triggers. Check it out for the whole demo.

Answer The Call – A Digital Learning Transformation Using Model-Based Testing – Greg Sypolt

EverFi is a socially-conscious educational platform with a large number of courses and individualized learner paths. Greg Sypolt joined them as their VP of Quality Assurance to solve a tricky testing challenge they had – with so many different courses and paths for learners to take, traditional testing just couldn’t cover it all, and it would only get worse as EverFi grew.

Greg’s solution was to launch a multi-pronged approach centered around model-based testing. In this eye-opening talk you’ll see the step-by-step approach Greg used to build his models. Cypress and Applitools as critical components of the process, but there’s a lot more to it. This one is hard to sum up in a couple of sentences but is definitely worth watching to get the full story.

Expert Panel: Successful Test Strategies

Stacy Kirk, CEO/Founder of QualityWorks Consulting Group, moderated this great panel with a trio of testing experts. Kristin Jackvony, Principal Engineer – Quality at Paylocity, Alfred Lucero, Senior Software Engineer at Twilio and Jeff Benton, Staff Software Engineer in Test at RxSaver share their experiences on a range of issues relevant to test engineers everywhere. Learn about the testing tools they used, tips for incorporating testing into the CI/CD process and how you can secure that crucial teamwide buy-in for testing. I won’t spoil it but the parting words from these experts make it clear that the first step for successful testing is to have the conversation with your team on the value of testing, and then just start – it’s ok if you start small with a quick win to get that buy-in quickly.

It’s a (Testing) Trap! – Common Testing Pitfalls and How to Solve Them – Ramona Schwering

How can we avoid trapping ourselves underneath tests that are hard to maintain or worse, don’t even deliver any value? Ramona Schwering, a Developer Core at shopware AG, shared her own mistakes here (and yes, her love of Star Wars) to try and make sure you don’t have to make them too. Ramona has worked as both a developer and in testing so she knows how to speak to both experiences, and this was a very easy to follow, relatable talk. She shared three main pain points (or traps) that tests can fall into – they can be slow, they can be tough to maintain, and they can be “Heisen tests” that are so flakey they don’t tell you anything. Check this one out to hear more about traps and solutions and how you can keep your tests simple.

Testing Beyond the DOM – Colby Fayock

Colby Fayock, Developer Advocate at Applitools, kicked off his talk with a game of “UI Gone Wrong,” taking us through some cringeworthy examples of UI bugs from major organizations that probably cost them revenue or customers. You all know the kind of bug – it happens to everyone sometimes, but does it need to? With Cypress and Applitools working together, Colby showed us that you can do better. He walked us through a live demonstration of how you can easily add Applitools to an existing Cypress test, enhancing the browser automation provided by Cypress with Visual AI to catch any visual bugs. Take a look and see how you can take your testing to the next level.

Our Journey to CI/CD: The Trials, Tribulations, & Triumphs – Hector Coronado and Joseph King

As projects get increasingly complex, they get harder to maintain and changes become slower to deploy. That was the issue Hector Coronado and Joseph King were running into as frontend and web application engineers, respectively, at Autodesk. They were working on a React app they had built, the “Universal Help Module,” that provides users several types of support while appearing in multiple locations with varying layouts and UIs. To keep up with the growing complexity, they set out to build a fast and thorough CI/CD pipeline that would include an automated testing strategy.

Hector and Joseph moved away from manual testing and tried many tools for automated functional and visual testing. In the end, Cypress won big as a free all-in-one testing framework that is fast an open source, and they loved Applitools for its blazing speed, simple Cypress SDK, strong cross-browser capabilities and excellent customer support. They put them together to achieve a dream they used to get buy-in – more coverage with less code! Check out their full journey below.

Practically Testing – Kent C. Dodds

You have limited time in your day – should you write that test or fix that bug? That’s the subhead for this talk by Kent C. Dodds, a JavaScript Engineer and Trainer at Kent C. Dodds Tech LLC. Unlike many of the presentations above, which are filled with awesome code examples and demos, Kent’s talk is intended to be a practical one with relatable examples to get you thinking about one key thing: How do you prioritize?

Kent describes his methodology for understanding what’s truly important to your company and its mission and how you can identify your role in pushing that forward. He also reminds all of us that we’re not simply hired as engineers to write code or tests, but as humans to advance a mission. Watch this video for some really humanizing inspiration and to spark some thoughts about how you can get more out of your day.

Looking for More? Learn about the Future of Mobile Testing

We’ve got you covered with another free event. Our next live Future of Testing: Mobile event takes place on August 10^th, and registration is officially open. Check it out and reserve your spot today.

Past Future of Testing: Mobile Events

You can also check out all the videos from our Future of Testing: Mobile event in June here or get a full recap of our Future of Testing: Mobile event from April right here.

Happy testing!

The post Front-End Test Fest 2021 Recap appeared first on Automated Visual Testing | Applitools.

What is a design system? Who would use it, and for what benefit?

In May 2020, Applitools had the pleasure of hosting Tyler Krupicka from Intuit for an hour-long webinar discussing design systems and testability.

Tyler works at Intuit, a 9,400 employee company headquartered in Mountain View, California, that specializes in accounting and tax preparation software. At Intuit, Tyler works on the “Player/Design Systems” team, where he focuses on design systems.

Tyler has been working on web development at Intuit since 2015. These days, he works on UI components, tools, and testing. He primarily uses TypeScript, React and Vue. He also focuses on accessibility, and he serves as a “Level 2” Accessibility Champion inside Intuit.As part of Tyler’s job, he ensures that design systems are testable. 

What Is A Design System

Tyler started asking this question. Some people think about large design systems:

• Material Design (Google’s design system)
• Human Interface Guidelines (Apple’s design system)
• Bootstrap (set of UI interface components and CSS that are popular on the web)

People look at these and think, “Is this overkill?” After all, Google uses Material Design for Android. How many people design an entire operating system. So, Tyler dove in to answer his question.

First, Tyler sees design systems as really “Design + Code.” One part of a design system are the “design” – the common components and patterns that get used throughout products. The other part of the design system, Code, involves chunks of functionality interacting with design developers can reuse across a product.  For example, CSS Frameworks, or React, or in some other way that lets teams reuse the same code.

Design

“Design” turns into a consistent voice for your product. Those are things like typography, colors, and spacing, as well as UI components (forms, charts, site navigation), Graphics and Animation standards. These help standardize the visualization – the “look” – of the application. One can imagine an application developed by a large team that lacked a standard design – it would be disorienting to customers who expect consistency from software. A design system lets  you take base pieces, document them well, and reuse them.

Code

“Code” lets you build and document components individually, test them once, and have a way to set expectations on whether or not this application will work.  For instance, you can build a component in the app for one persona and reuse that component multiple times inside the app without worrying that you must retest it. Reuse makes code standard and creates standards for interacting with design. If you ever change design elements, the standard code reduces the effort to validate the new design with existing code. 

Combining design and code into a design system results in increased developer productivity and a consistent voice for the application. And using a design system simplifies the management of an application.

Design Systems at Intuit: TurboTax Example

Next, Tyler dove into the use of design systems for the Intuit TurboTax product. 

Tyler explained how TurboTax works. By providing users with interview-style questions, TurboTax helps millions of people, primarily in the United States and Canada, file their income tax returns. Intuit offers both web and mobile versions of TurboTax.

Because TurboTax uses interview questions to create conditional flows through the product, there are thousands of screens users can encounter based on their unique needs. Screens can range in complexity from a single question to a complete form. 

Building the TurboTax Design System

To create the design, Intuit develops about 50 UI components needed to run the experience. Headers, radio buttons, tiles, multi-select, data entry, action buttons – all become part of the basic UI. 

Intuit also builds a set of about 500 “mocks”, or mock-ups, of what common screens look like for testing. These help show the UI components in common layouts for testing purposes.

By using a design system, Intuit can make changes to the application and ensure consistency for all these screens. 

Tyler gave the example of a decision to deliver a uniform increase in contrast in TurboTax for accessibility. The contrast changes required updates to typography weight, button color, and link color. This work would impact over 90% of all the screens. How could Intuit deliver these changes to the application and ensure they occurred consistently across the application? The team did not have the staff needed to manually check every possible screen. Instead, they made the design changes to the UI components and validated the mocks.

Testing A Design System

Next, Tyler dove into a discussion of testing a design system. 

Approach

Since the design system represents all the functionality and visual representation of the application, Tyler recommends going broad with testing across all the elements, including:

• Functionality
• Visual
• Performance
• Accessibility

From a functional perspective, unit tests serve as key building blocks for code. Integration tests ensure that the code works wiht the back end. Cross-vrowser tests help ensure that the code works on a bunch of different browsers and mobile devices. 

Visual testing validates the impact of UI component changes on the overall UI. If buttons change size or Intuit uses a new font size, does the page continue to render correctly? Do unexpected breaks happen on pages? Do space reductions make pages look too constrained?

Performance testing involves checking the impact of code changes on page loading times.  Specifically, do new packages or package upgrades change the amount of data transferred between the server and the browser – slowing the user experience? 

Finally, Tyler tests accessibility to make sure that any customer who wants to use the product can do so. Whether a user has a screen reader or navigates solely by keyboard, ensure that the user can get around.

Getting Organized

With testing goals articulated, Tyler introduced Storybook as the tool for creating documentation and initial testing. If you use projects like React, Vue and Angular that allow you to build user interfaces in small components, the Storybook project lets you build interactions for those components and document their behavior.  

Effectively, you demonstrate a  use case for your components and show how they all interact together on a page.  Each use case is one story – hence Storybook is a collection of these stories. 

Storybook allows for a range of tests, including accessibility tests. For instance, Tyler uses Storybook to test keyboard navigation and focus. He also tests for things like hover behaviors – where components appear while the cursor hovers over another element, which may be difficult to test with automation. Testing with Storybook lets the testing team run tests early across the range of design system dimensions.

Dog Fooding

To deliver successful products, Tyler’s team must support the construction and use of these UI components. So, they actually write code to try out the components – just to see how they work. Putting together checkboxes or radio buttons, or drop-down lists helps show how the code interacts with components and creates mocks for testing the design system.

Tyler showed the example of a page that looked like static boxes. Each box expanded when clicked to show hidden detail and slid the rest of the page down. His team dog foods this kind of behavior.

All of these activities help create a body of testable code, so that a design change in the design system can be mocked up and validated without impacting production code.

Testing Walkthrough

Tyler then walked us through a simple change that can have a signficant implications – resizing a button. 

Without a design system, you might have to make a button change like this ad-hoc across your application. As a result, you risk changing the page-to-page behavior and look-and-feel. Using a design system lets you make the change globally, and then focus on the implications.

What kinds of implications? A larger button might displace elements on a page. The button may not function as expected with existing elements on a given page. All these changes need to be tested.

Unit Tests

First, Tyler looked at unit tests. Does the button click? Can the button work with an icon associated with it? Will the button work with an transparent background? Does the button work with assigned ARIA (Accessible Rich Internet Applications) attributes? Testing puts these Storybook components through their paces.  

Intuit have a pretty standard way of running through these tests.  Tyler talked about using JEST and WebdriverIO and Selenium to automate this testing across multiple browsers. They have lots of users on multiple web browsers, as well as Mac users with Safari an iOS, so they pay special attention to those target platforms. 

Accessibility Tests

Next, Tyler talked about accessibility testing. His team uses AXE to perform static validation for ARIA standards on application pages, including contrast checking.  Tyler explained how AXE helps perform contrast checking. For instance, if a button is one color, does the background let the button stand out? AXE can do the contrast checking by comparing the two colors. Or, will the text be readable against its background – or is it too small or too thin? Again – AXE can make these calculations.

Tyler also talked about HTML customizations. For example, you want to use a special radio button design that isn’t included in the existing HTML. You build this behavior as a custom implementation – and then you have to validate this behavior across the browsers your customers use. Tyler pointed out that companies make these decisions to help brand their websites for customers. And, these custom behaviors must be validated for functionality and accessible behavior across multiple platforms.  At the same time, since you are replacing functionality built into HTML, keyboard navigation still needs to behave as expected. So, you need to test those behaviors. 

At Intuit, the team tests these behaviors manually – and with tools designed to help. By using popular screenreaders like Jaws, Voiceover, and NVDA, Tyler’s team ensures that these screen readers don’t get thrown off by customizations. Also through manual testing, the team checks to ensure that keyboard navigation results in expected page responses – for example, they make sure that focus states behave as expected. And, they also ensure that the page behaves correctly when a customer has selected “reduced motion”.

Performance Tests

From a design perspective, the design system simplifies app delivery. By using standard libraries and building standard behaviors, the design system creates this great look-and-feel. At the same time, developers need to consider the impact on performance. For instance, a specific package might add great mobile behavior – at the cost of an additional 1-2 seconds of page load time. Users likely won’t value the behavior over slower page load time.  

You can choose a great library that gives you great functionality – but at what cost? How does the size of your Javascript package impact your page downloads? Often, this problem creeps up on you.  You use an early version of the library and everything works well. But, when you update your application, your library update suddenly doubles in size. This happens quite often. New versions of libraries include new features – which add to the library load times. And, you have to rethink the software you’re using.

Tyler talked about how Intuit handles these kinds of challenges. If Intuit uses code from an open source project, they might contribute back to the project to make a library with reduced size. Or, they might use a different open-source package.  

Visual Testing

Each time a new component gets introduced or modified, Intuit runs tests through Applitools Ultrafast Grid. Intuit has been running Grid since its first beta tests. With Ultrafast Grid, Intuit can set checkpoints and take screenshots across a range of browsers and devices. They can set multiple screen sizes (from mobile to Full HD and  higher). Their goal – ensure that responsive pages behave as expected, and understand the impact of design changes on existing pages. Ultrafast grid gives them the capability to easily compare changes across different versions of an application.

Tyler talked about using Ultrafast Grid to test against Chrome, Edge, and Safari. He talked about setting different viewport sizes to check responsive page behavior against expectations. 

Component and Mock Level

Tyler explained how Intuit tests components and mocks. 

In Storybook, Intuit keeps the latest components, as well as page mocks for each component in use. When a change occurs in the design system, Intuit can do two kinds of checks:

1. They can check the components to see if the component changes have violated any specifications
2. They check the mocks, to determine how the component changes affect page rendering.

So, for instance, Intuit can see if a collection of multiple new components violate spacing guidelines or other design requirements on pages. Applitools Ultrafast Grid helps Intuit do this evaluation.

Also using the mocks, Intuit can also see if a change violates accessibility guidelines using AXE.  

Accessing Intuit Tools for Design Systems

Tyler discussed the two tools Intuit uses to ensure success in their design systems.

The first, the Intuit DS-CLI, makes it easy to implement Storybook for your designs. Just like Create-React-App lets you build a web app easily, DS-CLI contains all the code you need to utilize Storybook for component and mock management. This includes preconfigured Storybook, component templates with best practices, great Typescript support, and bundle size tracking.  Tyler mentioned that DS-CLI is free and open source. 

One great part about DS-CLI is size tracking.  Each component can be tracked over time for package size. You can see when your bundles are growing – and why.

With the second, Intuit Proof, you can automate the development of test running in Storybook. Proof is a tapable testrunner for Storybook. Proof helps you automate tests for components and mocks by helping you know which ones have tests. In addition, Proof has built-in hooks for Applitools and Axe. So, instead of fumbling about inside Storybook to see what has been tested and what remains, you know what tests exist, which still need to be written, and helps you track their status. 

Proof is also free and open source.

Conclusion

Tyler’s presentation made it clear that a design system can make your life much easier, but you need to invest in tools and methodology to help ensure consistency – the benefit of a design system. 

You need a component evaluation system. You need mockups of your pages to test. And you need a testing environment.  Intuit does all this inside Storybook.

Next, you need a testing strategy to get components evaluated and integrated into your pages. Invest in unit testing and test runners. Especially if you care about being perceived as open to all users – invest in accessibility tools. And, finally, know that rendering matters – invest in visual evaluation solutions like Applitools.

Finally, you need some pieces of infrastructure to make your design system work. Intuit shares theirs open source.

If you make these investments up front, a design system can serve your development and test strategy to help you easily develop incremental improvements across your application without fear of code changes. 

Read More

• Impact of Visual AI on Test Automation Report
• A/B Testing: Validating Multiple Variations
• Modern Dev Teams Are More Efficient Than Ever — Quality Engineering Should Be Too.
• The Next Generation of Cross Browser Testing is Ultrafast
• Sign up for a free Applitools account
• Request an Applitools Demo

The post Design Systems and Testability appeared first on Automated Visual Testing | Applitools.