As a test automation engineer, finding the right tools and frameworks is crucial to building a successful test automation strategy. Playwright is an end-to-end testing framework that provides a robust set of features to create fast, reliable, and maintainable tests.

In a recent webinar, Playwright Ambassador and TAU instructor Renata Andrade shared several use cases and best practices for using the framework. Here are some of the most valuable takeaways for test automation engineers:

Use Playwright’s built-in locators for resilient tests.
Playwright recommends using attributes like “text”, “aria-label”, “alt”, and “placeholder” to find elements. These locators are less prone to breakage, leading to more robust tests.

Speed up test creation with the code generator.
The Playwright code generator can automatically generate test code for you. This is useful when you’re first creating tests to quickly get started. You can then tweak and build on the generated code.

Debug tests and view runs with UI mode and the trace viewer.
Playwright’s UI mode and VS Code extension provide visibility into your test runs. You can step through tests, pick locators, view failures, and optimize your tests. The trace viewer gives you a detailed trace of all steps in a test run, which is invaluable for troubleshooting.

Add visual testing with Applitools Eyes.
For complete validation, combine Playwright with Applitools for visual and UI testing. Applitools Eyes catches unintended changes in UI that can be missed by traditional test automation.

Handle dynamic elements with the right locators.
Use a combination of attributes like “text”, “aria-label”, “alt”, “placeholder”, CSS, and XPath to locate dynamic elements that frequently change. This enables you to test dynamic web pages.

Set cookies to test personalization.
You can set cookies in Playwright to handle scenarios like A/B testing where the web page or flow differs based on cookies. This is important for testing personalization on websites.

Playwright provides a robust set of features to build, run, debug, and maintain end-to-end web tests. By leveraging the use cases and best practices shared in the webinar, you can power up your test automation and build a successful testing strategy using Playwright. Watch the full recording and see the session materials.

The post Power Up Your Test Automation with Playwright appeared first on Automated Visual Testing | Applitools.

The post 9 Reasons to Attend the 2023 Test Automation University Conference appeared first on Automated Visual Testing | Applitools.

const addition = (a, b) => {
 return a + b
}

addition(1, 2) // returns 3
addition('1', '2') // returns '12'

const addition = (a: number, b: number) => {
 return a + b
}


addition(1, 2) // returns 3
addition('1', '2') // shows an error

How TypeScript works

TypeScript cannot be read by the browser. In order to run TypeScript code, it needs to be compiled. In other words, everything you create using TypeScript will be converted to JavaScript at some point.

To run the compiler, we can open the terminal and run the following command:

tsc addition.ts

This command will point TypeScript compiler (tsc) to our addition.ts file. It will create a JavaScript file alongside the original TypeScript file. If there are any “type errors” in the file, we’ll get an error into our terminal.

But you don’t have to do this manually every time. There’s a good chance your code editor has a TypeScript compiler running in the background as you type your code. With VS Code, this functionality comes out of the box, which makes sense since both TypeScript and VS Code are developed and maintained by Microsoft. Having the compiler running in the background is incredibly useful, mostly because this allows us to immediately see errors such as the one shown in the last example. Whenever there is such an error, we get feedback and an explanation:

In this case, we are passing a string into a function that requires us to pass numbers. This information comes from a compiler that runs inside the editor (VS code in my case).

Additionally, some modern testing tools such as Playwright and Cypress run the compiler on the fly and convert your TypeScript code into browser-readable JavaScript for you.

How to use TypeScript as a tester

Now that you know what TypeScript is and how it works, it is time to answer the most important question: why? Is TypeScript even useful for someone who focuses on test automation?

My answer is yes, and I would like to demonstrate this in a few examples. I’ll also give you a couple of reasons why I think test automation engineers should start getting familiar with TypeScript.

Typed libraries

As test automation engineers, we often implement many different libraries for our work. There’s no tool that fits all the needs, and many times we deal with plugins, integrations, toolsets, extensions, and libraries. When you start, you need to get yourself familiar with the API of that tool, dig into the documentation, try it out, and understand different commands. It’s a process. And it can be exhausting to get through that first mile.

TypeScript can be helpful in speeding up that process. Libraries that contain type definitions can really get you up to speed with using them. When a library or a plugin contains type definitions, it means that functions in that library will have the same type of checking implemented as in the example I have given earlier.

This means that whenever you e.g. pass a wrong argument to a function, you will see an error in your editor. In the following example, I am passing a number into a cy.type() function that will only accept text:

Code autocompletion

Besides checking for correct arguments, TypeScript can help with giving autocomplete suggestions. This can act as a quick search tool, when you are looking for the right command or argument.

I have recently made a plugin for testing API with Cypress, and TypeScript autocompletion helps with passing different attributes such as url, method, or request body:

Handling imported data

Working with data can often get complicated. Especially when working with complex datasets in which you have to navigate through multiple levels of data structure. One mistake can make the whole test fail and cause a headache when trying to debug that problem.

When data is imported to a test, for example from a JSON file, the structure of that JSON file is imported to the test as well. This is something that the TypeScript compiler inside the editor does for us. Let’s take a look at a simple JSON file that will seed data into our test:

While creating our test, the editor will guide us through the fixture file and suggest possible keys that can be inferred from that file:

Notice how we not only get the key names, but also the type of the key. We can use this type inference for both seeding the data into our tests, as well as making assertions. The best part about this? The fixture file and test are now interconnected. Whenever we change our fixture file, the test file will be affected as well. If e.g. we decide to change the name property in our JSON file to title, TypeScript compiler will notice this error even before we decide to run our test.

Tightening source code with test code

Probably my strongest argument for using TypeScript is the connection between test code and source code. Being able to tie things together is a game changer. Whenever the source code changes, it can have a direct effect on tests, and with TypeScript, there’s a high chance it will show even before you run your tests on a pipeline.

Let’s say you have an API test. If your developers use TypeScript, chances are they have a TypeScript definition of the API structure. As a tester, you can import that structure into your test and use it e.g. for testing the API response:

import Board from "trelloapp/src/typings/board";


it('Returns proper response when creating new board', () => {
 cy.request<Board>('POST', '/api/boards', { name })
   .then(({ body }) => {
     expect(body.name).to.eq(name)
     expect(body.id).to.exist
   })
 })

Similarly to the previous example with fixture files, whenever something changes in our typings file, we will notice the change in the test file as well.

Checking errors on CLI

A really powerful feature of TypeScript is the ability to check all errors in the project. We can do this by typing following command in the terminal:

tsc --noEmit

The –noEmit flag means that the TypeScript compiler will not create JavaScript files, but it will check for any errors on our files. We can check all our files in the project, which means that even if we have worked on a single file, all the files will be checked for errors.

We can check the health of our tests even before they are run. Whenever we change files in our project, we can check if type checks are still passing. Even without opening any files.

This is sometimes referred to as “static testing”. It enables us to add an additional layer of checks that will help us make less mistakes in our code.

A great advantage of this is that it runs super fast, making it a good candidate for a pre-commit check.

In fact, setting up such a check is very easy and can be done in three simple steps:

First, install pre-commit package via npm using following command:

npm install pre-commit --save-dev

Then, create a lint script in package.json:

"scripts": {
 "lint": "tsc --noEmit"
}

As a final step, define lint as one of the pre-commit checks in package.json:

"pre-commit": [ "lint" ]

From now on, whenever we try to commit, our tsc –noEmit script will run. If it throws any errors, we will not be able to commit staged files.

The post Using TypeScript for Test Automation appeared first on Automated Visual Testing | Applitools.

Test Automation University (also called “TAU”) is one of the best online platforms for learning testing and automation skills. TAU offers dozens of courses from the world’s leading instructors, and everything is available for free. The platform is proudly powered by Applitools. As of November 2022, nearly 140,000 students have signed up! TAU has become an invaluable part of the testing community at large. Personally, I know many software teams who use TAU courses as part of their internal onboarding and mentorship programs.

So, which TAU courses are currently the most popular? In this list, we’ll count down the top 10 most popular courses, ranked by the total number of course completions over the past year. Let’s go!

#10: Selenium WebDriver with Java

Starting off the list at #10 is Selenium WebDriver with Java by none other than Angie Jones. Even with the rise of alternatives like Cypress and Playwright, Selenium WebDriver continues to be one of the most popular tools for browser automation, and Java continues to be one of its most popular programming languages. Selenium WebDriver with Java could almost be considered the “default” choice for Web UI test automation.

In this course, Angie digs deep into the WebDriver API, teaching everything from the basics to advanced techniques. It’s a great course for building a firm foundation in automation with Selenium WebDriver.

#9: Python Programming

#9 on our list is one of our programming courses: Python Programming by Jess Ingrassellino. Python is hot right now. On whatever ranking, index, or article you find these days for the “most popular programming languages,” Python is right at the top of the list – often vying for the top spot with JavaScript. Python is also quite a popular language for test automation, with excellent frameworks like pytest, libraries like requests, and bindings for browser automation tools like Selenium WebDriver and Playwright.

In this course, Dr. Jess teaches programming in Python. This isn’t a test automation course – it’s a coding course that anyone could take. She covers both structured programming and object-oriented principles from the ground up. After two hours, you’ll be ready to start coding your own projects!

#8: API Test Automation with Postman

The #8 spot belongs to API Test Automation with Postman by Beth Marshall. In recent years, Postman has become the go-to tool for building and testing APIs. You could almost think of it as an IDE for APIs. Many test teams use Postman to automate their API test suites.

Beth walks through everything you need to know about automating API tests with Postman in this course. She covers basic features, mocks, monitors, workspaces, and more. Definitely take this course if you want to take your API testing skills to the next level!

#7: Introduction to Cypress

Lucky #7 is Introduction to Cypress by Gil Tayar. Cypress is one of the most popular web testing frameworks these days, even rivaling Selenium WebDriver. With its concise syntax, rich debugging features, and JavaScript-native approach, it’s become the darling end-to-end test framework for frontend developers.

It’s no surprise that Gil’s Cypress course would be in the top ten. In this course, Gil teaches how to set up and run tests in Cypress from scratch. He covers both the Cypress app and the CLI, and he even covers how to do visual testing with Cypress.

#6: Exploring Service APIs through Test Automation

The sixth most popular TAU course is Exploring Service APIs through Test Automation by Amber Race. API testing is just as important as UI testing, and this course is a great way to start learning what it’s all about. In fact, this is a great course to take before API Test Automation with Postman.

This course was actually the second course we launched on TAU. It’s almost as old as TAU itself! In it, Amber shows how to explore APIs first and then test them using the POISED strategy.

#5: IntelliJ for Test Automation Engineers

Coming in at #5 is IntelliJ for Test Automation Engineers by Corina Pip. Java is one of the most popular languages for test automation, and IntelliJ is arguably the best and most popular Java IDE on the market today. Whether you build frontend apps, backend services, or test automation, you need proper development tools to get the job done.

Corina is a Java pro. In this course, she teaches how to maximize the value you get out of IntelliJ – and specifically for test automation. She walks through all those complicated menus and options you may have ignored otherwise to help you become a highly efficient engineer.

#4: Java Programming

Our list is winding down! At #4, we have Java Programming by Angie Jones. For the third time, a Java-based course appears on this list. That’s no surprise, as we’ve said before that Java remains a dominant programming language for test automation.

Like the Python Programming course at spot #9, Angie’s course is a programming course: it teaches the fundamentals of the Java language. Angie covers everything from “Hello World” to exceptions, polymorphism, and the Collections Framework. Clocking in at just under six hours, this is also one of the most comprehensive courses in the TAU catalog. Angie is also an official Java Champion, so you know this course is top-notch.

#3: Introduction to JavaScript

It’s time for the top three! The bronze medal goes to Introduction to JavaScript by Mark Thompson. JavaScript is the language of the Web, so it should be no surprise that it is also a top language for test automation. Popular test frameworks like Cypress, Playwright, and Jest all use JavaScript.

This is the third programming course TAU offers, and also the top one in this ranking! In this course, Mark provides a very accessible onramp to start programming in JavaScript. He covers the rock-solid basics: variables, conditionals, loops, functions, and classes. These concepts apply to all other programming languages, too, so it’s a great course for anyone who is new to coding.

#2: Web Element Locator Strategies

I’m partial to the course in second place – Web Element Locator Strategies by me, Andrew Knight! This was the first course I developed for TAU, long before I ever joined Applitools.

In whatever test framework or language you use for UI-based test automation, you need to use locators to find elements on the page. Locators can use IDs, CSS selectors, or XPaths to uniquely identify elements. This course teaches all the tips and tricks to write locators for any page, including the tricky stuff!

#1: Setting a Foundation for Successful Test Automation

It should come as no surprise that the #1 course on TAU in terms of course completions is Setting a Foundation for Successful Test Automation by Angie Jones. This course was the very first course published to TAU, and it is the first course in almost all the Learning Paths.

Before starting any test automation project, you must set clear goals with a robust strategy that meets your business objectives. Testing strategies must be comprehensive – they include culture, tooling, scaling, and longevity. While test tools and frameworks will come and go, common-sense planning will always be needed. Angie’s course is a timeless classic for teams striving for success with test automation.

What can we learn from these trends?

What other courses are popular?

The post The Top 10 Test Automation University Courses appeared first on Automated Visual Testing | Applitools.

Did you attend the Applitools workshop last week for You, Me, and Accessibility: Empathy and Human-Centered Design Thinking? If you missed it, don’t fret; the recording can be found on-demand and my recap of key learnings, Q&A, and poll results are right here!

Applitools workshops are designed to bring together talented people within software development teams to provide hands-on learning experiences. Folks shared with me after the workshop just how enjoyable learning about accessibility, empathy and human-centered design thinking is and discovered ways to share with their peers and product teams.

What we covered in this workshop

Human-centered design thinking

What it is and how we can use it!

Empathy sessions

What are they? Who are they for? The benefits they provide.

#A11y tools and resources

Manual and automated testing resources and much more!

Great products come from conversations, experiments, and learning. Human-centered design thinking is a wonderful process for problem solving. Here you can empathize, define, ideate, prototype, and test your solutions early and often! Empathy sessions fall right in line with the concepts of human-centered design thinking as an opportunity for exploratory testing of our users. 

We kicked things off with a base question of Web Content Accessibility Guidelines (WCAG) awareness. I am thrilled when I see a majority has knowledge of accessibility when it comes to digital content, and truly appreciate an opportunity to share with folks newer to the topic.

I spoke at length about common accessibility (often shortened to a11y) categories, where to consider them in the software development lifecycle (SDLC), and what the intersection of software and empathy looks like. We explored human-centered design thinking as a tool and demonstrated ways empathy drives innovation. Humans are emotional beings! There is value in proficiency in the technical side of software development as well as the human-centered side.

How does empathy drive innovation?

Empathy supercharges creativity and innovation by giving us insight into experiencing what someone else is thinking and feeling and allows us to more fully understand their motivations and anticipate their needs in ways the user may not be able to express themselves.

For the purpose of our workshop, we wanted to dig into what role folks play on their current product teams. To do this, we asked them! We went over what happens in the software development lifecycle and why it’s important to talk about how users are using technology. There we had opportunities to explore the different types of users who could potentially use our products or services. Then we discussed how those opportunities can be expanded upon by utilizing design thinking and in empathy sessions. 

What is human-centered design thinking?

Based on three principles:

1. Collaboration
2. Empathy
3. Experimentation

What happens in the software development lifecycle? We talk about the products and services that we want to make for our users. We make the products and services to the best of our ability. We test and ship those products and services to our users.

We wrapped up the workshop with some wonderful questions and answers that I wanted to also provide additional feedback and resources for. 

If you enjoyed this recap, be sure to follow me on social media for more content like it on LinkedIn and Twitter. You can also read more of my insight into empathy, software testing, and hot accessibility testing tools on medium!

The post You, Me, and Accessibility: Empathy and Human-Centered Design Thinking Webinar Recap appeared first on Automated Visual Testing | Applitools.

Learn about what’s new in iOS 16, including some new updates test engineers should be looking out for.

It’s an exciting time of the year for anyone who uses Apple devices – and that includes QA engineers charged with mobile testing. Apple has just unveiled iOS 16, and as usual it is filled with new features for iOS users to enjoy.

Many of these new features, of course, affect the look and feel and usability of any application running on iOS. If you’re in QA, that means you’ve now got a lot of new testing to do to make sure your application works as perfectly on iOS 16 as it did on previous versions of the operating system.

For example, Apple has just upgraded their iconic “notch” into a “Dynamic Island.” This is significant redesign of a small but highly visual component that your users will see every time they look at their phone. If your app doesn’t function appropriately with this new UI change, your users will notice.

If you’re using Native Mobile Grid for your mobile testing, no need to worry, as Native Mobile Grid already supports automated testing of iOS 16 on Apple devices.

With this in mind, let’s take a look through some of the most exciting new features of iOS 16, with a focus on how they can affect your life as a test engineer.

Customizable Lock Screen

The lockscreen on iOS 16 devices can now be customized far more than before, going beyond changing the background image – you can now alter the appearance of the time as well as add new widgets. Another notable change here is that notifications now pop up from the bottom instead of the top.

As a QA engineer, there are a few things to consider here. First, if your app will have a new lockscreen widget, you certainly need to test it carefully. Performing visual regression testing and getting contrast right will be especially important on an uncertain background.

Even if you don’t develop a widget, it’s worth thinking about (and then verifying) whether the user experience could be affected by your notifications moving from the top of the user’s screen to the bottom. Be sure and take a look at how they will appear when stacked as well to make sure the right information is always visible.

Notch –> Dynamic Island

As we mentioned above, the notch is getting redesigned into a “Dynamic Island.” This new version of the cutout required for the front-facing camera can now present contextual information about the app you’re using. It will expand and contract based on the info it’s displaying, so it’s not a fixed size.

That means your app may now be resizing around the new “Dynamic Island” in ways it never did with the old notch. Similarly, your contextual notifications may not look quite the same either. This is definitely something worth testing to make sure the user experience is still exactly the way you meant it to be.

Other New iOS 16 Features

There are a lot of other new features, of course. Some of these may not have as direct an impact on the UI or functionality of you own applications, but it’s worth being familiar with them all. Here are a few of the other biggest changes – check them carefully against your own app and be sure to test accordingly.

• Send, Edit and Unsend Messages: You can now send, edit and unsend content in the Messages app, and you can now send/unsend (as well as schedule delivery) in the Mail app as well
• Notifications and Live Activities: As mentioned, notifications now come up from the bottom. They can also “update” so that you don’t need to get repeated new notifications from the same app (eg: sports games scores, rideshare ETAs)
• Live Text and Visual Lookup: iOS users can now extract live text from both photos and videos, as well as copy the subject of an image out of its background and paste it elsewhere
• Focus Mode and Focus Filters: Focus mode (to limit distractions) can now be attached to custom lockscreens, and applied not just to an app but within an app (eg: restricting specific tabs in a browser)
• Private Access Tokens: For some apps and websites, Apple will use these tokens to verify that users are human and bypass traditional CAPTCHA checks
• Other improvements: The Fitness app, Health app, Maps app, iCloud, Wallet and more all got various improvements as well. Siri did too (you can now “speak” emojis ?). See the full list of iOS 16 updates.

Make Your Mobile Testing Easier

Mobile testing is a challenge for many organizations. The number of devices, browsers and screens in play make achieving full coverage extremely time-consuming using traditional mobile testing solutions. At Applitools, we’re focused on making software testing easier and more effective – that’s why we pioneered our industry-leading Visual AI. With the new Native Mobile Grid, you can significantly reduce the time you spend testing mobile apps while ensuring full coverage in a native environment.

Learn more about how you can scale your mobile automation testing with Native Mobile Grid, and sign up for access to get started with Native Mobile Grid today.

The post iOS 16 – What’s New for Test Engineers appeared first on Automated Visual Testing | Applitools.

Learn why cross-browser testing is so important and an approach you can take to make cross-browser testing with Selenium much faster.

What is Cross Browser Testing?

Cross-browser testing is a form of functional testing in which an application is tested on multiple browsers (Chrome, Firefox, Edge, Safari, IE, etc.) to validate that functionality performs as expected.

In other words, it is designed to answer the question: Does your app work the way it’s supposed to on every browser your customers use?

Why is Cross Browser Testing Important?

While modern browsers generally conform to key web standards today, important problems remain. Differences in interpretations of web standards, varying support for new CSS or other design features, and rendering discrepancies between the different browsers can all yield a user experience that is different from one browser to the next.

A modern application needs to perform as expected across all major browsers. Not only is this a baseline user expectation these days, but it is critical to delivering a positive user experience and a successful app.

At the same time, the number of screen combinations (between screen sizes, devices and versions) is rising quickly. In recent years the number of screens required to test has exploded, rising to an industry average of 81,480 screens and reaching 681,296 for the top 30% of companies.

Ensuring complete coverage of each screen on every browser is a common challenge. Effective and fast cross-browser testing can help alleviate the bottleneck from all these screens that require testing.

How to Perform Modern Cross Browser Testing in Selenium with Visual Testing

Traditional approaches to cross-browser testing in Selenium have existed for a while, and while they still work, they have not scaled well to handle the challenge of complex modern applications. They can be time-consuming to build, slow to execute and challenging to maintain in the face of apps that change frequently.

Applitools Developer Advocate and Test Automation University Director Andrew Knight (AKA Pandy Knight) recently conducted a hands-on workshop where he explored the history of cross-browser testing, its evolution over time and the pros and cons of different approaches.

Andrew then explores a modern cross-browser testing solution with Selenium and Applitools. He walks you through a live demo (which you can replicate yourself by following his shared Github repo) and explains the benefits and how to get started. He also covers how you can accelerate test automation with integration into CI/CD to achieve Continuous Testing.

Check out the workshop below, and follow along with the Github repo here.

More on Cross Browser Testing in Cypress, Playwright or Storybook

At Applitools we are dedicated to making software testing faster and easier so that testers can be more effective and apps can be visually perfect. That’s why we use our industry-leading Visual AI and built the Applitools Ultrafast Grid, a key component of the Applitools Test Cloud that enables ultrafast cross-browser testing. If you’re looking to do cross-browser testing better but don’t use Selenium, be sure to check out these links too for more info on how we can help:

• Testing Storybook Components in Any Browser – Without Writing Any New Tests!
• Running Lightning-Fast Cross-Browser Playwright Tests Against any Browser
• How to Run Cross Browser Tests with Cypress on All Browsers (and follow-up Q&A)
• Learn More about How to Use Applitools for Cross-Browser Testing

The post Ultrafast Cross Browser Testing with Selenium Java appeared first on Automated Visual Testing | Applitools.

Learn how to get started with web accessibility testing with this list of the best paid, free and open source accessibility testing tools.

If you visit the Web Accessibility Evaluation Tools List, there are a whopping 167 tools to choose from! As someone who is just starting in the world of accessibility, how do you decide which tools to choose?

There are of course various factors to consider when choosing an accessibility tool, but in this blog post, I’ll share a few tools that I have found really useful when it comes to accessibility testing. If you tuned in to my talk on Shifting Accessibility Testing to the Left (or read my blog post), you would know that I like to group the accessibility testing tools in different areas. These areas are:

• Tools requiring human assistance
• Semi automated tools in the form of browser extensions
• Automated tools

Accessibility Testing Tools Requiring Human Assistance

Even though there’s a growing number of automated tools out there, accessibility testing still requires human assistance to make sure that the experience we are testing closely matches the one our users will have. The following tools are my go-to when it comes to manually testing for accessibility.

Keyboards

The first tool is my very own keyboard. Making sure that your website is keyboard friendly and compatible already makes it more accessible than many other websites.

To get started with keyboard compatibility testing, you need to know basic keystrokes such as TAB, Enter, Arrow keys (← ↑ → ↓), just to name a few, to make sure that you can still interact with the website as if you are using a mouse.

When it comes to testing with a keyboard, this can surface accessibility considerations such as:

• Are your elements focusable when the user tabs through it?
• Do you have a skip to the main content link which is only visible when tabbed through with a keyboard?
• When presented with a modal or a pop-up, can the users exit the modal and interact with what they were presented before?

Screen Readers

Using a screen reader can be overwhelming for people who don’t use it but screen readers are a must when it comes to testing for accessibility. Depending on the operating system that you are using, there is an available screen reader software for you to use such as VoiceOver, JAWS, NVDA and TalkBack. Spend some time familiarizing yourself on how to use a screen reader to make sure that your websites are accessible for these users.

Zoom/Magnification

Users who have low vision need to have a way to easily perceive, navigate and interact with the content that is presented to them. By using the zoom or magnification too that’s built into browsers, you can zoom in up to 200% (or more) and verify if the elements are displayed nicely and are still interactable.

Semi-Automated Accessibility Testing Tools in the Form of Browser Extensions

Browser extensions are a quick way to help you surface any accessibility issues that your websites might have. Most of the browser extensions are provided to you for free with additional features unlocked if you purchase their commercial version. The common thing with the extensions below is they all provide an easy way to check violations. All of the extensions, apart from ColorBlindly, also provide an easy to digest output so you can share with your teams a list of accessibility issues.

Axe DevTools and WAVE

Axe DevTools and WAVE are two extensions that you can install and integrate easily in your browser of choice. These accessibility extensions scan a specific web page and report any accessibility violations that it finds. These are great tools to get started with accessibility, especially if you are a beginner. They provide useful information such as the description of the accessibility violation, its impact, how to fix it and elements that are impacted.

Google Lighthouse

Google Lighthouse, which is already built into Google Chrome’s developer tools, provides an easy way to perform accessibility audits. It can also measure web performance apart from accessibility and can provide recommendations on how to fix the issues caught. Don’t get too fixated though with the Lighthouse accessibility scores as this is not a complete indication that your website is accessible.

ColorBlindly

ColorBlindly is an easy to use Chrome extension that can simulate different types of color blindness with just one click. This extension can help you verify if the color schemes of your website are accessible from a wide range of color blindness.

Automated Accessibility Testing Tools

Now, in order to shift accessibility testing as early as possible, apart from having early conversations with your team, leveraging automation is key so that you can focus on areas where accessibility testing is needed the most. What’s common with these tools is that you can easily integrate them as part of your continuous integration pipelines and it can provide a safety net so your team can be confident in making changes or introducing new features.

Axe CLI

Command line lovers, this tool is for you! Axe CLI is a command line tool which provides a way to perform accessibility audits straight from your command line. This is particularly useful if you want to quickly scan various pages from your command line.

The scan can be configurable in the sense that you can disable certain accessibility rules, include or exclude certain elements and modify the accessibility report. 

If you’re looking for a quick tool that you can easily integrate as part of your pipelines, give Axe CLI a try.

Cypress and cypress-axe

Good news for Cypress users! Did you know that you can easily integrate accessibility tests just by installing a plugin called cypress-axe? Cypress-axe uses the axe-core library and lets you audit pages or components straight from your Cypress tests. I have discussed this in more detail in my course Test Automation for Accessibility so if you’re interested to find out how the plugin works, check out this course from Test Automation University.

Similarly, if you’re using other testing frameworks, you’re also in luck because axe-core can be integrated with other frameworks or testing libraries. Whether you are using Playwright, WebdriverIO, Selenium or others, axe-core has a library for you which can be found here: projects that use axe-core.

Applitools Contrast Advisor

Did you know that Applitools also supports accessibility testing? If you’re already using Applitools for visual testing, then you can also try their Contrast Advisor tool which can detect contrast violations using artificial intelligence. With Contrast Advisor, it can easily integrate into your existing workflow and pipelines already so no additional coding setup is needed. You can also validate the contrast of images and native applications easily with this tool.

Wrap Up

The tools above are by no means a complete list, but should help you get started when it comes to accessibility testing. Regardless of what tools you choose, you should have the same goal and that is to catch as many accessibility issues as possible before giving it to your real users. 

By using a combination of these tools as early as possible, along with other accessibility testing strategies, you can ensure that your user experience is inclusive. The above tools should not be a replacement for accessibility testing with real users but should complement it instead.

How Can I Help?

Accessibility doesn’t start and end with tools. It requires a change of culture and wider buy-ins to make sure that everyone is on the same page. If you or anyone from your team requires specific consultation help with regards to accessibility, I’m happy to have an introductory chat to help you nurture accessibility within your team. You can contact me via Twitter @mcruzdrake or via my personal blog at mariedrake.com.

The post Top 10 Accessibility Testing Tools for Websites appeared first on Automated Visual Testing | Applitools.

Learn about common localization bugs, the traditional challenges involved in finding them, and solutions that can make localization testing far easier.

What is Localization?

Localization is the process of customizing a software application that was originally designed for a domestic market so that it can be released in a specific foreign market.

How to Get Started with Localization

Localization testing usually involves substantial changes of the application’s UI, including the translation of all texts to the target language, replacement of icons and images, and many other culture, language, and country-specific adjustments, that affect the presentation of data (e.g., date and time formats, alphabetical sorting order, etc.). Due to the lack of in-house language expertise, localization usually involves in-house personnel as well as outside contractors, and localization service providers.

Before a software application is localized for the first time, it must undergo a process of Internationalization.

What is Internationalization?

Internationalization often involves an extensive development and re-engineering effort which goal is to allow the application to operate in localized environments and to correctly process and display localized data. In addition, locale-specific resources such as texts, images and documentation files, are isolated from the application code and placed in external resource files, so they can be easily replaced without requiring further development efforts.

Once an application is internationalized, the engineering effort required to localize it to a new language or culture is drastically reduced. However, the same is not true for UI localization testing.

The Challenge of UI Localization Testing

Every time an application is localized to a new language, the application changes, or the resources of a supported localization change, the localized UI must be thoroughly tested for localization and internationalization (LI) bugs.

Common Localization and Internationalization Bugs Most Testers can Catch

LI bugs which can be detected by testers that are not language experts include:

• Broken functionality – the execution environment, data or translated resources of a new locale, may uncover internationalization bugs can prevent the application from running or break some of its functionality.
• Untranslated text – text appearing in text fields or images of the localized UI is left untranslated. This indicates that certain resources were not translated or that the original text is hard-coded in the UI and not exported to the resource files.
• Text overlap / overflow – the translated text may require more space than available in its containing control, resulting with the text overflowing the bounds of the control and possibly overlapping or hiding other UI elements.
• Layout corruption – UI controls dynamically adjust their size and position to the expanded or contracted size of the localized text, icons or images, resulting with misaligned, overlapping, missing or redundant UI artifacts.
• Oversized windows and dialogs – multiple expanded texts and images can result with oversized tooltips, dialogs and windows. In extreme situations, expanded dialogs and windows may only be partially visible in low screen resolutions.
• Inadequate fonts – a control’s font cannot properly display some characters of the target language. This usually results with question marks or glyphs being displayed instead of the expected text.

Localization and Internationalization Bugs Requiring Language Expertise

Other common LI bugs which can only be detected with the help of a language expert include:

• Mistranslation – translated text that appears once in the resource files, may appear multiple times in different parts of the application. The context in which the text appears can vary its meaning and require a different translation.
• Wrong images and icons – images and icons were replaced with wrong or inappropriate graphics.
• Text truncation – the translated text may require more space than available in its containing control, resulting with a truncated string.
• Locale violations – wrong date, time, number and currency formats, punctuation, alphabetical sort order, etc.

Localization and Internationalization Bugs are Hard to Find

An unfortunate characteristic of LI bugs, is that they require a lot of effort to find. To uncover such bugs, a tester (assisted by a language expert) must carefully inspect each and every window, dialog, tooltip, menu item, and any other UI state of the application. Since most of these bugs are sensitive to the size and layout of the application, tests must be repeated on a variety of execution environments (e.g., different operating systems, web browsers, devices, etc.) and screen resolutions. Furthermore, if the application window is resizable, tests should also be repeated for various window sizes.

Why is UI Localization Testing Hard?

There are several other factors that contribute to the complexity of UI Localization testing:

• Lack of automation – most of the common LI bugs listed above are visual and cannot be effectively detected by traditional functional test automation tools. Manual inspection of the localized UI is slower than with a non-localized UI because it is  unreadable to the tester.
• Lack of in-house language expertise – since many of the common LI bugs can only be detected with the help of external language experts, which are usually not testers and are not familiar with the application under test, LI testing often requires an in-house tester to perform tests together with a language expert. In many cases, these experts work on multiple projects for multiple customers in parallel, and their occasional lack of availability can substantially delay test cycles and product releases. Similarly, delays can occur while waiting for the translation of changed resources, or while waiting for translation bugs to be fixed.
• Time constraints – localization projects usually begin at late stages of the development lifecycle, after the application UI has stabilized. In many cases, testers are left with little time to properly perform localization tests, and are under constant pressure to avoid delaying the product release.
• Bug severity – UI localization bugs such as missing or garbled text are often considered critical, and therefore must be fixed and verified before the product is released.

Due to these factors, maintaining multiple localized application versions and adding new ones, incurs a huge overhead on quality assurance teams.

Fortunately, there is a modern solution that can make localization testing significantly easier – Automated Visual Testing.

How to Automate Localization Testing with Visual Testing

Visual test automation tools can be applied to UI localization testing to eliminate unnecessary manual involvement of testers and language experts, and drastically shorten test cycles.

To understand this, let’s first understand what visual testing is, and then how to apply visual testing to localization testing.

What is Visual Testing?

Visual testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, visual testing focuses on validating the layout and appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.

Visual Test Automation tools can automate most of the activities involved in visual testing. They can easily detect many common UI localization bugs such as text overlap or overflow, layout corruptions, oversized windows and dialogs, etc. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation.

For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. Driving the AUT through its different UI states can be easily automated using a variety of open-source and commercial tools (e.g., Selenium, Cypress, etc.). If the tool is properly configured to rely on internal UI object identifiers, the same automation script/program can be used to drive the AUT in all of its localized versions.

So, how can we use this to simplify UI localization testing?

How Automated Visual Testing Simplifies UI Localization Testing

• Preparation – in order to provide translators with the context required to properly localize the application, screenshots of the application’s UI are often delivered along with the resource files to be localized. The process of manually collecting these screenshots is laborious, time consuming, and error prone. When a visual test automation tool is in place, updated screenshots of all UI states are always available and can be shared with translators with a click of a button. When an application changes, the tool can highlight only those screens (in the source language) that differ from the previous version so that only those screens are provided to translators. Some visual test automation tools also provide animated “playbacks” of tests showing the different screens, and the human activities leading from one screen to the next (e.g., clicks, mouse movements, keyboard strokes, etc.).  Such animated playbacks provide much more context than standalone screenshots and are more easily understood by translators, which are usually not familiar with the application being localized. Employing a visual test automation tool can substantially shorten the localization project’s preparation phase and assist in producing higher quality preliminary translations, which in turn can lead to fewer and shorter test cycles.

• Testing localization changes – visual test automation tools work by comparing screenshots of an application against a set of previously approved “expected” screenshots called the baseline. After receiving the translated resources and integrating them with the application, a visual test of the updated localized application can be automatically executed using the previous localized version as a baseline. The tool will then report all screens that contain visual changes and will also highlight the exact changes in each of the changed screens. This report can then be inspected by testers and external language experts without having to manually interact with the localized application. By only focusing on the screens that changed, a huge amount of time and effort can be saved. As we showed above, most UI localization bugs are visual by nature and are therefore sensitive to the execution environment (browser, operating system, device, screen resolution, etc.). Since visual test automation tools automatically execute tests in all required execution environments, testing cycles can be drastically shortened.

• Testing new localizations – when localizing an application for a new language, no localized baseline is available to compare with. However, visual test automation tools can be configured to perform comparisons at the layout level, meaning that only layout inconsistencies (e.g., missing or overflowing text, UI elements appearing out of place, broken paragraphs or columns, etc.) are flagged as differences. By using layout comparison, a newly localized application can be automatically compared with its domestic version, to obtain a report indicating all layout inconsistencies, in all execution environments and screen resolutions.

• Incremental validation – when localization defects are addressed by translators and developers, the updated application must be tested again to make sure that all reported defects were fixed and that no new defects were introduced. By using the latest localized version as the baseline with which to compare the newly updated application, testers can easily identify the actual changes between the two versions, and quickly verify their validity, instead of manually testing the entire application.

• Regression testing – whenever changes are introduced to a localized application, regression testing must be performed to make sure that no localization bugs were introduced, even if no direct changes were made to the application’s localizable resources. For example, a UI control can be modified or replaced, the contents of a window may be repositioned, or some internal logic that affects the application’s output may change. It is practically impossible to manually perform these tests, especially with today’s Agile and continuous delivery practices, which dictate extremely short release cycles. Visual test automation tools can continuously verify that no unexpected UI changes occur in any of the localized versions of the application, after each and every change to the application.

• Collateral material – in additional to localizing the application itself, localized versions of its user manual, documentation and other marketing and sales collateral must be created. For this purpose, updated screenshots of the application must be obtained. As described above, a visual test automation tool can provide up-to-date screenshots of any part of the application in any execution environment. The immediate availability of these screenshots significantly reduces the chance of including out-of-date application images in collaterals and eliminates the manual effort involved in obtaining them after each application change.

Application localization is notoriously difficult and complex. Manually testing for UI localization bugs, during and between localization projects, is extremely time consuming, error-prone, and requires the involvement of external language experts.

Visual test automation tools are a modern breed of test automation tools that can effectively eliminate unnecessary manual involvement, drastically shorten the duration of localization projects, and increase the quality of localized applications.

Applitools Automated Visual Testing and Localization Testing

Applitools has pioneered the use of Visual AI to deliver the best visual testing in the industry. You can learn more about how Applitools can help you with localization testing, or to get started with Applitools today, request a demo or sign up for a free Applitools account.

Editor’s Note: Parts of this post were originally published in two parts in 2017/2018, and have since been updated for accuracy and completeness.

The post Getting Started with Localization Testing appeared first on Automated Visual Testing | Applitools.

Wondering how to choose between Playwright vs Selenium for your test automation? Read on to see a comparison between the two popular test automation tools.

When it comes to web test automation, Selenium has been the dominant industry tool for several years. However, there are many other automated testing tools on the market. Playwright is a newer tool that has been gaining popularity. How do their features compare, and which one should you choose?

What is Selenium?

Selenium is a long-running open source tool for browser automation. It was originally conceived in 2004 by Jason Huggins, and has been actively developed ever since. Selenium is a widely-used tool with a huge community of users, and the Selenium WebDriver interface even became an official W3C Recommendation in 2018.

The framework is capable of automating and controlling web browsers and interacting with UI elements, and it’s the most popular framework in the industry today. There are several tools in the Selenium suite, including:

• Selenium WebDriver: WebDriver provides a flexible collection of open source APIs that can be used to easily test web applications
• Selenium IDE: This record-and-playback tool enables rapid test development for both engineers and non-technical users
• Selenium Grid: The Grid lets you distribute and run tests in parallel on multiple machines

The impact of Selenium goes even beyond the core framework, as a number of other popular tools, such as Appium and WebDriverIO, have been built directly on top of Selenium’s API.

Selenium is under active development and recently unveiled a major version update to Selenium 4. It supports just about all major browsers and popular programming languages. Thanks to a wide footprint of use and extensive community support, the Selenium open source project continues to be a formidable presence in the browser automation space.

What is Playwright?

Playwright is a relatively new open source tool for browser automation, with its first version released by Microsoft in 2020. It was built by the team behind Puppeteer, which is a headless testing framework for Chrome/Chromium. Playwright goes beyond Puppeteer and provides support for multiple browsers, among other changes.

Playwright is designed for end-to-end automated testing of web apps. It’s cross-platform, cross-browser and cross-language, and includes helpful features like auto-waiting. It is specifically engineered for the modern web and generally runs very quickly, even for complex testing projects.

While far newer than Selenium, Playwright is picking up steam quickly and has a growing following. Due in part to its young age, it supports fewer browsers/languages than Selenium, but by the same token it also includes newer features and capabilities that are more aligned with the modern web. It is actively developed by Microsoft.

Selenium vs Playwright

Selenium and Playwright are both capable web automation tools, and each has its own strengths and weaknesses. Depending on your needs, either one could serve you best. Do you need a wider array of browser/language support? How much does a long track record of support and active development matter to you? Is test execution speed paramount? 

Each tool is open source, cross-language and developer friendly. Both support CI/CD (via Jenkins, Azure Pipelines, etc.), and advanced features like screenshot testing and automated visual testing. However, there are some key architectural and historical differences between the two that explain some of their biggest differences.

Selenium Architecture and History

• Architecture: Selenium uses the WebDriver API to interact between web browsers and browser drivers. It operates by translating test cases into JSON and sending them to the browsers, which then execute the commands and send an HTTP response back.
• History: Selenium has been in continuous operation and development for 18+ years. As a longstanding open source project, it offers broad support for browsers/languages, a wide range of community resources and an ecosystem of support.

Playwright Architecture and History

• Architecture: Playwright uses a WebSocket connection rather than the WebDriver API and HTTP. This stays open for the duration of the test, so everything is sent on one connection. This is one reason why Playwright’s execution speeds tend to be faster.
• History: Playwright is fairly new to the automation scene. It is faster than Selenium and has capabilities that Selenium lacks, but it does not yet have as broad a range of support for browsers/languages or community support. It is open source and backed by Microsoft.

Comparing Playwright vs Selenium Features

It’s important to consider your own needs and pain points when choosing your next test automation framework. The table below will help you compare Playwright vs Selenium.

Should You Use Selenium or Playwright for Test Automation?

Is Selenium better than Playwright? Or is Playwright better than Selenium? Selenium and Playwright both have a number of things going for them – there’s no easy answer here. When choosing between Selenium vs Playwright, it’s important to understand your own requirements and research your options before deciding on a winner.

Selenium vs Playwright: Let the Code Speak

A helpful way to go beyond lists of features and try to get a feel for the practical advantages of each tool is to go straight to the code and compare real-world examples side by side. At Applitools, our goal is to make test automation easier for you – so that’s what we did! 

In the video below, you can see a head to head comparison of Playwright vs Selenium. Angie Jones and Andrew Knight take you through ten rounds of a straight-to-the-code battle, with the live audience deciding the winning framework for each round. Check it out for a unique look at the differences between Playwright and Selenium.

If you like these code battles and want more, we’ve also pitted Playwright vs Cypress and Selenium vs Cypress – check out all our versus battles here.

In fact, our original Playwright vs Cypress battle (recap here) was so popular that we’ve even scheduled our first rematch. Who will win this time? Register for the Playwright vs Cypress Rematch now to join in and vote for the winner yourself!

Learn More about Playwright vs Selenium

Want to learn more about Playwright or Selenium? Keep reading below to dig deeper into the two tools.

• Your Favorite Questions about Playwright, Answered [blog]
• What’s New In Selenium 4? [blog]
• Playwright: A New Test Automation Framework for the Modern Web [webinar w/team at Microsoft]
• Running Lightning-Fast Cross-Browser Playwright Tests Against any Browser [blog]
• Modern Cross Browser Testing with Selenium in Java [webinar]
• Playwright with JavaScript [free course]
• Selenium WebDriver with Java [free course]
• Playing with Playwright [blog]
• Playing with Playwright – Java API and Playwright vs Selenium [blog]
• 16 reasons why to use Selenium IDE in 2021 (and 1 why not) [blog]

The post Playwright vs Selenium: What are the Main Differences and Which is Better? appeared first on Automated Visual Testing | Applitools.

Curious about the latest updates in Storybook.js, including the upcoming Storybook 7? In this post, which will be continuously updated, we sum up the latest Storybook news.

The highly anticipated Storybook 7.0 is currently in alpha, and there is a lot to get excited about. Let’s take a look at everything we know so far.

What’s Coming in Storybook 7

Storybook 7 promises significant changes. In fact, the Storybook team describes it as “a full rework of Storybook’s core with fast build and next-generation interaction testing.” Interaction testing was first included in the most recent Storybook 6.5 release, but we can expect further development there as Storybook 7 develops.

A Storybook 7 Design Preview

Storybook’s developers have just revealed a “sneak peek” at the design and layout in Storybook 7 [update 8/18: these changes are now available in the latest alpha]. Here’s a few of the changes that were highlighted:

• 3.5% more screen space for the Canvas (where components are developed in isolation).
• A “Reload” tool to reload a selected story (component) without refreshing the whole browser.
• Better access for integrators to the same design patterns used to develop Storybook.
• 196 icons that can be used in all projects, each redrawn from scratch. This is 20 more icons than the previous set.
• Form components like Toggle and Slider conform to the new design language.
• Pre-bundling of Storybook to deliver faster start times and avoid dependency issues (dependencies were also audited to reduce bundle size).

Storybook 7 Release Date

We don’t yet know the exact release date for Storybook 7, but we can guess based on their recent development history.

Storybook Version 6 was originally released in August 2020. Since then, there have been 5 major updates, most recently ending in version 6.5 which was released in May 2022. On the journey to the Storybook 6 release, the development team hit the following milestones:

• 47 alphas between January 2020 and April 2020
• 47 betas between April 2020 and July 2020
• 30 release candidates between July 2020 and August 2020

Storybook 7 began its first alpha in June 2022, and as of this writing is on alpha-26. If it follows the same trajectory is Storybook 6, we can estimate that it will enter beta in September, RC in December and official release in January 2023. Of course, time will tell, and we’ll update this post when any concrete information becomes available.

How Visual Testing Helps Component Testing

Component testing is form of software testing that focuses on software components in isolation. Component testing takes each rendered state (or Storybook story) and tests it.

Visual testing of components allows teams to find bugs earlier – and without writing any additional test code. It works across a variety of browsers and viewports at speeds almost as fast as unit testing.

You can learn more about how you can save time by using Applitools and our AI-powered visual testing with Storybook here:

• Visual Testing with Storybook
• Testing Storybook Components in Any Browser – Without Writing Any New Tests!
• How to Add Applitools Eyes to Storybook

Stay Tuned

We’ll be sure to keep this page updated with the latest on what’s new in Storybook 7, so check back often. And of course, we’re working hard to ensure our own Applitools SDKs for Storybook React, Storybook Angular and Storybook Vue are always compatible with the latest Storybook features.

Last Updated: August 26th, 2022

The post What’s New in Storybook 7? appeared first on Automated Visual Testing | Applitools.

For some of us, it’s hard to believe how long smartphones have existed. I remember when the first iPhone came out in June 2007. I was working at my first internship at IBM, and I remember hearing in the breakroom that someone on our floor got one. Oooooooh! So special! That was 15 years ago!

In that decade and a half, mobile devices of all shapes and sizes have become indispensable parts of our modern lives: The first thing I do every morning when I wake up is check my phone. My dad likes to play Candy Crush on his tablet. My wife takes countless photos of our French bulldog puppy on her phone. Her mom uses her tablet for her virtual English classes. I’m sure, like us, you would feel lost if you had to go a day without your device.

It’s vital for mobile apps to have high quality. If they crash, freeze, or plain don’t work, then we can’t do the things we need to do. So, being the Automation Panda, I wanted to give mobile testing a try! I had three main goals:

1. Learn about mobile testing for Android – specifically how it relates to other kinds of testing.
2. Automate my own Appium tests – not just run someone else’s examples.
3. Add visual assertions to my tests with Applitools – instead of coding a bunch of checks with complicated locators.

This article covers my journey. Hopefully, it can help you get started with mobile testing, too! Let’s jump in.

Getting Started with Mobile

The mobile domain is divided into two ecosystems: Android and iOS. That means any app that wants to run on both operating systems must essentially have two implementations. To keep things easier for me, I chose to start with Android because I already knew Java and I actually did a little bit of Android development a number of years ago.

I started by reading a blog series by Gaurav Singh on getting started with Appium. Gaurav’s articles showed me how to set up my workbench and automate a basic test:

1. Hello Appium, Part 1: What is Appium? An Introduction to Appium and its Tooling
2. Hello Appium, Part 2: Writing Your First Android Test
3. Appium Fast Boilerplate GitHub repository

Test Automation University also has a set of great mobile testing courses that are more than a quickstart guide:

• Mobile Automation with Appium in Java
• Mobile Automation with Appium in JavaScript
• Automated Visual Testing with Appium
• Introduction to iOS Test Automation with XCUITest
• Android Test Automation with Espresso

Choosing an Android App

Next, I needed an Android app to test. Thankfully, Applitools had the perfect app ready: Applifashion, a shoe store demo. The code is available on GitHub at https://github.com/dmitryvinn/applifashion-android-legacy.

To do Android development, you need lots of tools:

• The Java Development Kit (JDK)
• Android Studio 
• Android SDK (can be installed with Android Studio)
• Android Emulator (can also be installed with Android Studio)

I followed Gaurav’s guide to a T for setting these up. I also had to set the ANDROID_HOME environment variable to the SDK path.

Be warned: it might take a long time to download and install these tools. It took me a few hours and occupied about 13 GB of space!

Once my workbench was ready, I opened the Applifashion code in Android Studio, created a Pixel 3a emulator in Device Manager, and ran the app. Here’s what it looked like:

I chose to use an emulator instead of a real device because, well, I don’t own a physical Android phone! Plus, managing a lab full of devices can be a huge hassle. Phone manufacturers release new models all the time, and phones aren’t cheap. If you’re working with a team, you need to swap devices back and forth, keep them protected from theft, and be careful not to break them. As long as your machine is powerful and has enough storage space, you can emulate multiple devices.

Choosing Appium for Testing

It was awesome to see the Applifashion app running through Android Studio. I played around with scrolling and tapping different shoes to open their product pages. However, I really wanted to do some automated testing. I chose to use Appium for automation because its API is very similar to Selenium WebDriver, with which I am very familiar.

Appium adds on its own layer of tools:

• Node.js for running Appium
• The Appium server (either the NPM module or Appium Desktop)
• Appium Doctor for validating your Appium setup
• Appium Inspector for inspecting mobile app pages

Again, I followed Gaurav’s guide for full setup. Even though Appium has bindings for several popular programming languages, it still needs a server for relaying requests between the client (e.g., the test automation) and the app under test. I chose to install the Appium server via the NPM module, and I installed version 1.22.3. Appium Doctor gave me a little bit of trouble, but I was able to resolve all but one of the issues it raised, and the one remaining failure regarding ANDROID_HOME turned out to be not a problem for running tests.

Before jumping into automation code, I wanted to make sure that Appium was working properly. So, I built the Applifashion app into an Android package (.apk file) through Android Studio by doing Build → Build Bundle(s) / APK(s) → Build APK(s). Then, I configured Appium Inspector to run this .apk file on my Pixel 3a emulator. My settings looked like this:

Here were a few things to note:

• The Appium server and Android device emulator were already running.
• I used the default remote host (127.0.0.1) and remote port (4723).
• Since I used Appium 1.x instead of 2.x, the remote path had to be /wd/hub.
• appium: automationName had to be uiautomator2 – it could not be an arbitrary name.
• The platform version, device name, and app path were specific to my environment. If you try to run this yourself, you’ll need to set them to match your environment.

I won’t lie – I needed a few tries to get all my capabilities right. But once I did, things worked! The app appeared in my emulator, and Appium Inspector mirrored the page from the emulator with the app source. I could click on elements within the inspector to see all their attributes. In this sense, Appium Inspector reminded me of my workflow for finding elements on a web page using Chrome DevTools. Here’s what it looked like:

Writing my First Appium Test

So far in my journey, I had done lots of setup, but I hadn’t yet automated any tests! Mobile testing certainly required a heftier stack than web app testing, but when I looked at Gaurav’s example test project, I realized that the core concepts were consistent.

I set up my own Java project with JUnit, Gradle, and Appium:

• I chose Java to match the app’s code.
• I chose JUnit to be my core test framework to keep things basic and familiar.
• I chose Gradle to be the dependency manager to mirror the app’s project.

My example code is hosted here: https://github.com/AutomationPanda/applitools-appium-android-webinar.

Warning: The example code I share below won’t perfectly match what’s in the repository. Furthermore, the example code below will omit import statements for brevity. Nevertheless, the code in the repository should be a full, correct, executable example.

My build.gradle file looked like this with the required dependencies:

plugins {
    id 'java'
}

group 'com.automationpanda'
version '1.0-SNAPSHOT'

repositories {
    mavenCentral()
}

dependencies {
    testImplementation 'io.appium:java-client:8.1.1'
    testImplementation 'org.junit.jupiter:junit-jupiter-api:5.8.2'
    testImplementation 'org.seleniumhq.selenium:selenium-java:4.2.1'
    testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.8.2'
}

test {
    useJUnitPlatform()
}

My test case class was located at /src/test/java/com/automationpanda/ApplifashionTest.java. Inside the class, I had two instance variables: the Appium driver for mobile interactions, and a WebDriver waiting object for synchronization:

public class ApplifashionTest {

    private AppiumDriver driver;
    private WebDriverWait wait;

    // …
}

I added a setup method to initialize the Appium driver. Basically, I copied all the capabilities from Appium Inspector:

    @BeforeEach
    public void setUpAppium(TestInfo testInfo) throws IOException {

        // Create Appium capabilities
        // Hard-coding these values is typically not a recommended practice
        // Instead, they should be read from a resource file (like a properties or JSON file)
        // They are set here like this to make this example code simpler
        DesiredCapabilities capabilities = new DesiredCapabilities();
        capabilities.setCapability("platformName", "android");
        capabilities.setCapability("appium:automationName", "uiautomator2");
        capabilities.setCapability("appium:platformVersion", "12");
        capabilities.setCapability("appium:deviceName", "Pixel 3a API 31");
        capabilities.setCapability("appium:app", "/Users/automationpanda/Desktop/Applifashion/main-app-debug.apk");
        capabilities.setCapability("appium:appPackage", "com.applitools.applifashion.main");
        capabilities.setCapability("appium:appActivity", "com.applitools.applifashion.main.activities.MainActivity");
        capabilities.setCapability("appium:fullReset", "true");

        // Initialize the Appium driver
        driver = new AppiumDriver(new URL("http://127.0.0.1:4723/wd/hub"), capabilities);
        wait = new WebDriverWait(driver, Duration.ofSeconds(30));
    }

I also added a cleanup method to quit the Appium driver after each test:

    @AfterEach
    public void quitDriver() {
        driver.quit();
    }

I wrote one test case that performs shoe shopping. It loads the main page and then opens a product page using locators I found with Appium Inspector:

    @Test
    public void shopForShoes() {

        // Tap the first shoe
        final By shoeMainImageLocator = By.id("com.applitools.applifashion.main:id/shoe_image");
        wait.until(ExpectedConditions.presenceOfElementLocated(shoeMainImageLocator));
        driver.findElement(shoeMainImageLocator).click();

        // Wait for the product page to appear
        final By shoeProductImageLocator = By.id("com.applitools.applifashion.main:id/shoe_image_product_page");
        wait.until(ExpectedConditions.presenceOfElementLocated(shoeProductImageLocator));
    }

At this stage, I hadn’t written any assertions yet. I just wanted to see if my test could successfully interact with the app. Indeed, it could, and the test passed when I ran it! As the test ran, I could watch it interact with the app in the emulator.

Adding Visual Assertions

My next step was to write assertions. I could have picked out elements on each page to check, but there were a lot of shoes and words on those pages. I could’ve spent a whole afternoon poking around for locators through the Appium Inspector and then tweaking my automation code until things ran smoothly. Even then, my assertions wouldn’t capture things like layout, colors, or positioning.

I wanted to use visual assertions to verify app correctness. I could use the Applitools SDK for Appium in Java to take one-line visual snapshots at the end of each test method. However, I wanted more: I wanted to test multiple devices, not just my Pixel 3a emulator. There are countless Android device models on the market, and each has unique aspects like screen size. I wanted to make sure my app would look visually perfect everywhere.

In the past, I would need to set up each target device myself, either as an emulator or as a physical device. I’d also need to run my test suite in full against each target device. Now, I can use Applitools Native Mobile Grid (NMG) instead. NMG works just like Applitools Ultrafast Grid (UFG), except that instead of browsers, it provides emulated Android and iOS devices for visual checkpoints. It’s a great way to scale mobile test execution. In my Java code, I can set up Applitools Eyes to upload results to NMG and run checkpoints against any Android devices I want. I don’t need to set up a bunch of devices locally, and the visual checkpoints will run much faster than any local Appium reruns. Win-win!

To get started, I needed my Applitools account. If you don’t have one, you can register one for free.

Then, I added the Applitools Eyes SDK for Appium to my Gradle dependencies:

   testImplementation 'com.applitools:eyes-appium-java5:5.12.0'

I added a “before all” setup method to ApplifashionTest to set up the Applitools configuration for NMG. I put this in a “before all” method instead of a “before each” method because the same configuration applies for all tests in this suite:

    private static InputReader inputReader;
    private static Configuration config;
    private static VisualGridRunner runner;

    @BeforeAll
    public static void setUpAllTests() {

        // Create the runner for the Ultrafast Grid
        // Warning: If you have a free account, then concurrency will be limited to 1
        runner = new VisualGridRunner(new RunnerOptions().testConcurrency(5));

        // Create a configuration for Applitools Eyes
        config = new Configuration();

        // Set the Applitools API key so test results are uploaded to your account
        config.setApiKey("<insert-your-API-key-here>");

        // Create a new batch
        config.setBatch(new BatchInfo("Applifashion in the NMG"));

        // Add mobile devices to test in the Native Mobile Grid
        config.addMobileDevices(
                new AndroidDeviceInfo(AndroidDeviceName.Galaxy_S21),
                new AndroidDeviceInfo(AndroidDeviceName.Galaxy_Note_10),
                new AndroidDeviceInfo(AndroidDeviceName.Pixel_4));
    }

The configuration for NMG was almost identical to a configuration for UFG. I created a runner, and I created a config object with my Applitools API key, a batch name, and all the devices I wanted to target. Here, I chose three different phones: Galaxy S21, Galaxy Note 10, and Pixel 4. Currently, NMG supports 18 different Android devices, and support for more is coming soon.

At the bottom of the “before each” method, I added code to set up the Applitools Eyes object for capturing snapshots:

    private Eyes eyes;

    @BeforeEach
    public void setUpAppium(TestInfo testInfo) throws IOException {

        // …

        // Initialize Applitools Eyes
        eyes = new Eyes(runner);
        eyes.setConfiguration(config);
        eyes.setIsDisabled(false);
        eyes.setForceFullPageScreenshot(true);

        // Open Eyes to start visual testing
        eyes.open(driver, "Applifashion Mobile App", testInfo.getDisplayName());
    }

Likewise, in the “after each” cleanup method, I added code to “close eyes,” indicating the end of a test for Applitools:

    @AfterEach
    public void quitDriver() {

        // …

        // Close Eyes to tell the server it should display the results
        eyes.closeAsync();
    }

Finally, I added code to each test method to capture snapshots using the Eyes object. Each snapshot is a one-line call that captures the full screen:

    @Test
    public void shopForShoes() {

        // Take a visual snapshot
        eyes.check("Main Page", Target.window().fully());

        // Tap the first shoe
        final By shoeMainImageLocator = By.id("com.applitools.applifashion.main:id/shoe_image");
        wait.until(ExpectedConditions.presenceOfElementLocated(shoeMainImageLocator));
        driver.findElement(shoeMainImageLocator).click();

        // Wait for the product page to appear
        final By shoeProductImageLocator = By.id("com.applitools.applifashion.main:id/shoe_image_product_page");
        wait.until(ExpectedConditions.presenceOfElementLocated(shoeProductImageLocator));

        // Take a visual snapshot
        eyes.check("Product Page", Target.window().fully());
    }

When I ran the test with these visual assertions, it ran one time locally, and then NMG ran each snapshot against the three target devices I specified. Here’s a look from the Applitools Eyes dashboard at some of the snapshots it captured:

The results are marked “New” because these are the first “baseline” snapshots. All future checkpoints will be compared to these images.

Another cool thing about these snapshots is that they capture the full page. For example, the main page will probably display only 2-3 rows of shoes within its viewport on a device. However, Applitools Eyes effectively scrolls down over the whole page and stitches together the full content as if it were one long image. That way, visual snapshots capture everything on the page – even what the user can’t immediately see!

Injecting Visual Bugs

Capturing baseline images is only the first step with visual testing. Tests should be run regularly, if not continuously, to catch problems as soon as they happen. Visual checkpoints should point out any differences to the tester, and the tester should judge if the change is good or bad.

I wanted to try this change detection with NMG, so I reran tests against a slightly broken “dev” version of the Applifashion app. Can you spot the bug?

The formatting for the product page was too narrow! “Traditional” assertions would probably miss this type of bug because all the content is still on the page, but visual assertions caught it right away. Visual checkpoints worked the same on NMG as they would on UFG or even with the classic (e.g. local machine) Applitools runner.

When I switched back to the “main” version of the app, the tests passed again because the visuals were “fixed:”

While running all these tests, I noticed that mobile test execution is pretty slow. The one test running on my laptop took about 45 seconds to complete. It needed time to load the app in the emulator, make its interactions, take the snapshots, and close everything down. However, I also noticed that the visual assertions in NMG were relatively fast compared to my local runs. Rendering six snapshots took about 30 seconds to complete – three times the coverage in significantly less time. If I had run tests against more devices in parallel, I could probably have seen an even greater coverage-to-time ratio.

Conclusion

My first foray into mobile testing was quite a journey. It required much more tooling than web UI testing, and setup was trickier. Overall, I’d say testing mobile is indeed more difficult than testing web. Thankfully, the principles of good test automation were the same, so I could still develop decent tests. If I were to add more tests, I’d create a class for reading capabilities as inputs from environment variables or resource files, and I’d create another class to handle Applitools setup.

Visual testing with Applitools Native Mobile Grid also made test development much easier. Setting everything up just to start testing was enough of a chore. Coding the test cases felt straightforward because I could focus my mental energy on interactions and take simple snapshots for verifications. Trying to decide all the elements I’d want to check on a page and then fumbling around the Appium Inspector to figure out decent locators would multiply my coding time. NMG also enabled me to run my tests across multiple different devices at the same time without needing to pay hundreds of dollars per device or sucking up a few gigs of storage and memory on my laptop. I’m excited to see NMG grow with support for more devices and more mobile development frameworks in the future.

Despite the prevalence of mobile devices in everyday life, mobile testing still feels far less mature as a practice than web testing. Anecdotally, it seems that there are fewer tools and frameworks for mobile testing, fewer tutorials and guides for learning, and fewer platforms that support mobile environments well. Perhaps this is because mobile test automation is an order of magnitude more difficult and therefore more folks shy away from it. There’s no reason for it to be left behind anymore. Given how much we all rely on mobile apps, the risks of failure are just too great. Technologies like Visual AI and Applitools Native Mobile Grid make it easier for folks like me to embrace mobile testing.

The post Mobile Testing for the First Time with Android, Appium, and Applitools appeared first on Automated Visual Testing | Applitools.