Update to One-line Setup Command

A small but effective update so far that made me smile was being able to set up a WebdriverIO project with one command `$ npm init wdio@latest ./`. After that command, it asks you all the questions needed to set up and run your tests. It provides you with predefined options for not only end-to-end testing of web and mobile applications but unit and component testing as well.

Transition from Common JS to EMCAScript modules (ESM)

To give some background, CommonJS and ESM refer to the module system that allows us to use code from a “next” developer/library or code/data that is in a different file. The CommonJS module system has been the default module system within the Node.js ecosystem. However, the ES module was added in Node.js v8.5.0 and has been stabilized and incorporated as standard from Node.js v13.2.0. 
You may be familiar with using `require` to use your page objects in your test files and `module.exports` or `exports` so that your page objects or other files can be accessed by your tests. This method of managing files and libraries is the CommonJS way.

With ESM you use `import` and `export` to manage files and libraries.

There are some other differences between them, but for you, the end user, this is the main one. 
This change should not affect your work as CommonJS is still supported. However, if you want to use ES modules then you can now do so. I recommend getting familiar with ESM and how it works before starting to use it. Hopefully, in the near future, there will be additional boilerplates from WebdriverIO on using ESM in your projects https://webdriver.io/docs/boilerplates/.

New Action Interface

This is a new interface that makes executing various actions much easier. Two new browser commands action and actions have been added. It is now much simpler and type-safe to run the right action, e.g. sending key events to the browser.

Read more about this in the WebdriverIO API docs.

Optional Globals

The WebdriverIO test runner would usually register the browser object or the $ and $$ commands to the global scope. With v8 update, the user can decide if they want to continue attaching these objects and methods to the global scope or prefer importing them directly.

To import them directly you would say : 

import { browser, $, $$, expect } from ‘@wdio/globals’

Importing the $ command allows you to use it from the @wdio/globals:

Using Visual Studio Code as my code editor, if I hover over the $ it shows that it is coming as a global import from WebdriverIO:

A new configuration property called injectGlobals (defaults: true) handles whether the test runner modifies the global scope or not. If your setup works fine using global objects, no change is needed to update to v8.

You don’t have to import from @wdio/globals as it will still work.

Update to WebdriverIO version 8 from version 7

Install a module to help update outdated npm modules

`npm install -g npm-check-updates`

Run `ncu` to show list of outdated npm packages:

Then run `ncu -u` to upgrade your package.json:

Then finally run npm install to install new versions:

I then reran all my tests and they passed successfully.

The post What’s New in WebdriverIO 8 appeared first on Automated Visual Testing | Applitools.

Learn how to use headless mode for your tests and why it’s so important. Understand when you should use it and see an example of headless testing with WebdriverIO.

Browser automation has been around for a long time. It is an important part of how we develop, test, and deploy web applications. Browser automation can be done on a headed browser and also headless browsers. A headless browser operates as you would expect a normal browser would, however, it does not have a Graphical User Interface (GUI). Therefore when you are running tests you will not see the browser GUI pop up and the actions being carried out. Interactions with a headless browser are done via the Command Line Interface.

Before the headless mode Chrome release in 2017,  in order to run your automated tests headlessly, you had to use browsers such as PhantomJS (now discontinued). Over time browser vendors included a headless browsing mode as a part of their releases. This started with Google Chrome 59 in 2017 and Firefox following with headless mode available on all their platforms starting with Firefox version 56. Headless Chrome quickly overtook PhantomJS upon its release. 

There are many uses of headless browsers by a wide range of individuals.

• Test Automation engineers use headless browsers to run their automated scripts faster and in Continuous Integration and Delivery pipelines. Scripts are executed faster as a headless browser does not need to load a GUI 
• Servers and tools such as Jenkins, GitlabCI, and Docker execute with a headed browser. When carrying out actions on these tools you will need to install headless browsers to complete the tasks
• SEO tools use it to analyze a web page and provide recommendations on how it can be improved
• Monitoring tools use headless mode to measure the performance of web applications
• Scraping tools also use the headless mode to carry out their tasks. Web scraping with a headless browser can be carried out very quickly as most times it does not require a Graphical User Interface
• Headless browsers can be used to extract metadata (e.g., the DOM) and generate bitmaps from page contents. It also allows users to tap into chrome devtools and make use of features such as network throttling, device emulating, website performance analysis, and more

As test automation engineers we can gain the following benefits from doing headless testing:

1. Providing the ability to run your tests on a server with no GUI which is the case with most cloud-based servers, tools such as Docker and CI pipelines where it is not always possible to install a GUI browser. 
2. Reduces the time it takes for your automated scripts to complete compared to a GUI browser. GUI browsers will have to load CSS, images, render HTML and this will increase the time your script takes to run.
3. When running tests in parallel, UI based browsers utilize a lot of memory compared to headless
4. Frees up your computer to continue doing tasks while your tests are running in the background. With GUI-based browsers when running scripts they can be on your main screen and prevent you from doing anything else in the time that it takes to be run.

Headless browsers have a lot of benefits, however, there are some things to consider when using headless browsers:

1. It may be difficult to debug failures when running your tests headlessly as there is no GUI for you to see failure points. There are logs available to help you debug, however, at times the failure is not easily spotted from the logs. This results in you having to add in commands to get screenshots to try and debug the issue or using other means.
2. Headless browsers aren’t mimicking exact user behavior and some tests may fail due to the speed at which they are being executed.
3. Regular users are not using the website in headless mode and so it is equally important to run test scenarios, do exploratory testing, visual regression, and other forms of testing on a headed browser. You want to ensure that the functionality and user experience of the web application remains consistent.

Different testing frameworks such as Selenium, Cypress, and WebdriverIO have commands that allow you to execute your scripts in headless browsers. In WebdriverIO you can execute your tests in the following way.

Let us quickly set up a demo WebriverIO project.

Setting up a WebdriverIO Project

In your code editor (I use VSCode) create a new project.

Initialize npm and install WebdriverIO with the following steps:

npm init wdio .

Select the following options from the configuration helper
=========================
WDIO Configuration Helper
=========================

Where is your automation backend located? On my local machine
Which framework do you want to use? mocha
Do you want to use a compiler? No!
Where are your test specs located? ./test/specs/**/*.js
Do you want WebdriverIO to autogenerate some test files? Yes
Do you want to use page objects (https://martinfowler.com/bliki/PageObject.html)? Yes
Where are your page objects located? ./test/pageobjects/**/*.js
Which reporter do you want to use? spec
Do you want to add a service to your test setup? selenium-standalone
What is the base url? http://localhost
Do you want me to run `npm install` Yes

To start the test, run: $ npm run wdio

This will execute tests in a headed Chrome browser.

Let us now set up the project to run headlessly.

Running Headless Chrome with WebdriverIO

1. In the wdio.conf.js file go to the capabilities section of the configurations and add:

   'goog:chromeOptions': {
               args:'headless',
             },
   

2. WebdriverIO allows you to use Devtools Service. If you are using that service instead of the Webdriver Protocol then add the following instead of step 1:

   'wdio:devtoolsOptions':{
               headless: true
           },
   

3. Rerun your tests with npm run wdio and they should run with a headless browser

Running Headless Firefox with WebdriverIO

1. In the wdio.conf.js file go to the capabilities section of the configurations and change browserName to Firefox 
2. Add the capability to run Firefox headlessly:

   "moz:firefoxOptions": {
               	args: ['-headless']
             	},

3. Rerun your tests with npm run wdio and they should run with a headless browser on Firefox

Here’s an example that demonstrates the potential speed difference between executing your tests in a headless browser vs a headed browser. I ran this test suite of 15 tests headed and headless (each having a max instance of 1) and the overall completion time was 18% faster from just that one change. The headed tests took 1 minute and 24 seconds to run and the headless tests ran in 1 minute and 9 seconds:

Headless Browsers offers many benefits and can be used to aid your test automation. You can start your browsers headlessly from the command line and also use it in your various test automation frameworks. You can check out Cypress, Selenium Webdriver , Puppeteer among others.

The post Using Headless Mode for Your Tests – When, Why and How (demonstration with WebdriverIO) appeared first on Automated Visual Testing | Applitools.

Wondering how to speed up your UI automation tests? Learn effective and actionable tips to help your automated UI tests run faster and more efficiently.

UI automation is the testing process where scripts are executed using an automated tool on a Graphical User Interface (GUI). It requires interaction with the browser and its elements to perform actions such as clicking and entering text to verify the functionality of the application. With the GUI, you are getting visual representation such as buttons and icons, and you communicate with those icons using the mouse or keyboard, rather than text-based or command-line types of communication to get things done or perform functions. 

Due to the nature of how it interacts with multiple elements and has a lot of dependencies, UI automation is considered slower and more brittle than other types of tests such as API, Database, and Unit tests. UI automation also tends to have a large variety of user scenarios, and the UI of an application, along with its locators, can change and cause failure. UI Automation also requires communicating with other dependencies which can cause bottlenecks. Despite all this, there are ways that you can speed up your UI automated tests. Here are some tips:

1. Use the API and Database Layers of the Application

The API Layer of an application executes commands thirty-five times faster than the UI Layer and can be used in the following ways to speed up the automated tests that you are executing:

• Create Test Data Using APIs – Data needed for test automation in a particular format can be created quickly using an API. For example, if it is that you want to spin up a certain amount of users with specific profile features, you can use the API to generate those users much more quickly than if you were to go to the Graphical User Interface and try to generate those 10 or a hundred types of users. Doing it via the API is going to save a whole lot of time. It’s also going to be less prone to errors, as if you are going to be creating 20 users and you have to run that profile creation test 20 times things may happen. The internet may be slow or the GUI may take a long time to load and affect the speed of a successful profile creation.
• Cleanup – Cleanup via API can help speed up your UI automated tests. If we use the scenario mentioned before where we had created 20 users, and let’s say for each of those users we had added 10 items to their cart. When doing test automation we want to ensure that whatever we do in one test does not adversely affect other tests. In this case, we need to remove the items from the 20 users’ carts. If we were to do that via the UI we would have to sign in to each user profile, navigate to their cart, and then remove each item from the cart (if there is no remove all items button). This is a cleanup step that could be done very quickly with a call or two with an API. 
• Verification – Test verification helps us to determine whether the actions we have taken were successful or had failed. We can do API calls to verify that a user was created as well as to verify that purchases were made. This would shorten the time our tests take instead of doing it via the UI.

In the same way, you can save time by creating test data, cleanup, and verification in the database layer. I have used database queries on projects to find specific types of users that I need for my tests to execute in certain conditions. For example, I was on a project where we were selling flowers for persons that have passed. There was different business logic per funeral home as well as the death date for the individual. I would use the database to get users that were in a particular funeral home and had passed in under 5 days. Using the results from that query I would then be able to generate a URL that took me directly to that person’s death page. From that setup, I could seamlessly carry out my tests.

If I had not done that query, I would have had to navigate to the home page of the site, search for the funeral home, wait for that to come up, and then filter for persons who had passed under 5 days, then click on that person and wait for the page to load. This would have taken more time than doing a database query.

If you add this technique to other tests having similar conditions that time adds up and you save so much time by using the database in this way. By using APIs, databases, and other test techniques to manage test setup and tear down you can speed up your UI tests.

2. Visual Regression Instead of Doing Certain UI Checks

Visual regression is the process by which the GUI of an application is checked to ensure that it maintains the defined look. This is done by comparing a baseline of original or reference images to what is currently being shown. These comparisons are done during regression tests or whenever something new has been added or some modifications have been done and new screenshots are gathered to compare against the original. Applitools is a leading automated visual regression tooling company that provides AI-powered visual testing coverage for your applications. 

Think about tests that you have in your automated UI test suite that it does not handle well or that would be better done using visual regression. Checking for font size, font color, spacing, alignment would be best done using visual regression rather than using UI automation. Visual regression will also be better able to find bugs for scenarios where:

• Font size, color, and placement differs
• An element is present on a screen but is in the wrong location, has the wrong color, or is being blocked by another element

If you want to get started with Visual Regression you can check out Applitools Eyes which provides a cross-environment testing feature that allows you to test your application on multiple platforms using a single, common baseline. 

3. Parallel Execution

Test automation can be run sequentially. That is one after the other. However, if you use parallel execution, you can vastly improve the speed of your automated tests. Parallel execution allows you to run multiple automated tests at the same time. Therefore, you’re going to have multiple browsers open and you’re going to see more of your test suite being run at the same time. 

With multiple suites such as a log-in suite, a checkout suite, and a search/filter suite instead of waiting for the login suite to finish and then you run the checkout suite and so on, you can have all three suites running at the same time. In most test automation frameworks, you can control the number of sessions that are created and therefore the amount of parallel execution that happens. You may want to have two sessions. You may want to have three, or you may want to have five. It all depends on the types of tests that you have.

The benefit of this is that it cuts down on the time it takes to run the overall test suite. If all your tests took 10 minutes to run, and you decided to do parallel execution and you split it into two sessions, then that will mean that your tests should take now five minutes to run. If you increase the number of parallels, then it’s going to continue to reduce the time to run.

This also helps you to test across more mobile devices and browser combinations. You could have one parallel execution dedicated to running on Google Chrome, one for Safari, Internet Explorer, Firefox, and so on. That would mean you have 4 different sessions for 4 different browsers and the time would be reduced compared to running all of those sequentially.

When doing parallel or distributed test execution, keep in mind tests that have dependencies. In WebdriverIO there is an option to group dependent tests so that when I am running them in parallel those tests will be run in the same instance/session in the order specified.

4. Distributed Execution

Distributed execution provides similar benefits and speed gains as when you run your tests in parallel. Distributed Testing or Distributed Execution is where your tests are executed across many different virtual machines or computers in the cloud. Cloud services such as Docker provide the option to spin up various servers or environments that you can use to execute your tests. 

Generally speaking end to end tests or UI tests are heavy processes and they can sometimes take up a lot of computing power. Distributed execution in the cloud enhances the speed of your tests; machines are more powerful, have more memory, space, and allow you to run tests in parallel as well, therefore your UI tests will run faster. Distributed execution also removes the management of servers and browser versions. It eliminates the time needed to set up new environments. These environments can be set up on those machines or resources in the cloud. These cloud services update when new mobile phones or OS versions come out so you don’t have to physically buy a new mobile device.

Overall parallel and distributed execution will help you to speed up your test and eliminate the time needed to set up those new environments.

How Parallel and Distributed Test Execution Saves Time

Parallel and Distributed test execution saves you time by allowing you to:

• Run multiple automated tests at the same time
  • cuts down on the time it takes to run a suite of tests
  • allows you to test across more mobile devices or desktop OS/browser combinations at once
• Run on multiple machines with faster resources or in the cloud
  • enhances the speed of the tests
  • eliminates the time needed for setup of new environments
  • removes management of servers and browser versions

5. Headless Browsers

Browser automation can be done on a headed browser and also headless browsers. A headless browser operates as you would expect a normal browser would, however, it does not have a Graphical User Interface. Therefore when you are running tests you will not see the browser GUI pop up and the actions being carried out. Interactions with a headless browser are done via the Command Line Interface.

If your tests are running in a CI (continuous integration) pipeline, like GitHub Actions, Jenkins, etc., then there is a very high probability that your UI tests are being executed headlessly.

Headless browsers do not require a visible UI and this makes headless tests way faster than on UI browsers. When running tests in parallel, UI-based browsers utilize a lot of memory compared to headless browsers.

As with all the tips that I have been sharing so far, you must make considerations for your specific test. Know that some application features may have unexpected behavior in headless. Headless browsers aren’t mimicking exact user behavior and some tests may fail due to the speed at which they are being executed. Also, regular users are not using the website in headless mode and so it is equally important to run test scenarios, do exploratory testing, visual regression, and other forms of testing on a headed browser. You want to ensure that the functionality and user experience of the web application remains consistent.

How Headless Browsers Speed Up UI Test Automation

Headless browsers increase the speed of UI test automation because they:

• Are way faster than GUI browsers.
• Reduce the time it takes for your automated scripts to complete compared to a GUI browser. GUI browsers will have to load CSS, images, render HTML and this will increase the time your script takes to run.
• Are useful when the test execution server has no GUI. This is the case with most cloud-based servers that you can use to execute parallel/distributed test execution.

You can try out headless browsers in various test automation frameworks such as WebdriverIO, Cypress, Selenium Webdriver, Puppeteer among others.

6. Manage Test Setup and Teardown Efficiently 

We’ve talked about how APIs and databases can be used to create a specific environment that you want for your tests. Using API, Databases, data files, custom functions and other test techniques to manage test setup and teardown is really going to impact the speed of your UI tests.

Create setup steps to execute and create the perfect scenarios needed for your tests. This is useful for unique scenarios where users/data need to be in a specific state before tests can be executed. You don’t want to be setting up the scenario on the GUI. You want to use other layers and other means to do that setup work for you, so that when you’re actually running your test, you just need to do the verification of the specific functionality that you’re checking for.

Think about how and when you do setup and teardown. Test setup and teardown can impact tests and cause failures if not done properly. It can also increase the time it takes for your tests to be executed.

Test setup is the process by which you create the needed data, users, or environments that your tests need to run. Teardown is the opposite where you remove or clean up actions that were carried out during your tests to ensure that they don’t adversely affect the state in which data or environments need to be in to facilitate other tests. Ideally, test setup should be done before your test run. Therefore, if you have a large database that you want to set up before to ensure that your environment, data and users are in a specific state, do that at the very beginning. 

Teardown should be done after scenarios where needed; otherwise do teardown when all tests have been completed. This saves time over setting up the same or similar scenario before each test and then tearing down after and repeating the process, thus increasing the time tests take to run. Setup and teardown take some planning, but if done correctly, can improve the speed at which your UI automation runs.

7. Use Cookies

Cookies allow browsers to track, personalize, and save information about each user’s session. Without cookies to streamline the user experience, you will have to login again after you leave a website, or recreate your shopping cart if you accidentally close the page. 

Cookies are an important part of the internet experience. They allow for session management, letting you know login information and preferences. Cookies are also used for personalization so that you can get custom advertisements and also for tracking. If you are on a shopping site and you previously viewed an item, then you may get a suggestion to purchase that item, it may come under things you liked or previously viewed. It helps to enhance the online shopping experience. 

Cookies can be used in web automation to make your UI test faster. It does that by using the information stored in cookies, such as logging information to preload test scenarios. If you don’t want to log in every time via the GUI, or you’re doing a specific test, you can use cookies to preload the login information.

A project where there are feature flags, such as A/B testing, is a good candidate for using cookies, as with A/B tests you aren’t sure which version of the A/B test is going to come up, and with test automation, you will only want to test a specific scenario. Cookies can be used to turn off feature flags and those specific features. Your tests will then know at all times which scenario is going to be loaded. Scenario A or B.

By storing information and turning on and off features and tracking and saving information about user sessions, you can use cookies to manipulate the state of the UI to increase the speed of your UI. 

Different frameworks provide the option to use cookies in your test automation. Selenium Webdriver has a command called addCookie. This is used to add a cookie to the current browsing context. It accepts a set of defined JSON objects and shows you examples in the different languages that Selenium Webdriver supports.

For this to work, you need to be on the domain that the cookie will be valid for. You can preset cookies before starting interaction with your site. Once it loads, then those cookies will be there. You can also manage the different cookies and there are other commands that you can use with Selenium Webdriver.

WebdriverIO also provides the option to manipulate and interact with cookies. The command is called setCookies. A cookie can be set for an arbitrary page without being on that page and it details how you go about setting different cookies. 

Look at the specific test automation framework that you’re using. See how you can integrate cookies into your test automation and speed up your UI tests

8. Use Navigation Shortcuts

Navigation shortcuts speed up the test as it cuts out unnecessary steps. Navigating directly to UIautomation.com/profile/edit is much faster than if you 

• navigate to uiautomation.com
• click login, then
• click profile
• Click edit
• Wait for all those pages to load

Navigating directly to the page is one navigation instead of navigating, and then doing a complete process before you get to the page.

Using all the tips mentioned above, such as setting up APIs and databases, to help generate URLs that you can navigate directly to will help, as will using cookies to preload the login information so that you are able to go directly to the website.

9. Create Atomic Non-Repeating Tests

In addition to those tips, overall, when doing test automation ensure that you create atomic non-repeating tests. Don’t have multiple of the same scenarios running over and over. You don’t want to have a login being repeated over and over in multiple places. Instead, you can use cookies and navigation shortcuts to reduce that. You don’t want to have a very long suite that is going to take a very long time to run and be more prone to failures. 

You want to keep your tests really small and focused on the specific thing that you’re trying to verify. Use other layers of the test automation pyramid, and use UI automation, just to verify those things that you need to.

How Will You Speed Up Your UI Automation Tests?

UI automation is a part of end-to-end tests that are slow and take longer to run when compared to integration and unit tests. To speed this up, evaluate your tests and select the technique that best serves the project. Think about other layers of the application that can be automated and don’t try to do everything in the UI layer. You can also use these other layers to aid your tests. Cookies, headless browsers, parallel/distributed execution, and navigation shortcuts can all help to speed up your UI automated tests. 

The post Top 9 Tips to Increase UI Automation Speed appeared first on Automated Visual Testing | Applitools.