There are many record and playback tools available, such as Selenium IDE, Testim, Katalon, and others.

Just recently Google has decided to launch its own Recorder tool embedded directly into Chrome. When Google joins the game it’s interesting to see. We decided to check it out.

The new tool is called Chrome DevTools Recorder.

What Is Chrome DevTools Recorder?

Chrome’s new Recorder tool allows you to record and replay tests from the browser, export them as a JSON file (and more), as well as measure test performance. The DevTools Recorder was released in November 2021, and you can read all about it here.

Right off the bat, we were excited to see that the tool is straightforward and simple. Since it is embedded in the browser we have the convenience of not having to context switch or deal with an additional external tool.

Let’s see what Google has in store with us with the tool and check out just how easily and quickly we can run our first test.

We’ll do so by recording a test on the Coffee cart website and exporting it as a Puppeteer Replay script. To top it off, we will be sprinkling some Applitools magic onto it and see how easy it is to integrate visual testing into the new tool. Let’s go!

First things first, let’s open up our new tool and record a test. 

How to Record a Test with Chrome DevTools Recorder

• Open this page. We will be using this demo page for recording
• Open Chrome DevTools
• Click on More options() > More tools > Recorder

• Click on the Start new recording button to begin
• Enter your recording name
• Click the Start a new recording button at the bottom of the recording window
• The recording is started (the panel is showing Recording… indicates the recording is in progress)
• Try to click around and order some coffees
• Every interaction with the webpage is now recorded – clicking on buttons, switching web pages, waiting for elements to load and much more

Once the recording is done, we have our first automation script ready to run.

Given the recording, we can see some options before us:

1. Replay the recording – playback what was recorded.
2. Measure performance – replays the tests and opens the new Performance insights panel or the Performance panel of the DevTools. This way we can analyze the performance of our test, the amount of time it took to load each resource.
3. Edit and add steps – we can manually edit our tests and add steps, even complex ones (e.g. wait until 9 images have been loaded and then continue the test). All without code.

Lastly, we have the option to export the test as a JSON file. This is a great feature as you can share the files with other users. 

You can also export it as a Puppeteer Replay script right away. It allows you to customize, extend and replay the tests with the Puppeteer Replay library, which makes the tool even more useful for more experienced users.

One of the main ‘weaknesses’ of Chrome’s Recorder tools is the very basic validation and a pretty standardized flow, with no option in the UI to add on top of it. 

The ability to quickly record a stable automated test and export it to make it more customizable is an incredible feature. It can help create tests quickly and efficiently.

Downloading the Puppeteer Script

• Click on the Export button in the Recorder panel
• Click on the Export as a @puppeter/replay script and save it as main.mjs (we will customize this file to add in Applitools visual testing)
• Also click on the Export as a JSON file as shown below (we will customize our script later to read from this JSON file)
  
  

Understanding the Puppeteer Replay script

Open the main.mjs file we exported just now. This is what the script looks like:

import url from 'url';
import { createRunner } from '@puppeteer/replay';

export const flow = {
    "title": "order-a-coffee",
    "steps": [
        {
            "type": "setViewport",
            "width": 380,
            "height": 604,
            "deviceScaleFactor": 1,
            "isMobile": false,
            "hasTouch": false,
            "isLandscape": false
        },
        ...
    ]
};

export async function run(extension) {
  const runner = await createRunner(flow, extension);
  await runner.run();
}

if (process && import.meta.url === url.pathToFileURL(process.argv[1]).href) {
  await run();
}

After we npm install all the dependencies, we can replay the script above with the node main.mjs command.

The Puppeteer Replay library provides us with an API to replay and stringify recordings created using the Chrome DevTools Recorder. 

The flow variable is our recorded test steps. It is a JSON object. You can replace the flow value to read from a JSON file instead. Here is an example:

/* main.mjs */
import url from 'url';
import fs from 'fs';
import { createRunner, parse } from '@puppeteer/replay';

// Puppeteer: read the JSON user flow
const recordingText = fs.readFileSync('./your-exported-file.json', 'utf8');
export const flow = parse(JSON.parse(recordingText));


export async function run(extension) {
  ...
}

...

Run the script again. It returns the same result.

Extend the Puppeteer Replay script

The Puppeteer Replay offers a way to customize how a recording is run using the PuppeteerRunnerExtension class, which introduces very powerful and simple hooks such as beforeEachStep and afterAllSteps. 

Puppeteer must be installed to customize the tests further. For example, the tests will launch in headless mode by default. In order for us to see how the browser runs the automated test we can turn it off.

Below you can see an example on extending this class and running in headful mode:

/* main.mjs */
...
import puppeteer from 'puppeteer';
import { PuppeteerRunnerExtension } from "@puppeteer/replay";

// Extend runner to log message in the Console
class Extension extends PuppeteerRunnerExtension {
  async beforeAllSteps(flow) {
    await super.beforeAllSteps(flow);
    console.log("starting");
  }

  async afterEachStep(step, flow) {
    await super.afterEachStep(step, flow);
    console.log("after", step);
  }
}

// Puppeteer: launch browser
const browser = await puppeteer.launch({ headless: false });
const page = await browser.newPage();

// Puppeteer: read the JSON user flow
..

// Puppeteer: Replay the script
if (process && import.meta.url === url.pathToFileURL(process.argv[1]).href) {
  // add extension
  await run(new Extension(browser, page));
}

// Puppeteer: clean up
await browser.close();

Now that we understand the code it’s time to kick it up a notch by adding Applitools Eyes to the mix to enable visual testing.

How to add Applitools Eyes to Chrome DevTools Recorder

Applitools Eyes is powered by Visual AI, the only AI-powered computer vision that replicates the human eyes and brain to quickly spot functional and visual regressions. Tests infused with Visual AI are created 5.8x faster, run 3.8x more stably, and detect 45% more bugs vs traditional functional testing.

Applitools also offers the Ultrafast Grid, which provides massively parallel test automation across all browsers, devices, and viewports. With the Ultrafast Grid, you run your test setup script once on your local machine, then the Applitools code takes a snapshot of the page HTML & CSS, and sends it to the grid for processing. This provides an out-of-the-box solution for cross-browser tests, so you don’t have to set up and maintain an in-house QA lab with multiple machines and devices.

Incorporating Applitools Eyes into Chrome DevTools Recorder only takes a few steps. Here’s an overview of the process, with the full details about each step below.

1. Install Applitools Puppeteer SDK
   using npm: npm i -D @applitools/eyes-puppeteer
2. Add the eyes-puppeteer dependency:
   const {Eyes, Target, VisualGridRunner, BrowserType, DeviceName} = require('@applitools/eyes-puppeteer')
   Eyes – the Eyes instance
   Target – Eyes Fluent API
   VisualGridRunner – using the Ultrafast Grid (UFG) with Eyes
   BrowserType – UFG browsers configuration
   DeviceName – UFG devices configuration
3. Initialize Eyes and set the desired configuration and browsers for the UFG
4. We must first open Eyes in order to perform visual validations – the browser must be defined for this step
5. Perform visual validation with Eyes in the afterEachStep hook
6. Finally, after closing the browser we can close Eyes and gather the test results
7. to run the test, run the command:
   node <path_to_test.js>
   

Step 1 – Install Applitools Puppeteer SDK

As indicated above, to install the Applitools Puppeteer SDK run the following command:

npm i -D @applitools/eyes-puppeteer

Step 2 – Dependencies 

/* main.mjs */
import { Eyes, Target, VisualGridRunner } from '@applitools/eyes-puppeteer';

/* applitools.config.mjs */
import { BrowserType, DeviceName } from '@applitools/eyes-puppeteer';

Step 3 – Instance and Configuration

We define an Eyes instance alongside a Visual Grid runner, which is used with Applitools Ultrafast Grid. We can use the runner at the end of the test to gather all the test results across all Eyes instances in the test, therefore, the runner is defined globally. Eyes is usually defined globally as well but may also be defined locally for a specific test case. The terminology for a test in Applitools is equivalent to opening Eyes, performing any number of visual validations, and closing Eyes when we’re done. This will define a batch in Applitools that will hold our test, meaning we can have multiple tests in a single batch.

/* main.mjs */

// Puppeteer: launch browser
...

// Applitools: launch visual grid runner & eyes
const visualGridRunner = new VisualGridRunner({ testConcurrency: 5 });
const eyes = new Eyes(visualGridRunner);

We then create a function, setupEyes, that will set our configuration to Eyes before starting the test and before opening Eyes.  

/* applitools.config.mjs */

import { BrowserType, DeviceName } from '@applitools/eyes-puppeteer';

export async function setupEyes(eyes, batchName, apiKey) {
  eyes.setApiKey(apiKey);

  const configuration = eyes.getConfiguration();

  configuration.setBatch({ name: batchName })
  configuration.setStitchMode('CSS');

  // Add browser
  configuration.addBrowser({ width: 1200, height: 800, name: BrowserType.CHROME });
  configuration.addBrowser({ width: 1200, height: 800, name: BrowserType.FIREFOX });
  configuration.addBrowser({ width: 1200, height: 800, name: BrowserType.SAFARI });
  configuration.addBrowser({ width: 1200, height: 800, name: BrowserType.EDGE_CHROMIUM });
  configuration.addBrowser({ width: 1200, height: 800, name: BrowserType.IE_11 });
  configuration.addBrowser({ deviceName: DeviceName.Pixel_2 });
  configuration.addBrowser({ deviceName: DeviceName.iPhone_X });

  eyes.setConfiguration(configuration);
};

Step 4 – Opening Eyes

In this step we open Eyes right after initializing the browser and defining the page. The page is required in order to communicate and interact with the browser.

/* main.mjs */

// Applitools: launch visual grid runner & eyes
...
const apiKey = process.env.APPLITOOLS_API_KEY || 'REPLACE_YOUR_APPLITOOLS_API_KEY';
const name = 'Chrome Recorder Demo';

await setupEyes(eyes, name, apiKey);
await eyes.open(page, {
  appName: 'Order a coffee',
  testName: 'My First Applitools Chrome Recorder test!',
  visualGridOptions: { ieV2: true }
});

This is a good opportunity to explain what a Baseline is – A Baseline is a set of images that represent the expected result of a specific test that runs on a specific application in a specific environment. A baseline is created the first time you run a test in a specific environment. This baseline will then be updated whenever you make changes to any of the pages in your app, and accept these changes in Applitools Eyes Test Manager. Any future run of the same test on the same environment will be compared against the baseline.

By default, creating a test on a specific browser for the first time (e.g. Firefox) will create a new Baseline, thus running the same test on a different browser (e.g. Chrome) will form a new baseline.

The baseline is a unique combination of the following parameters:

• OS
• Viewport Size
• Browser
• Application name
• Test name

This means that by default a new baseline will be created for every combination that was not used before.

Step 5 – Visual Validation

By calling eyes.check(), we are telling Eyes to perform a visual validation. Using the Fluent API we can specify which target we would like to capture. Here we are performing visual validation in an afterEachStep hook to validate each step of the replay along the way. The target is specified to capture the window (the viewport) without the fully flag, which will force a full-page screenshot.

/* main.mjs */
...
// Extend runner to take screenshot after each step
class Extension extends PuppeteerRunnerExtension {
  async afterEachStep(step, flow) {
    await super.afterEachStep(step, flow);
    await eyes.check(`recording step: ${step.type}`, Target.window().fully(false));
    console.log(`after step: ${step.type}`);
  }
}

Step 6 – Close Eyes and Gather Results

We must close Eyes at the end of our test, as not closing Eyes will result in an Applitools test running in an endless loop. This is due the fact that when Eyes are open, you may perform any amount of visual validations you desire.

By using the eyes.abortAsync functionality, we essentially tell Eyes to abort the test in case that Eyes were not properly closed for some reason.

/* main.mjs */
...

// Puppeteer: clean up
await browser.close();

// Applitools: clean up
await eyes.closeAsync();
await eyes.abortAsync(); // abort if Eyes were not properly closed

Finally, after Eyes and the browser are closed, we may gather the test results using the runner.

/* main.mjs */
...

// Manage tests across multiple Eyes instances
const testResultsSummary = await visualGridRunner.getAllTestResults()
for (const testResultContainer of testResultsSummary) {
  const testResults = testResultContainer.getTestResults();
  console.log(testResults);
}

You can find the full code in this GitHub repository.

Viewing Test Results in the Applitools Dashboard

After running the test, you’ll see the results populate in the Applitools dashboard. In this case, our baseline and our checkpoint have no visual differences, so everything passed.

Last but not Least – Export Cypress Tests from Google Chrome DevTools Recorder

As we have already mentioned, the ability to quickly record a stable automated test and export it to make it more customizable is an incredible feature. For advanced users, you may also customize how a recording is stringified by extending the PuppeteerStringifyExtension class.

For example, I’d like to introduce you to the Cypress Chrome Recorder library, where you can convert the JSON file into a Cypress test script with one simple command. The library is built on top of Puppeteer Replay’s stringified feature.

We can convert our JSON recording file to a Cypress test with the following CLI command:

npm install -g @cypress/chrome-recorder

npx @cypress/chrome-recorder <relative path to target test file>

The output will be written to the cypress/integration folder. If you do not have that folder, you can get it by installing Cypress with the npm install -D cypress in your project.

Once the test file is ready, we can simply run the test as we would run a standard Cypress test.

Conclusion

Although record and playback testing tools have their setbacks and challenges, this looks like a very simple and useful tool from Google. It can be a good solution for creating simple scenarios or quick tests, seeing how easy it is to use. 

What we loved most about the tool was its simplicity. Plain record and playback at the moment with no advanced features, it’s a great stepstone for beginners in testing or even non-code individuals. 

Like with any Record-Playback tool one of the challenges is validation. Combined with the ease and speed of adding and running Applitools Eyes you can start validating your UI in no time. Find all the visual regressions and make sure your application is visually perfect.

Applitools Eyes has many advanced features, including AI-powered auto-maintenance, which analyzes differences across all your tests and shows only distinct differences, allowing you to approve or reject changes that automatically apply across all similar changes in your test suite. Learn more about the Applitools platform and sign up for your own free account today.

The post Creating Your First Test With Google Chrome DevTools Recorder appeared first on Automated Visual Testing | Applitools.

With version 6.3.0, Cypress released a feature called “Cypress Studio.” It allows you to create a test script simply by opening the Cypress window and interacting with your app. If you are familiar with record & replay testing tools, you’ll probably get yourself familiar with the tool pretty quickly. So let’s see what Cypress Studio offers and what makes it different from other record & replay tools.

Getting started

Cypress Studio is still in experimental mode, so to enable it, you need to edit your cypress.json file. Just add "experimentalStudio": true and you are good to go.

To start working with Cypress Studio, create your first test file, and open it. You should now see a Cypress Studio icon in your test runner. At first, you might not even notice it, because the icon only appears when you hover over the test name.

Once you click the icon, the test runner will enter a “recording mode” and you can start interacting with your app. It will record clicks, typing, un/checking checkboxes and selecting options from a <select> element. If you accidentally do an unintended action, you can remove it.

Once you are happy with the flow you have created, you can save your commands right inside your test file. The flow you created is bounded by comments in the file, so you can clearly differ between what was generated and your own code.

Using Cypress Studio

First experience is really satisfying. You can see commands pop inside the test runner timeline. Being able to remove commands or accidental clicks is really neat and gives you a great control over the final script. Cypress Studio even seems to handle a situation when the application does an autofocus on an input and the user starts typing immediately. This is really nice, because this means that even if a user is not fully aware of this, it will still get added to the command chain.

Whenever you save your commands, your test is re-run. This gives you immediate feedback on your test flow. I have often found that my test is not really “repeatable” and I need to dive inside the code to handle this. That’s not always what I want, so to prevent automatic re-running of my test, I set "watchForFileChanges" to false in the cypress.json file.

One thing to keep in mind is that Cypress Studio does not record any assertions. Although you might be creating a great automation flow, you need to add some assertions to make your test really test something.

Cypress Studio builds on top of a Selector playground. This is another handy tool that helps you find the right selector in your application. It’s a great alternative to finding proper selectors via elements panel in Chrome developer tools.

What’s cool about Selector playground is that it has an API through which you can set up selector preference. The default preference can be found in docs, but if you wish to set up your own, you can do so by setting it up in your support/index.js file. For example, if you want to prefer using classes over any of the element attributes, you can set it up like this:

Who is Cypress Studio for?

To be completely honest, I wasn’t too excited when this feature was released. Trying to record my test flows will never be enough to generate a fully working test automation script. With a generated script you are just scratching the surface of what is happening in your app. There’s tons of network communication to be looked at and changes to be asserted.

But then I realized this was probably not built for someone who can already easily navigate themselves through Cypress or any other automation tool. It is actually a great learning tool for all those brave souls that want to start with test automation. And that’s where Cypress Studio becomes really empowering.

For years now, I held the opinion that Cypress is one of the greatest tools when it comes to starting with test automation. It has a really simple syntax and although you need to gather some JavaScript experience, you can get that thrill of your first running test very early on in your learning journey.

But even learning the basics of JavaScript might be too much in those beginnings. Especially when you are still learning what the web is made of. I started my first test automation efforts in Selenium IDE and there was a lot to figure out even with such a simple tool. Record and replay tools are great for getting that first time experience. Whenever you generate code, your natural next step is to examine how that code actually works. That can provide valuable insight.

E.g. you learn that for a normal user, typing into an input just a simple action. But for test automation script, it actually means:

1. selecting the element with proper selector
2. clearing the input
3. typing in the text
4. pressing an enter key

Not only that, but running a test multiple times introduces the problem of handling data that has been created, finding where the test should start and when should it end. This is just a fraction of things one needs to learn and it’s easy to forget these, once you’ve already passed a long way in your learning journey.

Cypress is a fairly new player in the test automation world and it seems to have gained a lot of love, and understandably, also some critique. Although the approach to testing might not sit well with everybody, there’s one area where the Cypress team pretty much nailed it. And that’s developer experience. With tools like Selector Playground and Cypress Studio, the Cypress team made sure that they open doors for everybody. At the time, when many make their first attempts in the tech world it makes perfect sense. Being one of those that started just a couple of years ago I really appreciate the effort to make test automation more accessible.

If you’ve enjoyed this post, visit my personal blog at filiphric.com, where I publish a new Cypress-related post every week.

The post Getting Started with Cypress Studio for Test Automation appeared first on Automated Visual Testing | Applitools.

Watch this webinar, where Selenium Guru Dave Haeffner and Software Developer Tomer Steinfeld, showcase the New Selenium IDE.

In this webinar, you’ll learn how the newly re-factored Selenium IDE is guaranteed to augment your testing efforts, regardless of your team’s automation “maturity” level, or your personal technical knowledge or experience.

Watch this session with Dave Haeffner and Tomer Steinfeld – full-time maintainers of Selenium IDE — and learn: 

• Why codeless solutions are worth paying attention to
• 10 traits of a great codeless testing tool
• Key features in the New Selenium IDE:
  • Reliable test playback with “locator fallback”
  • Enhance tests with control flow logic (e.g. if, else if, else, while, do, while, times)
  • Built-in debugger for diagnosing issues in your tests
  • Turn-key parallel, cross-browser execution that can be used on CI
  • Add/modify commands in the IDE, or integrate with a third-party provider through plugins
• Roadmap – what’s in store for the New Selenium IDE

Dave also shared how he went from a record-and-playback naysayer to the Selenium IDE maintainer.

In addition, this session includes 20 minutes of live Q-and-A with Dave and Tomer.

Full slide-deck:

Full webinar recording:

Demos (from the session):

Parallel execution with the Runner (slide 13 in the slide deck):

Reusable Test Case (slide 17 in the slide deck):

Additional Materials and Recommended Reading:

“10 Features Every Codeless Test Automation Tool Should Offer” — article by Angie Jones (as mentioned in the session by Dave)

Selenium IDE Page – on the Selenium Project website

Selenium Conf London – October 7-8, 2019, London UK – Join 600 test automation pros and an amazing lineup of speakers, including: Richard Bradshaw, Ash Coleman, and more. CFP and Early Bird Ticket Sales open on Feb 27!

Selenium IDE: The Next Generation — webinar with the Godfather of Selenium: Simon Stewart, inventor of Selenium WebDriver.

Release Apps with Flawless UI: Open your Free Applitools Account, and start visual testing today.

Test Automation U — the most-talked-about test automation initiative of the year: online education platform led by Angie Jones, offering free test automation courses by industry leaders. Sign up and start showing off your test automation badges!

— HAPPY TESTING —

The post The New Selenium IDE: See How It Can Turbo-charge Your Testing Efforts appeared first on Automated Visual Testing | Applitools.

Selenium IDE got a new long overdue overhaul, and Simon Stewart, Selenium Project Lead and Creator of WebDriver, unveiled the new Selenium IDE in a special live session, including capabilities, features, and roadmap — now available on-demand.  

The Selenium Project is composed of several different pieces.

For power users, Selenium Grid allows people to scale their tests horizontally, for developers, there are bindings available in almost every known programming language.

But what about the case where you’re not an experienced developer? Or when you want to bootstrap a new test suite? Or file a bug against a site and provide a reproducible test case? This is where Selenium IDE fits in perfectly.

In this webinar, Simon explored the new Selenium IDE, looking at where it fits into the Selenium ecosystem, how to install it, and how to use it. He will also look at planned features that are coming your way, and the roadmap from the current alphas to a release.

Useful Links:

• Selenium IDE for Chrome
• Selenium IDE for FireFox
• Instructions for creating a new command (or plugin) for IDE can be found here. 
• People interested in getting the code for IDE itself can go the Selenium IDE page on GitHub. 
• … And this is a bit off topic, but if you want to present a talk @ SeleniumConf Tokyo (taking place April 18-19), you can submit your CFP here (CFP closes on Dec 14).

Full webinar recording:

Advanced Materials & Additional Reading:

Test Automation U — is a free, community-driven education platform, focused on increasing test automation success rates. Angie Jones, who is leading Test Automation U, will be unveiling this exciting new initiative in a special live session — click here for more details.

JavaScript Asynchrony and async/await in Selenium WebDriver Tests – “how to” post by Gil Tayar

How to Automate Your Video Testing using Selenium – “how to” post by Justin Ison

Comparing JavaScript Browser Automation Frameworks: Selenium vs Webdriver.io vs Puppeteer – “how to” post by Gil Tayar

Cypress vs Selenium WebDriver: Better, or just different – post by Gil Tayar

Start visual testing today with Applitools Eyes – open your free account now.

The post Selenium IDE: The Next Generation – presented by Simon Stewart appeared first on Automated Visual Testing | Applitools.

Test Automation thought leaders gathered for a round-table discussion about the upcoming trends, best practices, tools, and ideas that will shape your Dev/Test environment in 2019.

Joe Colantonio hosted this all-star expert panel, including: Angie Jones, Dave Haeffner, and Gil Tayar – as they shared their thoughts and insights on the hottest topics in test automation and software quality, including:

• Shift left: how to get developers involved and invested in the testing process
• Selenium IDE: ghosts of the past, present, and future
• Avoiding common pitfalls: how to be successful with test automation
• Never bet against it: JavaScript’s role in test automation
• New automation frameworks Cypress and Puppeteer: the good, the bad and the ugly
• Keep calm and record-and-playback

Listen to the recording:

Slide Deck:

— HAPPY TESTING — 

The post Test Automation in 2019: Glimpse of the Near Future From the Experts Who Shape It appeared first on Automated Visual Testing | Applitools.

Test automation folklore is full of horror stories of failed attempts to apply Record & Playback (RPB) tools to perform GUI-based Functional Test Automation. In the first part of this post we explored the main causes for these failures. In this part we will show how RPB tools can be effectively used for automating Visual Testing. 

Visual Software Testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, Visual Testing focuses on validating the Layout and Appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.

Automated Visual Testing tools can automate most of the activities involved in Visual Testing. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation. For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. RPB tools excel at recording these sort of human interactions with the AUT and are therefore effective for driving automated visual tests.

Let’s take a second look at the main limitations of RPB tools (see part 1 for details), and their implications on Visual Testing:

• Validation: this is perhaps the main downside of RPB tools when applied to functional testing, since validation logic cannot be recorded, and its specification requires tool-specific or programming skills. In the case of Visual Testing, validation involves obtaining and approving screenshots of the application’s UI – a trivial task that only requires the tester to be familiar with the UI of the AUT.
• No data parameterization and control flow structures: although this is a key requirement for Functional Testing, in practice there is little value in repeatedly validating the visual aspects of the same screen. You might want to verify that a certain screen renders correctly when displaying diverse amounts of data, but even in extreme cases, 2-3 validation points are sufficient, and complex control flow mechanisms are not required.
• No test reuse: since visual tests include mostly linear traversals of UI screens, and do not include complex, manually-specified validation logic, there is little to reuse. When the AUT changes, affected tests can easily be corrected by re-recording the broken test steps.

Figure 1 illustrates Selenium IDE, a popular RPB tool for web browsers, extended with a toolbar button (circled in red) that adds a checkWindow command, which validates the correctness of the displayed web page. By using such a tool, a tester can easily browse the pages of a web application, add visual validation points where necessary by clicking the checkWindow button, and then run multiple visual tests by replaying the recorded test on various web browsers, operating systems, and screen resolutions. Recording a test can take only a few minutes, and save hours upon hours of manual visual testing. Since it is so easy to generate tests, the overall maintenance cost is negligible compared to the time saved in test execution.

There are, of course, situations in which an RPB tool will fail to record user interactions correctly (for example, clicking a complex, composite UI control). In such cases, tool specific skills or even programming skills are required to correctly automate the desired user interaction. However, for the majority of applications and the majority of screens of a “complex” application, an RPB tool can be effectively used to generate and execute automated visual tests.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Record Playback and Visual Testing – Part 2 appeared first on Automated Visual Testing | Applitools.

Test automation folklore is full of horror stories of failed attempts to apply Record & Playback tools to perform GUI-based Functional Test Automation. In this post we will explore the main causes for these failures. In part 2 of this post we will show how Record & Playback tools can be effectively used for automating Visual Testing. 

Many commercial and open source test automation tools provide record and playback (RPB) features, that allow testers to easily create tests by recording manual interactions with the UI of the Application Under Test (AUT). Recorded tests can then be played back to automatically test the AUT, possibly in different execution environments than the one used to create the test. For example, Selenium IDE, a popular open-source RPB test tool for web-applications, allows testers to record tests on a Firefox browser, and play them back on a variety of other desktop and mobile browsers.

The appeal of RPB test automation tools is clear: they allow testers to quickly and easily create automated tests while requiring little to no programming effort and tool specific skills. Since the introduction of the first RPB test automation tools in the early 90’s, the technologies underlying them have evolved substantially. Robust, multi-layered UI object identification replaced absolute cursor coordinates, implicit delays are automatically added when missing UI objects are accessed while an application window loads, the sensitivity of the recorded tester actions can be throttled, UI object maps allow UI objects identification details to be shared between tests to simplify maintenance, etc.

However, despite all these advancements, test automation folklore is full of horror stories of failed attempts to apply RPB tools to perform GUI-based Functional Test Automation. Let’s take a deeper look at some of the main causes for these failures:

• No test reuse: with RPB tools, if you want to add a test, you simply record it, resulting with test steps that invariably overlap step sequences in other tests, but have to be maintained separately. When the behavior of the AUT changes, the affected scenarios must be re-recorded and corrected individually in each of the tests.
• No data parameterization and control flow structures: when validating the functionality of an AUT, it is often required to repeatedly perform the same series of actions, each time using different data as input. A pure RPB tool requires a tester to record each and every one of these repeated interactions, hard-coding the input data inside the test, instead of looping through the repeated scenario using different input and expected output data (defined externally) in each iteration. Moreover, in order to add, modify or remove data entries, actions must be re-recorded, and when the AUT changes, each and every recorded “iteration” has to be updated separately.
• Validation: simulating user interactions is just a small part of creating and maintaining automated tests. The bulk of effort involved is validating the correctness of the AUT. A manual tester validates the functionality of the AUT by looking at the screen or by performing procedures that are external to the UI of the AUT. Of course, this sort of interaction cannot be recorded by the tool and must be manually defined by the tester. Different RPB tools provide different facilities to define validation commands and assertions, ranging from simple WYSIWYG abstractions and up to full-fledged programming. To accomplish this task effectively, tool specific skills and even programming skills are required, which contradicts the initial motivation to use an RPB tool. Combined with the little or no test reuse described above, there is little chance for large-scale RPB-based test automation to succeed.

Due to these limitations, RPB tools are hardly used in-practice for automating GUI-based functional testing. Most commercial and open-source test automation tools have evolved beyond RPB, to facilitate large-scale functional test automation, but at the expense of raising the bar of entry in terms of tool specific skills and programming skills. In part 2 of this blog we will show how a pure RPB approach can practically and effectively be used for automating Visual Testing.

Yeah! Take me to part 2 of this post!

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Record Playback and Visual Testing – Part 1 appeared first on Automated Visual Testing | Applitools.