Learn about common localization bugs, the traditional challenges involved in finding them, and solutions that can make localization testing far easier.

What is Localization?

Localization is the process of customizing a software application that was originally designed for a domestic market so that it can be released in a specific foreign market.

How to Get Started with Localization

Localization testing usually involves substantial changes of the application’s UI, including the translation of all texts to the target language, replacement of icons and images, and many other culture, language, and country-specific adjustments, that affect the presentation of data (e.g., date and time formats, alphabetical sorting order, etc.). Due to the lack of in-house language expertise, localization usually involves in-house personnel as well as outside contractors, and localization service providers.

Before a software application is localized for the first time, it must undergo a process of Internationalization.

What is Internationalization?

Internationalization often involves an extensive development and re-engineering effort which goal is to allow the application to operate in localized environments and to correctly process and display localized data. In addition, locale-specific resources such as texts, images and documentation files, are isolated from the application code and placed in external resource files, so they can be easily replaced without requiring further development efforts.

Once an application is internationalized, the engineering effort required to localize it to a new language or culture is drastically reduced. However, the same is not true for UI localization testing.

The Challenge of UI Localization Testing

Every time an application is localized to a new language, the application changes, or the resources of a supported localization change, the localized UI must be thoroughly tested for localization and internationalization (LI) bugs.

Common Localization and Internationalization Bugs Most Testers can Catch

LI bugs which can be detected by testers that are not language experts include:

• Broken functionality – the execution environment, data or translated resources of a new locale, may uncover internationalization bugs can prevent the application from running or break some of its functionality.
• Untranslated text – text appearing in text fields or images of the localized UI is left untranslated. This indicates that certain resources were not translated or that the original text is hard-coded in the UI and not exported to the resource files.
• Text overlap / overflow – the translated text may require more space than available in its containing control, resulting with the text overflowing the bounds of the control and possibly overlapping or hiding other UI elements.
• Layout corruption – UI controls dynamically adjust their size and position to the expanded or contracted size of the localized text, icons or images, resulting with misaligned, overlapping, missing or redundant UI artifacts.
• Oversized windows and dialogs – multiple expanded texts and images can result with oversized tooltips, dialogs and windows. In extreme situations, expanded dialogs and windows may only be partially visible in low screen resolutions.
• Inadequate fonts – a control’s font cannot properly display some characters of the target language. This usually results with question marks or glyphs being displayed instead of the expected text.

Localization and Internationalization Bugs Requiring Language Expertise

Other common LI bugs which can only be detected with the help of a language expert include:

• Mistranslation – translated text that appears once in the resource files, may appear multiple times in different parts of the application. The context in which the text appears can vary its meaning and require a different translation.
• Wrong images and icons – images and icons were replaced with wrong or inappropriate graphics.
• Text truncation – the translated text may require more space than available in its containing control, resulting with a truncated string.
• Locale violations – wrong date, time, number and currency formats, punctuation, alphabetical sort order, etc.

Localization and Internationalization Bugs are Hard to Find

An unfortunate characteristic of LI bugs, is that they require a lot of effort to find. To uncover such bugs, a tester (assisted by a language expert) must carefully inspect each and every window, dialog, tooltip, menu item, and any other UI state of the application. Since most of these bugs are sensitive to the size and layout of the application, tests must be repeated on a variety of execution environments (e.g., different operating systems, web browsers, devices, etc.) and screen resolutions. Furthermore, if the application window is resizable, tests should also be repeated for various window sizes.

Why is UI Localization Testing Hard?

There are several other factors that contribute to the complexity of UI Localization testing:

• Lack of automation – most of the common LI bugs listed above are visual and cannot be effectively detected by traditional functional test automation tools. Manual inspection of the localized UI is slower than with a non-localized UI because it is  unreadable to the tester.
• Lack of in-house language expertise – since many of the common LI bugs can only be detected with the help of external language experts, which are usually not testers and are not familiar with the application under test, LI testing often requires an in-house tester to perform tests together with a language expert. In many cases, these experts work on multiple projects for multiple customers in parallel, and their occasional lack of availability can substantially delay test cycles and product releases. Similarly, delays can occur while waiting for the translation of changed resources, or while waiting for translation bugs to be fixed.
• Time constraints – localization projects usually begin at late stages of the development lifecycle, after the application UI has stabilized. In many cases, testers are left with little time to properly perform localization tests, and are under constant pressure to avoid delaying the product release.
• Bug severity – UI localization bugs such as missing or garbled text are often considered critical, and therefore must be fixed and verified before the product is released.

Due to these factors, maintaining multiple localized application versions and adding new ones, incurs a huge overhead on quality assurance teams.

Fortunately, there is a modern solution that can make localization testing significantly easier – Automated Visual Testing.

How to Automate Localization Testing with Visual Testing

Visual test automation tools can be applied to UI localization testing to eliminate unnecessary manual involvement of testers and language experts, and drastically shorten test cycles.

To understand this, let’s first understand what visual testing is, and then how to apply visual testing to localization testing.

What is Visual Testing?

Visual testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, visual testing focuses on validating the layout and appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.

Visual Test Automation tools can automate most of the activities involved in visual testing. They can easily detect many common UI localization bugs such as text overlap or overflow, layout corruptions, oversized windows and dialogs, etc. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation.

For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. Driving the AUT through its different UI states can be easily automated using a variety of open-source and commercial tools (e.g., Selenium, Cypress, etc.). If the tool is properly configured to rely on internal UI object identifiers, the same automation script/program can be used to drive the AUT in all of its localized versions.

So, how can we use this to simplify UI localization testing?

How Automated Visual Testing Simplifies UI Localization Testing

• Preparation – in order to provide translators with the context required to properly localize the application, screenshots of the application’s UI are often delivered along with the resource files to be localized. The process of manually collecting these screenshots is laborious, time consuming, and error prone. When a visual test automation tool is in place, updated screenshots of all UI states are always available and can be shared with translators with a click of a button. When an application changes, the tool can highlight only those screens (in the source language) that differ from the previous version so that only those screens are provided to translators. Some visual test automation tools also provide animated “playbacks” of tests showing the different screens, and the human activities leading from one screen to the next (e.g., clicks, mouse movements, keyboard strokes, etc.).  Such animated playbacks provide much more context than standalone screenshots and are more easily understood by translators, which are usually not familiar with the application being localized. Employing a visual test automation tool can substantially shorten the localization project’s preparation phase and assist in producing higher quality preliminary translations, which in turn can lead to fewer and shorter test cycles.

• Testing localization changes – visual test automation tools work by comparing screenshots of an application against a set of previously approved “expected” screenshots called the baseline. After receiving the translated resources and integrating them with the application, a visual test of the updated localized application can be automatically executed using the previous localized version as a baseline. The tool will then report all screens that contain visual changes and will also highlight the exact changes in each of the changed screens. This report can then be inspected by testers and external language experts without having to manually interact with the localized application. By only focusing on the screens that changed, a huge amount of time and effort can be saved. As we showed above, most UI localization bugs are visual by nature and are therefore sensitive to the execution environment (browser, operating system, device, screen resolution, etc.). Since visual test automation tools automatically execute tests in all required execution environments, testing cycles can be drastically shortened.

• Testing new localizations – when localizing an application for a new language, no localized baseline is available to compare with. However, visual test automation tools can be configured to perform comparisons at the layout level, meaning that only layout inconsistencies (e.g., missing or overflowing text, UI elements appearing out of place, broken paragraphs or columns, etc.) are flagged as differences. By using layout comparison, a newly localized application can be automatically compared with its domestic version, to obtain a report indicating all layout inconsistencies, in all execution environments and screen resolutions.

• Incremental validation – when localization defects are addressed by translators and developers, the updated application must be tested again to make sure that all reported defects were fixed and that no new defects were introduced. By using the latest localized version as the baseline with which to compare the newly updated application, testers can easily identify the actual changes between the two versions, and quickly verify their validity, instead of manually testing the entire application.

• Regression testing – whenever changes are introduced to a localized application, regression testing must be performed to make sure that no localization bugs were introduced, even if no direct changes were made to the application’s localizable resources. For example, a UI control can be modified or replaced, the contents of a window may be repositioned, or some internal logic that affects the application’s output may change. It is practically impossible to manually perform these tests, especially with today’s Agile and continuous delivery practices, which dictate extremely short release cycles. Visual test automation tools can continuously verify that no unexpected UI changes occur in any of the localized versions of the application, after each and every change to the application.

• Collateral material – in additional to localizing the application itself, localized versions of its user manual, documentation and other marketing and sales collateral must be created. For this purpose, updated screenshots of the application must be obtained. As described above, a visual test automation tool can provide up-to-date screenshots of any part of the application in any execution environment. The immediate availability of these screenshots significantly reduces the chance of including out-of-date application images in collaterals and eliminates the manual effort involved in obtaining them after each application change.

Application localization is notoriously difficult and complex. Manually testing for UI localization bugs, during and between localization projects, is extremely time consuming, error-prone, and requires the involvement of external language experts.

Visual test automation tools are a modern breed of test automation tools that can effectively eliminate unnecessary manual involvement, drastically shorten the duration of localization projects, and increase the quality of localized applications.

Applitools Automated Visual Testing and Localization Testing

Applitools has pioneered the use of Visual AI to deliver the best visual testing in the industry. You can learn more about how Applitools can help you with localization testing, or to get started with Applitools today, request a demo or sign up for a free Applitools account.

Editor’s Note: Parts of this post were originally published in two parts in 2017/2018, and have since been updated for accuracy and completeness.

The post Getting Started with Localization Testing appeared first on Automated Visual Testing | Applitools.

It is no secret that many teams struggle with automated functional UI testing – some to the point where it is completely abandoned – even though the UI is the most significant interface of the system.

In this session, Adam Carmi — Applitools CTO and Co-founder — reviewed the main weaknesses of traditional approaches to UI testing, and how they negatively affect test stability, maintainability, coverage, execution speed, and overall ROI. He also discussed how these weaknesses become even more severe when running tests across multiple devices and browsers.

Adam demonstrated how Visual AI — the innovative technology powering Applitools’ testing engine — can be applied on your existing pipeline to efficiently implement functional UI testing with a fraction of the effort while drastically increasing test coverage and reducing test execution time.

Adam also showed a live coding session, where he converted a traditional UI test into a Visual AI-based test in minutes, and executed it across dozens of devices and browsers in seconds using the Applitools Ultrafast Grid.

Adam’s Functional UI Testing slide deck:

https://www.slideshare.net/slideshow/embed_code/key/pV803UfrsUZXOL

Full webinar recording:

Additional Recommended Resources:

• Overcome the challenges of Cross-Browser Testing using Applitools Ultrafast Grid!
• Applitools Root Cause Analysis: Found a Bug? We’ll Help You Fix It!
• How To Run 372 Cross-browser Tests In Under 3 Minutes
• Modern Functional Test Automation Through Visual AI [webinar]
• Angie Jones Presents: “Automated Visual Testing: A Fast Path To Test Automation Success” — ace automated visual testing with this free 9-chapter online course by Angie Jones, exclusively on Test Automation University
• 3 Steps to Visual Testing for PDF Files [with Code] — post by Angie Jones
• Angie Jones’s online Test Automation University course: “Setting a Foundation for Successful Test Automation” — In this TAU course, Angie explores the initial considerations for starting a new test automation initiative
• “Modern Functional Test Automation Through Visual AI” — take this online course — only on Test Automation University
• Start UI testing today — Open your free Applitools account right here

— HAPPY TESTING —

The post Cutting-edge Functional UI Testing Techniques – Live Coding Webinar w/ Adam Carmi appeared first on Automated Visual Testing | Applitools.

Once an application is internationalized, the engineering effort required to maintain its localized versions, and to localize it to new languages, is drastically reduced. However, the same is not true for UI localization testing.

In the first part of this post we reviewed common localization bugs, and the challenges involved in finding them. In this part, we show how Visual Test Automation tools can be applied to UI localization testing, eliminate unnecessary manual involvement of testers and language experts, and drastically shorten test cycles. 

Visual Software Testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, Visual Testing focuses on validating the layout and appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.

Visual Test Automation tools can automate most of the activities involved in visual testing. They can easily detect many common UI localization bugs such as text overlap or overflow, layout corruptions, oversized windows and dialogs, etc. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation.

For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. Driving the AUT through its different UI states can be easily automated using a variety of open-source and commercial tools (e.g., Selenium, UFT, etc.). If the tool is properly configured to rely on internal UI object identifiers, the same automation script/program can be used to drive the AUT in all of its localized versions.

So, how can visual test automation tools be used to simplify UI localization testing?

• Preparation – in order to provide translators with the context required to properly localize the application, screenshots of the application’s UI are often delivered along with the resource files to be localized.The process of manually collecting these screenshots is laborious, time consuming, and error prone. When a visual test automation tool is in place, updated screenshots of all UI states are always available and can be shared with translators with a click of a button. When an application changes, the tool can highlight only those screens (in the source language) that differ from the previous version so that only those screens are provided to translators. Some visual test automation tools also provide animated “playbacks” of tests showing the different screens, and the human activities leading from one screen to the next (e.g., clicks, mouse movements, keyboard strokes, etc.).  Such animated playbacks provide much more context than standalone screenshots and are more easily understood by translators, which are usually not familiar with the application being localized.Employing a visual test automation tool can substantially shorten the localization project’s preparation phase and assist in producing higher quality preliminary translations, which in turn can lead to fewer and shorter test cycles.

• Testing localization changes – visual test automation tools work by comparing screenshots of an application against a set of previously approved “expected” screenshots called the baseline.After receiving the translated resources and integrating them with the application, a visual test of the updated localized application can be automatically executed using the previous localized version as a baseline. The tool will then report all screens that contain visual changes and will also highlight the exact changes in each of the changed screens. This report can then be inspected by testers and external language experts without having to manually interact with the localized application. By only focusing on the screens that changed, a huge amount of time and effort can be saved. As we have shown in the first part of this post, most UI localization bugs are visual by nature and are therefore sensitive to the execution environment (browser, operating system, device, screen resolution, etc.). Since visual test automation tools automatically execute tests in all required execution environments, testing cycles can be drastically shortened.

• Testing new localizations – when localizing an application for a new language, no localized baseline is available to compare with. However, visual test automation tools can be configured to perform comparisons at the layout level, meaning that only layout inconsistencies (e.g., missing or overflowing text, UI elements appearing out of place, broken paragraphs or columns, etc.) are flagged as differences. By using layout comparison, a newly localized application can be automatically compared with its domestic version, to obtain a report indicating all layout inconsistencies, in all execution environments and screen resolutions.

• Incremental validation – when localization defects are addressed by translators and developers, the updated application must be tested again to make sure that all reported defects were fixed and that no new defects were introduced. By using the latest localized version as the baseline with which to compare the newly updated application, testers can easily identify the actual changes between the two versions, and quickly verify their validity, instead of manually testing the entire application.

• Regression testing – whenever changes are introduced to a localized application, it must be tested to make sure that no localization bugs were introduced, even if no direct changes were made to the application’s localizable resources. For example, a UI control can be modified or replaced, the contents of a window may be repositioned, or some internal logic that affects the application’s output may change. It is practically impossible to manually perform these tests, especially with today’s Agile and continuous delivery practices, which dictate extremely short release cycles. Visual test automation tools can continuously verify that no unexpected UI changes occur in any of the localized versions of the application, after each and every change to the application.

• Collateral material – in additional to localizing the application itself, localized versions of its user manual, documentation and other marketing and sales collateral must be created. For this purpose, updated screenshots of the application must be obtained. As described above, a visual test automation tool can provide up-to-date screenshots of any part of the application in any execution environment. The immediate availability of these screenshots significantly reduces the chance of including out-of-date application images in collaterals and eliminates the manual effort involved in obtaining them after each application change.

Application localization is notoriously difficult and complex. Manually testing for UI localization bugs, during and between localization projects, is extremely time consuming, error-prone, and requires the involvement of external language experts.

Visual test automation tools are a new breed of test automation tools that can effectively eliminate unnecessary manual involvement, drastically shorten the duration of localization projects, and increase the quality of localized applications.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Taking the Pain Out of UI Localization Testing – Part 2 appeared first on Automated Visual Testing | Applitools.

Listen to Adam Carmi, CTO and Co-founder at Applitools, as he takes a deep dive into the ecosystem of visual testing, including:

• Challenges with test automation
• Tools and test frameworks
• Best practices when testing the UI of your application
• A live demo of automation visual testing

In this session, we will take a deep dive into some of the technological challenges involved with visual test automation. We’ll also show how modern tools address these challenges using automated visual testing.

Watch this webinar, where we: 

• Review available open-source and commercial visual testing tools
• Demo cutting-edge technologies that enable running cross-browser and cross-device visual tests at large scale
• Show how visual test automation fits in the development/deployment lifecycle

You can watch The Missing Link in Test Automation on-demand webinar recording here:

And you can find Adam’s slide deck on the Introduction to Automated Visual Testing below:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account. Happy testing!

The post The Missing Link in Test Automation: Intro to Automated Visual Testing appeared first on Automated Visual Testing | Applitools.

So, how do you ensure that all of the visual elements of your app are functioning flawlessly without breaking the bank or delaying time-to-market?

The answer is Automated Visual Testing, an easy and reliable method to automatically validate that your app looks and acts as designed across devices, browsers, operating systems and form factors.

Watch this webinar with mobility expert Manish Mathuria (Infostretch) and automated UI validation expert Adam Carmi (Applitools), and discover:

• What Automated Visual Testing involves
• How it can help you helps shorten testing cycles while increasing coverage
• The landscape of tools and technologies available for this increasingly important type of testing
• Industry-leading best practices to quickly achieve maximum value with automated visual testing

What the on-demand recording of Best Practices of Automated Visual Testing:

Full slide deck can be found here:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Best Practices of Automated Visual Testing: Getting it Right for the Best UX appeared first on Automated Visual Testing | Applitools.

Watch Adam Carmi’s talk at SeConf India!

If you don’t know what automated visual testing is, or if you think that Sikuli is a visual test automation tool, or if you are already automating your visual tests and want to learn more on what else is out there, or if you are on your way to implementing Continuous Deployment or just interested in seeing how cool image processing algorithms can be – this talk is for you! 

Automated visual testing is a major trend in the Dev/Test community. In this talk, you will learn what visual testing is and why it should be automated. We take a deep dive into some of the technological challenges involved with visual test automation and show how modern tools address them.

We also review available Selenium-based open-source and commercial automated visual testing tools, demo cutting-edge technologies that enable running cross-browser and cross-device visual tests at large scale, and show how visual test automation fits in the development/deployment lifecycle.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How to do Advanced Automated Visual UI Testing With Selenium appeared first on Automated Visual Testing | Applitools.

Learn advanced skills for how to effectively automate tests for your responsive apps and sites – in this on-demand webinar.

Automatically testing responsive sites and apps can be a challenge, due to the need to cover all supported layouts, their respective navigation, and visible content.

Watch this advanced session, where Applitools co-founder and CTO Adam Carmi implements a complete Selenium-based automated test for a popular responsive website from scratch.

Watch this webinar and learn advanced techniques, including how to:

• Implement generic tests that work for all the different layouts of your app
• Control the browser’s viewport size in order to accurately target layout transition points
• Incorporate assertions that are specific to your layouts in your tests
• Effectively design responsive page objects
• Visually validate the correctness of your app’s layout
• Bonus: tips for planning and executing responsive website testing

Watch the Advanced Test Automation Techniques for Testing Responsive Apps and Sites webinar:

You can find Adam’s slide-deck here:

and his code examples here.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Advanced Test Automation Techniques for Testing Responsive Apps and Sites appeared first on Automated Visual Testing | Applitools.

We are rolling out a shiny new UI for Applitools Eyes, with a brand new look and feel, and improved user experience on all fronts of automated visual testing.

• The new test manager is designed to streamline large-scale baseline management.
• There is a new step thumbprints gallery for easier to search test validation.
• We have improved batch, test, and step searching and filtering for easier search and test analysis.
• We have improved automated maintenance capabilities, including grouping of similar differences flagged across entire batches of tests.
• There are substantially reduced wait times for tests to complete.

We also share the best practices for using the new UI in the test manager.

What the full on-demand recording of the webinar featuring Applitools CEO Adam Carmi sharing the new Applitools Eyes UI here:

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Applitools Eyes is Launching a New Shiny UI appeared first on Automated Visual Testing | Applitools.

Adam Carmi, Applitools’ co-founder and VP R&D, gave a talk at GTAC 2015, where he presented 5 tips for creating reliable and scalable automated visual tests, which help avoid visual regressions across multiple browsers and devices.

Watch Adam’s GTAC talk: (slide deck can be found here) 

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post How to Create Reliable and Scalable Automated Visual Tests appeared first on Automated Visual Testing | Applitools.

Adam Carmi, Applitools’ co-founder and CTO, gave a talk at last month’s Selenium Conference, where he presented Automated Visual Testing – and how it can easily be integrated into existing Selenium tests, to create a robust and scalable UI testing, which enables companies to avoid visual regressions and UI bugs across browsers and devices

Watch Adam’s talk right here: 

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Advanced Visual Testing with Selenium appeared first on Automated Visual Testing | Applitools.

Test automation folklore is full of horror stories of failed attempts to apply Record & Playback (RPB) tools to perform GUI-based Functional Test Automation. In the first part of this post we explored the main causes for these failures. In this part we will show how RPB tools can be effectively used for automating Visual Testing. 

Visual Software Testing is the process of validating the visual aspects of an application’s User Interface (UI).

In addition to validating that the UI displays the correct content or data, Visual Testing focuses on validating the Layout and Appearance of each visual element of the UI and of the UI as a whole. Layout correctness means that each visual element of the UI is properly positioned on the screen, is of the right shape and size, and doesn’t overlap or hide other visual elements. Appearance correctness means that the visual elements are of the correct font, color, or image.

Automated Visual Testing tools can automate most of the activities involved in Visual Testing. All a tester needs to do is to drive the Application Under Test (AUT) through its various UI states and submit UI screenshots to the tool for visual validation. For simple websites, this can be as easy as directing a web browser to a set of URLs. For more complex applications, some buttons or links should be clicked, or some forms should be filled in order to reach certain screens. RPB tools excel at recording these sort of human interactions with the AUT and are therefore effective for driving automated visual tests.

Let’s take a second look at the main limitations of RPB tools (see part 1 for details), and their implications on Visual Testing:

• Validation: this is perhaps the main downside of RPB tools when applied to functional testing, since validation logic cannot be recorded, and its specification requires tool-specific or programming skills. In the case of Visual Testing, validation involves obtaining and approving screenshots of the application’s UI – a trivial task that only requires the tester to be familiar with the UI of the AUT.
• No data parameterization and control flow structures: although this is a key requirement for Functional Testing, in practice there is little value in repeatedly validating the visual aspects of the same screen. You might want to verify that a certain screen renders correctly when displaying diverse amounts of data, but even in extreme cases, 2-3 validation points are sufficient, and complex control flow mechanisms are not required.
• No test reuse: since visual tests include mostly linear traversals of UI screens, and do not include complex, manually-specified validation logic, there is little to reuse. When the AUT changes, affected tests can easily be corrected by re-recording the broken test steps.

Figure 1 illustrates Selenium IDE, a popular RPB tool for web browsers, extended with a toolbar button (circled in red) that adds a checkWindow command, which validates the correctness of the displayed web page. By using such a tool, a tester can easily browse the pages of a web application, add visual validation points where necessary by clicking the checkWindow button, and then run multiple visual tests by replaying the recorded test on various web browsers, operating systems, and screen resolutions. Recording a test can take only a few minutes, and save hours upon hours of manual visual testing. Since it is so easy to generate tests, the overall maintenance cost is negligible compared to the time saved in test execution.

There are, of course, situations in which an RPB tool will fail to record user interactions correctly (for example, clicking a complex, composite UI control). In such cases, tool specific skills or even programming skills are required to correctly automate the desired user interaction. However, for the majority of applications and the majority of screens of a “complex” application, an RPB tool can be effectively used to generate and execute automated visual tests.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Record Playback and Visual Testing – Part 2 appeared first on Automated Visual Testing | Applitools.

Test automation folklore is full of horror stories of failed attempts to apply Record & Playback tools to perform GUI-based Functional Test Automation. In this post we will explore the main causes for these failures. In part 2 of this post we will show how Record & Playback tools can be effectively used for automating Visual Testing. 

Many commercial and open source test automation tools provide record and playback (RPB) features, that allow testers to easily create tests by recording manual interactions with the UI of the Application Under Test (AUT). Recorded tests can then be played back to automatically test the AUT, possibly in different execution environments than the one used to create the test. For example, Selenium IDE, a popular open-source RPB test tool for web-applications, allows testers to record tests on a Firefox browser, and play them back on a variety of other desktop and mobile browsers.

The appeal of RPB test automation tools is clear: they allow testers to quickly and easily create automated tests while requiring little to no programming effort and tool specific skills. Since the introduction of the first RPB test automation tools in the early 90’s, the technologies underlying them have evolved substantially. Robust, multi-layered UI object identification replaced absolute cursor coordinates, implicit delays are automatically added when missing UI objects are accessed while an application window loads, the sensitivity of the recorded tester actions can be throttled, UI object maps allow UI objects identification details to be shared between tests to simplify maintenance, etc.

However, despite all these advancements, test automation folklore is full of horror stories of failed attempts to apply RPB tools to perform GUI-based Functional Test Automation. Let’s take a deeper look at some of the main causes for these failures:

• No test reuse: with RPB tools, if you want to add a test, you simply record it, resulting with test steps that invariably overlap step sequences in other tests, but have to be maintained separately. When the behavior of the AUT changes, the affected scenarios must be re-recorded and corrected individually in each of the tests.
• No data parameterization and control flow structures: when validating the functionality of an AUT, it is often required to repeatedly perform the same series of actions, each time using different data as input. A pure RPB tool requires a tester to record each and every one of these repeated interactions, hard-coding the input data inside the test, instead of looping through the repeated scenario using different input and expected output data (defined externally) in each iteration. Moreover, in order to add, modify or remove data entries, actions must be re-recorded, and when the AUT changes, each and every recorded “iteration” has to be updated separately.
• Validation: simulating user interactions is just a small part of creating and maintaining automated tests. The bulk of effort involved is validating the correctness of the AUT. A manual tester validates the functionality of the AUT by looking at the screen or by performing procedures that are external to the UI of the AUT. Of course, this sort of interaction cannot be recorded by the tool and must be manually defined by the tester. Different RPB tools provide different facilities to define validation commands and assertions, ranging from simple WYSIWYG abstractions and up to full-fledged programming. To accomplish this task effectively, tool specific skills and even programming skills are required, which contradicts the initial motivation to use an RPB tool. Combined with the little or no test reuse described above, there is little chance for large-scale RPB-based test automation to succeed.

Due to these limitations, RPB tools are hardly used in-practice for automating GUI-based functional testing. Most commercial and open-source test automation tools have evolved beyond RPB, to facilitate large-scale functional test automation, but at the expense of raising the bar of entry in terms of tool specific skills and programming skills. In part 2 of this blog we will show how a pure RPB approach can practically and effectively be used for automating Visual Testing.

Yeah! Take me to part 2 of this post!

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Record Playback and Visual Testing – Part 1 appeared first on Automated Visual Testing | Applitools.