Test Automation University (also called “TAU”) is one of the best online platforms for learning testing and automation skills. TAU offers dozens of courses from the world’s leading instructors, and everything is available for free. The platform is proudly powered by Applitools. As of November 2022, nearly 140,000 students have signed up! TAU has become an invaluable part of the testing community at large. Personally, I know many software teams who use TAU courses as part of their internal onboarding and mentorship programs.

So, which TAU courses are currently the most popular? In this list, we’ll count down the top 10 most popular courses, ranked by the total number of course completions over the past year. Let’s go!

#10: Selenium WebDriver with Java

Starting off the list at #10 is Selenium WebDriver with Java by none other than Angie Jones. Even with the rise of alternatives like Cypress and Playwright, Selenium WebDriver continues to be one of the most popular tools for browser automation, and Java continues to be one of its most popular programming languages. Selenium WebDriver with Java could almost be considered the “default” choice for Web UI test automation.

In this course, Angie digs deep into the WebDriver API, teaching everything from the basics to advanced techniques. It’s a great course for building a firm foundation in automation with Selenium WebDriver.

#9: Python Programming

#9 on our list is one of our programming courses: Python Programming by Jess Ingrassellino. Python is hot right now. On whatever ranking, index, or article you find these days for the “most popular programming languages,” Python is right at the top of the list – often vying for the top spot with JavaScript. Python is also quite a popular language for test automation, with excellent frameworks like pytest, libraries like requests, and bindings for browser automation tools like Selenium WebDriver and Playwright.

In this course, Dr. Jess teaches programming in Python. This isn’t a test automation course – it’s a coding course that anyone could take. She covers both structured programming and object-oriented principles from the ground up. After two hours, you’ll be ready to start coding your own projects!

#8: API Test Automation with Postman

The #8 spot belongs to API Test Automation with Postman by Beth Marshall. In recent years, Postman has become the go-to tool for building and testing APIs. You could almost think of it as an IDE for APIs. Many test teams use Postman to automate their API test suites.

Beth walks through everything you need to know about automating API tests with Postman in this course. She covers basic features, mocks, monitors, workspaces, and more. Definitely take this course if you want to take your API testing skills to the next level!

#7: Introduction to Cypress

Lucky #7 is Introduction to Cypress by Gil Tayar. Cypress is one of the most popular web testing frameworks these days, even rivaling Selenium WebDriver. With its concise syntax, rich debugging features, and JavaScript-native approach, it’s become the darling end-to-end test framework for frontend developers.

It’s no surprise that Gil’s Cypress course would be in the top ten. In this course, Gil teaches how to set up and run tests in Cypress from scratch. He covers both the Cypress app and the CLI, and he even covers how to do visual testing with Cypress.

#6: Exploring Service APIs through Test Automation

The sixth most popular TAU course is Exploring Service APIs through Test Automation by Amber Race. API testing is just as important as UI testing, and this course is a great way to start learning what it’s all about. In fact, this is a great course to take before API Test Automation with Postman.

This course was actually the second course we launched on TAU. It’s almost as old as TAU itself! In it, Amber shows how to explore APIs first and then test them using the POISED strategy.

#5: IntelliJ for Test Automation Engineers

Coming in at #5 is IntelliJ for Test Automation Engineers by Corina Pip. Java is one of the most popular languages for test automation, and IntelliJ is arguably the best and most popular Java IDE on the market today. Whether you build frontend apps, backend services, or test automation, you need proper development tools to get the job done.

Corina is a Java pro. In this course, she teaches how to maximize the value you get out of IntelliJ – and specifically for test automation. She walks through all those complicated menus and options you may have ignored otherwise to help you become a highly efficient engineer.

#4: Java Programming

Our list is winding down! At #4, we have Java Programming by Angie Jones. For the third time, a Java-based course appears on this list. That’s no surprise, as we’ve said before that Java remains a dominant programming language for test automation.

Like the Python Programming course at spot #9, Angie’s course is a programming course: it teaches the fundamentals of the Java language. Angie covers everything from “Hello World” to exceptions, polymorphism, and the Collections Framework. Clocking in at just under six hours, this is also one of the most comprehensive courses in the TAU catalog. Angie is also an official Java Champion, so you know this course is top-notch.

#3: Introduction to JavaScript

It’s time for the top three! The bronze medal goes to Introduction to JavaScript by Mark Thompson. JavaScript is the language of the Web, so it should be no surprise that it is also a top language for test automation. Popular test frameworks like Cypress, Playwright, and Jest all use JavaScript.

This is the third programming course TAU offers, and also the top one in this ranking! In this course, Mark provides a very accessible onramp to start programming in JavaScript. He covers the rock-solid basics: variables, conditionals, loops, functions, and classes. These concepts apply to all other programming languages, too, so it’s a great course for anyone who is new to coding.

#2: Web Element Locator Strategies

I’m partial to the course in second place – Web Element Locator Strategies by me, Andrew Knight! This was the first course I developed for TAU, long before I ever joined Applitools.

In whatever test framework or language you use for UI-based test automation, you need to use locators to find elements on the page. Locators can use IDs, CSS selectors, or XPaths to uniquely identify elements. This course teaches all the tips and tricks to write locators for any page, including the tricky stuff!

#1: Setting a Foundation for Successful Test Automation

It should come as no surprise that the #1 course on TAU in terms of course completions is Setting a Foundation for Successful Test Automation by Angie Jones. This course was the very first course published to TAU, and it is the first course in almost all the Learning Paths.

Before starting any test automation project, you must set clear goals with a robust strategy that meets your business objectives. Testing strategies must be comprehensive – they include culture, tooling, scaling, and longevity. While test tools and frameworks will come and go, common-sense planning will always be needed. Angie’s course is a timeless classic for teams striving for success with test automation.

What can we learn from these trends?

What other courses are popular?

The post The Top 10 Test Automation University Courses appeared first on Automated Visual Testing | Applitools.

Learn about the latest updates in the Appium Java Client 8.0.0 release, and how it affects your Appium mobile testing today. Sai Krishna and Srinivasan are members of the Appium team.

The Selenium team released Selenium 4 in September 2021 to much anticipation with tons of great features. Since then, the Appium team has been working on upgrading the Selenium dependency in all the clients of Appium to provide a seamless experience. Most of the Appium clients are updated and now in early beta, but with a lot of breaking changes. Let’s go through the changes introduced in Appium Java client 8.0.0 beta2 and what you would need to do.

WebDriver Protocol Compliance

With Appium 2.0, we on the Appium team are making sure we are strictly W3C compliant. Since the Java client supports Selenium 4.1.1, it’s strictly W3C compliant too. Methods that are non-w3c complaint are removed and the details can be seen here.

Driver-Specific Options Classes

It is now recommended to use driver-specific W3C option classes instead of Desired Capabilities. Below is the list of driver-specific classes.

• XCUITestOptions to create an XCUITestDriver instance 
• UiAutomator2Options to create a UIAutomator2Driver instance
• EspressoOptions to create an EspressoDriver instance
• WindowsOptions to create a WindowsDriver instance
• Mac2Options to create a Mac2Driver instance
• GeckoOptions to create a GeckoDriver instance
• SafariOptions to create a SafariDriver instance

See the below example for changes required:

// From Appium Java Client version 8.0.0 Beta
UiAutomator2Options options = new UiAutomator2Options()
       .setDeviceName("Android Emulator")
       .setApp(System.getProperty("user.dir") + "/VodQA.apk")
       .eventTimings();
driver = new AndroidDriver(service.getUrl(), options);

// Older approach: Java Client version 7.X.X
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(MobileCapabilityType.DEVICE_NAME, "Android Emulator");
capabilities.setCapability(MobileCapabilityType.NEW_COMMAND_TIMEOUT, 700000);

Element Lookups

• Like Selenium, Appium also removed all its findBy methods.
• Since Appium supports both mobile and desktop platform applications, we replaced MobileBy with AppiumBy and introduced camelCase naming conventions. For example,  ​​MobileBy.AccessibilityId is now AppiumBy.accessibilityId. MobileBy is now deprecated.
• windowsAutomation locator strategy is deprecated.

Starting Appium Server Programmatically

• Java Client uses AppiumServiceBuilder to start a node server programmatically. With the updated version, it works slightly differently with different Appium server versions.
• The default URL for the server has been changed at the server-side and it does not contain the /wd/hub by default.
• Java Client starting from v8.0.0 has made required changes to align to Appium V2.
• If you still would like to use Java Client v8 with Appium 1.2.X then consider providing the –base-path explicitly while building the AppiumServiceBuilder.

// From Appium 2.0 Beta
AppiumDriverLocalService service;
service = new AppiumServiceBuilder()
       .withIPAddress("127.0.0.1")
       .usingPort(4723)
       .build();
service.start();

// Older approach: Appium 1.22.X
AppiumDriverLocalService service;
service = new AppiumServiceBuilder()
       .withIPAddress("127.0.0.1")
       .withArgument(GeneralServerFlag.BASEPATH, "/wd/hub")
       .usingPort(4723)
       .build();
service.start();

Changes in Driver Classes

• SelendroidDriver class is removed
• Both GeckoDriver and SafariDriver are newly introduced drivers
  • GeckoDriver is an officially supported Appium driver to automate Mobile browsers and web views based on the gecko engine
  • SafariDriver is also an official driver for automating Safari on macOS and iOS

Changes in Element Classes

• MobileElement classes including AndroidElement and iOSElement classes are removed. It is recommended to use WebElement instead.
• replaceValue method is now called replaceElementvalue in AndroidDriver class
• setValue method is removed in favor of the existing sendKeys method.

Touch Actions

The TouchActions and MultiTouchActions classes for automating gestures from your client code have been deprecated. Support for these actions will be removed from future Appium versions. It is recommended to use W3C Actions instead or the corresponding extension methods for the driver (if available). 

Point source = slider.getLocation();
PointerInput finger = new PointerInput(PointerInput.Kind.TOUCH, "finger");
Sequence sequence = new Sequence(finger, 1);
sequence.addAction(finger.createPointerMove(ofMillis(0),
       PointerInput.Origin.viewport(), source.x, source.y));
sequence.addAction(finger.createPointerDown(PointerInput.MouseButton.MIDDLE.asArg()));
sequence.addAction(new Pause(finger, ofMillis(600)));
sequence.addAction(finger.createPointerMove(ofMillis(600),
       PointerInput.Origin.viewport(), source.x + 400, source.y));
sequence.addAction(finger.createPointerUp(PointerInput.MouseButton.MIDDLE.asArg()));
driver.perform(singletonList(sequence));

Refer to the links below to know more about how to work with gestures:

• https://www.youtube.com/watch?v=oAJ7jwMNFVU
• https://appiumpro.com/editions/30-ios-specific-touch-action-methods
• https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/android/android-mobile-gestures.md
• https://github.com/appium/appium/blob/master/docs/en/writing-running-appium/ios/ios-xctest-mobile-gestures.md
• https://appiumpro.com/editions/29-automating-complex-gestures-with-the-w3c-actions-api for more details on how to properly apply W3C Actions to your automation context.

App Management

AppiumDriver methods like resetApp, closeApp and launchApp have been deprecated as they are going to be removed from the future Appium versions. Alternatively, the suggested approach is to use removeApp, installApp, and activateApp methods available here. The non-standard way for App Management in iOS is using the mobile: methods to remove the app then install the new application and launch it using the methods here. For Android UIAutomator2 backend, the non-standard mobile: methods like clearApp deletes all data associated with a package, and installMultipleApks allows users to install multiple applications at the same time.

Appium Event Listeners

The current event firing mechanism that Appium Java Client uses has been deprecated in favor of the one that Selenium 4 provides natively. Read The-event_firing.md for more details on how to use it.

Summary

There are a lot of great new changes in this latest Appium Java Client update. You can check out a detailed migration guide here in Appium’s Java Client for reference. Do let us know where there’s been a problem, and thank you. We only have a chance to improve when we know that there’s something that needs work!

The post What’s New in Appium Java Client 8.0.0 appeared first on Automated Visual Testing | Applitools.

REST has been the de facto standard for APIs for a while now, replacing the relatively cumbersome and XML-only SOAP-based APIs with an architecture that is more flexible, lightweight and easier to develop. However, REST is not without limits of its own. With pure REST, if you need to gather and combine data from multiple endpoints or entities, the API consumer typically needs to invoke multiple endpoints, collect the data from each of these endpoints and combine and filter that data into something useful.

Enter GraphQL. GraphQL tries to solve this problem by exposing all data through a single endpoint and allow the API consumer to write queries that retrieve the exact data required at any given moment, all with a single API call. The API consumer composes the query that should retrieve the data required, POSTs that query to the GraphQL API provider, which, if all goes well, then returns the requested data as a JSON document in the response.

Of course, when you’re developing GraphQL APIs, at some points you will want to write tests for them, too (right?). Developers and test automation engineers working with REST APIs and Java will probably have heard of or even worked with REST Assured, but can we use that for testing GraphQL APIs, too? Let’s find out.

Before we start to write some tests, we’ll need a GraphQL API to write tests for first. In this article, we’ll use the SpaceX GraphQL API, which exposes data about the SpaceX company and their space missions, launches and more. The API can be found at https://api.spacex.land/graphql/ and comes with a GraphiQL explorer allowing you to play around with the API and retrieve some SpaceX data yourself.

Let’s start with a basic example: retrieving some data about the SpaceX company itself, more precisely the company name, as well as the name of its CEO and COO. The GraphQL query required to retrieve this data looks like this:

To submit this query to the SpaceX GraphQL endpoint, we’ll need to create a JSON payload with a single element ‘query’ with our query as its value. I prefer to let REST Assured take care of creating the JSON payload instead of constructing it myself, so I’ve created a simple POJO that represents a GraphQL query, which can be serialized to JSON by REST Assured:

This POJO has two properties, ‘query’ and ‘variables’, and I’ve used the Lombok, and more specifically its @Data annotation, to create getters and setters for both properties. The ‘query’ property will hold our GraphQL query. We’ll talk about the ‘variables’ in a bit, let’s first see how we can use this class to send our first GraphQL query.

Now that we have set up a POJO that represents our payload, all we need to do in our test is create a new instance of it, fill the ‘query’ property with our GraphQL query and POST that to the SpaceX GraphQL endpoint. Since the response of a GraphQL API is plain JSON, we can assert on data returned by the API in the exact same manner with REST Assured as with ‘regular’ REST APIs. You can learn more about that in this free Test Automation University course: https://testautomationu.applitools.com/automating-your-api-tests-with-rest-assured.

Assuming we want to check that the API returns an HTTP status code 200 and that the data reflects that Elon Musk is the CEO of SpaceX, this is what a first GraphQL API test with REST Assured looks like:

Excellent! We have successfully written and run our first GraphQL API test with REST Assured. However, the typical query you will want to execute is probably not so static. GraphQL queries support variables, so let’s take a look at how we can incorporate those in our tests, too.

Say that for our next test, we want to query the launches performed by SpaceX, and retrieve the names of the missions these were associated with, but limit the number of results to 10. The GraphQL query to do that looks like this:

To make the actual limit a variable (maybe we’re only interested in 5 results another time we run this query), we’ll have to add a variable to the query:

and pass the value of the ‘limit’ variable separately:

This is where the ‘variables’ property in the GraphQLQuery class we created earlier comes into play. If we want to execute this parameterized GraphQL query with REST Assured, we’ll need to pass both the (parameterized) query itself, as well as its variables to the SpaceX API. We can pass the query itself as a string, just like we did in the first example. For the parameters, we can choose between two different approaches. We can:

1. create a new POJO that holds the value for the ‘$limit’ variable, just like we did for the query itself, or we can
2. dynamically add variables for our query using a JSONObject

In the first approach, our POJO for the $limit variable may look like this:

We can then pass that as part of our query using REST Assured like this, and assert on the mission name of the first launch like this:

If you’d prefer to use a JSONObject for the query variables instead, this is what that might look like:

Don’t forget to use .toString() on the JSONObject holding the variables before adding them to your query, otherwise the serialization mechanism has no idea how to create a JSON string from the JSONObject and no variables will be passed!

So, which approach is better? In my opinion, as with so many different things in test automation (and life in general): it depends. If you prefer the flexibility of adding variable names and their values ‘on the go’ in your tests, I’d recommend using a JSONObject and simply putting in there what you need. If you want to be a little more strict, and you don’t have a lot of different sets of variables to deal with, creating POJOs for them might be a good idea. As you’ve seen, both will work fine with REST Assured.

Finally, let’s take a look at a different type of query. So far, we have only retrieved data from the SpaceX GraphQL API, but it’s also possible to send data to it, for example to create your own user. Instead of a ‘query’ operation, you’ll need to perform a ‘mutation’ operation to do that:

This mutation inserts a new user, represented by an ‘id’ (a UUID) and containing a name (string) and a rocket (also a string). Because a user is probably an entity that is used more often in the test code (as well as in the application code, probably), I’ve chosen to use a POJO to create a user entity:

We can then use this POJO to create and send a new user to the SpaceX GraphQL API in our test, and check that the data returned in the response is the same as the data we have sent:

As you have seen in this article, with a couple of small steps, it is possible to use REST Assured not only to write tests for pure REST APIs, but also for GraphQL-based APIs. All code samples shown in this article can be found on GitHub for you to run and play around with.

The post Writing Tests for GraphQL APIs using REST Assured appeared first on Automated Visual Testing | Applitools.

What does it take to make a difference in the lives of 75,000 people?

Applitools has reached 75,000 students enrolled in Test Automation University, a global online platform led by Angie Jones that provides free courses on things test automation. Today, more engineers understand how to create, manage, and maintain automated tests.

What Engineers Have Learned on TAU

Engineers have learned how to automate UI, mobile, and API tests. They have learned to write tests in specific languages, including Java, JavaScript, Python, Ruby, and C#. They have applied tests through a range of frameworks including Selenium, Cypress, WebdriverIO, TestCafe, Appium, and Jest.

75,000 engineers would exceed the size of some 19,000 cities and towns in the United States. They work at large, established companies and growing startups. They work on every continent with the possible exception of Antarctica.

What makes Test Automation University possible? Contributors, who create all the coursework.

Thank You, Instructors

As of this writing, Test Automation University consists of 54 courses taught by 39 different instructors. Each instructor has contributed knowledge and expertise. You can find the list of authors on the Test Automation University home page.

Here are the instructors of the most recently added courses to TAU.

Author	Course	Details	Chapters
Corina Pip	JUnit 5	Learn to execute and verify your automated tests with JUnit 5	17
Matt Chiang	WinAppDriver	Learn how to automate Windows desktop testing with WinAppDriver	10
Marie Drake	Test Automation for Accessibility	Learn the fundamentals of automated accessibility testing	8
Lewis Prescott	API Testing In JavaScript	Learn how to mock and test APIs in JavaScript	5
Andrew Knight	Introduction to pytest	Learn how to automate tests using pytest	10
Moataz Nabil	E2E Web Testing with TestCafe	Learn how to automate end-to-end testing with TestCafe	15
Aparna Gopalakrishnan	Continuous Integration with Jenkins	Learn how to use Jenkins for Continuous Integration	5
Moataz Nabil	Android Test Automation with Espresso	Learn how to automate Android tests with Espresso	11
Mark Thompson	Introduction to JavaScript	Learn how to program in JavaScript	6
Dmitri Harding	Introduction to NightwatchJS	Learn to automate web UI tests with NightwatchJS	8
Rafaela Azevedo	Contract Tests with Pact	Learn how to implement contract tests using Pact	8
Simon Berner	Source Control for Test Automation with Git	Learn the basics of source control using Git	8
Paul Merrill	Robot Framework	Learn to use Robot Framework for robotic process automation (RPA)	7
Brendan Connolly	Introduction to Nunit	Learn to execute and verify your auotmated tests with nUnit	8
Gaurav Singh	Automated Visual Testing with Python	Learn how to automate visual testing in Python with Applitools	11

Thank You, Students

As engineers and thinkers, the students continue to expand their knowledge through TAU coursework.

Each course contains quizzes of several questions per chapter. Each student who completes a course gets credit for questions answered correctly. Students who have completed the most courses and answered the most questions successfully make up the TAU 100.

Some of the students who lead on the TAU 100 include:

Student		Credits	Rank
	Osanda Nimalarathna Founder @MaxSoft Ambalangoda Sri Lanka	44,300	Griffin
	Patrick Döring Sr. QA Engineer @Pro7 Munich Germany	44,300	Griffin
	Darshit Shah Sr. QA Engineer @N/A Ahmedabad India	40,250	Griffin
	Adha Hrustic QA Engineer @Klika Bosnia and Herzegovina	39,575	Griffin
	Ho Sang Principal Technical Test Engineer @N/A Kuala Lumpur Malaysia	38,325	Griffin
	Gopi Srinivasan Senior SDET Lead @Trimble Inc Chennai India	38,075	Griffin
	Ivo Dimitrov Sr. QA Engineer @IPD Sofia Bulgaria	37,875	Griffin
	Malith Karunaratne Technical Specialist – QE @Pearson Lanka Sri Lanka	36,400	Griffin
	Stéphane Colson Freelancer @Testing IT Lyon France	35,325	Griffin
	Tania Pilichou Sr. QA Engineer @Workable Athens Greece	35,025	Griffin

Join the 75K!

Get inspired by the engineers around the world who are learning new test automation skills through Test Automation University.

Through the courses on TAU, you’ll not only learn how to automate tests, but more importantly, you’ll learn to eliminate redundant tests, add automation into your continuous integration processes, and make your testing an integral part of your build and delivery processes.

Learn a new language. Pick up a new testing framework. Know how to automate tests for each part of your development process – from unit and API tests through user interface, on-device, and end-to-end tests.

No matter what you learn, you will become more valuable to your team and company with your skills on how to improve quality through automation.

The post Test Automation University is now 75,000 students strong appeared first on Automated Visual Testing | Applitools.

Playwright, Microsoft’s new end-to-end browser automation framework, is making quite the splash! I took Playwright for a spin only a few months ago when it was a JavaScript-only framework, and am pleasantly surprised to learn that language support has expanded to my beloved Java as well as Python and C#! ?

I’ve also started to think about how to compare Playwright vs Selenium.

Playwright vs Selenium

With the additional language support, as well as the ability to execute across modern browser engines Chromium, Firefox, and WebKit – this puts Playwright in the same category as Selenium WebDriver as viable testing solutions for all (not just JS) web testers who need cross-browser testing capabilities for complex applications.

Browsing the Playwright Javadocs was helpful, but I like to evaluate frameworks by actually using them to automate realistic scenarios. So in this post, I’ll share the steps of building a testing project with Playwright which includes Page Objects, and I’ll also compare the Playwright steps to their equivalents in Selenium WebDriver.

How to Install Playwright Java

The first step in getting started with Playwright is adding the dependency to your project. You can get the Playwright client from Maven Repository. I created a new pom.xml file and added the playwright dependency.

When it comes to comparing Selenium vs Playwright, it’s worth noting that like Selenium WebDriver, Playwright is a browser automation tool and not necessarily limited to a testing framework. In fact, neither of them provide any assertion methods. Therefore, you’ll need to add an assertion library as well. For this example, I’ll use TestNG.

How to Launch Browser in Playwright

The Playwright interface allows you to create specific types of Browser objects. The options are Chromium which is what Chrome and Edge are built upon, Firefox, and WebKit (the engine Safari is built upon). With this Browser object, a launch() method is available which will start a browser instance.

By default, Playwright launches the browser in headless mode, which means you won’t actually see the tests execute. If you want to see the browser open, you can disable headless mode by passing in a LaunchOption.

In addition to setting the headless mode, LaunchOptions provide several other methods including ones to set environment variables and open Chromium dev tools.

How to Launch a Website in Playwright

Now that we have a browser, we can load the application under test – Automation Bookstore. To do so, we need a Page object – which is similar to the WebDriver objects in Selenium. To create the Page object, we call browser.newPage() on line 8. This represents a single tab within the browser window. With this object, we can then navigate to our URL (line 9).

How to Create a Page Object in Playwright

We have our application loaded in the browser, and now we’d like to use the Page Object Model design pattern to create a Java class that represents the Search page of our application.

In order to interact with the web elements, the Page Object class will need access to the Playwright Page object we created above. Again, this is comparable to how we pass the Selenium WebDriver object to Page Object classes so that they can perform browser interaction methods.

I created that SearchPage class and the constructor that accepts the Page object.

The first method I’ll add to this class is search() which will take in text and enter it into a text field. The method to do this is fill(), which takes a locator and the text you’d like entered into the field. You can see the call on line 11.

Locators

The element locator for the search field is defined as a String. This is true for all locators used in Playwright; unlike Selenium which uses the By class to allow you to express locators as id, name, XPath, css selector, link text, etc. However, Playwright offers a variety of options to express the locators so all of the ones in Selenium are covered, plus extra flexibility for more expressive locators such as conditions of the elements, chaining, and even access to shadow DOM elements!

Wait Strategies

Playwright has auto-waiting for elements that you are directly interacting with. Selenium WebDriver often gets a bad rap for things that are out of its control, for example the need to wait for a desired state of the application. In my particular application, there is a slight delay between the time I enter the search query and the time that the results are filtered. Since the search results element is a different element than the one I used the automation tool to interact with (search field), I have to account for this wait in my code, and it doesn’t matter if I use Selenium, Cypress, or Playwright – this remains true.

To wait within Playwright, there’s a WaitForSelectorOptions class that allows you to specify the state you’re waiting for. The available states are ATTACHED, DETACHED, HIDDEN, and VISIBLE.

So here, after entering text into the field, I need to ask Playwright to wait until there are hidden books attached to the DOM. This is shown on lines 4-5.

To demonstrate the DETACHED selector option, I’ll also make a clear() method. In this method, I clear the text field by sending an empty String, then wait for the hidden books element to be detached from the DOM.

How to Get Elements in Playwright

Playwright provides the querySelector() mehod which returns a single element as an ElementHandle object. This is equivalent to Selenium’s WebElement object obtained by findElement(). To get multiple elements that match the locator, Playwright provides querySelectorAll() and this will return a list of elements: List<ElementHandle>.

This is demonstrated in our Page Object class as we get the number of visible books.

From an ElementHandle, you can take actions (e.g. click, fill, etc) or get information (getAttribute, isEnabled, isChecked, etc). The two methods that are JavaScripty and maybe not obvious at first glance to Java programmers are: innerText() and innerHTML(). The innerText() method is equivalent to Selenium’s getText() method; and there is no equivalent for innerHTML() in Selenium, but it’s used to get the entire HTML contained between an element’s opening and closing tags.

Here I use the innerText() method to get the titles of the visible books on line 4.

The Tests

Now that all of the heavy lifting has been done in the BaseTests and Page Object classes, the tests look exactly like they would if written in Selenium.

There’s Much More

What I’ve demonstrated here is how to use Playwright for some of the most common everyday tasks, however the API boasts a lot more! It can capture screenshots, mock geolocation, emulate mobile device settings, intercept network calls, and much more. When comparing Playwright vs Selenium, Playwright already contains a lot of the features that make Selenium 4 more modern. While Playwright is still very new, it looks extremely promising and I’m definitely keeping my eye on it.

The post Playing with Playwright – Java API and Playwright vs Selenium appeared first on Automated Visual Testing | Applitools.

There is a lot of buzz around Visual Testing these days. You might have read or heard stories about the benefits of visual testing. You might have heard claims like, “more stable code,” “greater coverage,” “faster to code,” and “easier to maintain.” And, you might be wondering, is this a hype of a reality?

So I conducted an experiment to see how true this really is.

I used the instructions from this recently concluded hackathon to conduct my experiment.

I was blown away by the results of this experiment. Feel free to try out my code, which I published on Github, for yourself.

Visual Assertions – my learnings

Before I share the details of this experiment, here are the key takeaways I had from this exercise:

1. Functional Automation is limiting. You only simulate known user behaviors, in predetermined conditions and in the process only validate & verify conditions what you know about. There has to be a more optimized and value generating approach!
2. The Automation framework should have advanced capabilities like soft assertions and good reporting to allow quick RCA.
3. Save time by integrating Applitools’ Visual AI Testing with the Ultrafast Grid to increase your test coverage and scale your test executions.

Let us now look at details of the experiment.

Context: What are we trying to automate?

We need implement the following tests to check the functionality of https://demo.applitools.com/tlcHackathonMasterV1.html

1. Validate details on landing / home page
   This should include checking headers / footers, filters, displayed items
2. Check if Filters are working correctly
3. Check product details for a specific product

For this automation, I chose to use Selenium-Java for automation with Gradle as a build tool.

The code used for this exercise is available here: https://github.com/anandbagmar/visualAssertions

Step 1 – Pure Functional Automation using Selenium-Java

Once I spent time in understanding the functionality of the application, I was quickly able to automate the above mentioned tests.

Here is some data from the same.

Refer to HolidayShoppingWithSeTest.java

Observations

A few interesting observations from this test execution:

1. I added only superficial validations for each test.
   • I only added validations for the number of filters and items in each filter. But I have not added the validations for actual content of the filters.
   • To add actual validations for each aspect of the page will take 8-10x the time taken for my current implementation, and the number of locators and assertions would also probably increase by 4-6x of the current numbers.
   • Definitely does not seem worth the time and effort.

2. The tests would not capture all errors based on the assertions added, as the first assertion failure would cause the test to stop.
3. In order to check everything, instead of hard assertions, the framework would need to implement and support soft assertions
4. The test implementation is heavily dependent on the locators in the page. Any small change in the locators will cause the test to fail. Ex: In the Product Details page, the locator of the Footer is different from that of the home page
5. Scaling: I was limited by how many browsers / devices I could run my tests on. I needed to write additional code to manage browser drivers, and that too only for browsers that I had available on my laptop

Step 2 – Add Applitools Visual Assertions to Functional Automation

When I added Applitools Visual AI to the already created Functional Automation (in Step 1), the data was very interesting.

Refer to HolidayShoppingWithEyesTest.java

Observations

Here are the observations from this test execution:

1. My test implementation got simplified
   • Less lines of code
   • Fewer locators and assertions
   • Test became easier to read and extend

2. Test became more stable
   • Fewer locators and assertions
   • It does not matter if the locators change for elements in the page as long as the user experience / look and feel remains as expected. (Of course, locators on which I need to do actions using Selenium need to be the same)

3. Comprehensive coverage of functionality and user experience
   • My test focuses on specific functional aspects – but with Visual Assertions, I was able to get validation of the functional change from the whole page, automatically

See these below examples of the nature of validations that were reported by Applitools:

Version Check – Test 1:

Filter Check – Test 2:

Product Details – Test 3:

4. Scaling test execution is seamless
   • I needed to run the tests only on any 1 browser which is available on my machine. I chose Chrome
   • With the Applitools Ultrafast Test Cloud, I was able to get results of the functional and visual validations across all supported platforms without any code change, and almost in the same time as a single browser execution.

Lastly, an activity I thoroughly enjoyed in Step 2 was the aspect of deleting code that now became irrelevant because of using Visual Assertions.

Conclusion

To conclude, the experiment made it clear – Visual Assertions are not a hype. The below table shows in summary the differences in the 2 approaches discussed earlier in the post.

Visual Assertions help in the following ways:

• Make your tests more stable
• Lower maintenance as there are less locators to work with
• Increase test coverage – you do not need to add assertions for each and every piece of functionality as part of your automation. With Visual Assertions, you will get the full – functional & user experience validation by 1 call
• Scaling is seamless – with the Ultrafast Test Cloud, you run your test just once, and get validation results across all supported browsers and devices

You can get started with Visual Testing by registering for a free account here. Also, you can take this course from the Test Automation University on “Automated Visual Testing: A Fast Path To Test Automation Success”

The post Visual Assertions – Hype or Reality? appeared first on Automated Visual Testing | Applitools.

As we complete 2020, we want to share our take on the past year. We had a number of achievements in 2020. And, we celebrated a number of milestones.

Any year in review article must include the effects of the pandemic, along with the threats on social justice. We also want to give thanks to our customers for their support.

Achievements: Product Releases in 2020

Ultrafast Grid

Among our achievements in 2020, Applitools launched the production version of Ultrafast Grid and the Ultrafast Test Cloud Platform. With Ultrafast Grid, you can validate your UI across multiple desktop client operating systems, browsers, and viewport sizes using only a single test run. We take care of the validation and image management, and you don’t need to set up and manage that infrastructure.

Ultrafast Grid works so quickly because we assume your application uses a common server response for all your clients. You only need to capture one server response. Ultrafast Grid captures the DOM state each snapshot and compares that snapshot in parallel across every client/operating system/viewport combination you wish to test. A single test run means less server time. Parallel validation means less test time. Ultrafast Grid simultaneously increases your test coverage while reducing both your test run time and infrastructure requirements.

A/B Testing

We introduced a new feature to support A/B testing. As more of our customers use A/B testing to conduct live experiments on customer conversion and retention, Applitools now supports the deployment and visual validation of parallel application versions.

GitHub, Microsoft, and Slack Integrations

Applitools now integrates with Slack, adding to our range of application and collaboration integrations. Applitools can now send alerts to your engineering team members, including highlights of changes and the test runs on which they occurred.

As a company, we also announced integrations with GitHub Actions and Microsoft Visual Studio App Center.  The integrations allow developers to seamlessly add Visual AI-powered testing to every build and pull request (PR), resulting in greater UI version control and improved developer workflows. As we have seen, this integration into the software build workflow provides visual testing at code check-in time. Instead of waiting for end-to-end tests to expose rendering problems and conflicts, developers can use Applitools to validate prior to code merge.

Auto Maintenance and Smart Assist

We also introduced major improvements with Auto Maintenance and Smart Assist. With Smart Assist, we help you deploy your tests to address unique visual test challenges, such as dynamic data and graphical tests. With Auto Maintenance, you can validate an intended visual change in one page of your application and then approve that change on every other page where that change occurs. If you update your logo or your color scheme, you can validate identical changes across your entire application in one click. Smart Assist and Auto Maintenance reduce the time and effort you need to maintain your visual tests – saving hours of effort in your development and release process.

Achievements: Milestones in 2020

Applitools also achieved a number of major milestones in 2020.

1,000,000,000 Page Images Collected

We recorded one billion page images collected across our customer base. Many of our customers now include Applitools validation as part of every CICD check-in and build. You will find out more in our customer insights discussion, below. We celebrated that achievement earlier in 2020.

Test Automation University

We launched Test Automation University (TAU) as a way to help expand test knowledge among practitioners. Among our achievements in 2020, TAU now has over 50 courses to teach test techniques and programming languages. You can take any of these courses free of charge. Whether you are an experienced test programmer or just getting started, you will find a range of courses to match your interests and abilities. We introduced 19 new courses in 2020. We also saw significant numbers of new students using Test Automation University. In early 2020, we announced that we had 35,000 students taking courses. Later in the year we celebrated reaching the 50,000 user milestone. Look forward to another announcement in early 2021.

Hackathons

In 2019, Applitools launched our Visual AI Rockstar Hackathon. Hackathon participants ran a series of test cases comparing legacy locator-based functional testing with Applitools visual validation. In 2020, we shared the results of that Hackathon.  Engineers wrote tests faster, wrote test code that ran more quickly, and wrote tests that required less maintenance over time. We were able to show your achievements in 2020.

Also in 2020, we hosted a cross browser test hackathon.  Participant results demonstrated that Ultrafast Grid sets up more easily than a traditional farm of multiple browsers. The real value of Ultrafast Grid, though, comes with test maintenance as applications update over time. In November, we hosted a hackathon based on a retail shopping application. We look forward to sharing the insights from that hackathon in early 2021.

Future of Testing

Lastly in 2020 Applitools launched the Future of Testing Conference. Applitools gathered engineering luminaries across a range of industries and companies – from brand names like Microsoft and Sony to tech leaders like GoodRX and Everfi. Their stories show how companies continue to deliver quality products more quickly by using the right approaches and the right tools. Applitools has planned more Future of Testing Conferences for 2021.

Achievement: Customer Growth In 2020

Another of our achievements in 2020 involved customers. We want to thank our customers for their commitment to using Applitools in 2020. Not only did we pass the 1,000,000,000 page capture mark, but we also learned about the many exciting ways our customers are using Applitools.

During the COVID-19 coronavirus pandemic, our customers have appreciated how we have worked to ensure that they continued to get full use and value from Applitools. Though our support team worked largely from home during the year, we used tools to ensure that our customers got the support they needed to succeed with Applitools.

We continued to see our existing customers use more and more page checks over time. A number of companies run Applitools to validate code check-in on daily, and even hourly, code builds. Our customers are also using Applitools to validate component libraries they are building and modifying in React, Angular, and Vue.

We also saw a large number of companies experimenting with and adopting Cypress for development validation. Some companies used Cypress in development to complement an existing Selenium test infrastructure. Others were starting their Cypress validation in new areas or on new products.

Our World in Review – 2020

While many issues affected the world in 2020, two dominated the Applitools world.

The first issue, the COVID-19 pandemic, required our team to work from home for much of the year. Dan Levy offered his suggestions on how to work from home more efficiently.  As we continued to work remotely, we saw how the pandemic affected our team and the world around us. At this point, all of us know people who have been infected. Some in our circles have been hospitalized. And, some have died.

As a company, we are fortunate that Applitools has provided its employees with the ability to work from home. As a company, we want to thank the first responders and health care workers who cannot shelter in safety. We thank them for risking their lives to make all of us safe.

And, we also share condolences with those of you who have lost family, friends, and other loved ones in 2020.

The second issue, social justice, has continued to capture the spirit of our company and its employees. For 8 minutes 46 seconds, the world saw one human’s casual indifference while kneeling on another human’s neck. While not the only incident of 2020, the video of George Floyd’s struggle to live affected all of us. How can there be justice if our civil guardians cannot treat all of us equally? If we want a just world, we need to support those who advocate for social justice.

Applitools and its employees support creating a more just world for all. We continue to encourage our employees to support social justice movements for all. They can support Black Lives Matter, or any other organization actively combatting racism and injustice.

We know there are some who sow division for their own gain. As a company, we think we are stronger together.

Next Up – Customer Insights in 2020

In our next blog post, learn more about Applitools customers. We will share some details we learned about Applitools driving our customers’ productivity. We will be sharing more details in 2021 with a series of customer success stories. Before we release those, read our next blog post on customer insights. Learn how your peers and colleagues benefit from using our highly-accurate Visual AI infrastructure.

The post Leading With Visual AI – Applitools Achievements In 2020 appeared first on Automated Visual Testing | Applitools.

I often get questions from those starting new test automation projects querying which programming language they should choose. I never have a cut and dry answer to this because it depends on a few factors such as which language the product features are being developed in, who will be writing the tests and which language are they most comfortable in, and how much support and tooling is available for a given language.

In this post, I’ll share which programming languages are most used for test automation as it gives some insight into industry usage. However, do not take this to mean “best programming language”, as the best language is the one that is best for your context.

The Data

You may be wondering where the data is from. Good question! More than half of the top companies in software, financial services, and healthcare verticals use Applitools for their web and mobile test automation needs. From the millions of tests that run in our cloud every week, I’ve analyzed which languages the tests were written in and aggregated the results at the team level (not the test level).

The Results

1. Java
   Java remains the most popular programming language for test automation. Java held its lead with 43% of our users opting to write their tests in this language. In last year’s review, Java was used by 44% of our customers, so a slight decline but nevertheless this language managed to keep the crown in 2020.
2. JavaScript
   Coming in as the 2nd most popular programming language for test automation is JavaScript with 35% of our users writing their tests in this language. This is a huge increase from last year where only 15% of our users tested in JS! According to StackOverflow, JavaScript is the most popular technology used by professional developers, so I expect to see an increased usage of JS for testing in the years to come.
3. C#
   With Java and JavaScript accounting for 78% of usage combined, there’s not much market share left for the other languages. So, we see quite the jump with the 3rd place language, C#, being used by 8.8% of our users. This is rather interesting because last year’s results showed 13% of our customers using C#, which means almost a third of these users have likely opted for a different language this year.
4. Python
   Right behind C# is Python, with 8% of our customers using Python as their language of choice for test automation. This is exactly the same percentage of usage we saw last year. What’s most surprising about this stat is that Python is gaining popularity year after year with professional developers and has become the fastest-growing major programming language – even edging out Java for the first time this year! Perhaps we’ll eventually see this trend in software testing as well.
5. Ruby
   Only 4.2% of our customers use Ruby for test automation. This is a stunning 40% decrease from Ruby test automation usage last year. StackOverflow shows Ruby’s popularity down to 8.9% with professional developers and it appears Ruby is even less popular in the testing space.

Remember…

While the data here doesn’t necessarily indicate which is the best programming language for test automation, it does highlight which ones are most used for testing amongst the hundreds of companies and open source projects surveyed.

The post 2020’s Most Popular Programming Languages for UI Test Automation appeared first on Automated Visual Testing | Applitools.

A data-centric view into the most popular tools for automated testing

Asking “what’s the best test framework?” is a sure way to get into a heated debate. Everyone has their favorite automation tool and wants to let the world know that it’s the best!

Instead of giving you yet another opinion on the matter, I am going to provide the statistics, because after all, numbers don’t lie.

What our data is saying

Many companies are using Applitools for the functional and visual testing of their web and mobile applications. With millions of tests running in our cloud every week, we’ve analyzed how widely used the most-talked about automation testing tools really are.

Most popular web test automation framework

Selenium WebDriver

There’s been a lot of discussion about the future of Selenium WebDriver when there are so many new kids on the block in terms of framework choices. Is Selenium WebDriver losing momentum?

According to our data, only slightly. However, make no mistake, Selenium Webdriver is by far, still the most widely used web automation tool with 75% of our users executing their tests with this framework.

Besides being the tried and true web automation framework for over a decade, Selenium WebDriver also supports all major browsers (Chrome, IE, Edge, Firefox, and Safari) and boasts support for Java, Python, C#, Ruby, JavaScript, and Kotlin whereas most other web automation tools only provide support for a limited number of browsers and a single programming language.

Given this, it’s expected that Selenium WebDriver will generally have more users, as it has a much larger target audience.

So to be fair, let’s look at the JavaScript usage, since most of the newer web automation frameworks are for that demographic.

JavaScript Automation Testing

If we only look at the JavaScript world, we see a much different story.

Cypress

Leading the pack is relative newcomer, Cypress, with just shy of half (47%) of our JavaScript users choosing this tool for their web testing. This is quite impressive, considering up until a couple of months ago (February 2020), Cypress’ browser support was limited to Chrome only.

WebdriverIO

WebdriverIO, which is an independent JavaScript implementation of the WebDriver API, comes in second place with 25% of our JS users choosing this framework for their test automation needs.

WebDriverJS

Selenium WebDriver’s official JavaScript implementation, WebdriverJS, is close behind with 23% of JavaScript tests being executed with this tool.

TestCafe

After the Big Three of JavaScript web automation tools, comes newcomer, TestCafe, with 5%.

Honorable Mention

Playwright

The youngest web automation framework in the race is Playwright by Microsoft. Playwright has only been available for a few months (May 2020) but we’re beginning to see usage. This is definitely one to keep your eye on.

Puppeteer

While Puppeteer is not on our list, we do realize that teams use this framework for their automation needs. However, from our observation, many are utilizing this for automating browsers (i.e. generating PDFs, crawling sites, etc) as opposed to test automation.

So, what’s the most popular test automation tool for web testing?

1. Selenium WebDriver

   Although technically a web automation framework (not a testing framework), Selenium WebDriver still reigns supreme. With support for a plurality of browsers and languages, WebDriver is by far the most popular choice overall and has seen a 9% increase this year.

2. Cypress

   Cypress rules the JavaScript web test automation world, and comes in second in overall web automation tools. Yet, it is a far way off from attracting the wide audience that Selenium WebDriver does as a whole. However, its growth is quite impressive, with an 41% increase so far this year.

3. WebDriverIO
   WebDriverIO is the third most popular web automation tool amongst our users, and saw an increase of 10% this year.

To hear more about the state of web automation tools, their similarities and differences, join the Future of Testing 2020 virtual event on September 17, where a panel of experts from Selenium, Cypress, Playwright, and Applitools will discuss the future of test frameworks.

For More Information

Applitools works with all popular test automation frameworks. We allow you to add accurate visual validation to your existing automated tests with ease. Check out some of these Applitools tutorials.

• Selenium Java Tutorial
• Selenium JavaScript Tutorial
• Selenium Python Tutorial
• Cypress Tutorial
• WebdriverIO 5 Tutorial
• Testcafé Tutorial
• Sign up for a free Applitools account
• Request an Applitools demo

The post 2020 Most Popular Front End Automation Testing Tools appeared first on Automated Visual Testing | Applitools.

Visual UI testing is more than just testing your app on Desktop browsers and Mobile emulators. In fact, you can do more with Visual UI testing to run your tests over physical mobile devices.

Visual UI testing compares the visually-rendered output of an application against itself in older iterations. Users call this type of test version checking. Some users apply visual testing for cross-browser tests. They run the same software version across different target devices/operating systems/browsers/viewports. For either purpose, we need a testing solution that has high accuracy, speed, and works with a range of browsers and devices. For these reasons, we chose Applitools.

Running your Visual UI testing across physical devices means having to set up your own local environment to run the tests. Imagine the number of devices, screen resolutions, operating systems, and computers you’d need! It would be frustratingly boring, expensive, and extremely time-consuming.

This is where Amazon’s AWS Device Farm comes into play. This powerful service can build a testing environment. It uses physical mobile devices to run your tests! All you do is upload your tests to Amazon, specify the devices you want, and it will take it from there!

In one of my recent articles, How Visual UI Testing can speed up DevOps flow I showed how you can configure a CD/CI service to run your Visual UI tests. The end result would be the same, whether you are running your tests locally, or via such services. Once the tests run, you can always check the results over the Applitools Test Manager Dashboard.

In this article, I will show you how you can run your Visual UI tests, whether you’ve written them for your mobile or web app, on real physical mobile devices in the cloud. For this, I will be employing Applitools, Appium, and AWS Device Farm.

AWS Device Farm for mobile visual testing

AWS Device Farm is a mobile app testing platform that helps developers automatically test their apps on hundreds of real devices in minutes.

When it comes to testing your app over mobile devices, the choices are numerous. Amazon helps to build a “Device Farm” on behalf of the developers and testers, hence the name.

Here are some of the major advantages and features for using this service:

1. Cross-platform. Android and iOS platforms (Native, Hybrid, and Web) are all supported. This includes native apps built with:
   • Java/Kotlin for Android.
   • Swift for iOS. 
   • PhoneGap.
   • Xamarin.
   • Unity.
   • And web apps built for mobile browsers.

2. Scale. AWS Device Farm supports hundreds of unique physical devices, categorized by make, model, and operating system. You may also choose to run your tests across multiple instances of the same device. All these devices are available for you with a few mouse clicks!

3. Safety and Security. AWS Device Farm provides full hardware and software isolation. The devices are physically isolated from one another! They cannot feed each other, so there’s no way for one phone to take a photo, video, or audio recording of a device sitting next to it. In addition, the devices are not visible to each other from a wireless or network point of view. Bluetooth and Wi-Fi traffic is not shared. On the software side, the devices are dynamically tethered to a host machine. When you run your test on a host machine, it has the device plugged into it over USB. That very host machine, that executes your code, is brought up on the fly, runs your code, and is then torn down. It’s never reused between customers.

4. Reporting. The test results, together with any screenshots, videos, logs and performance logs, are all logged, and saved in the cloud. The AWS Device Farm offers a rich Dashboard to allow you to browse any of these logs in order to debug your test runs.

AWS Device Farm supports a number of test runners. This includes Appium Java JUnit, Appium Python, Appium Ruby, and Appium Java TestNG. Back in January 2019, Amazon announced support for the Appium Node.js test runner. This means you can build your tests with Selenium Webdriver, for instance, and have it run on top of AWS Device Farm.

Now that you have an idea about AWS Device Farm, let’s move on, and discover the Appium automation testing framework.

Selenium Webdriver for browser app automation

Selenium WebDriver is a browser automation framework that allows a developer to write commands, and send them to the browser. It offers a set of clients with a variety of programming languages (Java, JavaScript, Ruby, Python, PHP and others). 

Figure 1 below shows the Selenium WebDriver architecture:

Figure 1: Selenium WebDriver Architecture

Selenium WebDriver architecture consists of:

• Selenium Language Bindings. These bindings are Selenium Client Libraries, offered in multiple programming languages, that developers use to send control commands to the browser. For instance, a developer can open a browser instance, and query for an element in DOM, among other tasks.

• JSON Wire Protocol. This is a REST API Protocol (JSONWP) that all WebDriver Server implementations adhere to and understand. Each of the queries and commands the developers write using the Selenium Client Library are converted to HTTP Requests, with the query or command, as payload in the JavaScript Object Notation (JSON) format, and is sent to the WebDriver Server.

• Browser Drivers. These are WebDriver Server implementations for a variety of browsers. A WebDriver Server implementation is nothing but an HTTP Server that receives requests from the Selenium Client Library using the JSON Wire Protocol format. It then analyzes the HTTP Request and prepares a browser-specific command to execute against the browser. If the request is a GET request, the browser-driver should return a response. Otherwise, a POST request is a one-way request to execute an action only without any response.

• Browsers. These are the browsers that have a corresponding WebDriver Server implementation. The WebDriver Server implementations communicate with the Browsers over HTTP via the DevTools Protocol of each browser.

Selenium 4 is obseleting the JSONWP in favor of the new W3C WebDriver standard.

Here’s a quick tutorial on using and learning Selenium WebDriver.

With that brief overview of Selenium WebDriver, let’s move on and explore Appium.

Appium for mobile app automation

Appium is an open-source tool to automate Mobile app testing. It’s a cross-platform that supports both OS (Android and iOS) test scripts. It is tested on simulators (iOS), emulators (Android) and real devices (iOS, Android).

It’s an HTTP Server written in Node.js that creates and handles WebDriver sessions. When you install Appium, you are actually installing the Appium Server. It follows the same approach as the Selenium WebDriver, which receives HTTP requests from the Client Libraries in JSON format with the help of JSONWP. It then handles those HTTP Requests in different ways. That’s why you can make use of Selenium WebDriver language bindings, client libraries and infrastructure to connect to the Appium Server. 

Instead of connecting a Selenium WebDriver to a specific browser WebDriver, you will be connecting it to the Appium Server. Appium uses an extension of the JSONWP called the Mobile JSON Wire Protocol (MJSONWP) to support the automation of testing for native and hybrid mobile apps. 

It supports the same Selenium WebDriver clients with a variety of multiple programming languages such as Java, JavaScript, Ruby, Python, PHP and others. 

Being a Node.js HTTP Server, it works in a client-server architecture. Figure 2 below depicts the Appium Client-Server Architecture model:

Figure 2: Appium Server Architecture

Appium architecture consists of:

• Appium Client. The Client Library that communicates with the Appium Server via a session by sending the commands and queries over HTTP in JSON format. These requests are eventually executed against the specific Mobile device (emulator or real device). 

• Mobile JSONWP. This is the communication Protocol that both Appium Clients and Appium Server understand and use to pass along the commands and queries to be executed. The Appium Server differentiates between iOS and Android Request using the Desired Capabilities argument. It’s a collection of keys and values encoded in a JSON object, sent by Appium clients to the server when a new automation session is requested. They contain all the information about the device to be used to run the tests against. Here’s a detailed tutorial on all possible desired capabilities to use with Appium. Desired Capabilities for Appium

• Appium Server. It’s a Node.js HTTP server that receives requests from the client libraries, translates them into meaningful commands and passes them over to the specific UI Automator. 

• UI Automators. They are used to execute commands against the Mobile device/emulator/simulator. Examples are: UI Automator2 and XCUITest (iOS)

The results of the test session are then communicated back to the Appium Server, and back to the Client in the form of logs, using the Mobile JSONWP.

Now that you are well equipped with knowledge for Selenium WebDriver and Appium, let’s go to the demo section of this article.

Demo

In this section, we will write a Visual UI test script to test a Web page. We will run the tests over an Android device both locally and on AWS Device Farm. 

I will be using both Selenium WebDeriver and Appium to write the test script.

Prerequisites

Before you can start writing and running the test script, you have to make sure you have the following components installed and ready to be used on your computer:

• Java installed
• JAVA_HOME environment variable is set to the Java SDK path
• Node.js installed

Assuming you are working on a MacOS computer, you can verify the above installations by running the following bash commands:

echo $JAVA_HOME // this should print the Java SDK path
node -v // this should print the version of Node.js installed
npm -v // this should print the version of the Node Package Manager installed

Component Installations

For this demo we need to install Appium Server, Android Studio / SDK and finally make sure to have a few environment variables properly set.

Let’s start by installing Appium Server. Run the following command to install Appium Server locally on your computer.

npm install -g appium

The command installs the Appium NPM package globally on your computer. To verify the installation, run the command:

appium -v // this should print the Appium version

Now let’s install Android Studio / SDK so that you can run the test script on an emulator or real device. You could install the Android SDK only but then you have to do additional advanced steps to properly configure the Android environment on your computer. I highly recommend installing the Android Studio as it makes your life easier.

Download the Android Studio executable. Follow the steps below to install locally on your computer:

Notice the location where the Android SDK was installed. It’s /Users/{User Account}/Library/Android/sdk.

Wait until the download and installation is complete. That’s all!

Because I want to run the test script locally over an Android emulator, let’s add one.

Open the Android Studio app:

Click the Configure icon:

Select the AVD Manager menu item.

Click the + Create Virtual Device button.

Locate and click the Pixel XL device then hit Next.

Locate the Q release and click the Download link. 

Read and accept the Terms and Conditions then hit Next.

The Android 10, also known as Q release, starts downloading.

Once the installation is complete, click the Next button to continue setting up an Android device emulator.

The installation is complete. Grab the AVD Name as you will use it later on in the test script, and hit Finish.

Finally, we need to make sure the following environment variables are set on your computer. Open the ~/.bash_profile file, and add the following environment variables:

APPLITOOLS_API_KEY={Get the Applitools API Key from Applitools Test Manager}
export APPLITOOLS_API_KEY
ANDROID_HOME=/Users/{Use your account name here}/Library/Android/sdk
export ANDROID_HOME
ANDROID_HOME_TOOLS=$ANDROID_HOME/tools
export ANDROID_HOME_TOOLS
ANDROID_HOME_TOOLS_BIN=$ANDROID_HOME_TOOLS/bin
export ANDROID_HOME_TOOLS_BIN
ANDROID_HOME_PLATFORM=$ANDROID_HOME/platform-tools
export ANDROID_HOME_PLATFORM
APPIUM_ENV="Local"
export APPIUM_ENV

Finally, add the above environment variables to the $PATH as follows:

export $PATH=$PATH:$ANDROID_HOME:$ANDROID_HOME_TOOLS:$ANDROID_HOME_TOOLS_BIN:$ANDROID_HOME_PLATFORM

One last major component that you need to download, and have on your computer, is the ChromeDriver. Navigate to the Appium ChromeDriver website, and download the latest workable ChromeDriver release for Appium. Once downloaded, make sure to move the file to the location: /usr/local/bin/chromedriver

That’s it for the installations! Let’s move on and explore the Visual UI test script in depth.

Run the Visual UI Test Script locally

You can find the source code demo of this article on this GitHub repo.

Let’s explore the main test script in this repo.

"use strict";

;(async () => {

    const webdriver = require("selenium-webdriver");
    const LOCAL_APPIUM = "https://web.archive.org/web/20221206000829/http://127.0.0.1:4723/wd/hub";

    // Initialize the eyes SDK and set your private API key.
    const { Eyes, Target, FileLogHandler, BatchInfo, StitchMode } = require("@applitools/eyes-selenium");

    const batchInfo = new BatchInfo("AWS Device Farm");
    batchInfo.id = process.env.BATCH_ID
    batchInfo.setSequenceName('AWS Device Farm Batches');
    
    // Initialize the eyes SDK
    let eyes = new Eyes();
    eyes.setApiKey(process.env.APPLITOOLS_API_KEY);
    eyes.setLogHandler(new FileLogHandler(true));
    eyes.setForceFullPageScreenshot(true)
    eyes.setStitchMode(StitchMode.CSS)
    eyes.setHideScrollbars(true)
    eyes.setBatch(batchInfo);

    const capabilities = {
        platformName: "Android",
        deviceName: "Android Emulator",
        automationName: "UiAutomator2",
        browserName: 'Chrome',
        waitforTimeout: 30000,
        commandTimeout: 30000,
    };

    if (process.env.APPIUM_ENV === "Local") {
        capabilities["avd"] = 'Pixel_XL_API_29';
    }
    
    // Open browser.
    let driver = new webdriver
        .Builder()
        .usingServer(LOCAL_APPIUM)
        .withCapabilities(capabilities)
        .build();

    try {
        // Start the test
        await eyes.open(driver, 'Vuejs.org Conferences', 'Appium on Android');

        await driver.get('https://us.vuejs.org/');

        // Visual checkpoint #1.
        await eyes.check('Home Page', Target.window());

        // display title of the page
        await driver.getTitle().then(function (title) {
            console.log("Title: ", title);
        });

        // locate and click the burger button
        await driver.wait(webdriver.until.elementLocated(webdriver.By.tagName('button.navbar__burger')), 2000).click();
        
        // locate and click the hyperlink with href='/#location' inside the second nav element
        await driver.wait(webdriver.until.elementLocated(webdriver.By.xpath("//web.archive.org/web/20221206000829/https://nav[2]/ul/li[3]/a[contains(text(), 'Location')]")), 2000).click();

        const h2 = await driver.wait(webdriver.until.elementLocated(webdriver.By.xpath("(//h2[@class='section-title'])[4]")), 2000);
        console.log("H2 Text: ", await h2.getText());

        // Visual checkpoint #2.
        await eyes.check('Home Loans', Target.window());

        // Close Eyes
        await eyes.close();
    } catch (error) {
        console.log(error);
    } finally {
        // Close the browser.
        await driver.quit();

        // If the test was aborted before eyes.close was called, ends the test as aborted.
        await eyes.abort();
    }
})();

The test script starts by importing the selenium-webdriver NPM package.

It imports a bunch of objects from the @applitools/eyes-selenium NPM package.

It constructs a BatchInfo object used by Applitools API. 

const batchInfo = new BatchInfo("AWS Device Farm");
batchInfo.id = process.env.BATCH_ID
batchInfo.setSequenceName('AWS Device Farm Batches');

It then creates the Eyes object that we will use to interact with the Applitools API.

// Initialize the eyes SDK
let eyes = new Eyes();
eyes.setApiKey(process.env.APPLITOOLS_API_KEY);
eyes.setLogHandler(new FileLogHandler(true));
eyes.setForceFullPageScreenshot(true)
eyes.setStitchMode(StitchMode.CSS)
eyes.setHideScrollbars(true)
eyes.setBatch(batchInfo);

It’s so important to set the Applitools API key at this stage. Otherwise, you won’t be able to run this test. The code above also directs the Applitools API logs to a File located at the root of the project under the name of eyes.log.

Next, we define the device capabilities that we are going to send to Appium.

const capabilities = {
        platformName: "Android",
        deviceName: "Android Emulator",
        automationName: "UiAutomator2",
        browserName: 'Chrome',
        waitforTimeout: 30000,
        commandTimeout: 30000,
    };
    if (process.env.APPIUM_ENV === "Local") {
        capabilities["avd"] = 'Pixel_XL_API_29';
    }

We are using an Android emulator to run our test script over a Chrome browser with the help of the UIAutomator 2 library.

We need to set the avd capability only when running this test script locally. For this property, grab the AVD ID of the Android Device Emulator we set above.

Now, we create and build a new WebDriver object by specifying the Appium Server local URL and the device capabilities as:

const LOCAL_APPIUM = "http://127.0.0.1:4723/wd/hub";
let driver = new webdriver
        .Builder()
        .usingServer(LOCAL_APPIUM)
        .withCapabilities(capabilities)
        .build();

Appium is configured to listen on Port 4723 under the path of /wd/hub.

The rest of the script is usual Applitools business. In brief, the script:

• Opens a new Applitools test session
• Sends a command to navigate the browser to https://us.vuejs.org/
• Grabs the page title and displays it on screen.
• Clicks the Burger Button to expand the menu on a Mobile device.
• Finds the Location section on the page.
• Finally, it prints the H2 text of the Location section.

Notice that the script asserts two Eyes SDK Snapshots. The first captures the home page of the website, while the second captures the Location section.

Finally, some important cleanup is happening to close the WebDriver and Eyes SDK sessions.

Open the package.json file, and locate the two scripts there:

"appium": "appium --chromedriver-executable /usr/local/bin/chromedriver --log ./appium.log",
"test": "node appium.js"

The first runs and starts the Appium Server, and the second to run the test script.

Let’s first run the Appium server by issuing this command:

npm run-script appium

Then, once Appium is running, let’s run the test script by issuing this command:

npm run-script test

Verify Test Results on Applitools Test Manager

Login to the Applitools Test Manager located at: https://applitools.com/users/login

You will see the following test results:

The two snapshots have been recorded!

Run the Visual UI Test Script on AWS Device Farm

Now that the test runs locally, let’s run it on AWS Device Farm. Start by creating a new account on Amazon Web Service website.

Login to your AWS account on this page: https://console.aws.amazon.com/devicefarm

Create a new project by following the steps below:

• Select the Mobile Device Project and name your project Appium.
• Click the Create project button.

• Locate and click the + Create a new run button.

• Select the HTML5 option since we are testing a Web Page on a mobile device. 
• Assign your test run a name. 
• Click the Next step button.

• Select the Appium Node.js test runner
• Upload your tests packaged in a zip file.

Let’s package our app in a zip file in order to upload it at this step.

Switch back to the code editor, open a command window, and run the following:

npm install

This command is essential to make sure all the NPM package dependencies for this app are installed.

npm install -g npm-bundle

The command above installs the npm-bundle NPM package globally on your machine.

Then, run the command to package and bundle your app:

npm-bundle

The command bundles and packages your app files, and folders, including the node_modules folder.

The output of this step creates the file with the .tgz extension.

The final step before uploading is to compress the file by running the command:

zip -r appium-aws.zip *.tgz 

Name the file whatever you wish.

Now you can upload the .zip file to AWS Device Farm.

Once the file uploads, scroll down the page to edit the .yaml file of this test run like so:

• Make sure you insert your Applitools API Key as shown in the diagram.
• Add the node appium.js command to run your test script.
• Click the Save Testspec file.

• You can name it anything you want.

• It’s time to select the devices that you want to run the test script against. I will pick up a customized list of devices. Therefore, click the Create a new device pool button.

• Give this new pool a name.
• Pick up the selected Android Devices. You may select others too.
• Click the Save device pool button.

• Now, you can see the new device pool selected with the devices listed. 
• Click the Next step button.

• Locate and click the Confirm and start run button.

• The test run starts!
• Select the test run listed.

• You can watch the progress on both devices as the test is running. Usually, this step takes a bit of time to run and complete.

• Finally, the results are displayed clearly, and in our case, all the green indicates a pass.

Verify Test Results on Applitools Test Manager

Switch back to the Applitools Test Manager, and verify the results of this second run via AWS Device Farm.

As expected, we get exactly the same results as running the test script locally.

Conclusion

Given the massive integrations that Applitools offers with its rich SDKs, we saw how easily and quickly we can run our Visual UI tests in the cloud using the AWS Device Farm service. This service, and similar services, enrich the Visual regression testing ecosystem, and make perfect sense when performing them.

For More Information

• Request an Applitools Demo
• Sign up for a free Applitool account
• Progressive Web Apps: Building Compelling Apps with the Latest Mobile Web Technology
• Learning Appium Visual Testing on TAU
• Cross-browser Testing With Cypress.io
• How Visual UI Testing can speed up DevOps flow

The post Visual Testing with Applitools, Appium, and Amazon AWS Device Farm appeared first on Automated Visual Testing | Applitools.

Ready For a Challenge?

If you are up for a challenge and enjoy learning new skills, join the world’s best quality engineers between now until June 30th to compete in the industry’s first next generation cross browser, cross device, and cross operating system hackathon. Focused on the use of Visual AI and Ultrafast Grid, this virtual event seeks to educate and upskill developers and test automation engineers all over the world. Test at incredible speeds, deliver higher quality software faster than ever, and earn a chance to win the $5,000 Diamond Prize.

Over 500 Winners. $50,000 in Cash Prizes.

So long as you are among the first 5,000 to qualify, you are eligible to win one of 500 prizes worth over $5,000. That’s at least a 10% chance to win! Since this hackathon is about testing at incredible speeds, the first 500 to submit a qualifying submission also earn a $25 ultrafast submission prize. Even better, you become eligible for one of the 100 cash prizes listed below if our panel of expert judges determines your test suites did the best job of providing efficient coverage and catching all the bugs.

As of June 8, almost 2,000 people have signed up, and we have been receiving initial submissions. If you want to qualify for an ultrafast submission prize, you still have time.

How Does the Hackathon Work?

Software developers, quality engineers, and QA professionals will compete for $50,000 in cash prizes. For those who qualify, you will be challenged to author cross browser and cross device tests against a real-world app using both your preferred legacy cloud testing solution and Applitools Ultrafast Grid powered by Visual AI. Contestants are free to use any major test framework, such as Cypress, Selenium, WebdriverIO, or TestCafe, and do so in their preferred language including Java, Javascript, Python, Ruby, or C#.

Here is what you need to do:

1. Apply here for access. Once you qualify, you will get access to the hackathon application, instructions on how to complete the challenge, and full access to Applitools Visual AI and Ultrafast Grid.
2. Submit when you’re ready. The instructions will guide you. We expect most submissions to take 4 to 6 hours to complete. There is plenty of help to get it done if you need it!
3. Your submission will be judged by a panel of experts. Those submissions that do the best job of catching all the bugs and doing so in the most efficient way possible will win.
4. All submissions are due by June 30th, 2020 at 11.59pm PT. No exceptions!
5. Winners will be announced no later than August 1st, 2020

That’s it! So why wait? Get started today.

The Next Generation of Cross Browser Testing is Ultrafast.

Our hackathons were created to make a point. There is a better way to automate your testing. Browsers do not suffer from the same executional bugs that plagued them five, 10, or 20 years ago. What does create problems, lots of problems, is the rendering of the application across various viewports and screens. This reality means amajor shift in how you need to test, and you will learn and see for yourself what we mean by competing.

In the Ultrafast Cross Browser Testing Hackathon, even more valuable than the prizes you might win, is the learning you will gain from competing. If you take on this challenge, you will learn how next generation cross browser testing works. If you want a quick summary — read this this blog post on next generation cross browser testing. 

We’ve Done This Before. The Visual AI Rockstar Hackathon.

In November 2019, the Visual AI Rockstar Hackathon was a huge success. Almost 3,000 quality engineers participated and the response was overwhelmingly positive. Here is what some of our winners had to say about their experience:

We expect this one to be even bigger, so what’s stopping you?

Take The Challenge

Why participate in the Applitools Cross Browser Testing Hackathon?

First, you will learn new skills. You get hands-on experience seeing how easily you can run tests once and evaluating behavior across the browsers that matter to your customers.

Second, you experience a new way of running application validation. If you have your own multi-browser lab today, or if you use a third-party service that requires multiple tests run on multiple setups in parallel, you can see the difference in running the Applitools Ultrafast grid in comparison. And, if you have not considered running tests across multiple browsers – due to cost or complexity – you can reevaluate your decision.

Finally, you can win prizes and bragging rights as a hackathon winner. To show the world, we will proudly display your name on our website. Your success will demonstrate your engineering acumen to your peers and anyone else that matters to you.

Your opportunity to learn something new and stand out in a crowd awaits. Sign up now.

The post Announcing the $50,000 Ultrafast Cross Browser Hackathon appeared first on Automated Visual Testing | Applitools.

As a quality engineer, what do you think about when you consider the return on investing an hour of your time? Or two hours? What if it could save you up to half the work hours you spent over the past year?

If you spend a lot of time writing or maintaining your end-to-end tests, our data shows that you’ll find lots of savings. What would you do with that time?

I have written previously about our Impact of Visual AI on Test Automation report. And, I have written about the benefits that many of your peers discovered by using Applitools Visual AI. They reduced the number of lines of code they had to write. Their code became more stable and easier to maintain. They caught more bugs with less code. And they learned how to do all this in about one to two hours.

So, what can an hour or two of your time give you?

ROI For A Quality Engineer Hour

Business people like me often talk about return on investment (ROI). If we invest this money today, what does the future payoff look like, and when does it come?

ROI offers a good model for thinking for everyone. Including a decision about how to spend your time to learn a new skill. If you spend an hour learning something now, what is the future benefit to you, and when do you get it?

The ROI of a quality engineer hour might be measured by:

• Increasing the number of test cases you can run
• Increasing the number of bugs you can uncover
• Decreasing the amount of low-value code you write.
• Decreasing your code maintenance effort.

So if you’re going to invest one or two hours into learning a new technology, like, say, Visual AI, you would like to see many hours worth of return on your time.

Visual AI frees you from writing inspection code in your end-to-end tests. Inspection code has the highest chance to contain errors, gets written selectively to provide limited coverage, and still can require a high degree of coding skill.  

You might think that your value as a software engineer comes from your coding skills, and that often devolves into the inane measure of the value of a software engineer by counting lines of code written. Truthfully, not all code has the same value. In fact, code in of itself has zero value. It’s what that code does to help you test more conditions that matters.

High-Value Code And Low-Value Code

Your end-to-end tests contain both high-value code and low-value code.  

The high-value code exercises your application. It sets the test cases. It runs execution variations. The high-value code correlates to source code and UI test coverage.

The low-value code inspects your results. And, if you’re using state-of-the-art coding, you’re inspecting the DOM for output web elements. Some you find with an ID. Others you find with a CSS selector. Some you find with a relative Xpath expression. Sometimes this code can involve incredible complexity. For example, writing all the assertions for reordering a table can involve elegant coding skills. And, yet…

If you review your end-to-end tests, much of your code determines whether the DOM contains the proper elements to let you conclude that the app passed. And, you have become good at triage. You cannot cover the state of every element in the DOM, so you become great at selecting code to inspect that tells you if your test passed or failed. Still, you write a lot of inspection code compared to execution code – and you only inspect a selected subset of the DOM.

But, the time you take writing this low-value code detracts from some of the high-value activity you can add to your end-to-end tests. For instance – you can write the preconditions for each test to ensure that they can be run independently – and, thus, in parallel. You can review the test conditions evaluated by each test to eliminate redundancy and improve automation accuracy.

What about maintenance? We didn’t even include code maintenance efforts you undertake between releases. That is yet more end-to-end coding effort you need – to validate and update existing tests, add new tests, and resolve failures – every time you modify the application. And, yes, some of that code provides new or modified test conditions. As well, some of that code needs to modify your inspection code.

Visual AI Replaces Low-Value Code

When we developed Visual AI, we recognized that a quality engineer makes trade offs. One engineer can only write a finite number of lines of code. An incomplete test has no value. So, every line of code needed to execute a test and validate the results makes a complete test.

We also recognized the limitations of DOM inspection for test results validation. Inspected thoroughly, the DOM represents the elements of the page to be rendered by the browser. And, one can write a detailed DOM inspection – and even account for variations in the DOM structure between releases. However, that depth of inspection involves complexity that rarely pays off for the coder. So, most coders spot-check their DOM – and can miss unexpected changes between releases unless validating the application through manual testing.

Visual AI uses one line of code to capture every visual element on your web page – and the state of the DOM that created the page. Once captured, Visual AI compares your captured version to your baseline. Then, Visual AI highlights for you the real user-noticeable differences between the baseline and the new capture. From there, you choose to accept the changes as expected new features or reject them as bugs. And, you can link those differences directly to the DOM code used to generate those differences.

Since inspection statements make up the bulk of your end-to-end test code, by adding Visual AI, you can eliminate the bulk of your inspection statements – letting you write code faster, making your code more stable, and allowing you more time to work on high-value test automation tasks.

How Long Does It Take To Learn Visual AI?

When we started the Applitools Visual AI Rockstar Hackathon, we directed participants to two courses on Test Automation University (TAU). TAU, offered exclusively by Applitools, offers classes on a range of technologies, including:

• Programming languages (like C and Python)
• Test frameworks (like Selenium and Cypress)
• Mobile testing
• API automation
• And, of course, Visual AI

We pointed participants to one course written by Raja Rao describing how to modernize your functional tests with Visual AI. Raja walked through the different test cases on the Hackathon in about an hour. We also pointed participants to a course by Angie Jones on how to add Visual AI to your test automation.  Each course took approximately an hour.

Hackathon participants got pretty amazing results. After an hour or two of classes, they applied their knowledge of Visual AI and found:

• Average test coverage jumped from 65% for coded inspection to 95% for Visual AI
• Test writing time, on average, dropped from a little over 7 hours to a little over 1 hour.
• The amount of code they wrote dropped significantly

So, for one or two hours of learning, Hackathon participants got faster test writing, more coverage, and less code.

Testing Visual AI Yourself

In the end, testing is a scientific activity. We run a series of tests and make observations – pass or fail.

In this blog, you’re reading a bunch of claims about Visual AI. These are based on data that we shared in our report about the Hackathon results.

What do these claims mean to you?

I recommend that you test these claims yourself. The claim – one or two hours of learning can help you write tests:

• With comprehensive inspection
• Requiring less code
• More easily maintained
• Let you focus your time on high-value test activity.

If true, would that be worth the investment?

It is up to you to find out.

Learn More

• Modern Functional Test Automation Through Visual AI
• Automated Visual Testing: A Fast Path To Test Automation Success

James Lamberti is CMO at Applitools

The post How Can 2 Hours Now Save You 1,000 Hours In The Next Year? appeared first on Automated Visual Testing | Applitools.