On September 9, 1947, Grace Hopper recorded the first computer bug ever in the Harvard Mark II computer’s logbook. The bug in question? Believe it or not, an actual bug – a moth – flew into the relay contacts in the computer and got stuck. Hopper duly taped the moth into the logbook. Then she added the explanation: “First actual case of bug being found.” (This might be the most famous moth in history.)

If only things were this simple today. As software continuously grows in complexity, so does the process of testing and debugging. Nowadays, the lifecycle of a bug in software can be lengthy, costly, and frustrating.

Finding and fixing bugs early on in the application development stage is cheaper and easier than doing so during QA, or worse, in production. This is because, as any developer knows, debugging is often a case of finding a needle in a haystack. The smaller the haystack (the code you’re looking through) the easier it is to find the needle (or bug). That haystack is going to be smaller when you’re looking at the code written by one developer, as opposed to several developers or more.

This dynamic is what’s driving the trend of ‘Shift-Left’ to do more testing – not just unit, but functional and visual – during the development phase of the app lifecycle. Recognizing the importance of this trend, we want to help frontend developers and have developed new visual testing capabilities as part of Applitools Eyes.

Expanding Left to Developers

When you adopt Shift Left, it impacts your software development lifecycle. Developers now need to do more testing as they code, to discover bugs as soon as they create them.

Shift-Left testing paired with an increasing velocity of releasing software to meet business demand, all while maintaining high quality, presents huge challenges for R&D teams. If you’re a developer, you have to do more. Write more code, do more testing, manage the merge, etc. All to deliver more features in less time and with better quality. Because of these challenges, we are seeing new tools being adopted by developers such as the Cypress test framework and Storybook UI component development framework, and more.

So we expanded our AI powered visual testing platform to integrate with Cypress and Storybook. Now you have the tools to do visual UI testing as you code — at the speeds you need for Agile development, continuous integration, and continuous delivery (CI-CD).

Ultrafast Cross-browser Visual Testing with Cypress

The Applitools Cypress SDK lets you instantly run visual UI tests on top of your functional tests by running browsers in parallel, in the cloud, to generate screenshots for multiple browsers and viewports. Our visual AI engine then compares these screenshots to find significant differences and instantly detect visual bugs.

If you’re making the move to Cypress, we made this process run ultra-fast so that you won’t experience any slowdown when adding Applitools Visual AI Testing into your work stream. We do this by using containers to test all options simultaneously, in parallel. We call this feature our Applitools Ultrafast Grid. It takes the DOM snapshot and renders the various screens on different browsers and various viewport sizes corresponding to multiple mobile devices, desktops, etc.  

For the first time, you can use Cypress to effectively test to see if you have any cross-browser bugs. And, you can now do responsive testing to see if any visual bugs occur on different screen sizes.

This increased speed lets you cover more tests cases, increase test coverage, and catch more bugs before they hit production.

Check out Applitools’ new Cypress SDK and sign up for a free Applitools Eyes account at (https://applitools.com/cypress)

Instantly Detect UI Component Bugs with Visual Testing

The Applitools Storybook SDK visually tests UI web components, without any test code and works with React, Angular or Vue.

The way it works is that we scan the Storybook library to inventory all components. Then we upload a DOM Snapshot for each component’s visual state to our Applitools Ultrafast Grid.

Once these DOM Snapshots are on our ultrafast grid, we run visual validations in parallel. Validations of all visualization states of all components, on all specified viewport sizes, on both Chrome and Firefox, happen at once.

All this lets you perform ultra-fast codeless validation of your Storybook component libraries in seconds. You can think of this as visual testing at the speed of unit testing. You can learn more in this blog post on visually testing React components in Storybook and this webinar on our Cypress and Storybook SDKs.

Check out Applitools’ new Storybook SDK and sign up for a free Applitools Eyes account at (https://applitools.com/storybook).

The Final Word: Expanding Left, Not Shifting Left!

I want to emphasize that, even though we’ve talked a lot about developers and Cypress in this post, we still remain as committed as ever to QA teams, and the frameworks they use, including Selenium.

Selenium remains an incredibly important part of the test automation world, and we continue to remain a sponsor of both the Selenium project itself and SeleniumConf.  In the coming weeks and months, you’ll see us release new functionality that backs this point up.

While we are delivering new visual testing tools to developers, we remain committed to serving test automation engineers and QA analysts. That’s why we are talking about expand left, rather than using the more common term shift-left.

If you have any questions or want to try the application visual testing approach, please reach out or sign up for a free Applitools account today!

How will you start visual UI testing with Cypress or Storybook? 

The post How to Implement Shift Left for your Visual Testing appeared first on Automated Visual Testing | Applitools.

There’s a new claim to the throne of functional test automation tools: Cypress.io. Is Cypress fast? Yes. Is Cypress interactive? Yep. Is Cypress reliable? You bet. And best of all… it’s cool!

But is Cypress an alternative to Selenium WebDriver? Does Selenium, the current king of web automation and testing frameworks, cringe in fear for its position, or is it smiling benevolently at the supposed usurper, knowing full well that, well, it’s just a kid!

Is Cypress better than Selenium WebDriver? I get asked this a lot. And frankly, the easiest path of a “this vs that” article is to try and find out which is “best”. But I will not take that path. Instead, I will try and explain how is Cypress different from Selenium WebDriver.

By describing what is the difference between Cypress and Selenium WebDriver, you will be able to understand how Cypress works. This post will also try and explain why Cypress took the path it did to web browser automation, a path that is remarkably different than the path WebDriver took.

How is Cypress similar to Selenium WebDriver? They both control and automate the web browser, thus enabling you to write functional tests that simulate the actions that a user does, and verifying that the results are correct. But that is where the similarity ends, and the differences begin. Hopefully, knowing the differences between these two testing tools, and knowing the reasons for that difference, will help you choose which tool to use, and when.

So let’s start exploring these differences.

It’s For Frontend Developers!

Selenium WebDriver was built for end-to-end regression tests of web applications. The Selenium front end testing framework is mostly used by QA developers, and not so much developers. This is because the idea of front-end developers testing their own code is not common. 

But a small revolution is starting to happen in the world of frontend developers. Frontend developers are starting to write their own tests. Yes, they are rising to the challenge of agile software methodologies. They understand that without writing their own tests, agile development cannot happen. They are writing unit tests, but also integration, and, yes, end-to-end functional tests that check their frontend using real browsers, just like the automation tests that QA developers write. The “Shift Left” movement has a lot of these trends.

Frontend developers’ needs for end to end tests are different than those of QA developers. Frontend developers don’t need a staging environment where the whole application—frontend, backend, and database systems—is deployed. They can get by with just the frontend part running locally, and mock the backend pretty easily.

This fact — the target audience of Cypress is frontend developers — lies at the heart of all the differences between Cypress and WebDriver. They’re different tools for different users.

It’s JavaScript Only!

Frontend web developers write in only one language—JavaScript. It is the only language that browsers can execute, and so they have no other choice. This limitation has its downsides and its upsides, but the fact exists—there is only one language for frontend developers (variants like TypeScript notwithstanding).

And, because Cypress is meant for frontend developers, Cypress tests can only be written in JavaScript. No other languages are supported. Compare that to Selenium WebDriver, which has an API with many language bindings, including Java, C#, Python, Ruby, R, Dart, Objective-C, and, yes, JavaScript.

So remember—if you want to do Cypress testing, you will have to learn JavaScript. Which is cool, because JavaScript today is not your grandparents’ JavaScript. It’s a modern, powerful, and concise language that is widely used today and has the largest open source package repository in the world (npm), a code repository that a few years back overtook what used to be the largest repository, the Java Maven repository.

It’s Mocha Only!

Cypress doesn’t only limit you in the language you can write, but also limits which testing framework you use. Mocha is the testing framework that you write your tests with (a testing framework’s role in JavaScript is similar to the role Junit has in Java, or NUnit in C#). You cannot write your tests in another JavaScript framework, such as Jest or Tape.

Selenium WebDriver, on the other hand, does not impose a testing framework on you. Moreover, you don’t even have to use a testing framework. You can write a regular program that is not a test, a program that just drives the browser. Many projects use WebDriver that way, for example to crawl web pages and gather information.

But Cypress decided early on that it is dedicated to the task of writing frontend tests, and can only be used within a testing framework, and that testing framework MUST be Mocha.

Enough already. Show Me Some Cypress Code!

“OK, OK, I got it,” I hear you saying. But can you please show me a Cypress test so that all this will be a bit less abstract? Sure, here you go:

View the code on Gist.

As you can see, this has examples of Cypress commands and assertions.

cy.visit will make the browser navigate to the url given.

cy.get will return a reference to the element in question, on which you can either type, click, or check assertions on using contains and other assertions.

As you can see, on the surface, it’s surprisingly similar to Selenium Webdriver, albeit simpler, and with much more power.

It’s Chrome Only!

The Cypress test runner only works on Chrome. It has no support for Firefox, Safari, Edge, or IE. This may come as a shock to QA developers that are used to WebDriver’s amazing support for all these browsers. But, yes, Cypress is Chrome only. There is an issue that was opened for cross-browser support, but that issue is almost a year old. Cypress’ priorities, it seems, lie elsewhere.

Why would frontend developers want to run their tests only in Chrome? I believe there are multiple answers to that question. First, the differences between Chrome, Firefox, Safari, and Edge these days are small, and getting smaller and smaller for most applications. So checking only in Chrome is probably good enough when testing in development.

Second, running all the tests on all the browsers takes time, and frontend developers, as we said above, don’t want to wait for their tests.

Third, many companies have QA developers that run end to end tests on all browsers, and so they are OK with the very rare bug slipping now and again and being caught by the QA team. And companies that don’t have extensive cross-browser tests? Then they are OK with those same rare bugs slipping and reaching users now and again.

Fourth, frontend developers do care, and would like Cypress to support more browsers, but it’s just not a priority for them, given the reasons described above.

It Runs in The Browser!

Besides the fact that frontend developers are used to JavaScript, there’s another reason, a more technical one, that only JavaScript is supported.

The code you write in your Cypress test scripts does not run outside of the browser, like in WebDriver. It runs inside the browser. It is the browser that is executing your test code. In fact, it’s executing your test code alongside the app code that you are testing!

What happens when you run your test is that, first, Cypress will prepare your code to run in the browser (technically speaking, it will bundle all the modules in your code to one JS file using webpack). Once ready, it will run Chrome, and inject your test code into a blank page. Then it will run that test code, in the browser.

That test code first of all navigates to the application page (using `cy.navigate(‘http://localhost:8080’)`), which is run in an iframe. Cypress will make the iframe navigate to that page, but your code is running in the browser, and (somewhat magically) has access to the same DOM as that iframe, and so all commands, such as “click” or “type”, happen inside the browser, using regular DOM APIs.

To compare, let’s review Selenium WebDriver architecture and what this means for how Selenium tests run. Take a look at the diagram below.

With Selenium WebDriver, you have three processes:

• Selenium WebDriver
• A browser driver, such as ChromeDriver, GeckoDriver (for Firefox), EdgeDriver, SafariDriver, etc.
• The browser itself

All the communications between these processes means that Selenium tests take a long time to run. Which takes us to our next point:

It’s Fast!

This in-browser execution is at the basis of the speed of Cypress testing. With Cypress, you have just one process: the browser itself. With Cypress, your test code is running alongside your application code.

So an automation command (e.g., clicking a button) does not send the command to the browser like WebDriver does through out-of-process communication. Instead, Cypress uses DOM events to send a click command to the button. Much MUCH faster.

And while WebDriver’s out of process automation involves asynchronous communication, Cypress’ automation commands are mostly synchronous and in-memory. This generates extremely fast tests.

I believe that it is this speed that is the main reason that frontend developers are enamored with Cypress. WebDriver, at least perception-wise, did not give them the speed that they needed. Frontend developers want to run their tests ALL THE TIME, and cannot afford the luxury of a suite that takes tens of minutes to run. They want to run their suite, and they need it to run in a minute or two, not more.

Is this speed difference really substantial as Cypress’ documentation claims? From my admittedly small experiments, then yes, but I am not convinced that the difference is worth the other benefits WebDriver has. Try it yourself on a small sample of tests, and see for yourself!

It Has Built-In Server Mocking!

But there is another reason that Cypress is perceived as faster. Let’s remind ourselves that Cypress is meant for frontend developers. Let’s also remind ourselves that frontend developers sometimes don’t use the real backend, but rather mock the XML HttpRequests to the server. This makes for extremely fast tests, on the order of seconds, regardless of whether you use Cypress or Selenium WebDriver.

Cypress understands that, and has built-in facilities for mocking server responses, facilities that are crucial for their target audience: frontend developers. Selenium WebDriver has no facilities for that, and to mock server responses, a WebDriver tests would need to run a mock server that returns the correct responses. While possible, it is a slower and much less convenient option: a fast and in-memory facility for mocking will always outperform a slow out of process one.

And it’s not only the speed of execution. The convenience of a built-in facility for mocking servers cannot be overstated in terms of speed of writing the tests. The fact that each test explicitly describes the mock responses is much easier to code (and understand!) than a separate server program that mocks the same responses.

It Has a Bizarre Execution Model!

Yeah, I know. Weird heading for this section. But it does! Cypress’ execution model is bizarre! To prove it, let me show you some Cypress sample code:

View the code on Gist.

“That’s not bizarre!”, you say, and you would be right if it did what you thought it did. But it doesn’t. When you call `cy.click` or `cy.navigate` or `cy.type`, you’re not executing that command. You’re actually recording what you want done when Cypress does run the test. The test doesn’t run inside the `it` function. It runs after the `it` function ends, based on the commands you recorded. `cy.nagivate/click/type` don’t execute, but rather record, to be replayed later.

What are the practical consequences of that decision for the developer? If you want to branch your code based on a value in the page, you can’t. For example, let’s say you want to do something like this, to execute some test code based on the value of text on the page:

View the code on Gist.

This won’t work. Because when running the test, you’re not actually executing the automation commands, and so checking for the value of the text is irrelevant.

Cypress does support this kind of test, but in a byzantine way:

View the code on Gist.

The `.then` facility enables you to supply code that “really” runs when Cypress executes the automation commands. This `.then` solves the problem of branched test execution in a “record-then-execute” model, but would not have been needed if Cypress had followed a regular execution model, like WebDriver does.

So why does Cypress have this byzantine execution model? The answer to that lies in the next section.

It’s Less Flaky!

According to Cypress, by delegating execution commands like navigate, click, and type, to Cypress itself, Cypress enables more control on how to execute those commands. It can, for example, when asked to click on a button, wait for that button to appear on the screen, stop animation, and then, and only then will it click it.

According to Cypress, this results in much less flaky tests.

I’m not convinced.

Waiting for an element to appear sounds a lot like WebDriver’s implicit wait, so I’m not sure why Cypress believes it is better in that regard.

But I do believe Cypress is less flaky, but this has nothing to do with controlling the execution. Because the tests on it are executed frontend only, with a mocking of all server responses, the test runs very fast. And the faster a browser test runs, the less flaky it is.

So, yes, I believe that Cypress is less flaky, but not because of control of execution. Too bad, the byzantine method of execution detracts somewhat from this wonderful tool.

But I may be wrong!

It’s Interactive!

One of the biggest coolnesses that Cypress has is also a big part of the reasons lots of developers flock to it: it’s an IDE of sorts. Because Cypress fully controls the execution environment (the browser), it can give developers a more interactive experience:

As you can see in the screenshot, when the test runs, the commands that are executed are shown in the side panel, while they are being executed. Moreover, once the test ends, you can click on each command in the side panel, and you can see a screenshot of how the page looked like when the command was executing. This is called Time traveling.

This is phenomenal for debugging a test! Cypress screenshots and seeing how your tests progress is a marvelous way to debug the tests, and a marvelous way to start to understand why the test failed. The whole experience helps in writing, understanding, and debugging tests, and in analyzing their results.

Its Documentation is Excellent!

Compare Cypress’ documentation…

…to Selenium’s documentation:

When you review Cypress documentation, it looks a lot nicer. But it’s not just what the documentation looks like. Cypress’ documentation is much more in-depth and broad. It covers most of what you want to know, without the need to go looking elsewhere on the web. You can see that the developers spent a lot of time on the documentation itself. Kudos to them, as it enhances the experience in a significant manner.

Selenium’s documentation is pretty good, especially given that it has a much more wider net to throw—both because it is multi-language, and because it is multi-browser. Unfortunately, it carries with it the weight of previous years and is not equal to Cypress’, both in depth and in width.

Conclusion

As you can see, on the surface, although the two tools Cypress and Selenium WebDriver—seem similar, there are actually many differences between the two. The main reason for the differences, in my opinion, stem from the different requirements from the two tools: Selenium was built as a multi-language, cross-browser tool, that fits multiple purposes, not just testing. Cypress, on the other hand, was built for one thing, and one thing only, executing frontend developers tests fast and consistently.

That is why Cypress seems so limited: it supports just one language (JavaScript), one framework (Mocha), and one browser (Chrome). Selenium WebDriver does not have these limitations. It cannot have these limitations because it is a general purpose tools for multiple target audiences. But these limitations are mostly not a problem for frontend developers, and enable Cypress to give frontend developers the advantages that they do need: speed and consistency.

Cypress is a classic example of less is more, while Selenium WebDriver is a classic example of a Swiss Army knife.

Which is better for you: Cypress or Selenium? You tell me in the comments!

One thing I do know is that my company, Applitools, has SDKs and tutorials for how to do visual UI testing with Cypress and Selenium. So whichever you choose for test automation, we got you covered.

The post Cypress vs Selenium WebDriver: Better, or just different? appeared first on Automated Visual Testing | Applitools.

As a genre, fantasy is often looked down upon by fans and writers of science fiction. There are some who argue that the worlds created by JRR Tolkien were influenced by the events of the early twentieth century. No matter how you interpret his works, one of the central themes of The Hobbit and The Lord of the Rings that remains timeless is that if you come into possession of and use something powerful, like an all-powerful magic ring, you should be prepared for the harm it can cause.

Before we get too carried away with Tolkien references, this post is about visual UI testing, not hobbits. Similar to Tolkien’s central theme, the goal of this post is to inform you that any time you use visual UI testing, you should consider the broader issues it involves. Specifically, this post explores the reasons why developers who use visual testing need to think about more than simply testing the parts of a web app that they can see. It’s the things you cannot see that will trip you up, not the things you can see. Many of these unseen page elements have very little to do with the application itself or its underlying code, but are caused by the display technologies that are used to present the application to users.

This post explains how the hardware and software technologies that are designed to make displayed content look better will cause your visual tests to fail. We also offer Applitools Eyes as a solution that can help you avoid these problems and their consequences.

A Brief Introduction to Visual UI Testing

The purpose of visual UI testing is to verify that the user interface is rendered so that it appears correct to its users. It factors in the many different environments, operating systems, devices, and sizes in which modern apps run. On the macro level, visual testing checks that the entire user interface is rendered properly. On the micro level, it verifies that elements appear in the right shapes, positions, and sizes. Visual tests are run on different platforms, devices, and display sizes to ensure that the application is displayed correctly within the target environment (e.g., desktop browsers running on PCs and laptops, smartphones, and tablets).

Using test frameworks, developers write code that simulates real user actions, such as typing text into a field or clicking a button. At relevant points in the test code, they will insert commands that capture a screenshot. The first time the test is run, an initial set of screenshots is captured. These screenshots serve as a baseline against which all changes are compared. Once the baseline is determined, the developer runs the test code in the background. When a change is detected, an image of the change is captured. The test runner compares this image to the baseline image for this area of the code, and if differences are detected between the images, the test is failed.

If you want a more in-depth introduction to visual testing, see this post or this YouTube video by Adam Carmi. For a developer’s perspective on this topic, read this post by Gil Tayar.

What You See Is Not Always What You Get

In the early days of visual computing, somebody coined the expression “what you see is what you get,” or WYSIWYG. No matter when this expression was created, the idea it expresses has never been true. No matter how good a GUI was, the technology used to display it was so primitive that it always looked terrible.

Over time, technology has improved. Modern display technologies are bigger and they pack more pixels into smaller spaces (pixel density) than ever before. Hardware can only go so far, so to overcome any remaining hardware-related display issues, advanced algorithms and clever software were created. To better understand the full meaning of this post’s title, we need to know what this software is doing, how it fools us into believing that the quality of the displayed image is never as good as we think, and why it causes false positives.

False positives are one of the major reasons why one does not simply do bitmap comparisons. A false positive is generated when a test performs as expected, but it fails a test that should have been passed. There are many reasons why image comparison tools generate false positives. In most cases, we get a false positive when the tool detects something that is invisible to humans.

To illustrate this point, let’s look at how an application renders text and how this can generate a false positive test result. Suppose we have created a web page and stylesheet that specifies that all text must be black. In order to hide any jagged lines, the browser has applied a text smoothing, anti-aliasing algorithm. The text appears black to the human eye, but if we magnify the text, we’ll see that extra colors have been added to each letter at the pixel level. Since we can’t see the changes with our own eyes, we expect the test to be passed. But, since the tool sees the added color pixels, the test is failed. By default, the easiest way to compare two images is to count the total number of similar pixels. Assuming that two images of the same size are being compared, if the number of colored pixels in each image is the same, the test is passed.

Unlike machines, human beings perceive the world in relative and not absolute terms. To a human tester, if the text appears to be black, it’s black. To limit visual tests from generating unexpectedly high numbers of false positives, we try to make the tools a little bit more human. The easiest way to do this doesn’t involve machine learning or AI—instead, it involves adding a defined margin of error to the image comparison algorithm.

The margin of error allows the two images to differ by an amount that is small enough to avoid a false positive result. This margin of error is defined by counting the number of pixels in a defined area of the image. The number of pixels is divided by the total number of pictures in the image and expressed as a percentage of the total area. This margin of error is referred to as the error ratio. The image comparison tool lets the developer set a threshold for this ratio.

If the error ratio for two compared images is below the threshold, the test is passed; otherwise, it is failed. Ideally, this number should be as small as possible. By using the error ratio threshold, developers can compensate for minor differences between the baseline and captured images that have been captured by the testing tool. The problem is that, for the sake of convenience, many testers will increase this threshold beyond where it should be. Over time, as more changes are detected, the threshold will be raised to the point that a serious defect could escape detection.

The Applitools Solution

Applitools Eyes offers a number of solutions for false positive visual testing results. It solves the issues that occur during the image capture, comparison, and reporting stages of a visual test run. Applitools Eyes also has the ability to analyze the structure and layout of images, and it uses this analysis as the basis for passing or failing a test (as opposed to counting pixels or measuring image area).

To understand how this would be helpful in practice, let’s say you have captured your baseline images of a web page using the previous generation of a mobile device and the tests are run on the latest version of the device. The older device has a wider screen, but the newer one is slightly longer than its predecessor. Even if both devices run the exact same operating system, each device will display the web page differently.

Applitools is able to determine which differences are caused by the way a page is rendered, as well as the display-related constraints of the device they are rendered on. It has been designed to account for the differences caused by screen size, layout, and many other factors that generate unnecessary false positives. It also ensures that the visual test reports include the issues that need your attention.

Conclusion: Minimize False Positives and Maximize Test Results

Visual UI testing involves more than just comparing two images. As we have seen, tests that only compare the absolute differences between a baseline and current image can generate a large number of false positives. For small projects, it isn’t too hard to review each individual false positive instead of relying on an error ratio.

However, as your project grows and you create more visual tests, reviewing each false positive will require increasing amounts of time and energy. At this point, you might switch to an error ratio-based approach. In the early stages, this might yield good results, but this approach introduces its own, new set of issues.

There are no magic solutions in software testing. Even with advanced tools like Applitools Eyes, your visual tests might still generate false positive results. However, Applitools Eyes handles most of these issues automatically, and it only shows you cases that actually require your attention. With the issue of false positives out of the way, you can concentrate on writing code and developing new features.

–HAPPY TESTING–

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Visual UI Testing: One Does Not Simply Do Bitmap Comparison appeared first on Automated Visual Testing | Applitools.

As front end developers, our job is to present visual information that looks good and gives users cues to help them understand content quickly. Even a static, text-only web page is more than a collection of words. The page has a title that appears at the top. The title may be bold or in a large font size. In the body of the text, words are grouped into sentences that end with periods, and related sentences are grouped into paragraphs. Certain words in a sentence may be emphasized with bolding, underlining, or italics. These examples barely scratch the surface of the many, minute details of visual web page design.

These components may seem basic and unimportant, but for many people, accessing and reading text on a web page is difficult. For example, individuals with visual impairments often struggle to read the text. When we think of visual impairment, most of us automatically picture a person who is blind, but there are many other types of visual impairment to consider, including color blindness, low vision, sight problems caused by degenerative disorders, etc. The range of disabilities and handicaps our users may have extends far beyond the visual category to include hearing, cognitive, motor, and other types of issues. Therefore, making sure your website or application is accessible rises to the utmost importance. 

Since front end developers handle the visual presentation and functionality of web pages and applications, we need to be aware of and address the accessibility issues that exist for those who have disabilities and handicaps. While we all have a moral imperative to do our best to make websites and applications accessible, front end developers in many countries also have a legal obligation to do so.

This post will define what accessibility means and will discuss what kinds of issues front end developers need to be conscious of in terms of providing accessibility. We’ll also review some of the available methods, tools, and resources to address these issues.

What Is Web Accessibility and Why Do You Need to Be Involved?

The World Wide Web was created for the purpose of sharing research amongst atomic physicists at CERN. As the web grew, the World Wide Web Consortium (WC3) was created to govern it. The WC3’s mission states that the web was created to enable human communication to benefit all people, no matter where they live or what physical and/or mental abilities they may have.

Web accessibility ensures that people with physical and non-physical disabilities have equal access to the web. It helps people with impaired auditory, cognitive, neurological, physical, speech, or visual functions perceive, understand, navigate, interact with, and contribute to the internet. To help web developers and designers make their pages and applications accessible, the WC3 launched the Web Accessibility Initiative (WAI), which provides guidelines for making the web available to people with disabilities. Providing accessibility also helps other groups, such as users of mobile technologies, older people, people who have temporary disabilities (e.g., someone who is recovering from an injury caused by a physical accident), and people in certain locations and countries with limited bandwidth.

Beyond our basic obligation to help our fellow human beings, there are also practical reasons for providing equal accessibility. No matter what type of organization we work for, be it commercial, non-profit, governmental, etc., we need to accommodate all users and customers. If we understand what our users need and provide methods to meet those needs, any person, no matter their physical or other limitations, can buy our products and use our services.

As the reach of the internet spreads to individuals with limited income and to less developed countries, we risk losing huge numbers of potential customers if we don’t make our products accessible to them. According to the World Health Organization (WHO), one out of seven people of the world’s population is disabled or handicapped in some capacity. So, if we fail to meet this population’s needs, we’re missing out on approximately one billion potential consumers.

In some countries, front end developers are also legally required to provide accessibility. Many governments around the world have adopted the WC3’s Web Content Accessibility Guidelines (WCAG) 2.0. WCAG 2.0 provides guidelines for implementing solutions, but it isn’t tied to any specific technologies. The UN and local initiatives have also used WCAG 2.0 as the basis for their accessibility legislation. Countries that currently have laws governing web accessibility include Australia, Brazil, Israel, the United Kingdom, the United States, and the member states of the European Union.

How to Provide Web Accessibility

Since we have a limited amount of space in this post, we’ll concentrate on vision and cognitive impairments for our specific examples of how to provide web accessibility.

One of the simplest ways to accommodate these users is to provide good page semantics. Semantics are the elements and structure of a page that give it meaning to a human reader. A page’s semantics are created by applying the correct HTML elements in the right sequence (e.g., using page headings H1 and H2 in the correct order). You can also differentiate between different types of content by using bulleted and numbered lists. Page semantics matter because assisted reading technologies, such as screen readers, use them to process the page and read it aloud to people with visual impairments.

Page semantics are also important for helping individuals with cognitive disorders. For example, people who have dyslexia can see words and pictures, but they have difficulty extracting meaning from the raw, visual input because of difficulties with processing contextual information like letters and word order.

There are a variety of ways to make information accessible for users with disabilities and handicaps. To help people with low vision and dyslexia, you can use a high-contrast color palette. You can achieve this contrast by using a dark color like black as the background color and a bright color like white or yellow for the foreground text.

Front End Development and Accessibility

As front end developers, we have the power to make equal accessibility a reality. There are a number of tools you can use to build and test web accessibility. We discuss some of the best tools below.

ARIA

The Angular, React, and Vue frameworks all provide support for the Accessible Rich Internet Applications (ARIA) toolkit. 

ARIA is a WIA platform that standardizes how developers can build web-accessible applications and present content for people with disabilities. It provides the required functionality for assistive technologies (e.g., means to control input devices for people with limited/impaired motor functions and screen readers for people who have visual impairments).

ARIA can be used as an indicator of the state of a web component to a screen reader or an input device. In a web application, components indicate their current state visually. If the state of the component changes, the object’s presentation changes. For example, a checkbox indicates the values are true by displaying a checkmark, and false by leaving the box blank. To support assistive technologies, additional semantic information is required so that the component’s state can be presented to the user.

To provide this information, you can add the ARIA-checked label to the component. If the page is read by a screen reader, it will use a label to inform the user if the box is checked or not. If the page is being used by a person with limited mobility, component state changes can be indicated via haptic feedback.

Ally.js

While ARIA can help you identify components and provide state information to users, it doesn’t have any built-in support for navigating web pages. In this context, navigating a page refers to two things: moving between page components and indicating the order in which they should be used. For example, if a website requires a user to change their password, it will ask them to type a password into one text box, retype it into a second text box, and then press the submit button. After each step is completed, visual cues in the application tell the user what to do next and where to place the required input. For a user who is unable to use conventional input devices, it’s difficult to enter the information in the correct location and navigate to the next component in the correct order.

Ally.js is a library that enables users with restricted mobility and limited vision to navigate web pages. It does this by providing an accessibility tree that shows the structure of the page and indicates any objects and components that have semantic significance. The accessibility tree defines a web page’s Sequential Navigation Focus Order, which is a list that determines the sequence in which components are accessed via an input device.

Test Automation

There are many available tools for testing web accessibility, but testing often involves boring and repetitive tasks. So, there are many tools and frameworks for automating functional, regression, and visual testing. Testing accessibility can be done by using the same methods and tools. The best tool for the job is the aXe platform, which is an open-source project that provides assertions for accessibility testing and integrates with a large range of testing and CI systems.

Conclusion: Committing to Making the Web Accessible to All

We all agree that everybody, no matter what mental or physical restrictions they may have, should have equal access to the web. There are clear moral, economic, and legal reasons to make the web accessible, and there are also guidelines, techniques, and technologies available that can help front end developers actually make it happen.

With the many tools we have at our disposal, creating accessible web pages and applications has never been easier. Accessibility support is baked into the major platforms via ARIA for front end developers. If you need extra functionality that isn’t provided by your framework of choice, you can use ally.js. You can even do automated testing with the aXe platform to test your applications for web accessibility. With empathy, understanding, and the proper tools, front end developers have the power and the responsibility to make the web accessible to all.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Making Your Website/Application Accessible: a Guide for Front End Developers appeared first on Automated Visual Testing | Applitools.

In the days before web applications, programmers wrote applications for a specific platform and used that platform’s native development environment and user interface controls. Before the application’s final release, a tester would check that the application was ready. Some testers wrote elaborate documents that described complicated scenarios they performed on the software manually. Other, more adventurous testers used fancy tools that were the quality assurance version of Microsoft’s Word and Excel macro recorders. A tester would record a series of actions on the software under test. These steps were recorded in a high-level language in which the tester could edit the code and add test conditions that would verify that the actual results of the test matched the expected results. Each time a test was run, a report was generated that showed which conditions had been passed or failed.

With web applications, the time it takes to write and deploy code has been reduced to seconds. In this environment, much of the responsibility for testing is now in the hands of developers, who have since created many code-based test automation platforms.

As good as these tools are, and as widespread as they have become, it isn’t safe to assume that all developers are using them. Developers who aren’t writing test code often use these excuses: they don’t have time, the learning curve is too steep, they’ve tried and it didn’t work, or their management doesn’t care.

However, these excuses aren’t good enough reasons for developers to avoid writing test code. Automated developer tests help detect defects and issues at a much earlier stage in the development process. Automated tests can also be written for a wide range of scenarios. And once a testing suite has been created, it can be reused for existing tests and future scenarios.

Getting started with test automation is easy with the proper tools and frameworks. Automated developer testing is simpler than ever before. All you need to do is pick two frameworks: a test automation framework for writing and running test code, and a browser automation framework for simulating a web browser. This post explains what these frameworks do, examines the most popular options, and discusses the advantages and disadvantages of each respective framework.

Choosing a Test Automation Framework

Most test automation frameworks are based on a common set of agile testing practices. Generally speaking, the development team collects user stories that map to a feature. A user story is a single sentence that states the feature’s persona/role, action, and reason. If you want to enter your user credentials to access the system, each story must have clearly defined acceptance criteria that ensure the implemented feature meets the stakeholder’s/target user’s expectations.

There are a number of competing test runners in use today, but most front end developers use either Jest or Mocha. The good news is that they’re so similar that it doesn’t matter which one you pick.

Below, we have provided a sample Mocha test. The same thing can be written with Jest, and hardly anyone would notice or care. If you’re a React developer and use the create-react-app app builder, the framework automatically generates an initial Jest test for each new project.

Let’s say you’re testing a calculator web application. Start by creating a describe block that will hold one or more tests.

describe('calculator', function () {})

Next, add a test using the it function. The test includes a description of what you want to test.

describe('calculator', function () {
  it('should show initial display correctly', () => {})
})

Now, add an assertion using the expect function. This assertion checks that the calculator display shows the expected information on launch. The test is only passed if the calculator display shows a zero.

it('should show initial display correctly', () => {
    	  expect(calculator.initialState.display).to.equal('0')
  	})

In addition to allowing you to write the code, the testing framework lets you run it. To run your test code, type npm test at the command line. When the framework detects changes to your source code, it executes your tests. Each time the framework runs the tests, it indicates if the test was passed or failed.

Comparing Browser Automation Frameworks

A browser automation framework does much more than just simulate a web browser. For starters, these frameworks can launch either full or headless browsers. A headless browser is a browser without a graphical user interface (GUI). Headless browsers are lighter and faster than full browsers, which makes them the ideal tool for automated developer testing.

These frameworks also provide an API that can be used in your test code. This API can simulate users’ browser interactions, capture screenshots, find out the value of input fields, and perform other automation tasks. Unfortunately, choosing a browser automation framework will require a bit more work. Three of the most popular options are presented below.

Selenium Webdriver

If you’re looking for the safest choice, Selenium Webdriver is the way to go. Out of the three framework options we present here, Selenium Webdriver has been around the longest. It’s an open-source project with a very active developer and support community. One of the best things about Selenium Webdriver is that it’s part of the much wider Selenium ecosystem. This provides developer bindings not just for JavaScript, but also for languages such as Java, C#, Python, Ruby, PHP, and Perl.

Over time, Selenium Webdriver has collected a wide range of features, and it’s the only one of the three frameworks presented here that supports all five of the major web browsers (Chrome, IE, Edge, Firefox, and Safari). Selenium Webdriver’s builder API makes it easy to configure and launch the browser instance that you want and set the configuration options that are relevant. It also lets you write code that simulates a wide range of browser user interactions, such as typing text into a text box or pressing a button.

Although Selenium Webdriver has great versatility, features, and integrations, it’s losing popularity since it operates so much slower than its competitors, Puppeteer and Cypress.io.

Puppeteer

One of the newer frameworks is Puppeteer. Puppeteer is excellent for simulating human-like browser interaction. Like Selenium Webdriver, it simulates a wide range of browser user interactions.

Puppeteer also has a range of features that Selenium Webdriver lacks. It can capture screenshots and generate PDFs of specific screens. This makes it a great choice if you want to use it for visual testing. Its speed is also a major benefit for many developers who are choosing a testing tool.

Puppeteer’s other major selling point is that it’s highly integrated with Chrome and its debugging and developer tools. This should come as no surprise since, like Chrome, Puppeteer was developed and is supported by Google. So, if you develop and test your apps only in Chrome, Puppeteer is a great option. However, if you develop with or test against Firefox, Edge, or Safari, Puppeteer won’t work for you and you should go with Selenium Webdriver.

Cypress.io

Cypress.io is a free, open-source front end testing tool. Unlike the two previous frameworks, it’s a complete testing solution. It makes no claims of being a general test automation solution because its main focus is performing end-to-end testing of a web application’s user experience.

Cypress.io’s biggest selling point is its blazing speed. It achieves this speed by running inside the browser. This means that it runs in the same execution loop as your application’s source code (as opposed to Puppeteer and Selenium, which run in a different process than the browser process that renders the application code). This means that Cypress.io doesn’t need to support the control protocols that are required for inter-process communication. This also resolves any system or network latency that slows down sending and receiving commands and responses.

When it comes to discussing any disadvantages of Cypress.io, their documentation is refreshingly open about the product’s benefits and trade-offs. The three most significant cons that may influence your decision relate to the way in which Cypress.io runs tests inside the browser. First, Cypress.io’s method makes it harder to communicate with backend servers, which will mean that you’ll need to install additional modules. Second, it doesn’t support testing multiple browser tabs. Finally, it can only control a single browser instance at a time.

Conclusion: Choose the Right Tool for the Job

When it comes to choosing a test automation framework, you can choose either Jest or Mocha. However, of the browser automation frameworks we looked at, you’ll need to decide between Selenium Webdriver and Puppeteer. If you want a fast, all-in-one solution, Cypress.io is the best choice.

Now that we’ve presented you with the research, all you need to do is pick an option and get started. Once you put the pieces in place and start writing test code, the rest is easy.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Comparing JavaScript Browser Automation Frameworks: Selenium Versus Webdriver.io Versus Puppeteer appeared first on Automated Visual Testing | Applitools.

It’s not often that software testing makes the news, but when it does, it’s usually because something within the test automation has gone spectacularly wrong. At 8:05 AM on January 13, 2018, the Hawaii Emergency Management Agency sent out a false missile alert. While there is much that is still unclear regarding exactly what happened, most reports agree that an internal test was treated as a real event. According to The Washington Post, part of the problem may have been caused by a human operator who chose the “Missile alert” instead of the “Test missile alert” from an application menu.

This event revealed a number of design flaws within the emergency alert software. On top of a bad user interface design, the system had no fail-safe mechanisms or human overrides in place. So, once the alert was sent, it couldn’t be retracted. As a result, thousands of people panicked because they believed that a North Korean ICBM, possibly armed with a nuclear warhead, was headed their way. Nearly 40 minutes elapsed between the time the message was sent and the time the initial cause was located. During that time, no message was sent to the public to indicate that this was a false alarm.

To be clear, this event was the result of a system being tested, not software being tested. However, had the system been subjected to rigorous software testing, the flaws that caused the incident could have been found and fixed. Using current technology, much of this testing process could have been handled by automated testing software.

Any organization that relies on computer software should and must perform the appropriate testing. This post describes the role of test automation as part of the software development process, and why everyone in your organization should actively participate in testing. We provide an introduction to software testing and its importance, the rationale behind automating testing processes, best practices, and recommendations on how to proceed.

Who Is Responsible for Software Testing?

In addition to the standard questions of what to test, when to test, and how long to test is the question of who is responsible for testing the software? Does this job belong to a specific person, role, group, or department? Shouldn’t everybody with a stake in a product/throughout an organization be involved in ensuring quality? The answer to all of these questions is “yes”—the role of assuring software quality is the responsibility of developers, testers, managers, stakeholders, and even users.

Much of today’s development and testing practices have been highly influenced by Agile development. As is codified in the first twelve principles of the Agile Manifesto, Agile development prioritizes satisfying customers through early and continuous delivery of valuable software. In Agile projects, quality and testing are part of a wider conversation between creators and consumers. Broadly speaking, the responsibility for testing is shared between project teams and its stakeholders. In addition, tests are written as early as possible, and they run continuously before, after, and during the development process.

However, recent changes to the ways in which we access and use applications have caused a move in the opposite direction, concentrating the responsibility of some testing tasks to a much smaller group of individuals. Due to the widespread availability of broadband internet and the rapid adoption of smart mobile devices (phone/tablets), some computing tasks that once required execution by powerful back-end servers are now performed in a desktop browser or in a native mobile app. As a result, a large part of the responsibility for testing client applications has shifted to the developers who build front-end web apps or client mobile apps.

Developers also perform more of the operational tasks that were once handled primarily by corporate IT departments. Traditionally, IT and operations have also been involved in the testing process. As these separate roles merge together with DevOps, operational testing tasks (e.g., performance, load, and security testing) are performed by developers.

How Software Is Tested

There are many types of software tests that can be used. We review four of the most important ones below.

Developer Testing

Unit tests are performed by developers, and they are written before and during the development process. These tests perform a variety of functions in the development process. They are used to validate stakeholder-defined acceptance criteria, and they can also be used to ensure that any changes made by a developer to existing code don’t create new problems. Many developers will test their code by running it after they have written or modified it. Developers that use Test Driven Development (TDD) write unit tests that test individual code blocks (functions) and modules.

Functional Testing

Functional testing checks that code work, an application feature, or a specific user action is functioning as expected. Functional tests are based on user stories, each of which includes a persona, task, and reason for performing the task. The functional test validates the assumptions of the story and checks that the expected and actual outcomes are the same. The test also verifies that the developers implemented the feature or scenario according to the requirements expressed in the user story.

Regression Testing

Regression tests are used to locate issues after a major change has been implemented. They are designed to test changes that result from writing code for a new feature or fixing existing defects. As the name suggests, they are designed to see if the software has regressed as a result of these changes. Regression tests examine the intended and unintended consequences of changes, such as introducing new defects or reintroducing old ones.

Visual Testing

Visual testing involves testing how applications actually look. Most visual testing systems are based on capturing and comparing images. By using a tool or by writing a test script, the tester captures an initial set of screens and their component elements. Whenever the application’s user interface changes, the tests are rerun and new images are captured. After each test run, the latest images are compared with the previous test run. If differences are detected, the test is marked as failed.

When Should Automated Testing Be Used?

Testing that involves uncertainty, active participation, or interpretation is best performed by people. Other tasks—especially those that involve monitoring system behavior and performance—are better handled by computers.

Test automation is often used to perform repetitive, routine tasks. Automated testing makes economic sense, and it relieves testers from having to do these mundane tasks, allowing them to spend their time on more productive activities. Automation advocates also claim that test automation can lead to faster and more frequent releases, and that it lets organizations build smaller, more productive development teams.

However, automated testing doesn’t solve all software quality-related issues. If it isn’t done right, it can make a bad situation worse. To use test automation effectively, you need to know which parts of your development and testing process will benefit from it the most. You need to do the necessary research, find the right tools for the job, and plan your deployment carefully.

You also need to train developers to use the tools properly and provide guidelines and standards to ensure they write effective, relevant, and high-quality tests. Some tasks simply can’t be done by machines, so you have to recognize when humans are necessary to get the job done. This is especially true in cases when the outcome of a test can’t be measured in terms of success or failure.

Test Automation Tools

Today, many automated testing tools are free and open-source. There are also a variety of tools for writing, running, and managing tests.

Automated tests are written by people and executed by machines. A number of test-specific languages have been created to describe and organize testing tasks. For example, Jasmine is used to write specifications, which are human-readable blocks of text that translate user stories’ acceptance criteria into machine-executable tests. Each test specification includes an assertion that tests the feature and sets a condition to be passed or failed. In most cases, the specifications are written in as a single file that is used to test a specific module and/or application functionality.

Once you’ve written a test, you need a tool to execute it. This is handled by test runners, which are tools that read the test specifications and assertions for the targeted module. They read these specifications, run the assertions, collect results, and display them to the user. Test runners also handle common configuration and cleanup tasks that are performed before or after a test run. Popular test runners include Jest, Karma, and Protractor. 

When you run tests against a web browser, you need a tool to interact with them. Tools like Puppeteer and Chromeless provide a headless browser that lets your test perform the same interactions as a human user. Puppeteer also allows for easy automation of browser tasks, and it captures a timeline trace that is useful for diagnosing performance issues. Some frameworks offer support for visual testing and can generate screenshots and PDFs of web pages. 

Conclusion: Test Automation Is for Everyone

Test automation may seem complicated, but at this point, it’s actually within the reach of most organizations. There will be a learning curve, but since most of the tools are free and open-source, the types of automation covered in this post are available for anyone to use.

Before you automate your testing processes, you need to understand the different types of testing and determine which of your processes do and don’t need to be automated. Once you’ve done that, you must train your development teams, testers, and support staff, and choose the tool that best fits your organization’s needs. If you follow these steps, you’ll be able to reap the full benefits of test automation.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post A Decision-Maker’s Guide to Test Automation appeared first on Automated Visual Testing | Applitools.

At some point while each of us were growing up, we wished that the adults in our lives would just disappear. They made our lives miserable with their arbitrary rules and restrictions, but they got to do all kinds of fun things. After all, how hard could it be to drive a car, and why did we need them to watch an R-rated movie or to cross the street?

Well, front end developers have a similar fantasy. Their wish is that one day, all back end developers will move out of their way and let them take control. Front end developers are responsible for the things people see and use. All the back end developers need to do is create REST APIs and HTTP endpoints that work and return well-formed JSON, and the front end developers do the rest.

One way you can get what you want in a timely manner is to go over to the dark side and become a full-stack developer, but where’s the fun in that? Well, there have been some interesting things happening in the world of cloud services, and soon, the tables may finally turn and put front end developers in the driver’s seat.

A new development that is benefiting front end developers is called serverless computing. Of course, the term “serverless computing” is deceptive. In this article, we take a look at the server behind the curtain, discuss the rise of serverless computing and what that means for front end development, and show you how you can get started.

What is Serverless Computing?

Serverless computing could just be a fad, and it might eventually join the long list of tech marketing buzzwords like Agile, virtualization, containers, DevOps, AI, machine learning, etc. In his article about serverless architecture, Mike Roberts defines serverless computing as applying to two different, but overlapping, models. These models are Backend as a Service (BaaS) and Functions as a Service (FaaS).

BaaS refers to third-party cloud services such as authentication, file storage, messaging, and data management. These services are accessible via an easy-to-use, web-based graphical user interface (GUI). Sometimes, they also have a command line interface (CLI) for building the services. The service also takes care of operational tasks such as maintenance, security, and scaling. Developers call these services directly from their client code via an HTTP endpoint and a rest API.

FaaS enables the developer to create a function that combines a number of exposed BaaS endpoints. These functions provide a container that can be deployed to provide specific backend services. The functions are triggered by an event such as a request sent from a client application.

In order to run the functions, FaaS services provide a compute platform and a gateway. The compute platform lets you write and deploy code that receives requests and performs the relevant tasks, such as user authentication, running queries against a database, and uploading files to storage. The gateway publishes the functions and provides an API proxy that receives requests from client applications, authenticates them, passes them to the compute platform for processing, and sends the response back to the client. In addition, all of the services provide a developer console, management tools, and various ways to analyze usage and performance.

Getting Started with Serverless Computing

One of the best places to start exploring serverless computing is Google’s Firebase service. 

Below, we provide examples of how to use Firebase for BaaS and FaaS. Please note that the examples are all written in JavaScript and ReactJS.

Creating a Project

Before exploring the BaaS and FaaS aspects of Firebase, you must create a project. Creating a project with Firebase is very straightforward. First, you open the Firebase console web page and follow the instructions. Once you create the project, Firebase generates a JSON object that contains all of the configuration information. Then, you copy the object and include it as part of the project. Here is a sample of the generated code:

BaaS Services

Firebase provides a range of BaaS services, but for the purposes of this post, we will just review the authentication and data management services.

Authentication

Even if you do not use any other service, there is no excuse for not using commercial authentication. The provider has done most of the leg work, so you do not need to reinvent the wheel.

Firebase gives you everything you need to create and manage users. More importantly, it lets you create the rules that control permissions and data access. On the client side, it provides the Firebase UI library, which abstracts away the annoyance of building and maintaining client-side authentication code. For React developers, there is a Firebase UI React library that provides additional wrappers. The libraries allow you to create an object that lists the relevant authentication methods, and they show you how to handle the authentication process.

const uiConfig = {
  signInOptions: [ firebase.auth.GoogleAuthProvider.PROVIDER_ID,
    firebase.auth.FacebookAuthProvider.PROVIDER_ID ]
};

All you need to do is embed the Firebase authentication component in your class object’s render method.

class Login extends React.Component {
  render() {
    return 
  }
}

Data Management

Firebase provides a cloud-based document database called Firestore. A document is a JSON object that stores data in key-value pairs. Like other popular document databases such as MongoDB and CouchBase, Firestore lets you create and manage collections (groups of related documents like customer and sales data). In the following example, a user creates a product that manages a collection of users, and locates a document with the information for a specific member.

firebase.firestore().collection("members")
.where("member", "==", "jonnygold").get().then((snapshot) => {})

FaaS Services: Cloud Functions

Cloud Functions allow developers to write code that responds to specific events. These events can be handled by any Firebase or GCP service. For example, they can be used to handle authentication and query data, send messages related to changes for specific data, manipulate files, or integrate with third-party services. In the following example, the function is triggered when a user updates their profile:

exports.updateUser = functions.firestore
  .document('users/{userId}')
  .onUpdate(event => {
    // Retrieves user document
    let newVal = event.data.data();
    // access relevant field
    let name = newValue.name;
});

The Serverless Framework

In addition to exploring serverless offerings from Firebase, Google Cloud Platform, Microsoft Azure, and Amazon Web Services, you should also check out the Serverless Framework. It was built to provide tools for deploying and operating serverless architectures. The framework is vendor agnostic. It supports all the major vendors, but is designed so that you won’t get locked into a single provider’s solution. In fact, you can start building your app on any of the supported platforms, such as AWS, and at a later stage migrate it to a different platform, such as GCP or Azure. Serverless provides a command line interface for creating and deploying applications, and lets you use the language your project uses to handle configuration and application management.

Conclusion: Onwards and Upwards

We only scratch the surface of the serverless computing revolution in this short post. There is currently a lot of marketing hype about serverless computing, but it is true that this development represents a significant change in how we build apps. It gives front end developers the power and opportunity to build what they want, when they want, and how they want.

Firebase is a good place to start, but all of the platforms more or less offer the same services, and they all offer a developer/free tier to get you started. So whatever your idea is, you now have the tools to make it a reality.

In addition, once you get started you may want to migrate your serverless application to a new platform. This is where the Serverless Framework comes in. It will help you build a serverless application that does not depend on a single vendor’s tools or infrastructure. It will let you deploy your serverless system without the vendor lock-in that typifies cloud vendors and platforms.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post A Front End Developer’s Guide for Creating Serverless Applications appeared first on Automated Visual Testing | Applitools.

Have you ever asked yourself why all apps aren’t web apps? Well, this is actually already the case on desktops and most of the apps you use at home and/or at work. When it comes to productivity apps, Microsoft, Apple, and Google all have full-featured web apps that are just as capable as locally installed desktop applications. As long as you have good wired or wireless internet, you can do nearly all of your work in Chrome, Edge, or Safari. Apart from specific categories, such as graphics, video editing, software development, and high-end gaming, you can do nearly everything in a web browser.

Mobile, however, is a different story.

Whether you’re trying to tether a laptop or use a mobile device, using a native app is preferable to using a web app. Native desktop and mobile apps have access to device hardware (such as storage) and can work offline. When it comes to mobile, it doesn’t matter whether you use the mobile versions of Chrome (Android) or Safari (iOS)—they’re pale shadows of the desktop versions, and they aren’t worth the effort.

The good news is that things are starting to change, and a new generation of Progressive Web Apps (PWAs) are changing the status quo. They’re the missing piece that will allow any browser-based app to look, act, and respond like a native app. They work offline, are able to access native device hardware (e.g., camera, storage, network, etc.), and provide a user experience that’s indistinguishable from native applications.

At this point, Google and Microsoft are fully committed to supporting PWAs across their platforms and browsers. Apple hasn’t made an official commitment yet, but there’s evidence that both desktop and mobile Safari will offer partial or full PWA functionality in the near future.

In this post, we discuss how the mobile revolution and the rise of smartphones led to the creation of PWAs. We explain what PWAs are, describe their component parts, and provide instructions for how to build and test them.

The Rise and Fall of Web Apps

Until 2007, it seemed like the rise of web apps was unstoppable, but then the iPhone changed everything. Google quickly followed Apple’s example and launched Android. Both platforms came with a built-in browser that did a reasonable job of rendering web pages. Due to security and performance concerns, the browsers had limited access to a device’s hardware. Developers voted with their feet—they preferred writing mobile versions of their apps using each platform’s native SDK and distributing them via app stores.

During the rise of native apps on mobile devices, it appeared that web apps were becoming irrelevant at best, and obsolete at worst. Desktop browsers and web apps may have been overshadowed, but in this period, a number of new developments made the web browser the platform of choice for app development and delivery. Ironically, given that we think of Apple as the iPhone company, we forget that they were instrumental in creating the web standards that resulted in new and better web apps.

Following the launch of Safari on the Mac, Apple created the WebKit rendering engine. In 2008, Google adopted WebKit and started to implement the new web standards in their Chrome browser. Chrome enabled developers to deliver web apps that could access device hardware such as storage, camera, and location. Recently, Google has been adding these features to the latest versions of mobile Chrome on Android and Chrome OS. Alex Russell, a Google Chrome team member, coined the term Progressive Web App (PWA) to define these new developments.

What is a Progressive Web App?

On Google PWA homepage, a PWA is described as a web-based user experience that’s reliable, fast, and engaging. Highlighting these three qualities draws attention to the fact that many web apps are unreliable, slow, and not engaging, which has the potential to tar PWAs with the same brush.

When it comes to reliability, a web app is only as reliable as its network connection. A bad connection means that a web app will either load slowly or not load at all. So PWAs should load quickly and reliably even when network connectivity is uncertain. If network connection is lost during a session, you should be able to rely on the PWA to save data and any changes you made.

You could say that reliability and speed are two sides of the same coin. This is partly because they’re dependant on sending requests to and receiving responses from remote servers. It’s also because many front end frameworks that are used to create web applications perform less efficiently than similar native frameworks.

A PWA that’s reliable and fast will engage users and ensure that the PWA user experience feels as natural as using a native app. As far as the user is concerned, a PWA is no different from a native app, and it should provide a shortcut icon that allows launching from the home screen. PWAs are progressive because, no matter what limitations a browser has, they can be progressively enhanced to include any missing functionality.

In addition, PWAs must have the following characteristics: 

Discoverable

What use is writing a great app if no one can find it? And once you’ve found it, how do you install it? There are many sites that provide curated lists of available apps (e.g., PWA Rocks). In the near future, PWAs will also be available from app stores. 

To install a PWA, you must locate and download the relevant assets to your device. In order to find and install these assets, developers create an app manifest. The manifest is a JSON file that stores web application metadata and related information. This includes its name, links to assets (e.g., icons, images, video), the preferred URL to launch or open the web app, and configuration settings (orientation, full screen, etc.). The browser uses the information to provide support for device-native services and hardware.

Provide Offline Support

One of the biggest problems with using the mobile web is that it must be connected to the internet. In order to provide a completely app-like experience, a PWA must be available and work offline. PWAs provide offline support via service workers, which are scripts that a web browser runs separately from and in the background of a web page. The service worker sends requests and caches the responses. In other words, a service worker doesn’t just store application data in random access memory; instead, it persists this data to the device’s file system or the browser’s internal databases. In the event that the browser’s internet connection fails, it can continue to operate offline. Service workers can also provide support for push notifications and updates.

Secure

The HTTP protocol enables you to pass unencrypted data over an internet connection. Since the world is a dangerous place and PWAs will be used to handle sensitive personal and financial information, it’s better to be safe than sorry. PWAs ensure security by being served over HTTPS. While you technically can create a PWA and serve it via a non-secure HTTP, using HTTPS provides an extra layer of protection for sensitive user data.

Building and Testing Your First PWA

The first step for creating a PWA is writing a manifest file. Next, write a service worker for caching pages and handling offline data. The last step involves integrating the service worker into your application. This requires writing code that registers and downloads the service worker. For a detailed, step-by-step guide, see this tutorial from Google. If you want to convert an existing web app, see this guide from Chris Love. Another approach is to use an online service, such as the Manifold Online PWA Builder. 

As with any web or mobile application, it’s not enough to just write the code—you also need to test it before it’s released for users. A PWA is a web app, so its elements can be tested using existing manual and automated testing techniques. Additional tests will be required to measure online and offline performance and presentation. For a comprehensive list of PWA features and functionality, you should test, see Google’s PWA checklist. 

PWAs are also excellent for visual testing. For example, you can test your PWA using Applitools Eyes. Applitools Eyes allows you to define a visual baseline of your app. If a test run detects differences between the expected/baseline image and the actual image that’s detected in the test, the test is failed. Oftentimes, the difference doesn’t come from an issue or a defect but is instead caused by a newly introduced feature or cosmetic change. 

Conclusion: The Best Is Yet to Come

Over the last ten years, we’ve witnessed an impressive evolution of the mobile web. In that time, we’ve progressed from primitive, barely usable mobile browsers running on features and pre-iPhone smartphones to today’s highly capable mobile browsers that run on phones and tablets.  

PWAs are the next step in the evolution. They will blur the lines between mobile and native development. While developing and deploying mobile web apps may be easier, the issue of testing mobile apps becomes increasingly important. To ensure the best possible experience for your users, the case for using visual testing tools like Applitools Eyes has never been stronger.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Progressive Web Apps: Building Compelling Apps with the Latest Mobile Web Technology appeared first on Automated Visual Testing | Applitools.

It is probably safe to assume that any developers reading this article are already testing their applications. At the bare minimum, you’re testing the feature you’re working on as you develop it and checking that it works with any existing code. Many of you are probably using some form of automated functional testing as well. These types of testing are certainly important and they help to ensure that your applications work as expected, but adding visual testing into the mix will improve your testing significantly.

Today, users access applications in a variety of ways, including wearables, smartphones, tablets, laptops, desktop PCs and an increasing range of smart devices. Now, developers need to account for the ways in which these new and diverse technologies change how their software is presented to and perceived by their users. The typical forms of testing we’ve relied upon for so long simply can’t detect the problems that may exist on every possible device. This is why developers now need to test the visual aspects of their applications. In recent years, developers have started using visual testing methods, tools, and frameworks to meet these new challenges.  

In this post, we make the case that it’s wise to use visual testing. Even if you, your team, or your organization are old hands at automated functional testing, we’ll explain why you should add visual testing into your existing infrastructure. We also offer advice about available tools, frameworks and solutions that will help you get started. Whether you’re a novice developer or a seasoned veteran, the information presented in this post will help you deliver the best experience for your users.

What Is Visual Testing and Why Is it Important?

We are taught from a very early age that looks shouldn’t matter. Most developers test their code with a similar assumption—that what matters is how the code works, not how it looks. Visual testing verifies that the user interface appears correct to its users.

Even though you invested many hours to make your application work and wrote tests to prove that your code functions as expected, and even if everything appears to be working on your end, none of that matters if the application isn’t working properly for your users. Don’t assume that your tests see exactly what your customers do. At best, your tests are a crude approximation of a real human user’s interactions with your software.

The developer’s job is to solve real problems for real people with their software. If developers do a good job, users will pay for their product. Problems that appear to be superficial or cosmetic on your end can make it difficult—or even impossible—for users to achieve their desired end results. From their perspective, even if the reason for the problem is out of your control and is really being caused by the way a certain device is rendering your application, it’s still broken.

Imagine that you created a new insurance company website. The company wants to improve the user experience by letting them file claims online. They believe that their existing paper-based system is inefficient and costly, and that the new system will both attract new customers and reduce operational expenses. You were responsible for creating the online claims form. Your automated developer tests and internal tests in many designated sites showed that the system works so well that when the site goes live, the old paper-based system will be shut down.

When the system went live, the insurance company’s phone lines were immediately overwhelmed by irate customers. They all had the same complaint: they were trying to file a claim, they entered all of the relevant information, but they can’t find a button to submit the form. You investigate the problem and you find that when your form is displayed in a mobile browser, the button that should be visible is being obscured by a large text box because of the device’s space restrictions. This problem did not show itself in your testing, but when the system went live, the users were let down by your software.

This example may seem exaggerated, but it’s an entirely plausible situation. It highlights one of the key reasons for visually testing your software: even if your current tests show that the software is functioning as intended on your device/s, it must also run as expected on the users’ devices. You need to check that the software functions as expected on all possible target devices your users have. This involves testing individual page elements and checking that each one appears in the right shape, position, and size.

By using visual testing combined with your existing developer tests, you’ll be able to detect visual issues in the early stages of your development cycle. Visual testing factors in how different environments, operating systems, devices, and sizes will affect your software.

How Visual Testing Works

Today, there are more than 30 open-source tools and commercial visual test automation tools available. Most of these tools work the same way, but they follow a different workflow depending on the target audience. For front end web developers, this will involve using two types of frameworks. First, you need a test runner that enables you to write and run the tests. Second, you need a browser automation framework to simulate user interactions. For the sake of brevity, we’ll treat these two types of frameworks as a single unit and refer to them as (visual) test code.

Using the test frameworks, developers write code that simulates real user actions, such as typing text into a field or clicking a button. At relevant points in the test code, they’ll insert commands that capture a screenshot. The first time the test is run, an initial set of screenshots is captured. These screenshots serve as a baseline against which all changes are compared.

Once the baseline has been established, the developer runs the test code in the background. When a change is detected, an image of the change is captured. The test runner compares this image to the baseline image for this area of the code. If differences are detected between the images, the test is failed.

When all of the test code has been run, a report is automatically generated. A human tester reviews all of the images in the report that are different than the baseline. Many testing tools will generate an image that highlights the differences between the expected baseline and actual image captured in a test run. The image highlights all of the differences in pink. If the difference was caused by a bug, the developer can fix the bug and rerun the test to check that the bug was, indeed, fixed. If the difference was caused by a real change in the UI, the developer can review the screenshot and update the baseline so that future test runs will be passed.

Getting Started with Visual Testing

We can’t cover all of the available tools for visual testing in this short post, but here’s a brief overview of what you need to get started.

As we previously mentioned, the first thing you’ll need is a test runner, which will let you write tests to be executed by machines. Test runners, such as Jest or Mocha, let you write Javascript code to describe and organize testing tasks. Each test includes assertions that define a condition that either passes or fails. Tests can be organized into blocks, and these blocks are often put into a single file that is used to test a specific module and/or application functionality.

You will recall that once you have a way to write and run tests, you need a tool to interact with the browser. Tools like Puppeteer, Webdriver.io, and Cypress.io provide a headless browser that allows your test to perform the same interactions as a human user. They also support visual testing and can generate screenshots and PDFs of web pages.

You will also need a tool that lets you manage the testing process. Currently, there are a number of companies providing web-based solutions in this space. The best place to start is with Applitools Eyes. To effectively manage visual testing, Applitools manages your baseline images and the potentially large number of images generated during each test run. It generates reports that not only show you what passed and what failed, but that also give you useful insights from current test runs and from all previous test runs.

Any visual testing solution must also help you effectively maintain your baseline. In addition, it must ensure that any misleading information is removed from specific test runs, but that it isn’t propagated in a way that leads to information overload or hides potentially damaging issues. Applitools checks the boxes for all of these qualities and more.

Take Your First Steps Toward Visual Testing

Visual testing can’t replace your current testing practices, tools, or infrastructure, but it can help you test areas that you’re currently unable to with functional testing alone. In today’s world, where you don’t know where and how your software will run, visual testing will help you discover problems that matter to today’s users.

There are many tools that can help you get started with visual testing. Applitools Eyes is an excellent option for managing your visual testing process. In addition to the features that were previously mentioned in this post, Applitools is designed to handle the many labor-intensive tasks and obstacles that exist with other tools. Considering the many benefits of visual testing, and with the help of accessible, web-based tools like Applitools, your organization should be ready to take its first steps into the world of visual testing.  

— HAPPY TESTING —

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post Visual Testing: A Guide for Front End Developers appeared first on Automated Visual Testing | Applitools.

The agile development movement is making testers shift their tests to the left. From a practical perspective, this means that the testers are getting closer to the frontend developers, and the testers’ automated tests need to be integrated with the developers’ tests (I discussed this in considerable length here, if you’re interested).

This is interesting. The shift-left movement now ties together the tester’s tests and the developer’s tests. This begs the question — should the tests be written in the same language?

I will try to convince you that answering yes to that question has many benefits to both sides.

Writing tests in a different language than the developer

When the tester writes their tests in a different language than the developers do, what naturally happens is that a barrier dividing the two tests exists. This barrier exists because:

• The developer will not want to add tests to the tester’s tests.
• The tester will find it difficult to both help with the developer tests and to add to them.
• The developer and tester can’t share code and processes between them.

This is totally natural and would happen even between two developers on the same team that speak different languages. But in an agile company, the testers and developers should be working together as close as possible in order to deliver the software. And this barrier impedes this.

If the company is not agile, the testers are in a separate team, and receive the “release” to ensure it’s quality, test it, and create a set of issues in the issue tracker for the developers to fix, build, and create a new release for the testers to test. In this configuration, it doesn’t really matter whether the teams are speaking the same language.

Writing tests in the same language as the developer

But in an agile company, this barrier, this wall, should not exist if the company wants to stay agile. And the first thing you need to do to enable communication between developers and testers, is to have them speak the same language.

(For the remainder of the post, I will focus on developing web applications, and ignore the other types of applications).

So what languages should the tester use? What languages do the developers use? This depends on whether the developer is coding in the backend or the frontend.

Backend developers use a multitude of languages, but the mains ones are: Java, C#, PHP, Ruby, and Python (not necessarily in order of popularity). For frontend developers, things are much simpler: they all use JavaScript! (And small variants like TypeScript.) There really are no other mainstream choices.

So if a tester needs to choose between a language to write the tests in, which one should they choose? Should they choose the language the backend developer uses (e.g. Java) or should they choose the frontend choice, which is JavaScript.

The answer up to a few years ago was simple — there was no way one could write tests in JavaScript, so the tester has to choose the backend language. (e.g. Java or C#)

Using Javascript as a testing language

But today there is an alternative — NodeJS — that is extremely popular and has multiple testing frameworks to choose from. You can write tests in JavaScript, and as we will soon see, it’s actually rather easy.

So now that we have the privilege of choosing the frontend developer’s language or the backend developer’s language, which shall we choose? And the answer, in my opinion, is to choose the language that is closer to the code you are testing.

If you are testing a web service using a REST API, then the code you are testing is backend code, and should be tested using the language the developers use for backend development, but if you are testing the web app’s UI, then the code you are testing is frontend code, and should be tested using JavaScript.

While there is a ton of information out there for testers on how to write tests using Java, there isn’t a lot of information about writing tests in JavaScript. Well, actually, there is, but it is all geared towards developers. But the situation is getting better! Not because there are more articles about testing in JavaScript for testers, but because increasingly developers are doing more and more tests that are UI automation tests.

This can be evidenced by the amount of UI automation frameworks out there for JavaScript. To name a few: Selenium WebDriver, Puppeteer, Cypress, NightmareJS, WebdriverIO. And this is just a small subset.

Advantages of using JavaScript as a testing language

Deciding to invest in JavaScript will bring many advantages. Let’s name a few of them:

First and foremost, the ability to work with the frontend developers:

• The tester tests can side by side with the developer tests. Green for the application, even for the developer, is only after both sets of tests pass.
• If the developers change something that fails the tests, the developers themselves can change the test to conform to these changes.
• The testers can aid the developers in writing unit and end to end tests.
• If the tester needs to test something, and the test is more easily written as a unit or integration test, the tester can easily do that.
• When a project becomes big, the test code also becomes big, and a lot of it is actually shared libraries that aid in testing. Now, these shared libraries can be shared between the developer and the tester.

But there are advantages in using JavaScript itself:

• JavaScript has the largest repository of ready-made packages out there. Bigger than Java’s maven repository, and much bigger than Python’s pip and Ruby’s Gem. You can find today everything you need there.
• The current version of JavaScript (ES2017) is tremendously useful. This is not your grandparents’ JavaScript. It has most of the things you would want in a modern language. And, most important of all, the dreaded callbacks, which made JavaScript a language to be avoided by many, are gone — you don’t need to use callbacks anymore…
• Because it is mostly a frontend language, the amount of frontend testing tools is much bigger than in any language.
• The JavaScript community is at least as large (if not larger) than communities like the Java or Python community, and very welcoming. You could probably find answers to all of the questions that you ask.
• Many backend projects are now written using NodeJS, which is a JavaScript runtime, so you could find yourself using the same language when writing backend tests.

I’m convinced! How do I try it?

Yes! Let’s write a simple test. We’re going to write a test for the Todo list application that you can find here. And hopefully, I’ll convince you that it’s actually pretty easy.

First, install NodeJS. We need version 8 or later. You can find download links here: https://nodejs.org/en/.

Next, we’ll create a directory for our project:

mkdir todo-tests && cd todo-tests

Now we’ll initialize our project:

npm init -y

This creates a package.json with information about our project’s dependencies.

Now let’s install the dependencies we need for our test:

npm install --save-dev jest selenium-webdriver chromedriver

(this might take a minute)

Now let’s write the test. Create a file named todo-list.test.js:

View the code on Gist.

That’s it. That’s all the code we need to test that the todo list runs, and shows the “todos” header.

Let’s run it:

./node_modules/.bin/jest

And… the test passes:

Cool…

Adding Visual Assertions

But we’ve just checked the “todo” title. We didn’t really check the whole page. We could go over each element in the page, to ensure that it is correct. Alternatively, we can use visual assertions (to understand what visual assertions are, read Applitool’s “Visual Testing as an Aid to Functional Testing” post). Let’s do that.

First, we’ll add theApplitool’s Selenium Eyes SDK to our project:

npm install --save-dev eyes.selenium

Let’s modify the code a bit. We will add initialization code for Eyes, and one line to visually assert that the page is the same. The final code is this:

View the code on Gist.

To use the Eyes SDK, you need an API Key, which you can get by registering at the Applitools site, here. Once you have that, log in and go to the Eyes admin tool, here. You will find the API using the following menu:

To make Eyes Storybook use the API key, set the environment key APITOOLS_API_KEY to the key you have:

// Mac, Linux:
$ export APITOOLS_API_KEY=your-api-key

// Window
C:\> set APITOOLS_API_KEY=your-api-key

Now we can run the test again:

./node_modules/.bin/jest

And… it passes!

With this test, we checked all elements in the page and not just the todo.

Summary

An agile company should shift the testing process left, to be close to the code. To do that, testers need to be “close” to the developers and to the code they’re writing. To do that, it would be beneficial to both testers and developers that they both spoke the same language.

In the frontend world, that language is JavaScript. Fortunately, JavaScript as a language and platform has advanced tremendously and is a wonderful language to write tests in. In a few lines of code, one can write tests that compete in clarity and conciseness with the best Java, C#, and Python code out there.

So if you are a tester wanting to make the move to the left, consider using Javascript as the language to write your tests in.

To learn more about Applitools’ visual UI testing and application visual management (AVM) solutions, check out the tutorials on the Applitools website. To get started with Applitools, request a demo, or sign up for a free Applitools account.

The post How Testers can Speak the Language of Frontend Developers appeared first on Automated Visual Testing | Applitools.

In a previous blog post of mine “Embracing Agility: A Practical Approach for Software Testers”, I discussed how an agile team changes the role of the software tester. Instead of being a gatekeeper that tests the product after it is developed, the software tester is part of the development process.

I also described how the shift-left movement enables the software tester to concentrate on the more interesting tests, and leave the boring tests (“this form field can only accept email addresses”) to the software developer. But those interesting tests, at least the ones we decided to automate, need to be written, and written quickly.

In this blog post I will discuss a technique whereby you can use visual UI testing to make writing those tests much easier. 

Visual UI Testing vs Functional Testing

Visual tests are usually used to check the visual aspects of your application. Makes sense, no? To automate tests that verify that your application looks OK, you write visual tests, and use visual testing tools such as Applitools Eyes. But visual testing tools can also be used for functional tests.

Really? How can visual testing help us write functional tests, tests that check how the application behaves? First, let’s understand how visual testing and functional testing of applications work.

Functional Testing

A functional test (in this blog post! I am sure that you can find other people that define it differently) is a test that starts an application in a certain state, performs a set of actions, and tests that the application is in the correct state after those actions.

So a functional test looks like this:

1. Load a specific set of application data
2. Start the application
3. Execute a set of actions on the UI (or the REST API!) of the application
4. Assert that the actual state of the application is the expected one

Yes, you might argue that it is not the test that starts the application, and that the test should always create the specific set of data using other actions. You could, but that’s bikeshedding. True, no functional tests look exactly like this, but the idea is correct:

• action, action, action
• assertion, assertion, assertion
• action, action, action
• assertion, assertion, assertion

Let’s take an example of a to-do list application:

One functional test would be:

• Action: check a todo
• Assertion: it is checked in the “All” tab
• Assertion: it is gone from the “Active” tab
• Assertion: it is checked in the “Completed” tab

Implementing functional assertions

How would one go about implementing the assertions? If it’s a web application, and we’re using a browser automation library like Selenium Webdriver, then the tester would need to check whether the specific todo’s checkbox is checked, and also check that the other checkboxes are not checked. Also, check that the text of the todo is striked out, while the others are not striked out.

Oh, and don’t forget to check the “2 items left” text in the bottom left of the list.

And each such check is a pain in the butt — the tester needs to determine what the id of the control they are checking is. And sometimes there is no specific id (or class), and so the tester needs to go to the developer and ask them to add an id for testing, or if the tester is in an agile team, maybe add it themselves!

Also: this needs to be done across hundreds of tests, and thousands of assertions. This is practically impossible, and definitely impossible to maintain. What usually happens is that the developer chooses a subset of assertions, and tests just those. In our example, it would only assert that the checkbox of the todo item is checked, and forego the other tests.

Visual Testing

We’ve seen what functional tests are, and one of the problems associated with them. Let’s move on to visual tests: a functional test (in this blog post! I am sure that you can find other people that define it differently) is a test that starts an application in a certain state, performs a set of actions, and tests that the application looks good after these actions.

See how similar they are? A visual test looks like this:

1. Load a specific set of application data
2. Start the application
3. Execute a set of actions on the UI (or the REST API!) of the application
4. Assert that the application lookscorrect

This is exactly like functional tests, except that the assertions are different. So let’s look at how assertions are performed in visual tests. How does a visual test go about testing that an application looks correct?

Implementing visual assertions

How do visual assertions work? Well, first of all, we take a screenshot of the screen — we are doing a visual assertion after all. But this screenshot, alone, is not enough. What we do is compare this to a baseline screenshot that we took in a previous run of the test. If the algorithm comparing the current screenshot to the baseline one tells us that they’re the same, then the assertion passes, otherwise it fails.

And how do we generate the screenshot? In a two phase process:

1. We define the screenshots that are created by the assertion of the first test the baseline screenshots. The tester manually checks each one of those baseline screenshots.
2. Each time the test failsit could be because there is a bug in the software, but it could also be because the software changed and this is the new The tester in this case manually checks each one of those new screenshots and declares them as the new baseline.

How does visual comparison work?

Unfortunately, visually comparing two screenshots seems easy, but it’s not. Theoretically, it should be as easy as comparing pixels, but this, unfortunately, is not the case. Different operating systems, browsers, even different versions of the same browser generate slightly different images. Moreover what if the screenshot includes the current time? You would need to check everything except that region of the screen.

There are even more concerns, which we should outline in a future blog post. Fortunately, tools like Applitools Eyes hide us from these concerns.

Using Visual Assertions in Functional Tests

Now that we understand how functional testing works, we understand that there’s no difference between functional and visual tests in how they are structured — the only difference is in what they test and in how they implement the assertions:

• Functional assertions check each and every control on the page to ensure that they have the correct value
• Visual assertions compares a screenshot of the page against a manually verified baseline screenshot.

Visual assertions seem pretty easy to use. Just manually verify the first screenshot, and just take screenshots of the pages after the actions in the test. No need to check each control, determine the id of the control, or ask the developers to add ids to controls. Just take a screenshot and let visual testing frameworks like WebDriverCSS or Applitools Eyes compare them.

And we’ve seen how functional assertions are hard. But what if we can use visual assertions in functional tests? Something like this:

1. Load a specific set of application data
2. Start the application
3. Execute a set of actions on the UI (or the REST API!) of the application
4. Take a screenshot to verify that the data is correct

Think of the todos page after checking the todo item:

With one screenshot, we can check that the check was checked (ouch, too many “check”-s), verify that the other todo items weren’t, that the “2 items left” is correct, that the todo item is in strike-out, and probably other stuff which we didn’t even notice.

Many front-end developers use this kind of “testing against a baseline”, but what they test is the HTML of the components, and not how they look like. But the idea is similar. The difference is that it is much nicer to check the difference from a baseline between two images, than to compare a textual diff of HTML.

Pleasant side benefit of using visual assertions: you get to check the visual look of your application for free!

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Visual UI Testing as an Aid to Functional Testing appeared first on Automated Visual Testing | Applitools.

The Agile Manifesto is 16 years old. In the home page, it defines a very simple set of values:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

These four simple values have sparked a revolution in how we write and deliver software. I like to call them the “cut the bulls**t” values. To understand how these values affect testers, let’s dispense with the ceremony, let’s dispense with lengthy design documents, and, yes, lengthy test plan documents, and focus on understanding the core — writing the software.

And the way to do that is to deliver working software from the start (this is the “Working software over documentation” part of the manifesto). The agile manifesto asserts that the product needs to work from the beginning, and not wait for a lengthy and cumbersome “integration phase” when all the work on the product is integrated together. Instead of many team silos, where everybody is working on one piece of the puzzle (and the architect is the only one aware of how everything will be pieced together), we start with a small team that builds an initial working software (MVP, or minimal viable product) and gradually grows it to a bigger product with a bigger team.

This team includes everybody — the customer, product manager, developers, and testers (this are the “Individuals and interactions” and “customer collaboration” values of the agile manifesto). This integrated team delivers a minimal version of the product in a very short time, and… waits. Waits for what? For the customer to start working on the product and figure out the next step for the product. In other words, agile teams don’t build a specification for the whole product, but rather start small and incrementally determine how the product should look like (this is the “Responding to change over following a plan” part).

Yeah, yeah, yeah. Bla, bla, bla. I got that. But how does it affect my work as a software tester? It does. Immensely. I have one sentence for you:

Embrace agility: shift left and shift right.

But, unfortunately, to explain this, and to see how this affects your work as a software tester, I need a bit more explanations, so hang in there.

Testing in the Days of “Release Early, Release Often”

The agile manifesto also says: “Deliver working software frequently”. And today’s software development community has taken it to the extreme. Some companies release every week, some every day, some don’t even have the concept of “release”. They just release anytime a feature is ready.

Moreover, agility forced the backend to come up with an alternative to the “monolithic backend”. The concept of “microservices” has replaced the “monolith”. This concept enables large companies to break up their backend into many small services, each one responsible for a small piece of the puzzle, and to deploy each microservice independent of each other.

These two things means that in many companies there is no “release”. It’s a fluid set of changes that just happen to the product in ongoing process.

But if there is no release, how will software testers test? I have one sentence for you:

Embrace agility: shift left and shift right.

Shifting Left

What does shifting left mean?

The software development software, in the days before the process became agile, looked like this:

Software testers used to be at the right of this process. They used to be the gatekeepers that bar the gate to releases that are not qualified for deployment.

This process actually hasn’t changed much in the days of agile, except that now we have many of these pipelines in parallel, and each takes days, and not months:

But how can a software tester be a gatekeeper in this model? It’s practically impossible. Once the implementation is done, there is no “time” for testing, as the team has gone on to the next thing it needs to do. In the agile model, delivering software incrementally is paramount. Time for testing is not included.

To solve this, the tester needs to shift the testing left. It needs to be part of the implementation, and not a separate step:

This has many practical consequences, consequences that affect you as a software tester:

Software developers need to test too

Shifting left means that a lot of the burden of testing needs to be dealt with by the software developer and not by the tester. A tester should not need to check that a certain form field needs to be numeric. A tester should not need to check that the value in this table is correct. All those boring tests that occupy a lot of the tester’s time need to sit “close to the code”, and should not be dealt with by the tester. The software developer can use unit testing to check those boring checks in much faster and more reliable manner than the tester, and free up the tester’s time for the more interesting and holistic tests, whether manual or automated.

Automated tester tests need to sit close to the code too

As discussed above, agile software is now deployed piecemeal-wise, and not in one big deployment bang. This means that the software testers tests need to test not just the whole big product, but also parts of it.

This means that if the microservice your team is writing is a web service that communicates via REST API, then your tests should check that REST API in isolation from the rest of the product. And if your team is building UI components that the other parts of the company uses, those components should be tested in isolation from the rest of the product.

In other words, software testers should not only do end to end tests, but should also check parts of the software, and not the whole thing. Which parts? A good rule of thumb is that if your team is building components and microservices that are part of a bigger whole, you should test those components in isolation and not only as parts of the bigger whole.

Automated tester tests need to be part of the CI process

Software developers that operate as part of a true agile process frequently use CI tools to help them cope with the fantastic pace of software development. For software developers, the CI tool isthe gatekeeper for their software change.

What is a CI tool? CI means Continuous Integration, and a CI Tool is a tool that listens to changes in the source code, builds it, runs the tests, and if the tests pass, mark the changes “green”. If they fail, they usually email the person that submitted the software changes. The CI tool helps the software developer ensure that all the tests they write are executed for those specific changes. Examples of CI Tools are Jenkins, TeamCity, Travis, Bamboo.

These are fantastic tool, and are definitely part of the shift-left movement, as the tests runs for every code change. A software developer should ensure that not only do the software developer tests run, but also their own tests. A code change needs to be green only if the developer tests pass and the tester tests pass.

The three rules above are, for me, the essence of the shift-left movement. To be agile, automated tests should not be a separate stage of the development, but should be part and parcel of it. And to implement this should be the responsibility of both the tester and the developer.

And this responsibility brings us to what is for me, the holy grail of the shift-left movement:

Automated tester tests need to be part of the source code

There should be no difference between where the developer tests reside and where the tester tests reside. They should be part of the same source code, and as we said above, run as part of the same CI process. I will dedicate a blog post in the near future to this important idea (“Speaking the language of frontend developers”).

Remember, in an agile process, the tests become as important and integral as the development itself. Embrace agility.

Shifting Right

But if the shift-left movement means that a tester does not check the whole product holistically, then does that mean that it does not need to be checked as a whole?

Of course it has to be checked as a whole. The big question is when? If there is no release, then when do we test the whole product? The answer may surprise you and is part of the shift-right movement.

And the answer is — whenever you want to. Once there is no release, and software changes keep pouring out of the pipe, and they are well-tested due to the shift-left movement, then you can do E2E testing whenever you want to.

Run your end to end tests continuously

If you can run your end to end tests whenever you want, then just run them continuously. Yes, source code keeps pouring in from the left, and deployments keep pouring out from the right, but that doesn’t mean that we should give up and not test. It actually means that we should run them all the time.

No, we will never be testing what is exactly going to be in production in an hour. We will always be testing an approximation. But if this run of e2e tests doesn’t uncover the bug, the next one will. And, yes, it will be too late, because that bug is already in production, but that’s OK — because we’re agile, fixing the bug also takes a very short amount of time.

In an agile world, it’s OK for bugs to sometimes reach production. This is what is meant by Facebook when they say “Move fast and break things”. While an extreme sentence, it does hold the important truth that trying to stop all bugs from reaching production doesn’t help development, it harms it.

Run your end to end tests in production

Some companies take this to the extreme, and have software testers test the full product not in a testing or staging environment, but actually in production, after it’s released! (this is why it’s called “shift right”). And incredibly enough, this is not as dumb as it sounds. Testing in production enables us to uncover bugs that are due to:

• bad data that can only be found in production
• the scale and amount of people using the application in production
• the deployment system used for production
• and bugs that are in the infrastructure (servers, networks, database, etc) used in production

Embracing Agility

So yes, embrace the agile manifesto. Agility changes and amplifies the role of the software tester. The shift left movement encourages the software developer to take on the role of the tester, and encourages the software tester to participate in the software development process, while the shift right movement encourages the software tester to take on some of the responsibility of monitoring the application.

And this is a good thing — the tester should not be a gatekeeper, a judge that is aloof and separate from the software development and deployment process. The tester should not be the bad guy saying, “thou shall not pass” and delaying the release of the process.

What the tester should be is an integral part of the software development process. And embracing agility allows them to do that.

Yes, But How?

Embrace agility. Shift left:

• Software developers need to test too: shifting tests to the left means that software developers should test their code too. Educate your consumers. Work with them to make this happen. Fortunately, ongoing education is moving software developers to this understanding.
• Automated tester tests need to sit close to the code too: stop thinking as gatekeepers and start thinking as developers. Test parts of the code and not just the final product.
• Automated tester tests need to be part of the CI process: the CI tool is the new gatekeeper, so your tests should be part of the CI.
• Automated tester tests need to be part of the source code: if you can, have your tests be part of the source code, and not a separate piece.

Embrace agility. Shift right:

• Run your end to end tests continuously: there is no release. Continuously test the “current” product, no matter whether it is not yet in production or already there.
• Run your end to end tests in production: there is no release. The best place to run your tests is in production, and you will get a ton of added benefits from it.

What’s in it for Applitools?

In the end, I am a developer advocate for Applitools, a company that builds a Visual UI Testing platform for testers and developers. So, what’s in this movement for Applitools?

Plenty. We believe visual UI testing tools are crucial for shifting left and shifting right. But that’s for another future blog post.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

The post Embracing the Agile Manifesto: A Practical Approach for Software Testers appeared first on Automated Visual Testing | Applitools.