GitHub Archives - Automated Visual Testing | Applitools

AI-Powered Test Automation: How GitHub Copilot and Applitools Can Help

Applitools Team — Tue, 22 Aug 2023 21:23:00 +0000

Test automation is crucial for any software engineering team to ensure high-quality releases and a smooth software development lifecycle. However, test automation efforts can often be tedious, time-consuming, and require specialized skills. New AI tools are emerging that can help accelerate test automation, handle flaky tests, increase test coverage, and improve productivity.

In a recent webinar, Rizel Scarlett and Anand Bagmar discussed how to leverage AI-powered tools like GitHub Copilot and Applitools to boost your test automation strategy.

GitHub Copilot can generate automated tests.

By providing code suggestions based on comments and prompts, Copilot can help quickly write test cases and accelerate test automation development. For example, a comment like “validate phone number” can generate a full regular expression in seconds. Copilot also excels at writing unit tests, which many teams struggle to incorporate efficiently.

Applitools Execution Cloud provides self-healing test capabilities.

The Execution Cloud allows you to run tests in the cloud or on your local machine. With self-healing functionality, tests can continue running successfully even when there are changes to web elements or locators. This helps reduce flaky tests and maintenance time. Although skeptical about self-healing at first, the experts found Applitools to handle updates properly without clicking incorrect elements.

Together, tools like Copilot and Applitools can transform your test automation.

Copilot generates the initial test cases and Applitools provides a self-healing cloud environment to run them. This combination leads to improved productivity, reduced flaky tests, and increased coverage.

Applitools Eyes and Execution Cloud offer innovative AI solutions for automated visual testing. By leveraging new technologies like these, teams can achieve test automation at scale and ship high-quality software with confidence. To see these AI tools in action and learn how they can benefit your team, watch the full webinar recording.

The post AI-Powered Test Automation: How GitHub Copilot and Applitools Can Help appeared first on Automated Visual Testing | Applitools.

Our Best Test Automation Videos of 2022 (So Far)

Danny Shain — Fri, 20 May 2022 21:07:55 +0000

We’re approaching the end of May, which means we’re just a handful of weeks the midpoint of 2022 already. If you’re like me, you’re wondering where the year has gone. Maybe it has to do with life in the northeastern US where I live, where we’ve really just had our first week of warm weather. Didn’t winter just end?

As always, the year is flying by, and it can be hard to keep up with all the great videos or events you might have wanted to watch or attend. To help you out, we’ve rounded up some of our most popular test automation videos of 2022 so far. These are all top-notch workshops or webinars with test automation experts sharing their knowledge and their stories – you’ll definitely want to check them out.

Cross Browser Test Automation

Cross-browser testing is a well-known challenge to test automation practitioners. Luckily, Andy Knight, AKA the Automation Panda, is here to walk you through a modern approach to getting it done. Whether you use Cypress, Playwright, or are testing Storybook components, we have something for you.

Test Automation with GitHub or Chrome DevTools

GitHub and Chrome DevTools are both incredibly popular with the developer and testing communities – odds are if you’re reading this you use one or both on a regular basis. We recently spoke with developer advocates Rizel Scarlett of GitHub and Jecelyn Yeen of Google as they explained how you can leverage these extremely popular tools to become a better tester and improve your own testing experience. Click through for more info about each video and get watching.

Make Testing Easy with GitHub

For more, see this blog post: Using GitHub Copilot to Automate Tests.

Automating Tests with Chrome DevTools Recorder

For more, see this blog post: Creating Your First Test With Google Chrome DevTools Recorder.

Test Automation Stories from Our Customers

When it comes to implementing and innovating around test automation, you’re never alone, even though it doesn’t always feel that way. Countless others are struggling with the same challenges that you are and coming up with solutions. Sometimes all it takes is hearing how someone else solved a similar problem to spark an idea or gain a better understanding of how to solve your own.

Accelerating Visual Testing

Nina Westenbrink, Software Engineer at a leading European telecom, talks about how the visual time to test the company’s design system was decreased and simplified, offering helpful tips and tricks along the way. Nina also speaks about her career as a woman in testing and how to empower women and overcome biases in software engineering.

Continuously Testing UX for Enterprise Platforms

Govind Ramachandran, Head of Testing and Quality Assurance for Asia Technology Services at Manulife Asia, discusses challenges around UI/UX testing for enterprise-wide digital programs. Check out his blueprint for continuous testing of the customer experience using Figma and Applitools.

This is just a taste of our favorite videos that we’ve shared with the community from 2022. What were yours? You can check out our full video library here, and let us know your own favorites @Applitools.

The post Our Best Test Automation Videos of 2022 (So Far) appeared first on Automated Visual Testing | Applitools.

Everything You Need to Know About Integrating Applitools with GitHub

Daniel Levy — Wed, 16 Jun 2021 20:10:57 +0000

Overview

Applitools is used by the world’s top organizations to help expedite time to market with Visual AI-powered functional & visual testing. Applitools easily fits into existing toolchains, turbo-charging CI/CD pipelines with AI-infused cross browser testing reducing test maintenance and slashing build times, allowing developers to test more, earlier.

With GitHub boasting over 56 million users as of September of 2020, it’s no surprise that a deep integration with Microsoft’s popular source control solution is critical to our customers’ app-dev workflows and thus success.

This article provides an overview of all the ways in which Applitools integrates with GitHub & GitHub Actions.

Use cases covered in this article:

What is UI Version Control? (read more)
Integration with GitHub via GitHub app (read more)
Integration with On-prem GitHub server (read more)
[Legacy Integration] with GitHub cloud and on-prem via direct integration (read more)
Integration with GitHub Actions (read more)

What is UI Version Control?

By enabling the Applitools / GitHub integration, you instantly unlock UI Version Control. With UI Version Control enabled, developers can easily correlate code changes with UI/UX updates for Visual UI version control across web and mobile apps. As branches are created in GitHub, Applitools UI Version Control automatically mirrors the branch and allows for a new set of baseline images. This process simplifies the automated testing of feature branches without impacting the trunk and its tests. Next, as PRs are merged back, UI Version Control automatically updates the target branch with the source branch’s baseline images while ensuring that no conflicts exist between branches, eliminating test maintenance overhead.

Integration with GitHub via GitHub app

With Eyes 10.11, Applitools now officially supports visual source control via the GitHub app. This interface requires fewer permissions than the Applitools direct integration and is thus the officially recommended integration by the Applitools product team. For step by step instructions on enabling the GitHub integration via GitHub all, visit the integration documentation.

Integration with GitHub server which is installed on-prem

Another great addition to Eyes 10.11 is the ability for GitHub on-prem servers to leverage a new proxy service to eliminate the security challenges IT teams previously encountered with the legacy “direct connect” integration. Setup is quick and easy, simply enable the integration, get the GitHub app, and install the Applitools GitHub Proxy Service.

[Legacy Integration] with GitHub cloud and on-prem via direct integration

Existing cloud & on-prem deployments who have our legacy direct integration already enabled can continue to enjoy backward compatibility.

Integration with GitHub Actions

Microsoft describes GitHub Actions as an easy way to automate all your software workflows including with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

The Applitools integration requires your Applitools API Key and for you to pass Git Commit SHA as Applitools Batch ID. Once it’s all configured, GitHub Actions workflow understands when changes are made to code, and it automatically kicks off the workflow complete with your Visual AI powered test suite.

For more information, check out the article “How Do I Link GitHub Actions with Applitools?” from Satish Mallela

Additional Links on GitHub & Applitools

Additional Source Control Integrations to unlock UI Version Control

(heart icon in header from Kiranshastry at www.flaticon.com)

The post Everything You Need to Know About Integrating Applitools with GitHub appeared first on Automated Visual Testing | Applitools.

Get a Jump Into GitHub Actions

Applitools Team — Tue, 02 Mar 2021 16:24:17 +0000

On January 27, 2021, Angie Jones of Applitools hosted Brian Douglas, aka “bdougie”, Staff Developer Advocate at GitHub, for a webinar to help you jump into GitHub Actions. You can watch the entire webinar on YouTube. This blog post goes through the highlights for you.

Introductions

Angie Jones serves as Senior Director of Test Automation University and as Principal Developer Advocate at Applitools. She tweets at @techgirl1908, and her website is https://angiejones.tech.

Brian Douglas serves as the Staff Developer Advocate at GitHub. Insiders know him as the “Beyoncé of GitHub.” He blogs at https://bdougie.live, and tweets as @bdougieYO.

They ran their webinar as a question-and-answer session. Here are some of the key ideas covered.

What Are GitHub Actions?

Angie’s first question asked Brian to jump into GitHub Actions.

Brian explained that GitHub Actions is a feature you can use to automate actions in GitHub. GitHub Actions let you code event-driven automation inside GitHub. You build monitors for events, and when those events occur, they trigger workflows.

If you’re already storing your code in GitHub, you can use GitHub Actions to automate anything you can access via webhook from GitHub. As a result, you can build and manage all the processes that matter to your code without leaving GitHub.

Build Test Deploy

Next, Angie asked about Build, Test, Deploy as what she hears about most frequently when she hears about GitHub Actions.

Brian mentioned that the term, GitOps, describes the idea that a push to GitHub drives some kind of activity. A user adding a file should initiate other actions based on that file. External software vendors have built their own hooks to drive things like continuous integration with GitHub. GitHub Actions simplifies these integrations by using native code now built into GitHub.com.

Brian explained how GitHub Actions can launch a workflow. He gave the example that a team has created a JavaScript test in Jest. There’s an existing test using Jest – either npm test, or jest. With GitHub Action workflows, the development team can automate actions based on a starting action. In this case, the operator can drive GitHub to execute the test based on uploading the JavaScript file.

Get Back To What You Like To Do

Angie pointed out that this catchphrase, “Get back to what you like to do,” caught her attention. She spends lots of time in meetings and doing other tasks when she’d really just like to be coding. So, she asked Brian, how does that work?

Brian explained that, as teams grow, so much more of the work becomes coordination and orchestration. Leaders have to answer questions like:

What should happen during a pull request?
How do we automate testing?
How do we manage our build processes

When engineers have to answer these questions with external products and processes, they stop coding. With GitHub Actions, Brian said, you can code your own workflow controls. You can ensure consistency by coding the actions yourself. And, by using GitHub Actions, you make the processes transparent for everyone on the team.

Do you want a process to call Applitools? That’s easy to set up.

Brian explained that GitHub hosted a GitHub Actions Hackathon in late 2020. The team coded the controls for the submission process into the hackathon. You can still check it out at githubhackathon.com.

The entire submission process got automated to check for all the proper files being included in a submission. The code recognized completed submissions on the hackathon home page automatically.

Brian then gave the example of work he did on the GitHub Hacktoberfest in October. For the team working on the code, Brian developed a custom workflow that allowed any authenticated individual to sign up to address issues exposed in the Hackathon. Brian’s code latched onto existing authentication code to validate that individuals could participate in the process and assigned their identity to the issue. As the developer, Brain built the workflow for these tasks using GitHub Actions.

What can you automate? Informing your team when a user does a pull request. Send a tweet when the team releases a build. Any webhook in GitHub you can automate with GitHub Actions. For example, you can even automate the nag emails that get sent out when a pull request review does not complete within a specified time.

Common Actions

Angie then asked about the most common actions that Brian sees users running.

Brian summarized by saying, basically, continuous integration (CI). The most common use is ensuring that tests get run against code as it gets checked in to ensure that test suites get applied. You can have tests run when you push code to a branch, push code to a release branch or do a release, or even when you do a pull request.

While test execution gets run most frequently, there are plenty of tasks that one can automate. Brian did something specific to assign gifts to team members who reviewed pull requests. He also used a cron job to automate a GitHub Action which opened up a global team issue each Sunday US, which happens to be Monday in Australia, and assigned all the team members to this issue. Each member needed to explain what they were working on. This way, the globally-distributed team could stay on top of their work together without a meeting that would occur at an awkward time for at least one group of team members.

Brian talked about people coming up with truly use cases – like someone linking IOT devices to webhooks in existing APIs using GitHub Actions.

But the cool part of these actions is that most of them are open source and searchable. Anyone can inspect actions and, if they don’t like them, modify them. If a repo includes GitHub Actions, they’re searchable.

On github.dom/bdougie, you can see existing workflows that Brian has already put together.

Jump Into GitHub Actions – What Next?

I shared some of the basic ideas in Brian’s conversation with Angie. If you want to jump into GitHub Actions in more detail, you can check out the full webinar and the slides in Addie Ben Yehuda’s summary blog for the webinar. That blog also includes a number of Brian’s links, several of which I include here as well:

Enjoy jumping into GitHub Actions!

Featured Photo by Aziz Acharki on Unsplash

The post Get a Jump Into GitHub Actions appeared first on Automated Visual Testing | Applitools.

How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow

Colby Fayock — Mon, 01 Mar 2021 20:37:37 +0000

Applitools provides a number of SDKs that allows you to easily integrate it into your existing workflow. Using tools like Cypress, Espresso, Selenium, Appium, and a wide variety of others, web and native platforms can get automated visual testing coverage with the power of Applitools Eyes.

But what if you’re not looking to integrate it directly to an existing testing workflow because maybe you don’t have one or maybe you don’t have access to it? Or what if you want to provide blanket visual testing coverage on a website without having to maintain which pages get checked?

We’ll walk through how we were able to take advantage of the power of Applitools Eyes and flexibility of GitHub Actions to create a solution that can fit into any GitHub-based workflow.

Note: if you want to skip the “How it Works” and go directly to how to use it, you can check out the Applitools Eyes GitHub Action on github.com. https://github.com/colbyfayock/applitools-eyes-action

What are GitHub Actions?

To start, GitHub Actions are CI/CD-like workflows that you’re able to run right inside of your GitHub repository.

Running build, test, and publish on a branch with GitHub Actions

Using a YAML file, we can set up our project to run tests or really any kind of script based on events such as a commit, pull request, or even on a schedule with cron.

name: Tests

on:
  push:
	branches: [ main ]
  pull_request:
	branches: [ main ]

jobs:
  test:
	runs-on: ubuntu-latest
	steps:
	- uses: actions/checkout@v2
	- uses: actions/setup-node@v2
	  with:
		node-version: '12'
	- run: npm ci
	- run: npm test

Setting up simpler workflows like our example above, where we’re installing our dependencies and running our tests, is a great example of how we can automate critical code tasks, but GitHub also gives developers a way to package up complex scripts that can reach beyond what a configurable YML file can do.

Using custom GitHub Actions to simplify complex workflows

When creating a custom GitHub Action, we unlock the ability to use scripting tools like shell and node to a greater extent, as well as the ability to stand up entire environments using Docker, which can allow us to really take advantage of our allotted environment just like we could on any other CI/CD platform.

Building a container in a GitHub Action

In our case, we want to allow someone to run Applitools Eyes without ever having to think about setting up a test runner.

To achieve this, we can include Cypress (or another test runner) right along with our Action, which would then get installed as a dependency on the workflow environment. This allows us to hook right into the environment to run our tests.

Scaffolding a Cypress environment in a GitHub Action workflow with Docker

Setting up Cypress is typically a somewhat simple process. You can install it locally using npm or yarn where Cypress will manage configuring it for your environment.

Installing Cypress with npm via cypress.io

This works roughly the same inside of a standard YAML-based GitHub Action workflow. When we use the included environment, GitHub gives us access to a workspace where we can install our packages just like we would locally.

It becomes a bit trickier however when trying to run a custom GitHub Action, where you would want to potentially have access to both the project and the Action’s code to set up the environment and run the tests.

While it might be possible to figure out a solution using only node, Cypress additionally ships a variety of publicly available Docker images which let’s us confidently spin up an environment that Cypress supports. It also gives us a bit more control over how we can configure and run our code inside of that environment in a repeatable way.

Because one of our options for creating a custom Action is Docker, we can easily reference one of the Cypress images right from the start:

FROM cypress/browsers:node12.18.3-chrome87-ff82

In this particular instance, we’re spinning up a new Cypress-supported environment with node 12.18.3 installed along with Chrome 87 and Firefox 82.

Installing Cypress and Action dependencies

With our base environment set up, we move to installing dependencies and starting the script. While we’re using a Cypress image that has built-in support, Cypress doesn’t actually come already installed.

When installing Cypress, it uses cache directories to store download binaries of Cypress itself. When using Docker and working with different environments, its important to have predictable locations of where these caches exist, so that we’re able to reference it later.

Cypress verifies that it can correctly identify an installation path from cache

In our Docker file, we additionally configure that environment with:

ENV NPM_CACHE_FOLDER=/root/.cache/npm
ENV CYPRESS_CACHE_FOLDER=/root/.cache/Cypress

When npm and Cypress goes to install, they’ll use those directories for caching.

We also need to set up an entrypoint which tells Docker what script to run to initiate our Action.

Inside of our Dockerfile, we add:

COPY entrypoint.sh /entrypoint.sh

ENTRYPOINT ["/entrypoint.sh"]

We’ll use a shell script to initiate our initial installation procedure so we can have a little more control over setting things up.

Finally, inside of our referenced shell script, we include:

#!/bin/sh -l

cd $GITHUB_WORKSPACE

git clone https://github.com/colbyfayock/applitools-eyes-action
cd applitools-eyes-action

npm ci

node ./src/action.js

This will first navigate into our GitHub’s Workspace directory, to make sure our session is in the right location.

We then clone down a copy of our custom Action’s code, which is referencing itself at this point, but it allows us to have a fresh copy in our Workspace directory, giving us access to the script and dependencies we ultimately need.

And with our Action cloned within our working environment, we can now install the dependencies of our action and run the script that will coordinate our tests.

Dynamically creating a sitemap in node

Once our node script is kicked off, the first few steps are to find environment variables and gather Action inputs that will allow our script to be configured by the person using it.

But before we can actually run any tests, we need to know what pages we can run the tests on.

To add some flexibility, we added a few options:

Base URL: the URL that the tests will run on
Sitemap URL: this allows someone to pass in an already sitemap URL, rather than trying to dynamically create one
Max Depth: how deep should we dynamically crawl the site? We’ll touch on this a little more in a bit

With these settings, we can have an idea on how the Action should run.

If no sitemap is provided, we have the ability to create one.

Example sitemap

Specifically, we can use a Sitemap Generator package that’s available right on npm that will handle this for us.

const generator = SitemapGenerator(url, {
  stripQuerystring: false,
  filepath,
  maxDepth
});

Once we plug in a URL, Sitemap Generator will find all of the links on our page and crawl the site just like Google would with it’s search robots.

We need this crawling to be configurable though, which is where Max Depth comes in. We might not necessarily want our crawler to drill down link after link, which could cause performance issues, but it could also include pages or other websites that we aren’t interested in including in our sitemap.

Reduced sitemap of applitools.com

With Max Depth, we can tell our Sitemap Generator how deep we want it to crawl. A value of 1 would only scrape the top level page, where a value of 2 would follow the links on the first page, and then follow the links on the second page, to find pages to include in our dynamically generated sitemap.

But at this point, whether dynamically generated or provided to us with the Sitemap URL, we should now have a list of pages that we want to run our tests on.

Running Cypress as a script in node

Most of the time when we’re running Cypress, we use the command line or include it as a script inside of our package.json. Because Cypress is available as a node package, we additionally have the ability to run it right inside of a node script just like we would any other function.

Because we already have our environment configured and we’ve determined the settings we want, we can plug these values directly into Cypress:

const results = await cypress.run({
  browser: cypressBrowser,
  config: {
	baseUrl
  },
  env: {
	APPLITOOLS_APP_NAME: appName,
	APPLITOOLS_BATCH_NAME: batchName,
	APPLITOOLS_CONCURRENCY: concurrency,
	APPLITOOLS_SERVER_URL: serverUrl,
	PAGES_TO_CHECK: pagesToCheck
  },
  headless: true,
  record: false,
});

If you notice in the script though, we’re setting a few environment variables.

The trick with this is, we can’t directly pass in arguments that we may need inside of Cypress itself, such as settings for Applitools Eyes.

The way we can handle this is by creating Cypress environment variables, which end up roughly working the same as passing arguments into the function, we just need to access it slightly differently.

But beyond some Applitools-specific configurations, the important bits here are that we have a basic headless configuration of Cypress, we turn recording off as ultimately we won’t use that, and we pass in PAGES_TO_CHECK which is an array of pages that we’ll ultimately run through with Cypress and Applitools.

Using Cypress with GitHub Actions to dynamically run visual tests

Now that we’re finally to the point where we’re running Cypress, we can take advantage of the Applitools Eyes SDK for Cypress to easily check all of our pages.

describe('Visual Regression Tests', () => {
  const pagesToCheck = Cypress.env('PAGES_TO_CHECK');

  pagesToCheck.forEach((route) => {
	it(`Visual Diff for ${route}`, () => {

	  cy.eyesOpen({
		appName: Cypress.env('APPLITOOLS_APP_NAME'),
		batchName: Cypress.env('APPLITOOLS_BATCH_NAME'),
		concurrency: Number(Cypress.env('APPLITOOLS_CONCURRENCY')),
		serverUrl: Cypress.env('APPLITOOLS_SERVER_URL'),
	  });

	  cy.visit(route);
	  
	  cy.eyesCheckWindow({
		tag: route
	  });

	  cy.eyesClose();
	});
  });
});

Back to the critical part of how we ran Cypress, we first grab the pages that we want to check. We can use Cypress.env to grab our PAGES_TO_CHECK variable which is what we’ll use for Eyes coverage.

With that array, we can simply run a forEach loop, where for every page that we have defined, we’ll create a new assertion for that route.

Running Applitools Eyes on each page of the sitemap

Inside of that assertion, we open up our Eyes, proceed to visit our active page, perform a check to grab a snapshot of that page, and finally close our Eyes.

With that brief snippet of code, we’re uploading a snapshot of each of our pages up to Applitools, where we’ll now be able to test and monitor our web project for issues!

Configuring Applitools Eyes GitHub Action into a workflow

Now for the fun part, we can see how this Action actually works.

To add the Applitools Eyes GitHub Action to a project, inside of an existing workflow, you can add the following as a new step:

steps:
- uses: colbyfayock/applitools-eyes-action@main
  with:
	APPLITOOLS_API_KEY: ${{secrets.APPLITOOLS_API_KEY}}
	appName: Applitools
	baseUrl: https://applitools.com

We first specify that we want to use the Action at its current location, then we pass in a few required input options such as an Applitools API Key (which is defined in a Secret), the name of our app, which will be used to label our tests in Applitools, and finally the base URL that we want our tests to run on (or we can optionally pass in a sitemap as noted before).

With just these few lines, any time our steps are triggered by our workflow, our Action will create a new environment where it will run Cypress and use Applitools Eyes to add Visual Testing to the pages on our site!

What’s next for Applitools Eyes GitHub Action?

We have a lot of flexibility with the current iteration of the custom GitHub Action, but it has a few limitations like having an environment already deployed that can be accessed by the script and generally not having some of the advanced Applitools feature customers would expect.

Because we’re using node inside of our own custom environment, we have the ability to provide advanced solutions for the project we want to run the tests on, such as providing an additional input for a static directory of files, which would allow our Action to spin up a local server and perform the tests on.

As far as adding additional Applitools features, we’re only limited to what the SDK allows, as we can scale our input options and configuration to allow customers to use whatever features they’d like.

This Action is still in an experimental stage as we try to figure out what direction we ultimately want to take it and what features could prove most useful for getting people up and running with Visual Testing, but even today, this Action can help immediately add broad coverage to a web project with a simple line in a GitHub Action workflow file.
To follow along with feature development, to report issues, or to help contribute, you can check out the Action on GitHub at https://github.com/colbyfayock/applitools-eyes-action.

The post How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow appeared first on Automated Visual Testing | Applitools.

Intro to GitHub Actions for Test Automation — with Angie Jones and Brian Douglas [webinar recording]

Addie Ben-Yehuda — Mon, 01 Feb 2021 11:42:26 +0000

Watch this on-demand webinar to learn how the new GitHub Actions can help you build, test and deploy faster and easier!

Curious about GitHub Actions and how this feature can be used for test automation? So is test automation guru Angie Jones!

GitHub Actions makes it easy to automate all your software workflows, now with CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want.

Watch this on-demand webinar, where Angie Jones chats with Brian Douglas — staff developer advocate at GitHub — about this exciting new offering and how it can be utilized for test automation.

Brian also showed a demo illustrating how easy it is to add automated tests to a project and kick them off with GitHub Actions!

Angie’s Slide Deck

https://slides.com/angiejones/github-actions

Full Webinar Recording

Additional GitHub Action Links Provided by Brian

Additional Resources and Reading Materials

Top 10 Most Popular Free Test Automation Courses of 2020 — post by Angie Jones
Selenium vs Cypress: The Rematch — post by Angie Jones
Where To Learn Test Programming – July 2020 Edition (post)
Start visually testing your app and site today, with your free Applitools account — Click here

— HAPPY TESTING —

The post Intro to GitHub Actions for Test Automation — with Angie Jones and Brian Douglas [webinar recording] appeared first on Automated Visual Testing | Applitools.

Learn from Google – Use Google Cloud Build

Ranjith Reddy — Thu, 01 Oct 2020 20:51:44 +0000

We all want to catch and fix problems as early as possible. In software development, this mindset helps development teams create a culture of continuous improvement. To automate best practices, like Continuous Integration & Continuous Delivery (CI/CD), teams are adopting tools that help automate continuous improvement. Google has created a great tool in Google Cloud Build.

By creating Google Cloud Build, Google has created a platform that helps every software developer from every walk of life, on teams of every size. Google Cloud Build is a fully managed cloud application development environment. It includes an associated set of build and test management tools that also encompasses automation and workflow controls, along with privacy and vulnerability analysis intelligence.

Regardless of the speed or scale that teams are able to build, test and deploy software. This is very much aligned with the vision of Applitools specially after the launch of the Ultrafast Visual Grid. Supporting an integration between Google Cloud Build and Applitools provides every software developer the ability to run their tests across all environments at the speed of light and get the feedback seamlessly in second as a part of the build process.

Integrate Applitools with Google Cloud Build

To get started, follow the following steps to integrate Applitools into your Google Cloud Build.

Step 1: Create an account on Google Cloud Platform.
Step 2: Navigate to the Cloud build.

Create the trigger and select your source code management tool (in this example I have chosen GitHub ) and connect your repository.

Allow the required access to perform the trigger runs on every commit on each branch or each pull request.

Edit your trigger and select the Build Configuration as Cloudbuild.yaml or Dockerfile.

Once you make a small change in application and push it up to GitHub this will kick off the Cloud Build trigger you’ve just created.

Continue With Applitools Setup

Now follow these steps to configure your Applitools setup.

Step 1: Navigate to admin panel in the Applitools eyes dashboard

Step 2: Go to Teams and select your team

Step 3: Select your team and Navigate to Integrations Section

Step 4: Integrate your GitHub repo with Applitools

Run Your Cypress Tests

Now, run your Cypress tests on Google Cloud build.

Step 1: Let us clone the demo Cypress project from Applitools Cypress

https://applitools.com/tutorials/cypress.html

Step 2: Create a cloudbuild.yaml file to pass the build steps.

View the code on Gist.

To run a Cypress end-to-end test let’s use a Docker.

In the Docker file we have defined the steps to copy the Docker images, files, Cypress folder, add configurations and execute the scripts.

Make sure to set the APPLITOOLS_API_KEY as ENV variable or access it from secrets in the Docker file.

View the code on Gist.

Applitools build status is reported based on COMMIT_SHA, hence we need to access the git commit_sha in the Docker and assign the COMMIT_SHA to Applitools BatchID as ENV variable.

Once the test is completed the curl command in Docker will perform a post call to update the build status.

Once you build your project. Applitools can visually validate each commit and reports the build status.

Watch the video to try out the integrations & Keep Testing!

The post Learn from Google – Use Google Cloud Build appeared first on Automated Visual Testing | Applitools.

Review – Automated Visual Testing With WebdriverIO

Michael Battat — Mon, 17 Aug 2020 22:08:24 +0000

I took Nyran Moodie’s course on Test Automation University: Automated Visual Testing with WebdriverIO. If you want the explicit link to the course, here it is: https://testautomationu.applitools.com/automated-visual-testing-javascript-webdriverio/index.html

Course Introduction

If you use WebdriverIO regularly, and you are unfamiliar with the basics of using Applitools for automated visual testing, you will appreciate this course. Nyran focuses you on how to add visual test automation with Applitools to your WebdriverIO tests.

Nyran expects you to know WebdriverIO. If you want to learn WebdriverIO, check out Julia Pottinger’s excellent course – UI Automation with WebdriverIO on Test Automation University.

Nyran uses JavaScript for his examples and Visual Studio Code as his IDE. In case you didn’t know, you can use Applitools with a range of test languages. No matter what your test language of choice or IDE, I think you will find Nyran’s code and coding approach fairly intuitive.

Course Structure

Nyran breaks the course into eight main chapters. He gives this description in the course overview:

1 – I am going to start by giving you an introduction to visual testing.
2 – We are going to look at how we can get our environment set up to start using Applitools eyes.
3 – We are going to create and run our first visual test using Applitools
4 – I want to introduce you to the different match levels that Applitools has and the concept of viewport sizes
5 – I will be talking about checkpoints.
6 – I will be looking at how we can organize our visual tests using batches
7 – We will look at how we can analyze our test results using the Test Manager.
8 – We will be looking at the integration and collaborations that Applitools provides.

Each of these chapters provides a methodical approach to getting going with Applitools. I’ll give a quick overview of each.

Chapter 1 – Why Visual Testing

If you read this review or take the course, you know why visual testing. Your UI and end-to-end tests result in rendered output. You can write all the functional tests that grab locators, enter data, effect action, and cause the appropriate output locators to have the appropriate values. But, until you actually look at the result, you cannot tell if the input and output conform to design and usability expectations.

Nyran did not explain the most frequently-experienced reason for visual testing – unintended consequences of code changes over time. Our experience shows us that most expected application changes get tested, but unintended changes cause problems.

Chapter 2 – Getting Started With Applitools

Nyran does a nice job explaining how to get started. You need an Applitools API key, which you can get from the Applitools console. Nyran explains why you set up a local environment variable for your API key (so you do not need to include your API key in your test code directly). He also points to the github repo he uses for all the examples in the course.

Chapter 3 – Create And Run A Visual Test with Applitools

Chapter 3 involves the first coding examples for setting up Applitools. With a simple:

npm install @applitools/eyes.webdriverio

You get the the node instructions for installing the Applitools Eyes to your WebdriverIO setup. After this you can install the Applitools Eyes service to your tests. He shows code examples of what test code looks like when calling Applitools:

Once he walks you through a test page example and fills in the tests, he gets code that looks like a full test.

https://github.com/NyranMoodie/WebdriverIO-Applitools/blob/master/test/specs/visual.js

Finally, he shows you the Applitools UI and how it highlights differences found during a test. To do this, he shows a test site with the ability to show different content, and he shows how Applitools highlights the differences.

Nyran makes it clear that Applitools can find and highlight all the visual changes on a comparison page (the checkpoint) versus the original capture (the baseline). And, he explains that Applitools lets you accept the checkpoint as the new baseline, or reject it.

Chapter 4 – Viewports and Match Levels

Nyran breaks chapter 4 into two sections. In Section 4.1, Nyran goes through the idea of viewport sizes. If you build a responsive app, you want to run your app for 4K down to mobile device sizes. How do you validate the different device size views? Applitools makes it easy to add a JavaScript file that specifies all the viewport sizes you want to validate. Applitools runs all the captures for you.

Next, Nyran writes about match levels. Applitools default comparison level, called “strict”, compares visually noticeable changes between a checkpoint and baseline. Strict uses Applitools Visual AI to break items on a page into elements that it then compares.

However, sometimes strict is too strict. In “content” match level, Applitools checks text and structures but ignores color variations. This match level helps when you apply a global change to color and want to ensure that – color outstanding – no other changes have taken place. And, if they have, you want those changes highlighted quickly.

In “layout” match level, Nyran shows, Applitools lets you validate pages that have dynamic content sharing a common structure. For example, you might have a retail shoppping page that shows user-specific and hot items updating from test run to test run. Or, you have a news site that updates top stories regularly. Layout match level pays attention to the layout structure (relationship of sections and text sizes) without comparing the specific contents of any set of elements within that structure.

Chapter 5 – Checkpoints

Nyran spends this chapter reviewing the ways Applitools lets you make visual captures. First, ou can capture the visible page – the current viisble screen. Alternatively, you can capture the full page. Applitools runs through the app horizontally and vertically and stitches the images into a single screen baseline and checkpoint.

Next, you can capture individual web elements. Finally, you can capture frames on a page. Your flexibility in setting checkpoints gives you plenty of power to control the details of your inspection at different times in your testing life cycle.

Chapter 6 – Batches

Batches provide a way of organizing common tests inside the Applitools Test Manager. Nyran explains how to code tests into batches. He also shows how batched tests get grouped inside the Test Manager. When you use batches, your test results become easier to interpret.

Nyran implies two things about batches. First, grouping tests into batches make your testing much easier to understand. Second, If you want the benefit of batches, you need to code those batch definitions yourself.

Chapter 7 – Using the Test Manager

The Test Manager is the Applitools service user interface. Your user login provides you an API key, and tests run under your API key collect in your view in Test Manager.

Nyran shows you the basics, as well as some cool tricks in the Test Manager. He shows you how to compare differences between the checkpoint and the baseline. You can approve expected changes and update the baseline to the checkpoint. Or, you can reject changes and have them routed back to development as bugs.

Next, Nyran shows you how tests get grouped and how to use statistics. He also shows you how to override the existing match level on all or part of a test. Finally, he shows you Automated Test Maintenance.

Automated Test Maintenance gives you huge powers of scale for validating changes to your application. When you find a change in one checkpoint and approve it as a valid change, Applitools finds all other similar changes and gives you the power to approve those identical changes at the same time. For example – you change your menu bar at the top of your app, and the change affects 145 pages. Following your validation of a change on one page, Applitools asks you if you want to approve the other 144 pages with the identical change. That’s powerful.

Chapter 8 – Integrations

In the last chapter, Nyran shows how Applitools integrates with other tools in development – especially your CICD workflows.

Nyran shows the range of Applitools test framework and language SDKs that work on web and mobile test frameworks. Applitools lets you capture screenshots to use Visual AI if you use an unsupported framework. Applitools even lets you compare PDFs to ensure that your document generators behave correctly even as you update your application.

Next, you see how to link Applitools with Jira. You can link issues found in Applitools with incidents in Jira. Also, you can link GitHub and Applitools to tie your image comparisons to the source code GitHub pull requests. Finally, you see the standard approach to having Applitools link with your favorite CICD workflow manager.

Conclusion

Nyran wrote a nice course on how to use Applitools for automated visual testing. He makes clear that he used WebdriverIO and JavaScript because he knew these well. However, he knows the range of choices available to you.

Nyran really doesn’t cover:

Setting up tests in WebdriverIO,
Approaches for running and managing tests and test results
Managing WebdriverIO in your CICD workflow

You can find other courses to address these issues.I enjoyed taking Nyran’s course. Having taken other JavaScript testing courses, I think Nyran provides good examples of Applitools. He wants you to know how to get the most out of Applitools when you use WebdriveriO.

As always, I include my certificate of completion from Test Automation University:

For More Information

The post Review – Automated Visual Testing With WebdriverIO appeared first on Automated Visual Testing | Applitools.

Vue.js Development with Storybook and Applitools

Bilal Haidar — Tue, 17 Sep 2019 01:06:03 +0000

At one point in time, the server-side code was responsible for generating and serving every page in our application. Nowadays, Single Page Apps (SPA) are taking a bigger slice of the pie, as more clients are leaning towards building their applications as an SPA.

Developers spend most of their time debugging a SPA inside the browser, by debugging the JavaScript files, CSS styles and the DOM, hoping to find the root cause of any visual bug appearing in their application.

Applitools realizes the complexity of visually debugging an application and introduced the Root Cause Analysis (RCA) module on their Test Manager Dashboard to help quickly identify the root cause behind visual UI changes.

In this article, I will demonstrate the Applitools RCA feature by walking you through a complete step by step guide to visually test a Vue.js app with Storybook and Applitools.

If you’re not already familiar, Applitools is an automated visual regression testing framework. It focuses on the visual aspects of your app and plays a major role in exposing the visual differences between baseline snapshots and both current and future snapshots.

Applitools integrates with dozens of development and testing frameworks, such as Cypress.io, Storybook and Selenium. It provides SDKs for use in your projects to seamlessly communicate and interact with Applitools, in order to send both baseline and test screenshots.

So, get started with visual UI testing. Your visual testing project will depend on React, Storybook and Applitools. You will also need the node.js tools. So, firstly, make sure you know the following software tools and install on your machine:

Before we immerse ourselves in writing code, let’s look at how Applitools and Storybook work together.

How Applitools Works With Storybook

Whether you use Vue.js, React or Angular, Storybook doesn’t offer any extension points for third-party frameworks, such as Applitools, to integrate with. The Applitools team worked around this limitation by providing the Applitools Eyes Storybook SDK.

Once installed, this SDK collects all stories in the application, runs them all via the Storybook engine, generates DOM snapshots for each story and uploads these snapshots to the Applitools Eyes server. Here’s a visual of this flow:

The Applitools server runs the tests generates image snapshots. Applitools compares each snapshot to a baseline snapshot (if one exists) and displays the results on the Test Manager Dashboard.

This kind of integration provides a smooth experience in terms of writing Storybook stories. Nothing changes in the way you write your Vue.js apps.

Automated Root Cause Analysis

The concept of Root Cause Analysis stems from the world of management, where it’s defined as a method of problem solving, used for identifying the root causes of faults or problems.

So far, Applitools has been focusing on visual testing by comparing the baseline and the current test run snapshot through their AI engine. It finds the visual differences in order to map them graphically. Previously, there was no way to search your codebase for a reason behind any visual testing bug. As you may know, searching through code is nightmarish, time-consuming and tedious!

This is where Applitools steps in with RCA, showing you the exact DOM and CSS changes between the baseline snapshot and the current test run snapshot.

Why is this so important for both developers and QA testers? Let’s take a look.

Step By Step Guide

For this demonstration of visual testing a Vue.js application with Storybook and Applitools, I’ve chosen the Bootstrap Vue open-source library hosted on GitHub under this repo. This library, with over 40 plugins and more than 75 custom components, provides one of the most comprehensive implementations of Bootstrap v4 components and grid system for Vue.js. With extensive and automated WAI-ARIA accessibility markup.

The plan is to start by installing Storybook onto the Bootstrap Vue project. Write and run a few Storybook stories, install the Applitools Eyes SDK for Storybook and finally demonstrate the RCA feature.

Now, let’s get on with it and start coding …

Step 1: Clone the Bootstrap Vue GitHub repo locally on your machine by running this command:

git clone git@github.com:bootstrap-vue/bootstrap-vue.git

Step 2: Install all NPM package dependencies by running this command:

yarn

Step 3: Install the Storybook NPM package as a dev-dependency by issuing this command:

yarn add @storybook/vue -D

Step 4: Install all peer dependencies required by Storybook to function properly by issuing this command:

yarn add vue-loader babel-loader babel-preset-vue -D

Step 5: Install css-related NPM packages in order to load CSS styles inside your Storybook stories by issuing this command:

yarn add css-loader style-loader -D

Step 6: Add the following NPM script to your package.json file in order to start the Storybook:

{

  "scripts": {

    "storybook": "start-storybook"

  }

}

Step 7: Create the Storybook config.js file inside a .storybook folder located at the root of the solution folder as follows:

import { configure } from '@storybook/vue';

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/);

  req.keys().forEach(filename => req(filename));

}

configure(loadStories, module);

Step 8: Create a scss-loader.scss file inside the .storybook folder to include all the styles required by Bootstrap Vue library components to load properly inside our Storybook stories:

@import ‘../node_modules/bootstrap/scss/bootstrap.scss’

@import ‘../src/index.scss’

Then import this file inside the config.js file as follows:

import { configure } from '@storybook/vue';

import ‘!style-loader!css-loader!sass-loader!./scss-loader.scss’

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/);

  req.keys().forEach(filename => req(filename));

}

Step 9: Create the navbar.stories.js file inside the stories folder, located at the root of the solution folder, to hold our Storybook stories for the NavBar component:

import { storiesOf } from '@storybook/vue'

const navbarStories = storiesOf('BNavBar', module)

navbarStories.add('Full Navbar', () => ({

  template: `

  

    NavBar

    

    

      

        Link

        Disabled

      

      

        

          

          Search

        

        

          EN

          ES

          RU

          FA

        

        

          

          Profile

          Sign Out

        

      

    

  

`

}))

The story groups all NavBar stories under the name BNavBar and assigns the first story the name of Full Navbar.

There are two ways to tell Storybook stories about your Vue.js components. Either define all used components in a components object property on the story itself or define the components globally inside .storybook/config.js file so that they are available to all running stories in your solution.

Let’s follow the latter option and define all components needed by NavBar story inside .storybook/config.js file using the Vue.component() helper method:

import { configure } from '@storybook/vue'

import Vue from 'vue'

import '!style-loader!css-loader!sass-loader!./scss-loader.scss'

import {

  BNavbar,

  BNavbarNav,

  BNavbarBrand,

  BNavbarToggle,

  BButton,

  BCollapse,

  BNavItem,

  BFormInput,

  BNavForm,

  BNavItemDropdown,

  BDropdownItem

} from '../src'

Vue.component('b-navbar', BNavbar)

Vue.component('b-navbar-nav', BNavbarNav)

Vue.component('b-navbar-brand', BNavbarBrand)

Vue.component('b-navbar-toggle', BNavbarToggle)

Vue.component('b-button', BButton)

Vue.component('b-collapse', BCollapse)

Vue.component('b-nav-item', BNavItem)

Vue.component('b-form-input', BFormInput)

Vue.component('b-nav-form', BNavForm)

Vue.component('b-nav-item-dropdown', BNavItemDropdown)

Vue.component('b-dropdown-item', BDropdownItem)

function loadStories() {

  const req = require.context('../stories', true, /\.stories\.js$/)

  req.keys().forEach(filename => req(filename))

}

configure(loadStories, module)

Step 10: Now the story is ready, let’s run it with Storybook using this command:

yarn storybook

The Storybook engine runs successfully and renders our Navbar story.

Now let’s install the Applitools Eyes SDK for Storybook.

Step 11: Install the Applitools Eyes SDK for Storybook NPM package as a dev-dependency by issuing this command:

yarn add @applitools/eyes-storybook -D

In order to authenticate via the Applitools server, you need to supply the Eyes-Storybook SDK with the API key you got from Applitools. Read more about how to obtain the API key here.

To do this, set the environment variable APPLITOOLS_API_KEY to the API key before running your tests. For example, on Linux/Mac:

export APPLITOOLS_API_KEY=

And on Windows:

set APPLITOOLS_API_KEY=

Step 12: Let’s run the first cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Once the tests are done running, let’s check back with the Applitools Test Manager Dashboard and verify the results.

The left side of the Dashboard lists all batch test runs that you have previously performed. Currently, there is a single test run with a status of Passed. This is the first test run and so Applitools engine automatically passes this test and all snapshots thereafter as baseline snapshots.

Click on the single batch test to view the snapshots available. For each and every Storybook story, a snapshot is generated by Applitools.

Expand the BNavBar: Full Navbar test run and click the snapshot to view it in a bigger scale:

As you can tell, the test run snapshot is simply an image snapshot for the story that runs inside the Storybook playground application.

At this moment of time, Applitools stores all the generated snapshots as baseline snapshots. The future test runs would generate a new set of snapshots and Applitools would compare them to the baseline ones and report the results accordingly.

You have noticed that Applitools Eyes SDK for Storybook is prepackaged with some default settings allowing you to do visual testing for your stories without having to specify any test configuration settings. However, that doesn’t mean that Applitools doesn’t offer you the means to customize your tests by specifying test configuration settings either through environment variables or storing your settings inside the applitools.config.js configuration file.

Let’s switch gears and explore a unique gem of Applitools that is the Regression UI testing. The traditional regression testing verifies recent source code changes or additions and that the functionality of the software has not changed or been affected in any way.

Applitools allows you to perform visual UI Regression testing. You can run multiple cycles of test runs and all visual changes collected from one to another, are detected by Applitools servers and logged into the Dashboard.

Step 13: Let’s introduce a visual change in the Navbar component by including the following CSS selector inside .storybook/scss-loader.scss file and run our tests again.

@import "../node_modules/bootstrap/scss/bootstrap.scss";

@import "../src/index.scss";

a {

  background-color: yellow;

}

The change causes all the hyperlinks on the Navbar component to render with a background color of yellow.

Step 14: Let’s run the second cycle of the visual UI testing by issuing this command:

npx eyes-storybook

Notice the logs on the terminal windows after running the command:

The test failed and that’s exactly what we expect. Applitools detected the change we introduced to the background color of the hyperlinks and reported it.

Let’s switch back to the Applitools Test Manager Dashboard to check the results.

The new test run is marked as Unresolved. Expand on the Navbar test run to view the visual differences highlighted to you by Applitools:

Make sure both snapshots are selected so that you can easily view the differences. If not, locate the following menu and choose the option Both:

The snapshot above highlights all the hyperlinks, indicating that those hyperlinks have changed.

Step 15: Click on the RCA tool icon (highlighted in the red rectangle) to get a full analysis of the source of this visual bug.

Next, you select the visual difference you want RCA to assess.

I’ve selected to click on the Lang hyperlink on the Navbar. Notice how the RCA tool detects a css style change on the tag represented by the Yellow background.

The RCA tool runs and shows the root cause of this visual difference, which is a change in the CSS Rules.

Step 16: The RCA tool can also detect Attribute changes. Let’s change the tag of the Search box to have a size of lg as follows and run the test once more.

The RCA tool detects a change in the attributes of the Search box.

It also shows the change of CSS rules as follows:

Step 17: Another category RCA detects is the DOM changes category. Let’s wrap the Search button text with a paragraph tag as follows and run the test again:

Search

The RCA tool detects a change in the text inside the Search button. Instead of rendering the text as is, the new test snapshot renders the text inside a

tag.

Any test run with status Unresolved requires someone’s attention and manual intervention to accept or reject the change. In this case, you may accept the new change(s) and thus update the baseline snapshots.

On the other hand, if the change detected is undesirable, then this implies a bug in the source code and you can reject this new test run to keep the original baseline snapshot without any changes.

In Conclusion

Applitools RCA is both powerful and convenient. It can pinpoint the exact DOM/CSS change. If you did not intend the change, you can identify the code the responsible code quickly and make appropriate modifications to pass your tests.

RCA supports a set of DOM/CSS categories of change, including:

Textual changes
CSS Property changes
Attributes changes
Bounding Box changes
Tag changes (for example, when you replace a button with a hyperlink)

Happy Testing!

Video Demonstration

More about Applitools

The post Vue.js Development with Storybook and Applitools appeared first on Automated Visual Testing | Applitools.

How to update Jenkins build status in GitHub pull requests [Step-by-Step Tutorial]

Sophie Tagar — Tue, 27 Nov 2018 01:25:11 +0000

I regularly use Jenkins CI and GitHub to help our customers integrate Applitools into their continuous integration workflow. Even though I’m pretty familiar with both tools, there’s one thing that drives me nuts.

Here’s the problem:

I can’t see Jenkins build status in my GitHub repo pull request. Below is a screenshot of a typical GitHub pull request. As you can see, there’s no indication of my Jenkins build status.

This forces me to leave GitHub and go to Jenkins every time I merge, just so I can check the build status.

Doing this once is no big deal. But after doing this for a hundred pull requests, I feel like pulling my hair out. (And that would just look weird.)

This is especially frustrating since other continuous integration servers, like Travis and Circle CI, make this easy.

Since Jenkins is the third most popular continuous integration tool used with GitHub (after Travis and Circle CI), you’d think this issue would have been solved.

But when I started looking online for a way to solve this issue. I found some good resources, but not one document that described the process from beginning to end with all the configurations needed.

So, to spare you the pain I’ve gone through trying to make this work, I wrote this end-to-end tutorial on how to update Jenkins build status in the GitHub pull request page.

So let’s dive in!

The Big Picture

To get a better understanding of the interaction between those tools and the problem they present, see the following diagram. The red marks are the parts that are missing and this post is attempting to fix.

Get the basics in place

To follow the steps below, you just need a couple of things:

A GitHub account.
A Jenkins server up and running on a static URL.

What you will do in this tutorial:

Setup GitHub
1. Install Jenkins plugin
2. Generate a GitHub access token
Setup Jenkins
1. Install Hudson Post Build Task plugin
2. Call the GitHub API

3) Bask in the awesomeness of your technical prowess

Setup GitHub

So, how to get GitHub setup with Jenkins? These steps explain.

What you’ll do here is install the Jenkins plugin on your GitHub repository in order to trigger a build in Jenkins when you commit a change in GitHub. This way, Jenkins can run tests to make sure your code is working as expected.

On your GitHub account, go to your repository and configure a webhook to the Jenkins server:

In your browser, go to your GitHub account. The URL should be something like www.github.com/your_username
Go into one of your repositories
Click on Settings, on the upper right part of the page
Click on Webhooks
Click on Add Webhook
Set the payload url to /github-webhook/
Set Content type to application/json
Select the event you would like the webhook to be triggered by (I set my webhook to be triggered by “Just the push event”.
Click on Add webhook

Here’s what those steps look like visually:

So, how to get GitHub setup with Jenkins? These steps explain.

What you’ll do here is install the Jenkins plugin on your GitHub repository in order to trigger a build in Jenkins when you commit a change in GitHub. This way, Jenkins can run tests to make sure your code is working as expected.

On your GitHub account, go to your repository and install the Jenkins plugin:

In your browser, go to your GitHub account. The URL should be something like www.github.com/your_username
Go into one of your repositories
Click on Settings, on the upper right part of the page
Click on Integrations and services
Click on Add service
Search for Jenkins
Add your Jenkins static URL or IP.
Click on Add service

Here’s what those steps look like visually.

Generate a GitHub access token

You will need to generate a GitHub access token for security purposes. This token can be used by an application to use an API, here the application is Jenkins and the API is the GitHub API to update the Jenkins build status in the pull request.

Ok, now you’re on to some non-obvious items.

Go to your GitHub account.
Under your profile, open the drop-down menu and click on Settings.

3. Go to Developer settings on the left side menu, all the way on the bottom:

4. Click on Personal access tokens:

5. On the top right corner of the screen click on the Generate new token button.

6. Insert your password.

7. Under repo check the repo:status checkbox.

9. Click on Generate token at the bottom of the page.

10. Make sure to save your token somewhere so you can use it later, such as in the curl commands above.

Jenkins setup

So, how to get Jenkins working with GitHub? Let’s show you how.

What you’ll do here is set up your Jenkins project to run your automation tests and call the GitHub API to update the status of the commit with success or failure.

Install GitHub plugin:

In your browser, go to your Jenkins URL
Click on Manage Jenkins
Click on Manage Plugins
Go to Available tab and search for GitHub integration plugin
Check the checkbox on the left and click on download and install after restart

Setup GitHub configuration:

Go to your GitHub project
Click on Configure
On the top bar, click on Source code management
Check the Git checkbox
Paste the repository URL that you would like to integrate with under repository URL.
Set the Branch to integrate with under Branch specified.

Now that you have GitHub setup, do the following:

In your browser, go to your Jenkins URL
Click on Manage Jenkins
Click on Manage Plugins
Click on Install Hudson Post Build Task

Here’s what you’ll see when you run each of those commands. First, the Manage Jenkins command, in the menu bar on the left side:

Next, the Manage Plugins command.

And finally, the Install Hudson build task command.

Hudson post-build task

(In case you’re wondering why you’re installing a Hudson plugin and not a Jenkins plugin, it’s because of this interesting bit of CI historical trivia.)

Install Hudson Post Build Task plugin

The next step is to go into your Jenkins project post build actions, and add two post build tasks. One task for success, and another for failure. You will need those actions to distinguish between a successful build and a build that failed.

The plugin name could also be called “post build task”

Define those build actions as follows:

What you are doing is looking for the words “SUCCESS” or “FAILURE” in the Jenkins log. When you find “SUCCESS”, you run a script that uses the curl command to call the GitHub webhook API to set the build status of the commit that started the build in GitHub pull request. The commit identifier, the SHA, is accepted by Jenkins within an environment variable – $GIT_COMMIT. Similarly, if you find “FAILURE” in the Jenkins log, you use curl to let GitHub know that the Jenkins build failed.

Either way, the status will be updated in the pull request. And, if you run Applitools tests, the Applitools status will also be updated.

Call the GitHub API

For log entries containing “SUCCESS”, here’s the post-build command that you want Jenkins to run:

View the code on Gist.

And for log entries containing “FAILURE”, here’s the post-build command that you want Jenkins to run::

View the code on Gist.

Notice that you will need to replace the following with your relevant info:

– from GitHub
– from GitHub
– your Jenkins project name

Let’s dig into these post-build commands, using the curl man pages.

As you can see, these are having curl request a URL from api.github.com. This URL has your GitHub username, repository name, and access token in it.

You’re sending a JSON snippet in the header of the HTTP request, as specified by the -H option. And you’re sending an HTTP POST request, as indicated by the -X flag. And the data that post command is sending, indicated by -d, is a bit of JSON with either success or failure, your Jenkins URL, and your Jenkins project name.

(That’s some pretty complex curl you got going!)

Once you get it setup be sure to give yourself a pat on the back. Or a nice Belgian waffle with whipped cream and strawberries. (I know which one I’d pick!)

So, where do you get all this information?

To get this info, you’ll do the following:

GitHub Username – This should be your GitHub username. Hopefully this is obvious.
GitHub Repository Name – You can find the name of any particular GitHub repo at the top of its page. In the example below, the repo name is Applitools-GitHubIntegration.

Jenkins URL – This should be the base URL of your Jenkins server. This is simply the URL you put into your browser to access Jenkins, without anything to the right of the domain name. For example, if you have a Jenkins server setup on Amazon EC2, it might look something like this:

http://ec2-53-70-132-187.compute-1.amazonaws.com/

In order for GitHub to find your Jenkins server, you should plug in your fully qualified domain name.

Jenkins Project Name – which you can see in the Jenkins UI as shown below.

Conclusion

Now that you’ve configured both Jenkins and GitHub, the commit status will be shown in the GitHub pull request page when you merge a PR:

As you can see, the Jenkins build status is now appearing in the pull request page.

Pretty nice, right?

The post How to update Jenkins build status in GitHub pull requests [Step-by-Step Tutorial] appeared first on Automated Visual Testing | Applitools.

How to improve your user interfaces with UI Version Control

Al Sargent — Wed, 12 Sep 2018 01:24:38 +0000

You know what’s crazy about software?

It’s the fact that, for many businesses, applications are a big revenue driver — maybe even the ONLY revenue driver. But we have no system of record to manage the development of our application user interfaces.

Let me explain.

Suppose your company’s revenue drops off. You learn that over the past quarter, revenue coming from mobile app has declined significantly.

You dig further. App downloads are unchanged. App performance is unchanged. App crash rates are unchanged and low.

In fact, the only change is in the UI. You’ve released several new versions over the past few months. So, some of these UI changes must be causing the revenue dropoff, you guess.

You decide to do some user research. Show them old and new versions of the UI and ask which they prefer.

But no one has a record of what the old UI looked like. Those designs are lost, washed away by the passing of time like a sand castle in surf.

It’s crazy that, in 2018, we have accounting systems to track our company finances, CRM to manage sales process — heck, we even have apps to manage our houseplants!

But there’s nothing to manage the application user interfaces that drive revenue for modern businesses.

Like I said: crazy.

So how do we fix this?

Introducing UI Version Control

We were amazed that, with so many businesses driving the bulk of their revenue through web and mobile apps, there wasn’t a way for product teams to track the visual evolution of their apps. So we built one.

We call this UI Version Control. It’s like a wayback machine for your apps.

UI Version Control lets you build software smarter by capturing the entire visual history of your apps, so you have a record of what’s been changed, by whom, and when — and what should be rolled back if the need arises.

Move Fast and Fix Things

Most important, UI Version Control directly links changes in the UI to underlying code changes in your GitHub code repository. This clarifies what code commits need to be undone in order to fix a user interface change that has hurt the user experience or revenue generation.

This helps your company on a number of levels.

They can release more frequently, secure in the knowledge that they know what code changes to make to roll back a feature that didn’t pan out.

They can begin to view application features as experiments — some of which will work, and some won’t — and iterate their way to an optimal UI.

They can de-risk the user-facing aspects of their Digital Transformation projects — many of which are at risk, due in part to lack of a system of record for digital transformation — because they have a way to back out of digital dead ends.

So how do you derisk Digital Transformation?

With a UI System of Record

Before UI Version Control, Applitools showed only baselines in the context of test results from the test results manager, and you weren’t able to view the history of a test baseline.

That’s all changed.

With UI Version Control, you can see the history of all your test baselines in each branch, compare them to prior versions of your baselines, and revert to an earlier version if necessary. You can do this by selecting ‘Save as latest’ from the baseline options menu. So, if you accidentally accepted a baseline that you shouldn’t have, you can undo your mistake.

Applitools Baseline History

UI Version Control works just like source code version control that you, or your developers, are already familiar with. Visual baseline branches can be merged in the same way that code changes are merged.

User Interface Version Control branch merge

For each branch in your GitHub project repository, you can see the history of all your test baselines, compare them to prior baseline versions, and if necessary, revert to an earlier version. You can then reject and/or revert to any baseline modified by accident or by design.

Applitools Baseline Comparison

Here’s a demonstration from our CTO, Adam Carmi, of UI Version Control:

Conclusion

Applitools UI Version Control is now immediately available to all Applitools customers. So, while we might not be able to prevent you from building bad UIs, at least we’ll help you undo them faster.

To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.

What recent UI changes to your app are you worried about?

The post How to improve your user interfaces with UI Version Control appeared first on Automated Visual Testing | Applitools.

How to Test Your GitHub Pull Requests so You Don’t Break Your UI

Al Sargent — Tue, 24 Jul 2018 16:41:58 +0000

Many people think of GitHub as a niche website that hosts open-source projects. In June 2018, that niche website with a mere 28 million users got more media attention than Apple’s Worldwide Developer Conference (WWDC). Somehow, a site that provides free services was so valuable that Microsoft acquired them for $7.5 billion US dollars.

Since its launch in 2007, GitHub has grown far beyond its roots as a web-based service that provides a hosted version of the Git-distributed version control system. The site hosts open-source projects and also offers paid plans for developers and companies. With the adoption of agile development and DevOps practices, it has become an essential part of the continuous integration/delivery/deployment pipeline for many companies. Not only do these companies host their source code in private GitHub repositories, but they also integrate their repositories with their continuous integration (CI) systems.

An important part of any CI system is the ability to run automated tests on any code committed to a source control repository. The automated tests help prevent new code from breaking the software built by the CI system. Now, with the release of the Applitools Eyes GitHub integration, you can include automated tests that perform visual UI testing as part of your build process. Visual UI testing lets you create tests that accurately simulate real users interacting with your software. The integration also takes care of uploading test results to Applitools’ advanced Automated Visual Management (AVM) system for analysis.

This post explains how you can use the Applitools Eyes GitHub integration to perform visual validations for pull requests. Integrating Applitools Eyes with GitHub prevents visual defects from merging into your production software. For each pull request, it merges the baseline screenshots of the branch along with your source code, and runs automated visual validation tests.

Applitools Eyes GitHub Integration

Applitools Eyes ensures your application looks and functions exactly as expected across all browsers, operating systems, devices, and screen resolutions. In addition to handling application testing, the Applitools Eyes GitHub integration lets you use Applitools Eyes as an integral part of your build automation process.

The Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines. It displays the status of your tests directly in the pull request page, right next to the merge status. The integration lets you view the history of an application’s user interface (UI) and see how it has changed, what has been changed, and by whom. This will show you how your product’s UI has evolved over the product’s lifetime. The feature allows you to merge visual testing baseline branches in the same way you merge changes to your test and source code. For each branch in your GitHub project repository, you can see the history of all your test baselines, compare them to prior baseline versions, and if necessary, revert to an earlier version. This feature lets you reject and/or revert to any baseline you modified by accident or by design.

Visually Validating a Pull Request

In order to illustrate how to use the Applitools Eyes GitHub integration, we created a complete demo that includes all the relevant source code files and project-related assets. The demo is called merge-demo, and we made it available as a GitHub repository. The demo runs visual validation against the Applitools Hello World test page. The test navigates to the Hello World page, visually validates it, clicks the Click Me button, and visually validates the page. In this demo, the Hello World test code has been modified to test the page in two different viewport sizes.

Configuring the Integration

Before you can run the demo, you need to install and configure the Applitools Integration application. In the merge-demo repo, open the Settings > Integration & Services page. The page indicates that the Applitools Integration application has been installed. The page also indicates that the Travis CI service has been configured to build and run tests.

Creating the Pull Request

To create the pull request on your PC, open a terminal window and clone the merge-demo locally. Create a new branch, called mybranch, and modify the tests. Commit the changes and push them back to the remote branch. Now, create a pull request to merge the changes with the master branch.

Building the Request

After receiving the Pull Request, Travis CI initiates a build for the pushed pull request. For the sake of simplicity, we recommend you disable the Builds for the Pushed Branches option; otherwise, Travis CI will build two individual branches.

When the build is complete, open the Pull Request page. The Test Applitools status indicates the status of all visual tests that ran as part of the build. We can see the two tests detected differences. If you click the details link, it will take you to the corresponding page in the Applitools Test Manager. The batch contains two tests with different viewport sizes. The test name includes the branch name, the name of the pull request, the source branch, the commit message, and the SHA. Using the page, you can accept the changes made to the first test and reject the changes made to the second test.

Managing and Merging Baselines

Reopen the Pull Request page and you will see the Status Description now indicates that one of the visual tests has been failed. If you accept the changes in the second test, you can see the status changes to green to indicate that all of the visual tests were passed. By looking in the Test Details, you can see the tests were configured to run in the source branch. After reviewing the test results, you save them to the baselines of the source branch. The updated baselines will not affect any tests running in other branches.

After updating your baselines, you can check that other team members did not make conflicting changes to the baseline. You can do this by checking the SCM status of the Applitools request, which is currently pending. Next, open the Pull Request page. This page displays the details of all the changes in the baselines, compares them, and allows you to edit the source baseline by porting ignore regions and locations from the target baseline, or by adding new ones. The page shows that the Applitools SCM status changed to green and no baseline conflicts were found. This means the two changes in the source branch are ready to be merged with the target branch. As soon as your source code is fully merged, Applitools will automatically merge the changed baseline in the source branch with the baseline of the target branch.

Visual UI Version Control

In a very short time, GitHub has grown from an obscure developer-centric website into an essential tool for a wide range of enterprises. It serves the needs of freelance contractors, large corporations, and governments. GitHub may have started as a cloud-based version control system, but today, it provides additional services such as project management, online documentation, web hosting, and defect tracking. For many companies, GitHub has replaced their on-premises source control management systems, and it functions as an essential piece of their CD/CI tooling.

In this post, we looked at how to use GitHub as part of your DevOps stack. We explained how the Applitools Eyes GitHub integration automatically configures the baselines used for obtaining and saving test baselines, how to create pull requests, and how you can add visual test validation to your CI pipeline.

This solution gives you powerful tools that ensure each time a pull request triggers a build and merges your target and source branches, it also gives you control and insight into the process. The integration also simplifies the process of reviewing and managing visual test baselines. More importantly, it is a form of UI version control that shows how your application’s UI has changed and evolved over time. These Applitools Eyes GitHub integration benefits will ensure the quality of your products and will benefit your end users and customers.

The post How to Test Your GitHub Pull Requests so You Don’t Break Your UI appeared first on Automated Visual Testing | Applitools.

GitHub Archives - Automated Visual Testing | Applitools

AI-Powered Test Automation: How GitHub Copilot and Applitools Can Help

GitHub Copilot can generate automated tests.

Applitools Execution Cloud provides self-healing test capabilities.

Together, tools like Copilot and Applitools can transform your test automation.

Our Best Test Automation Videos of 2022 (So Far)

Cross Browser Test Automation

Modern Cross Browser Testing with Cypress

Modern Cross Browser Testing in JavaScript Using Playwright

Modern Cross Browser Testing for Storybook Components

Test Automation with GitHub or Chrome DevTools

Make Testing Easy with GitHub

Automating Tests with Chrome DevTools Recorder

Test Automation Stories from Our Customers

Accelerating Visual Testing

Continuously Testing UX for Enterprise Platforms

Everything You Need to Know About Integrating Applitools with GitHub

Overview

Use cases covered in this article:

What is UI Version Control?

Integration with GitHub via GitHub app

Integration with GitHub server which is installed on-prem

[Legacy Integration] with GitHub cloud and on-prem via direct integration

Integration with GitHub Actions

Additional Links on GitHub & Applitools

Additional Source Control Integrations to unlock UI Version Control

Get a Jump Into GitHub Actions

Introductions

What Are GitHub Actions?

Build Test Deploy

Get Back To What You Like To Do

Common Actions

Jump Into GitHub Actions – What Next?

How to Setup GitHub Actions with Cypress & Applitools for a Better Automated Testing Workflow

What are GitHub Actions?

Using custom GitHub Actions to simplify complex workflows

Scaffolding a Cypress environment in a GitHub Action workflow with Docker

Installing Cypress and Action dependencies

Dynamically creating a sitemap in node

Running Cypress as a script in node

Using Cypress with GitHub Actions to dynamically run visual tests

What’s next for Applitools Eyes GitHub Action?

Intro to GitHub Actions for Test Automation — with Angie Jones and Brian Douglas [webinar recording]

Angie’s Slide Deck

Full Webinar Recording

Additional GitHub Action Links Provided by Brian

Additional Resources and Reading Materials

— HAPPY TESTING —

Learn from Google – Use Google Cloud Build

Integrate Applitools with Google Cloud Build

Continue With Applitools Setup

Run Your Cypress Tests

Watch the video to try out the integrations & Keep Testing!

Review – Automated Visual Testing With WebdriverIO

Course Introduction

Course Structure

Chapter 1 – Why Visual Testing

Chapter 2 – Getting Started With Applitools

Chapter 3 – Create And Run A Visual Test with Applitools

Chapter 4 – Viewports and Match Levels

Chapter 5 – Checkpoints

Chapter 6 – Batches

Chapter 7 – Using the Test Manager

Chapter 8 – Integrations

Conclusion

For More Information

Vue.js Development with Storybook and Applitools

How Applitools Works With Storybook

Automated Root Cause Analysis

Step By Step Guide

In Conclusion

Video Demonstration

More about Applitools

How to update Jenkins build status in GitHub pull requests [Step-by-Step Tutorial]

The Big Picture

Get the basics in place

What you will do in this tutorial:

Setup GitHub

Generate a GitHub access token

Jenkins setup