How to organize your tests with batches

In the Test Manger, test results are organized by batches, and tests within those batches. Batching tests together can make analysis and management of your test results easier and more convenient. Eyes also has features that operate on the tests in a batch as a whole, such as filtering, deletion, and grouping by similarity.

In the screen shot below you can see a list of batches in the left panel. Scrolling through the list you can see all the batches run by your team. When you click on a batch it becomes the selected batch and the right panel displays the tests that are associated with that batch.

Assigning tests to batches

You use the Eyes SDK to associate tests with a batch when the test is run. When using the SDKs that support the ClassicRunner and VisualGridRunner classes, by default, the results of all tests executed by a runner will be in the same batch. On other SDKs, each test execution will by default be in its own batch. In either case, the SDK provides a flexible interface that allows you to assign tests to batches, both when the tests run in the same execution run and when they run in separate execution runs. For details of how to do this see the article How to group tests into batches using the SDK. If you generate tests using Selenium IDE then you can read in the article Organizing tests with Selenium IDE how the entities used in Selenium IDE such as Project and test suites relate to batches.

Using batches in the Test Manager

After the tests run, you can use the Eyes Test Manager Test Results page to search for and select the batch you are interested in and see the results of all the tests that are part of the batch.

The Test Manager Test Results page is divided into two main parts. The left panel displays a Batch List which lists your Teams batches. Clicking on an entry in the batch list makes it the “current batch”, and updates the right panel with information about that batch. At the top, of the panel, there is summary information about the batch as a whole, and underneath the panel you can choose between various views of the batches tests that focus on the batch summary, tests, steps, or remarks and issues as well as filter by various criteria. For more information see Test Manager Views.

The batch selection panel

The main part of the batch selection panel in the left panel is a scrollable list of batch entries. Initially, 30 entries are displayed, if you scroll down to the end of the list, then you get access to older batches. Each entry in the list displays the name of the batch, when the first test in the batch ran, and a summary of the test status. It also shows if the test information has been changed and not saved. When you click on any element in this list Test Manager loads the information about that batch into the right panel.

Above the Batch list, Test Manager displays the number of batches currently displayed in the batch list. It also displays a “Load latest batches” icon which will flash if new batches are available that are not yet in the list. Clicking on this button will update the list with the latest batches. There are additional controls that allow you to filter the batches in the batch list and to delete selected batches.

You can drag the right edge of the batch list to the right to enlarge it if your batch names are large. The panel is extended when your cursor is hovering over the batch list.

Further reading

You can read more about batch related features in the following articles:

  • How to group tests into batches using the SDK. This article describes how to use command in the Eyes SDK to define that multiple tests should be assigned to the same batch.
  • Batch completion notifications. This article describes how to setup Eyes so that you get a notification via email or Slack when the batch is complete.
  • Test and batch status.This article describes the various values that the Test Manager displays as the batch status and what they mean.
  • Viewing the batch details. This article describes how to view details about a specific batch, including it's name, when it started, it's duration, batch ID and batch sequence name, as well as any user defined batch properties.
  • How to filter and search batches.This article describes how to use the Test Manager to look for particular batches based on criteria you choose such as its name or status.
  • Using batch properties. This article describes how you can use the SDK to associate user defined properties with a batch, and then in the Test Manager see these properties for a particular batch, or search for batches with specific properties.
  • How to rename a batch. This article describes how to change the name of a particular batch.
  • How to delete batches.This article describes how to delete one or more batches from the Eyes system.