Insights batch statistics

Eyes groups all batches that are associated with the same Batch sequence name into a chronological sequence of batch executions. A Batch sequence. Eyes collects, analyzes and presents batch and test execution status information and statistics for each batch sequence that was executed within the Scope and Date Range values that you set on the Batch statistics tab.

For more information see Setting the Scope and Setting the Date Range.

Viewing batch sequence status information

Each Batch sequence is presented in a separate row in the Batch sequence list. Each Batch sequence row indicates the status and execution time of the last batch in the sequence, and shows a summary of all the batch runs of the sequence for a defined time period including the number of passed, failed and unresolved batches and the total number of batch runs.

Batch sequence statistics

The Insights Batch Statistics tab displays a list of batch sequences and their statistics over time. Clicking a batch sequence in the list, expands the row to show a collection of charts that deal with various pieces of Batch sequence status information and statistics. All statistics are based on the current status information for a particular metric.

Each of the charts is colored coded and accompanied by a legend. Hovering the mouse pointer over part of a chart displays explanatory information on the data metrics or status for that point of the chart.

Test results status

The Test Results Status is a Pie chart that displays the pass rate for all the tests that were executed over a set period of time. The Pass rate represents the ratio of the number of tests that passed as compared to the number of tests that did not pass, expressed as a percentage.

Diff status

A Pie chart that displays the percentage of Diffs that are currently accepted, rejected and unresolved for the selected batch sequence.

Baseline status

Indicates the status of steps and changes on the baseline. Shows the number of Saved steps, Unsaved steps and Unsaved changes.

1. Saved steps refers to steps that at some point were saved to the baseline.
2. Unsaved steps refers to steps that were never saved to the baseline.
3. Unsaved changes refers to steps that contain changes that are yet to be saved to the baseline.

Tests

Indicates the total number of tests in the batch and the number of those tests that are Completed, Running and Aborted.

Checkpoints

Browsers

A bar chart that shows a bar for each browser on which batches were executed as part of the batch sequence.

Devices

A bar chart that shows a bar for each emulated device for which tests were rendered as part of the batch sequence.

Operating systems

A bar chart that shows a bar for each operating system on which batches were executed as part of the batch sequence.

Viewports

A bar chart that shows a separate bar for each Viewport size for which tests were rendered as part of the batch sequence.

Test assignment

A bar chart that shows a bar for each team member and indicates the number of tests that were assigned toe each team member. Tests that were not assigned to any team member are represented by a bar that is labeled as Unassigned.

Issues and remarks

A bar chart that shows two bars, one for Issues and one for Remarks, that indicate the number of Issue and Remark regions that are applied in steps that were checked as part of the batch sequence. The Issues and Remarks bars are color-coded to show the number of Issues or Remarks that are New or that were already Existing. If Issue or Remark regions were applied to a batch sampled at a point in time within the set Date Range then those Issues and Remarks are listed as New. If the regions were applied at a point in time before the set Date Range then they are listed as Existing.

Setting the Batch sequence name

To group batches into a Batch sequence Eyes requires that batches be associated with a Batch sequence name. The Batch sequence name can be set and associated with a batch in several ways.

1. You can set the Batch sequence name explicitly ins the SDK using the Eyes SDK method class$batchinfo$setsequencename.Do this when setting up the batch and before calling class$eyes$open
2. If you do not set the Batch sequence name explicitly in the SDK and the environment variable APPLITOOLS_BATCH_SEQUENCE is defined then the SDK uses the value of that environment variable.
3. If you use an Applitools supported SCM or CI integrated plugin then they automatically set the environment variable.

No batch sequence information page

If no sequenced batches were run in the specified time period, for a given team, then Eyes cannot display any status information or batch statistics in the Batch Statistics tab.

In this case, the Batch Statistics tab displays information about batch sequences and a link to the documentation that describes how to define batch sequence names.