Some questions I get asked a lot from customers are: Can I automate visual testing of videos? What are some techniques for automated video testing? What software testing tools should I use to test video?
They’re asking because manual video testing can consume a lot of time — hours, if not days. It can be a bottleneck in your testing efforts and slow down release cycles.
My answer: you CAN and should automate video testing! In this post, I’d like to show you how.
Video testing tip #1: Check every few seconds
This technique works well if you have a lot of video content throughout your website, and want to check if the correct video is displayed on a given page, as well as whether the videos actually play correctly.
With Selenium, Applitools, Ruby, and JQuery, video testing can be done easily. If you look at this example Ruby script (this will work in any language), you’ll see that with JavaScript you can extract details about a video like the duration, playback rate, and autoplay, amongst other things. See the full list of available methods.
So how does Applitools come into play here? With JavaScript, you can set the video to whatever time you want to capture a frame on the video. For example, if you wanted to capture the frame five seconds into the video you can use Selenium and JavaScript and set the “currentTime = 5”, then take a visual check and upload to Applitools. On calling eyes.close, if Applitools reports back any differences, you then know you video changed or is incorrect!
Furthermore, looking at this other example Ruby script you can see it grabs all videos on a given page and iterates through them capturing an image at the first, middle and last frames. This gives you further assurance your video plays correctly and it is exactly the correct video and play length.
Video testing tip #2: Ensure correct conversion
With Applitools, Ruby, and FFmpeg this kind of video testing is straightforward. If you look at this example Ruby script (this one as well will work in any language), you’ll see that with FFmpeg you can extract frames from a video at a given rate.
(What is FFmpeg? It’s a set of libraries that let you programmatically record, edit, convert, and stream videos.)
Anyways, for this script, I’m using an FFmpeg feature to extract “keyframes” (sometimes called index frames that are pictures used as a reference in video files).
Whatever you call them, they should select the same frames consistently on every extraction and reduce the overall amount of images generated. However, FFmpeg provides you the flexibility to pull out frames at any rate you want. FFmpeg is pretty nice — you can read more in their documentation.
So now that you have extracted images from a video, you can upload these to Applitools using our Images SDK. If Applitools reports back any differences, you then know your video has changed without a human ever having to view it. Additionally, if you ever need to edit a videos attributes or convert it from one format to another (e.g. WAV to MP4, AVI to MOV, etc.), you can use this method to thoroughly validate that the conversion happened properly. This saves you time without having to watch the entire video yourself!
To read more about Applitools’ visual UI testing and Application Visual Management (AVM) solutions, check out the resources section on the Applitools website. To get started with Applitools, request a demo or sign up for a free Applitools account.
