Last week, Applitools hosted Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps, where I explored how artificial intelligence, specifically ChatGPT, can revolutionize the field of test automation. In this article, I’ll share the audience Q&A as well as some of the results of the audience polls. Be sure to read my previous article, where I summarized the key takeaways from the webinar. You can also find the full recording, session materials, and more in our event archive.

Audience Q&A

Our audience asked various questions about the data and intellectual property when using AI and adding AI into their own test automation processes.

Intellectual properties when using ChatGPT

Question: To avoid disclosing company intellectual properties to ChatGPT, is it not better to build a “private” ChatGPT / large language model to use and train for test automation inside the company while securing the privacy?

My response: The way a lot of organizations are proceeding is setting up private ChatGPT-like infrastructure to get the value of AI. I think that’s a good way to proceed, at least for now.

Data privacy when using ChatGPT

Question: What do you think about feeding commercial data (requirements, code, etc.) to ChatGPT and/or OpenAI API (e.g. gpt-3.5-turbo) data privacy connected to the recent privacy issues like with Samsung, exposure of ChatGPT chats, and so forth?

My response: Feeding public data is okay, because it’s out in the public space anyway, but commercial data could be public or it could be private and that could become an issue. The problem is we do not understand enough about how ChatGPT is using the data or the questions that we are asking it. It is constantly learning, so if you feed a very unique type of question that it has never come across before, the algorithm is intelligent to learn from that. It might give you the wrong answer, but it is going to learn based on your follow-up questions, and it is going to use that information to answer someone else’s similar question.

Complying with data regulations

Question: How can we ensure that AI-driven test automation tools maintain compliance with data privacy regulations like GDPR and CCPA during the testing process?

My response: It’s a tough question. I don’t know how we can ensure that, but if you are going to use any AI tool, you must make sure you are asking very focused, specific questions that don’t disclose any confidential information. For example, in my demo, I had a piece of code pointing to a website asking it a very specific question how to implement it. That question could be implemented using some complex free algorithms or anything else, but taking that solution, you make it your own and then implement it in your organization. That might be safer than disclosing anything more. This is a very new area right now. It’s better to be on the side of caution.

Adding AI into the test automation process

Question: Any suggestions on how to embed ChatGPT/AI into automation testing efforts as a process more than individual benefit?

My response: I unfortunately do not have an answer to this yet. It is something that needs to be explored and figured out. One thing I will add is that even though it may be similar to many others, each product is different. The processes and tech stacks are going to vary for all these types of products you use for testing and automation, so one solution is not going to fit everyone. Auto-generated code will go up to a certain level, but at least as of now, the human mind is still very essential to use it correctly. So it’s not going to solve your problems; it is just going to make solving them easier. The examples I showed are ways to make it easier in your own case.

Using AI for API and NFRs

Question: How effective would AI be for API and NFR?

My response: I could ask a question to give me a performance test implementation approach for the Amazon website, and it gives me a performance test strategy. If I ask it a question to give me an implementation detail of what tool I should use, it is probably going to suggest a few tools. If I ask it to build an initial script for automating this performance test, it is probably going to do that for me as well. It all depends on the questions you are asking to proceed from there, and I’m sure you’ll get good insights for NFRs.

Using AI a dedicated private cloud instance

Question: Our organization is very particular about intellectual property protection, and they might deny us from using third-party cloud tools. What solution is there for this?

My response: Applitools uses the public cloud. I use a public cloud for my learning and training and demos that I do, but a lot of our customers actually use the dedicated cloud instance, which is hosted only for them. Only they have access to that, so that takes care of the security concerns that might be there. We also work with our customers to ensure from compliance and security perspectives that all the questions are answered and to make sure everything conforms as per their standards.

Using AI for mobile test automation

Question: Do you think AI can improve quality of life for automation engineers working on mobile app testing too? Or mostly web and API?

My response: Yes, it works for mobile. It works for anything that you want. You just have to try it out and be specific with your questions. What I learned from using ChatGPT is that you need to learn the art of asking the questions. It is very important in any communication, but now it is becoming very important in communicating with tools as well to get you the appropriate responses.

Audience poll results

In the live webinar, the audience was asked “Given the privacy concerns, how comfortable are you using AI tools for automation?” Of 105 votes, over half of the respondents would be somewhat or very comfortable with using AI tools for automation.

• Very comfortable: 16.95%
• Somewhat comfortable: 33.05%
• Not sure: 24.59%
• Somewhat comfortable: 14.41%

Next steps

You can take advantage of AI today by using Applitools to test web apps, mobile apps, desktop apps, PDFs, screenshots, and more. Applitools offers SDKs that support several popular testing frameworks in multiple languages, and these SDKs install directly into your projects for seamless integration. You can try it yourself by claiming a free account or request a demo.

Check out our events page to register for an upcoming session with our CTO and the inventor of visual AI Adam Carmi. For our clients and friends in the Asia-Pacific region, be sure to register to attend our upcoming encore presentation of Unlocking the Power of ChatGPT and AI in Test Automation happening on April 26th.

Editor’s note: This article was written with the support of ChatGPT.

Last week, Applitools hosted Unlocking the Power of ChatGPT and AI in Test Automation: Next Steps, when I discussed how artificial intelligence – specifically ChatGPT – can impact the field of test automation. The webinar delved into the various applications of AI in test automation, the benefits it brings, and the best practices to follow for successful implementation. With the ever-growing need for efficient and effective testing, the webinar is a must-watch for anyone looking to stay ahead of the curve in software testing. This blog article recaps the key takeaways from the webinar. Also, you can find the full recording, session materials, and more in our event archive.

Takeaways from the previous webinar

I started with a recap of the takeaways from the previous webinar, Unlocking the Power of ChatGPT and AI in Testing: A Real-World Look. The webinar focused on two main aspects from a testing perspective: testing approach mindset (strategy, design, automation, and execution) and automation perspective. ChatGPT was able to help with automation by guiding me to automate test cases more quickly and effectively. ChatGPT was also able to provide solutions to programming problems, such as giving a solution to a problem statement and refactoring code. However, there were limitations to ChatGPT’s ability to provide answers, particularly in terms of test execution, and some challenges when working with large blobs of code.
If you didn’t catch the previous webinar, you can still watch it on demand.

What’s new in AI since the previous webinar

Since we hosted the previous webinar, there have been many updates in the AI chatbot space. A few key updates we covered in the webinar include:

• ChatGPT has become accessible on laptops, phones, and Raspberry Pi, and can be run on own devices.
• Google Bard was released, but it is limited to English language, cannot continue conversations, and cannot help with coding.
• ChatGPT 4 was released, which accepts images and text inputs and provides text outputs.
• ChatGPT Plus was introduced, offering better reasoning, faster responses, and higher availability to users.
• Plugins can now be built on top of ChatGPT, opening up new and powerful ways of interaction.

During the webinar, I gave a live demo of some of ChatGPT’s updates, where it was able to provide code implementation and generate unit tests for a programming question.

Using AI to address common challenges in test automation

Next, I discussed the actual challenges in automation and how we can leverage AI tools to get better results. Those challenges include:

• Slow and flaky test execution
• Sub-optimal, inefficient automation
• Incorrect, non-contextual test data

Using AI to address flakiness in test automation

The section specifically focused on flaky tests related to UI or locator changes, which can be identified using consistent logging and reporting. I advised against using a retry listener to handle flaky tests and instead suggest identifying and fixing the root cause. I then demonstrated an example of a test failing due to a locator change and discussed ways to solve this challenge.

Read our step-by-step tutorial to learn how to use visual AI locators to target anything you need to test in your application and how it can help you create tests that are more resilient and robust.

Using AI to address sub-optimal or inefficient information

Next, I discussed sub-optimal or inefficient automation and how to improve it. I use GitHub Copilot to generate a new test and auto-generate code. I explained how to integrate GitHub Copilot and JetBrains Aqua with IntelliJ and how to use Aqua to find locators for web elements. Then, I showed how to implement code in the IDE and interact with the application to perform automation.

Using AI to address incorrect, non-contextual test data

Next, I discussed the importance of test data in automation testing and related challenges. There are many libraries available for generating test data that can work in the context of the application. Aqua can generate test data by right-clicking and selecting the type of text to generate. Copilot can generate data for “send keys” commands automatically. It’s important to have a good test data strategy to avoid limitations and increase the value of automation testing.

Potential pitfalls of AI

AI is not a magic solution and requires conscious and contextual use. Over-reliance on AI can lead to incomplete knowledge and lack of understanding of generated code or tests. AI may replace certain jobs, but individuals can leverage AI tools to improve their work and make it an asset instead of a liability.

Data privacy is also a major concern with AI use, as accidental leaks of proprietary information can occur. And AI decision-making can be problematic if it does not make the user think critically and understand the reasoning behind the decisions. Countries and organizations are starting to ban the use of AI tools like ChatGPT due to concerns over data privacy and accidental leaks.

Conclusion

Overall at first, I was skeptical about AI in automation, but that skepticism has reduced significantly. You must embrace technology or you risk being left behind. Avoid manual repetition, and leverage automation tools to make work faster and more interesting. The automation cocktail (using different tools in combination) is the way forward.

Focus on ROI and value generation, and make wise choices when building, buying, or reusing tools. Being agile is important, not just following a methodology or procedure. Learning, evolving, iterating, communicating, and collaborating are key to staying agile. Upskilling and being creative and innovative are important for individuals and teams. Completing the same amount of work in a shorter time leads to learning, creativity, and innovation.

Be sure to read my next article where I answers questions from the audience Q&A. If you have any other questions, be sure to reach out on Twitter or LinkedIn.

My answers to questions I received at TAU Conference 2023

Recently it was a pleasure to give a talk about the role of automation in Mobile Continuous Testing at the TAU Conference powered by Applitools, and actually, it was an excellent opportunity to meet different test automation experts and attendees from all over the world.

If you missed the session, you could find the on-demand recording here: applitools.info/kte

During the session, I got various questions about mobile testing and CI, so I wrote this blog to answer them all.

Let’s get started!

Question 1: Automating tests for different devices

Question: What is the most efficient approach when automating tests for the same mobile app on different (Android/iOS) devices? Automate with a tool that works for both or a tool for each one individually?

My answer: The best answer always depends on the team’s needs and goals. Using a tool that works for both platforms is a good choice, because this approach saves time and effort in test automation as it allows you to write test scripts once and run them on multiple devices and platforms such as using Appium. 

But sometimes, the team decides to use platform-specific testing tools such as Espresso for Android and XCUITest for iOS, because they need the mobile developers to work closely with the test engineers in writing the UI tests because they believe it’s a shared responsibility. 

Additionally, they can use the same programming language. Test scripts will be in the same repositories, and using the native locators from the app will be directly accessible in this case.

Question 2: Testing multi-platform apps

Question: Is the approach to/process for testing a multi-platform app like Evernote (mobile, desktop, web) different from a mobile-only app?

My answer: Testing a multi-platform app like Evernote requires a different approach because the following reasons:

• Each platform’s user interface, operating system, hardware, and software requirements (mobile, desktop, web) differ, so testing must account for these differences and ensure that the app operates as intended.
• To ensure consistency across all platforms, including features like note-taking, search, sync, and collaboration, each platform must be tested separately, as well as its integration.
• Testing the app across various platforms ensures the design, navigation, and features are consistent and intuitive.

In order to know how to deal with different platforms, these things should be included in the test plan and strategy at the beginning.

Question 3: Upcoming course content

Question: Is the outline described in your presentation today part of an existing course? Or will it be part of a future course re-haul of your existing course on TAU?
My answer: No, it’s not in my existing courses, but maybe we can have it in the future.

Question 4: Manual testing for mobile CI

Question: When in this mobile CI workflow would manual testing fit in? Or is it suggested that manual testing is not needed?

My answer: If your team would like to achieve the fully automated CI/CD to build, test, package, and release the mobile apps to the App Stores, manual testing will not fit in the lifecycle because the goal is to make the process fully automated as much as we can. But if the target is to have continuous integration only, you can build and package the apps and then send them to the manual testers to do the testing activities. After that, if there are no issues in the build or the release, you can continue with the release process of manually automating it. 

It always depends on the team goal and the release cadence. Some teams or companies release every one or two weeks; in this case, manual testing will be a blocker and releases can take a long time.

Question 5: Manual testing with an automation setup

Question: Is there still a need for manual testing, even with an automation setup?

My answer: Yes, manual testing can be used to identify issues not covered by automated tests, sometimes, different scenarios require a human eye, interaction, or decision. 

It always depends on the type of mobile app and the functionalities we need to cover; this can be added to the test plan and the test strategy from the beginning.

Question 6: Making apps testable

Question: Isn’t it also necessary that apps are made testable?

My answer: Yes, I totally agree, as a mobile development team, we should consider making the mobile app testable by doing the following tips:

• By writing modular code, you can test individual components independently.
• Make your app’s logic, data, and presentation layers separate using design patterns like MVVM or MVP. This will make testing easier.
• Decouple dependencies between classes through dependency injection. This will make it easier to swap out components during testing.

Question 7: Using tools for mobile test automation

Question: If you’re not strong in mobile app automation, for something like a React Native app, would you stick with something like Appium (which is familiar with native automation/Selenium web), or would it be better to try the Maestro (or something similar)?

My answer: The POC (Proof of Concept) project is always the best, in my opinion. If you already have a React Native app, you can try to create a project and try to automate the app with Appium, as you know, Appium is a black-box testing tool so at the end you need only the .apk or .ipa files to be able to automate them, which is not different with React Native apps because at the end you will have already two native apps (iOS and Android). Or you can try Maestro. It’s a new framework, and it may be beneficial in your case.

Question 8: Staying up-to-date on industry skills

Question: Does the coming of flutter make the tester’s life easy?

My answer: I cannot guarantee that, as a tester, we face new challenges every day. For instance, we must learn about new testing tools and technologies, which requires us to stay on top of our game. In contrast, as we can see these days, AI is booming and is affecting our daily work, which makes acquiring unique skills indispensable.

Question 9: Preparing for ISTQB Foundation Certification

Question: How best to prepare for an ISTQB Foundation Certification?

My answer: It depends on which ISTQB certificate you want to achieve, for instance, if it’s the foundation level, you need at least 6 months of experience with software testing to understand the fundamentals and the concepts in the Syllabus. 

As you can see in the following image, the certificates have different tracks and levels. Each level required specific requirements and a different level of experience. You can find all the details on the ISTQB official website, including the Syllabus and the sample questions with answers.

Besides reading the Syllabus, your background will be enough.

Wrapping up

I hope you enjoyed these answers to your questions, and I look forward to seeing you at the TAU and Applitools conferences in the near future.

Please do not hesitate to contact me if you have any other questions.

My LinkedIn: https://www.linkedin.com/in/moataz-nabil/

My Twitter: @moatazeldebsy

Thank you for reading.

Happy testing!

Test Automation University (TAU) is the premier platform for learning about software testing and automation. Powered by Applitools, TAU offers dozens of courses on all testing topics imaginable from the world’s best instructors for FREE.

In my previous article, I shared a breakdown of TAU by the numbers. I covered all the stats and metrics about TAU today. Now, in this article, I’d like to share the future by publicly announcing all the new content we have scheduled in our development pipeline!

New courses

Test Automation Design Patterns by Sarah Watkins

Up first is Test Automation Design Patterns by my good friend Sarah Watkins. Anyone who’s ever worked on a test project knows that good test cases don’t just happen – they’re built intentionally. In this upcoming course, Sarah teaches everything she knows about designing and delivering test automation projects at scale, from formulating tests to follow Arrange-Act-Assert, to using dependency injection for handling test inputs and other objects, to deciding between page objects or the Screenplay Pattern. This will be an advanced course to really help you become the ultimate SDET, or “Software Development Engineer in Test.”

GitHub Actions for Testing by Matthias Zax

Next up is GitHub Actions for Testing by Matthias Zax. Automation means much more than merely scripting test case procedures. It includes automating repetitive segments of your development process, like running tests and reporting results. GitHub Actions are a marvelous, lightweight way to add helpful workflows to manage your source code repositories. In this course, Matthias teaches how to create GitHub Actions specifically for testing and quality concerns. I know GitHub Actions have transformed how I run my own GitHub projects, and I’m sure they’ll make a big difference for you, too!

Behavior-Driven Development with SpecFlow by Bas Dijkstra

Next is a course for a topic very near and dear to my own heart: Behavior-Driven Development with SpecFlow by Bas Dijkstra. Behavior-Driven Development, or “BDD” for short, is a set of pragmatic practices that help you develop high-quality, valuable behaviors. Not only will Bas walk through the phases of Discovery, Formulation, and Automation, but he will show you how to use SpecFlow – one of the best BDD test frameworks around – to automate Gherkin scenarios. He will also teach how to integrate SpecFlow with popular libraries like Selenium WebDriver and RestAssured.NET.

I hope you’re excited for those three upcoming courses. Course development takes time, so we don’t have a hard publishing date set, but be on the lookout for them this year. I know Sarah, Matthias, and Bas are all hard at work recording their chapters.

New learning paths

That’s not all our new content. We have a lot more to announce today, so let’s keep going.

Visual Testing Learning Path

We have a new learning path to announce: Visual Testing! Applitools’ very own Matt Jasaitis is developing a three-part series on visual testing. The example code uses Selenium WebDriver with Java, but the concepts he teaches are universal. The first two courses in this learning path are already available. The third course, with the tentative title Expert Visual Testing Practices, will be available by April.

Leadership Learning Path

Historically, Test Automation University has focused almost exclusively on testing and automation skills. As we grow in our career as professionals, we often find we need to develop other kinds of skills as well. That’s why I’m excited to announce that TAU will be developing a new learning path for leadership!

Managing Test Teams by Jenny Bramble

The first course in our new leadership track will be Managing Test Teams by Jenny Bramble. At some point in our careers, many of us face the question, “Should I become a manager?” Becoming a good manager requires a different skill set than becoming a good tester or a good engineer. It comes with a new set of challenges that you might not expect. In this course, Jenny will share everything she’s learned about leading teams of testers, holding crucial conversations, and deciding if a career path in management is right for you. Jenny is also one of our speakers later today, so be sure to join her session.

Building Up Quality Leaders by Laveena Ramchandani

The second course will be Building Up Quality Leaders by Laveena Ramchandani. While Jenny’s course will focus primarily on being a good manager, Laveena’s course will focus on attributes of leadership that apply to any role within software quality. Laveena will cover leadership skills like servanthood, charisma, decisiveness, and gratitude, and how they all apply to the testing world.

Creating Effective Test Strategies by Erin Crise

The third course will be Creating Effective Test Strategies by Erin Crise. The key to leading successful testing initiatives is a solid, well-grounded, well-balanced strategy. The strategy must balance automation with exploration. It must include a variety of areas such as visuals, accessibility, and performance. It must be communicated effectively across teams. And it must be agile enough to adjust to changes. With her wealth of experience from several software projects, Erin will teach how to plan and execute effective test strategies in this course.

Just like for the other upcoming courses I just announced, these leadership courses aren’t ready yet. Expect them to drop later this year. We are also planning a few more courses in this leadership path, so stay tuned for more details!

Refreshing old courses

Now, we said earlier how TAU is over four years old. In the world of tech, that’s a long time. I hate to say it, but some of our courses are becoming outdated. Tools and frameworks evolve – as they should. It’s time to refresh many of our courses.

Last year, we published our first refresh courses: Introduction to Robot Framework by Paul Merrill and UI Automation with WebdriverIO v7 by Julia Pottinger. Both refresh courses were well received by our community of students.

After auditing our whole course catalog, I decided that it’s time to refresh some more courses!

Cypress courses by Filip Hric

Our intro and advanced Cypress courses are among our most popular. In fact, Introduction to Cypress is currently ranked #8 for monthly completions. This is no surprise given Cypress’s incredible popularity with front end developers. However, our mainstay courses were published before many of Cypress’s current features, like component testing and the new directory structure introduced with Cypress 10.

Filip Hric will take up the challenge to refresh both the intro and advanced Cypress courses. He will redevelop them to flow together cohesively as parts 1 and 2. We will also create a new Cypress learning path for them once we publish them.

Playwright courses by Renata Andrade

As a Playwright Ambassador, I can’t let Cypress have all the fun. Playwright is so hot right now. The community is booming, the downloads are skyrocketing, and even within Applitools and TAU, we see its adoption growing significantly. That’s why we are not only refreshing our introduction-to-Playwright course, but we are also going to publish an all-new Advanced Playwright course!

Our instructor Renata Andrade, like Filip, will develop these courses to be parts 1 and 2. She will also develop them in TypeScript. And you’d better believe we’ll create a Playwright learning path for them as well.

Mobile Automation with Appium in Java by Moataz Nabil

Let’s not forget mobile test automation as well. Moataz Nabil is going to split his course, Mobile Automation with Appium in Java, into two parts. At six and a half hours, this course is currently the longest on TAU. By splitting it into two parts, we hope to make it easier for students to complete.

Web Element Locator Strategies by Andrew Knight

And finally, there’s one more course we plan to redevelop: my very first course, Web Element Locator Strategies. In my refresh, I intend to develop new subchapters to show how to write locators in different frameworks like Playwright and Cypress, not just in Selenium Java. I’ll include new types of locators as well.

Retired courses

We also needed to retire a few of our courses that are out of date:

1. Automated Visual Testing with Java by Angie Jones
2. AI for Element Selection by Jason Arbon
3. From Scripting to Framework with Selenium and C# by Carlos Kidman

In case you missed it, we removed these courses from TAU at the beginning of March. I want to thank these instructors for making these courses.

A full curriculum

I know I’ve talked a lot about learning paths today. One of the most requested new features we receive is certificates for completing learning paths. I’m sorry to say that we are not going to add certificates for learning paths. Learning paths are somewhat dynamic. They can change over time as we publish new courses. So, giving certificates for completing them doesn’t make sense.

Instead, we are toying with a stronger idea: a curriculum. A learning path concentrates on one narrow testing topic, like leadership or visual testing. I want to create a comprehensive program of courses covering a wide breadth of testing topics, and to recognize completion of that program with something like a diploma. We have a rich catalog of courses for building a strong curriculum. Right now, this is still in the idea stage. I’m hoping to bring it into reality later this year.

One more thing…

TAU is four years old, and while I hate to admit it, the TAU platform itself is starting to show its age. With so many courses available, it’s becoming hard to find the ones you want. On the backend, it actually takes quite a bit of manual labor to publish a new course, too.

It’s time to refresh the TAU platform. It’s time for TAU version 2.

Over the coming months, Matt Jasaitis and I will redevelop the whole TAU web app. Yes, that’s the same Matt Jasaitis who is creating the Visual Testing learning path. Our goal is to build a sustainable learning platform to serve Test Automation University for years to come. We will also build a better support system into the platform. Unfortunately, this project is so new that I don’t even have mockups to show you yet, but at least I can announce that TAUv2 is coming.

So, what do y’all think about all this new content coming to TAU? We’d love to hear from you. Let us know through TAU Slack.

Test Automation University (TAU) is the premier platform for learning about software testing and automation. Powered by Applitools, TAU offers dozens of courses on all testing topics imaginable from the world’s best instructors for FREE.

In this article, I’d like to share a breakdown of Test Automation University by the numbers: all the stats and metrics that we gather for insights into the program. I also shared this information in my keynote address for Test Automation University’s 2023 conference, What’s New With TAU? Let’s dive in!

Students, courses, and achievements

Believe it or not, it’s been four years since Applitools launched TAU! Four years is a long time. That’s as long as a typical college undergraduate degree.

I remember feeling thrilled when Applitools first launched TAU. Before then, there wasn’t any cohesive, community-oriented education platform for testing and automation. Most schools barely included testing topics in their programs. While the Web had a wealth of information, it was difficult to know which resources were both comprehensive and trustworthy. Test Automation University provided a central platform that could be trusted with literally the best and the brightest instructors in the world.

Now, I can’t imagine a world without TAU. At least once a week, someone I’ve never met before slides into my DMs and asks me, “Andy, how can I start learning about testing and automation?” I literally just tell them, “Have you heard of Test Automation University?” Boom. Done. I don’t have to check my bookshelf for titles or do a quick search for blog articles. I just say go to TAU. They’re always grateful.

And when they do go to TAU, they join our ever-growing community of students! Just last month, we crossed 150 thousand registered “TAU-dents”! That’s a huge size for any community.

Every time we’ve hit a milestone, we’ve celebrated. When we hit 50 thousand students in June 2020, we threw our first TAU Party. Then, when we hit 100 thousand students, we hosted TAU: The Homecoming, our first virtual conference. That’s when I first joined Applitools. Now, at 150 thousand students, we’re hosting our second virtual conference.

Our growth is also accelerating. Historically, we have grown by 50 thousand students about every year and a half. This most recent milestone happened a little sooner than expected.

Every week, about 800 to 1000 new students sign up for TAU. That’s sustained, consistent growth. It’s awesome to see all the new students joining every week.

Over 4 years, we have published a total of 73 courses. That’s a lot of content! A single course has at least 4 chapters with at least 30 minutes of video lessons. Most courses average about 8 chapters over an hour of content, and our longest courses stretch to 6 hours! We try to publish a new course about once a month. And the best part is that every one of our courses is completely, totally, absolutely FREE!

Our courses would be nothing without the excellent instructors who teach them. To date, we have 48 different instructors who have produced courses for TAU. These folks are the real superstars. They are bona fide software testing champions. Let me tell you, it is no small effort to develop a TAU course.

When Applitools first launched TAU back in 2019, I remember thinking to myself as a younger panda, “Man, that’s awesome. I wish I could be part of that. Maybe someday I can develop a TAU course and be like them.” A few months later, Angie Jones slid into my DMs and asked me if I’d want to develop my first course: Web Element Locator Strategies. In all honesty, TAU is one of the main reasons I work at Applitools today!

Let’s look at achievements next. In total, all students have completed 162 thousand courses! That’s 162 thousand certificates and 162 thousand badges, averaging slightly more than 1 course completion per student.

The credit total is even more impressive. Our students have earned about 116 million course credits. That’s just under 800 credits per student, which correlates to about one course’s worth of credits. 116 million credits earned is a mind-blowing number, and that total is only going to increase with time.

Course completion analysis

Those are just some raw numbers on TAU. I also did some deeper analysis on our courses to learn what topics and formats are most valuable to our community. Let’s take a look at that together.

When evaluating courses, the main metric I measure is course completion – meaning, how many students completed the course and earned a certificate. Anyone can start a course, but it takes dedication to complete the course. Completion signals the highest level of engagement with TAU: a student must dedicate hours of study and coding to earn a course badge.

My goal for any course we publish – and how I determine if a course is “successful” in most basic terms – is to consistently hit 30 completions per month, or 1 completion per day. Very popular courses hit 2 completions per day, and our most popular courses hit at least 3 completions per day.

Our top 10 courses are all in that upper echelon of completion rate. Our most popular course by far is Angie Jones’ Setting a Foundation for Successful Test Automation, followed by my course, Web Element Locator Strategies.

All three of our programming courses – Java, JavaScript, and Python – appear in the top 10. Rounding out the top 10 are courses on Selenium WebDriver, Cypress, API testing, and IntelliJ IDEA. If you want to see more details about the most popular TAU courses, check out the article I wrote about it a few months ago on the Applitools blog. Note that some small shifts have happened since I wrote that article, but the information is still mostly accurate.

When partitioning the full catalog of courses by the programming languages they use, we can see that Java and JavaScript dominate the landscape. This should be no surprise, since those two languages are by far the most popular languages for test automation. Applitools product data also backs this up. TAU offers a good number of Python and C# courses that are reasonably popular. I personally developed 3 of those 7 Python courses. TAU also offers a handful of courses that use Ruby and other languages. Unfortunately, however, we have discovered that those courses, especially for Ruby, have very low completion rates. Again, I surmise that this reflects broader industry trends.

I also analyzed the completion rate for a course versus its length. What I found is probably not surprising: shorter courses have higher completion rates, while longer courses have lower completion rates. The average completion rate for all courses is 57%. The breaking point appears to be at about two hours, with a sweet spot between 60-90 minutes. Since we want to encourage course completions, we are going to encourage instructors to produce courses in the hour-to-hour-and-a-half range moving forward.

So, what’s next?

These numbers describe TAU as it is today. In my next article, I’ll share all the new plans we have for TAU, including upcoming courses!