There’s a new claim to the throne of functional test automation tools: Cypress.io. Is Cypress fast? Yes. Is Cypress interactive? Yep. Is Cypress reliable? You bet. And best of all… it’s cool!
But is Cypress an alternative to Selenium WebDriver? Does Selenium, the current king of web automation and testing frameworks, cringe in fear for its position, or is it smiling benevolently at the supposed usurper, knowing full well that, well, it’s just a kid!
Is Cypress better than Selenium WebDriver? I get asked this a lot. And frankly, the easiest path of a “this vs that” article is to try and find out which is “best”. But I will not take that path. Instead, I will try and explain how is Cypress different from Selenium WebDriver.
By describing what is the difference between Cypress and Selenium WebDriver, you will be able to understand how Cypress works. This post will also try and explain why Cypress took the path it did to web browser automation, a path that is remarkably different than the path WebDriver took.
How is Cypress similar to Selenium WebDriver? They both control and automate the web browser, thus enabling you to write functional tests that simulate the actions that a user does, and verifying that the results are correct. But that is where the similarity ends, and the differences begin. Hopefully, knowing the differences between these two testing tools, and knowing the reasons for that difference, will help you choose which tool to use, and when.
So let’s start exploring these differences.
It’s For Frontend Developers!
Selenium WebDriver was built for end-to-end regression tests of web applications. The Selenium front end testing framework is mostly used by QA developers, and not so much developers. This is because the idea of front-end developers testing their own code is not common.
But a small revolution is starting to happen in the world of frontend developers. Frontend developers are starting to write their own tests. Yes, they are rising to the challenge of agile software methodologies. They understand that without writing their own tests, agile development cannot happen. They are writing unit tests, but also integration, and, yes, end-to-end functional tests that check their frontend using real browsers, just like the automation tests that QA developers write. The “Shift Left” movement has a lot of these trends.
Frontend developers’ needs for end to end tests are different than those of QA developers. Frontend developers don’t need a staging environment where the whole application—frontend, backend, and database systems—is deployed. They can get by with just the frontend part running locally, and mock the backend pretty easily.
This fact — the target audience of Cypress is frontend developers — lies at the heart of all the differences between Cypress and WebDriver. They’re different tools for different users.
It’s JavaScript Only!
Frontend web developers write in only one language—JavaScript. It is the only language that browsers can execute, and so they have no other choice. This limitation has its downsides and its upsides, but the fact exists—there is only one language for frontend developers (variants like TypeScript notwithstanding).
And, because Cypress is meant for frontend developers, Cypress tests can only be written in JavaScript. No other languages are supported. Compare that to Selenium WebDriver, which has an API with many language bindings, including Java, C#, Python, Ruby, R, Dart, Objective-C, and, yes, JavaScript.
So remember—if you want to do Cypress testing, you will have to learn JavaScript. Which is cool, because JavaScript today is not your grandparents’ JavaScript. It’s a modern, powerful, and concise language that is widely used today and has the largest open source package repository in the world (npm), a code repository that a few years back overtook what used to be the largest repository, the Java Maven repository.
It’s Mocha Only!
Cypress doesn’t only limit you in the language you can write, but also limits which testing framework you use. Mocha is the testing framework that you write your tests with (a testing framework’s role in JavaScript is similar to the role Junit has in Java, or NUnit in C#). You cannot write your tests in another JavaScript framework, such as Jest or Tape.
Selenium WebDriver, on the other hand, does not impose a testing framework on you. Moreover, you don’t even have to use a testing framework. You can write a regular program that is not a test, a program that just drives the browser. Many projects use WebDriver that way, for example to crawl web pages and gather information.
But Cypress decided early on that it is dedicated to the task of writing frontend tests, and can only be used within a testing framework, and that testing framework MUST be Mocha.
Enough already. Show Me Some Cypress Code!
“OK, OK, I got it,” I hear you saying. But can you please show me a Cypress test so that all this will be a bit less abstract? Sure, here you go:
As you can see, this has examples of Cypress commands and assertions.
cy.visit will make the browser navigate to the url given.
cy.get will return a reference to the element in question, on which you can either type, click, or check assertions on using contains and other assertions.
As you can see, on the surface, it’s surprisingly similar to Selenium Webdriver, albeit simpler, and with much more power.
It’s Chrome Only!
The Cypress test runner only works on Chrome. It has no support for Firefox, Safari, Edge, or IE. This may come as a shock to QA developers that are used to WebDriver’s amazing support for all these browsers. But, yes, Cypress is Chrome only. There is an issue that was opened for cross-browser support, but that issue is almost a year old. Cypress’ priorities, it seems, lie elsewhere.
Why would frontend developers want to run their tests only in Chrome? I believe there are multiple answers to that question. First, the differences between Chrome, Firefox, Safari, and Edge these days are small, and getting smaller and smaller for most applications. So checking only in Chrome is probably good enough when testing in development.
Second, running all the tests on all the browsers takes time, and frontend developers, as we said above, don’t want to wait for their tests.
Third, many companies have QA developers that run end to end tests on all browsers, and so they are OK with the very rare bug slipping now and again and being caught by the QA team. And companies that don’t have extensive cross-browser tests? Then they are OK with those same rare bugs slipping and reaching users now and again.
Fourth, frontend developers do care, and would like Cypress to support more browsers, but it’s just not a priority for them, given the reasons described above.
It Runs in The Browser!
Besides the fact that frontend developers are used to JavaScript, there’s another reason, a more technical one, that only JavaScript is supported.
The code you write in your Cypress test scripts does not run outside of the browser, like in WebDriver. It runs inside the browser. It is the browser that is executing your test code. In fact, it’s executing your test code alongside the app code that you are testing!
What happens when you run your test is that, first, Cypress will prepare your code to run in the browser (technically speaking, it will bundle all the modules in your code to one JS file using webpack). Once ready, it will run Chrome, and inject your test code into a blank page. Then it will run that test code, in the browser.
That test code first of all navigates to the application page (using `cy.navigate(‘http://localhost:8080’)`), which is run in an iframe. Cypress will make the iframe navigate to that page, but your code is running in the browser, and (somewhat magically) has access to the same DOM as that iframe, and so all commands, such as “click” or “type”, happen inside the browser, using regular DOM APIs.
To compare, let’s review Selenium WebDriver architecture and what this means for how Selenium tests run. Take a look at the diagram below.
With Selenium WebDriver, you have three processes:
- Selenium WebDriver
- A browser driver, such as ChromeDriver, GeckoDriver (for Firefox), EdgeDriver, SafariDriver, etc.
- The browser itself
All the communications between these processes means that Selenium tests take a long time to run. Which takes us to our next point:
It’s Fast!
This in-browser execution is at the basis of the speed of Cypress testing. With Cypress, you have just one process: the browser itself. With Cypress, your test code is running alongside your application code.
So an automation command (e.g., clicking a button) does not send the command to the browser like WebDriver does through out-of-process communication. Instead, Cypress uses DOM events to send a click command to the button. Much MUCH faster.
And while WebDriver’s out of process automation involves asynchronous communication, Cypress’ automation commands are mostly synchronous and in-memory. This generates extremely fast tests.
I believe that it is this speed that is the main reason that frontend developers are enamored with Cypress. WebDriver, at least perception-wise, did not give them the speed that they needed. Frontend developers want to run their tests ALL THE TIME, and cannot afford the luxury of a suite that takes tens of minutes to run. They want to run their suite, and they need it to run in a minute or two, not more.
Is this speed difference really substantial as Cypress’ documentation claims? From my admittedly small experiments, then yes, but I am not convinced that the difference is worth the other benefits WebDriver has. Try it yourself on a small sample of tests, and see for yourself!
It Has Built-In Server Mocking!
But there is another reason that Cypress is perceived as faster. Let’s remind ourselves that Cypress is meant for frontend developers. Let’s also remind ourselves that frontend developers sometimes don’t use the real backend, but rather mock the XML HttpRequests to the server. This makes for extremely fast tests, on the order of seconds, regardless of whether you use Cypress or Selenium WebDriver.
Cypress understands that, and has built-in facilities for mocking server responses, facilities that are crucial for their target audience: frontend developers. Selenium WebDriver has no facilities for that, and to mock server responses, a WebDriver tests would need to run a mock server that returns the correct responses. While possible, it is a slower and much less convenient option: a fast and in-memory facility for mocking will always outperform a slow out of process one.
And it’s not only the speed of execution. The convenience of a built-in facility for mocking servers cannot be overstated in terms of speed of writing the tests. The fact that each test explicitly describes the mock responses is much easier to code (and understand!) than a separate server program that mocks the same responses.
It Has a Bizarre Execution Model!
Yeah, I know. Weird heading for this section. But it does! Cypress’ execution model is bizarre! To prove it, let me show you some Cypress sample code:
“That’s not bizarre!”, you say, and you would be right if it did what you thought it did. But it doesn’t. When you call `cy.click` or `cy.navigate` or `cy.type`, you’re not executing that command. You’re actually recording what you want done when Cypress does run the test. The test doesn’t run inside the `it` function. It runs after the `it` function ends, based on the commands you recorded. `cy.nagivate/click/type` don’t execute, but rather record, to be replayed later.
What are the practical consequences of that decision for the developer? If you want to branch your code based on a value in the page, you can’t. For example, let’s say you want to do something like this, to execute some test code based on the value of text on the page:
This won’t work. Because when running the test, you’re not actually executing the automation commands, and so checking for the value of the text is irrelevant.
Cypress does support this kind of test, but in a byzantine way:
The `.then` facility enables you to supply code that “really” runs when Cypress executes the automation commands. This `.then` solves the problem of branched test execution in a “record-then-execute” model, but would not have been needed if Cypress had followed a regular execution model, like WebDriver does.
So why does Cypress have this byzantine execution model? The answer to that lies in the next section.
It’s Less Flaky!
According to Cypress, by delegating execution commands like navigate, click, and type, to Cypress itself, Cypress enables more control on how to execute those commands. It can, for example, when asked to click on a button, wait for that button to appear on the screen, stop animation, and then, and only then will it click it.
According to Cypress, this results in much less flaky tests.
I’m not convinced.
Waiting for an element to appear sounds a lot like WebDriver’s implicit wait, so I’m not sure why Cypress believes it is better in that regard.
But I do believe Cypress is less flaky, but this has nothing to do with controlling the execution. Because the tests on it are executed frontend only, with a mocking of all server responses, the test runs very fast. And the faster a browser test runs, the less flaky it is.
So, yes, I believe that Cypress is less flaky, but not because of control of execution. Too bad, the byzantine method of execution detracts somewhat from this wonderful tool.
But I may be wrong! 🙂
It’s Interactive!
One of the biggest coolnesses that Cypress has is also a big part of the reasons lots of developers flock to it: it’s an IDE of sorts. Because Cypress fully controls the execution environment (the browser), it can give developers a more interactive experience:
As you can see in the screenshot, when the test runs, the commands that are executed are shown in the side panel, while they are being executed. Moreover, once the test ends, you can click on each command in the side panel, and you can see a screenshot of how the page looked like when the command was executing. This is called Time traveling.
This is phenomenal for debugging a test! Cypress screenshots and seeing how your tests progress is a marvelous way to debug the tests, and a marvelous way to start to understand why the test failed. The whole experience helps in writing, understanding, and debugging tests, and in analyzing their results.
Its Documentation is Excellent!
Compare Cypress’ documentation…
When you review Cypress documentation, it looks a lot nicer. But it’s not just what the documentation looks like. Cypress’ documentation is much more in-depth and broad. It covers most of what you want to know, without the need to go looking elsewhere on the web. You can see that the developers spent a lot of time on the documentation itself. Kudos to them, as it enhances the experience in a significant manner.
Selenium’s documentation is pretty good, especially given that it has a much more wider net to throw—both because it is multi-language, and because it is multi-browser. Unfortunately, it carries with it the weight of previous years and is not equal to Cypress’, both in depth and in width.
Conclusion
As you can see, on the surface, although the two tools Cypress and Selenium WebDriver—seem similar, there are actually many differences between the two. The main reason for the differences, in my opinion, stem from the different requirements from the two tools: Selenium was built as a multi-language, cross-browser tool, that fits multiple purposes, not just testing. Cypress, on the other hand, was built for one thing, and one thing only, executing frontend developers tests fast and consistently.
That is why Cypress seems so limited: it supports just one language (JavaScript), one framework (Mocha), and one browser (Chrome). Selenium WebDriver does not have these limitations. It cannot have these limitations because it is a general purpose tools for multiple target audiences. But these limitations are mostly not a problem for frontend developers, and enable Cypress to give frontend developers the advantages that they do need: speed and consistency.
Cypress is a classic example of less is more, while Selenium WebDriver is a classic example of a Swiss Army knife.
Which is better for you: Cypress or Selenium? You tell me in the comments!
One thing I do know is that my company, Applitools, has SDKs and tutorials for how to do visual UI testing with Cypress and Selenium. So whichever you choose for test automation, we got you covered.
