Over my career I’ve dabbled in various forms of testing, both on the backend and front-end. I’ve tried various frameworks and experimented with different approaches, types of tests, and philosophies. From unit tests to Gherkin behaviour tests to E2E tests with Selenium in the good ol’ days, I’ve seen a lot.
And yet despite all this, I don’t consider myself good at testing because I can be very lazy, and I tend to get discouraged quickly when writing tests becomes too hard. I don’t do TDD as much as I should nor do I care about things like code coverage or testing every edge case.
I’m a very optimistic kinda gal and often get satisfied with purely testing the happy path or ensuring things just broadly work. All of these are consequences of successes and hardships encountered in my journey, to the point where I know I should write this or that kind of tests, but never do because past experiences taught me that doing these things right was hard.
My opinion on testing, unit tests, integration tests and end-to-end-tests.
Eventually my opinions crystallised to the following (maybe incorrect) statements:
Unit tests are easy to write because they purely care about input and output. But they are very often insufficient when it comes to testing web applications and things that aren’t libraries because an application is much more than the sum of its pure functions.
Integration tests are already much better when it comes to testing full systems, and, with modern tools, they’re often relatively easy to write (like in a Laravel application). But they often omit part of the story, or mock it, and at the end of the day they can miss problems just as crucial as the ones unit tests can miss. This wildly depends on how close to reality your integration tests are, how much you stub out to ease writing them. Other than unit tests, they’re the type of tests I’ve probably written the most in recent years.
Full end-to-end tests are the holy grail of testing for me because they ensure your application actually works in what — at the end of the day — matters: they ensure your users can use your application and its features properly without encountering issues. They’re not to be used exclusively of course, but if I were to get my pick, they’re the kind of tests I would write most, because they’re also the ones least prone to breakage and refactors, and the ones most honest about what you want to test and the results you want to achieve.
And yet… they’re also the kind of tests I’ve written the least in my career, because every time I’ve attempted them in the past, I hit walls and walls of technical hurdles and implementation limitations. Because replicating an actual user interacting with an actual app comes with such a unique set of challenges, it took years and years for us to collectively get it right and user friendly. So I’ve always pushed it off to the next day, and the next, convinced that, were I to try again, I would hit the same barriers, the same browser bugs, the same CI setup issues, and so on.
Now don’t get me wrong, we’re still not fully there yet, things still are a bit like that here and there. But they’re also miles ahead of what I thought was the current state of things.
Should you use Cypress for testing your applications?
I jumped into Cypress after hearing so much praise about it, and I was very much convinced I would struggle like back in the days of Selenium. But then something happened: not only did I not struggle but things were intuitive and easy to set up. The library was good. The tooling around it was good. Things worked “just like that” and I didn’t hit limitation after limitation. And I felt more stupid than ever before because I realised I had procrastinated jumping back into end-to-end tests for years in fear of a terrible experience that was long gone.
What Cypress got right
Instead of simply listing all the features and bells and whistles of the tool, I’m just going to go over what it got right, in the form of a few pain points I’ve had to deal with over the years.
Setting up an E2E library is hard
This was one of my first struggles with E2E libraries. Before we had Puppeteer and headless browsers and all that fancy stuff, it used to be incredibly hard to actually get a browser to run tests in. As such, setting up a library was often a bit of a dependency hell, and even when things started to smoothen with prebuilt PhantomJS binaries and such, I still regularly hit issues.
Cypress on the other hand, is always just one install command away. You run yarn add cypress and you’re good to go. It comes with everything you will need; it doesn’t require special images with this or that OS dependency present; and so on. You install it and it just works. This was the first thing that surprised me.
E2E tools require a lot of configuration
To continue on that point, here again we can see the impact left by the last few years of “minimal configuration” testing tools such as Jest. It used to be a whole headache to configure the capabilities of your testing browser, and how to run it, how to preserve sessions and so on.
With Cypress, the only thing I had to configure was which URL my application was running on and that was it. I already had a working setup. I didn’t need to configure 300 browser options and capabilities. I didn’t need to install and configure separate assertion libraries. Cypress comes with everything you will need under the hood and tries to use smart defaults for most things.
E2E browsers have limited capabilities
This is one of the problems I encountered the most: up until recently whenever you decided to write end-to-end tests, you had to account for the fact that the browser you were running was only a truncated version of the real thing. Even relatively recent projects like PhantomJS were full of things you needed to polyfill, things that behaved differently from a real browser, or things that were straight up not working.
Cypress on the other hand is built on the shoulders of giants and can both use any real browser you have locally installed (Chrome, Firefox, etc.) or headless browsers such as Headless Chrome or Electron. All of those behave almost exactly like a real browser except for Electron which has more limitations. And I didn’t have to update the code of my application to add workarounds or skip tests because they were just plainly not possible.
Writing E2E tests is hard
This varies wildly depending on the solution, but my experience writing E2E tests was usually not that great. Because most of those tools aimed for completeness over good developer experience, you often had confusing or extensive APIs that took a lot of time to learn.
Cypress instead has a very, very minimal API with which you can yield a lot of power. A standard Cypress test is structured just like you would write a basic Jest test, and you can accomplish most things with a handful of methods:
describe("Add product to basket", () => {
beforeEach(() => {
cy.visit("/homepage");
cy.loginWithUser(); // Also supports custom methods
});
it('can add product to basket', () => {
cy.get('.product:first').click();
cy.url().should('include', '/products/my-product')
cy.contains('Add to basket').click();
cy.get('input[type=number]').type('23');
cy.get('#size').select('Large');
cy.get('form').submit();
cy.url().should('equal', '/cart')
cy.get('#total').should('contain', '€23');
// You can even do visual testing!
cy.get('.cart').toMatchImageSnapshot();
});
});As you can see, with its chainable API and polyvalent get method you can already do most things, and most of the methods are named how you would expect them too. Every assertion passes through a flexible should method which underneath gives you access to assertions from myriad libraries such as Sinon or Chai. It also supports callbacks to interact with the DOM elements themselves through a then method for more complicated scenarios.
Another thing you might notice is that at no point do I wait for various elements and refreshes and UI changes and so on. By default Cypress smartly waits for the right elements to be accessible and visible before interacting with them which cuts most of the fat of other E2E tests I’ve written. It will also retry assertions by default when they fail and so on.
You can also very easily add custom commands to Cypress to abstract common patterns through commands:
Cypress.Commands.add('login', () => {
cy.visit('/login');
cy.get('input[name=email]').type('[email protected]');
cy.get('input[name=password]').type('password');
cy.get('form').submit();
});
// Anywhere else
cy.login();But more than its API, Cypress also makes it very easy to do TDD and to write tests incrementally. For this it comes with a cypress open command which will launch a Cypress application. This application will show you Cypress interacting with the browser live and self reload every time you touch the test. It allows you to actually interact with the browser, open the devtools, inspect elements to see what went wrong, and so on.
This makes writing tests super easy because when something fails or goes wrong you can debug it like you would any problem, and it also allows time-travelling between each and every step of the test to see the state of the browser at that point in time. It even comes with helpers to write the tests themselves, such as a very useful selector tool.
Debugging E2E tests is hard
Often in the past, whenever I managed to write proper E2E tests, a big source of pain was that whenever something went wrong it would become a nightmare to understand it, even more if the error only happened on CI.
Thankfully Cypress is not just a library but an ecosystem, and a big part of that is Cypress Dashboard which allows you to easily have the same power as that Cypress application whenever and wherever, even on CI. It allows you to see a replay of the test run, to optimize them, to compare the results between browsers, and so on. And it’s very easy to plug in as you only need to create your project there, specify the project key in your Cypress configuration, and you’re good to go.
Even if you’re not using the Dashboard, Cypress by default generates screenshots and videos of the test runs so you can easily see what happened step by step. Cypress also logs various bits of information to the console for every step of a test, as well as gives you advice on the most likely reason for failed assertions.
Making an app E2E-ready is hard
Even if you have a real browser, and it’s easy to write a test, the problem remains that to make an app ready to be testable you sometimes still have to bypass certain systems or mock certain parts.
Thankfully Cypress comes built in with a mocking system which allows you to not only stub network requests, but also functions and modules, as well as environmental information such as the date and time and so on. The API is not dissimilar to what you would find in Jest and it makes it a breeze to replace specific systems or individual functions with temporary versions of them.
E2E tests are hard to run on CI
This was another big pain point for me. Often I would manage to write my tests, to make them pass, and I would be happy. But then on the first push to CI all hell would break loose because the environment would be missing this or that dependency.
Cypress thankfully not only provides dozens and dozens of prebuilt images for everything you might need, but it also provides prebuilt actions to easily add it to your pipeline.
When you couple this with the Dashboard which gives you great insight into your test runs even on CI, it solves most of the major issues you could run into.
Cypress to the rescue: all the tools for your E2Ea nd A/B testing
Just Cypress the button
There are a million things I didn’t mention from IDE/editor plugins to Typescript support to built-in code coverage to test analytics. Cypress is not just a library, it’s a fully-featured ecosystem that provides you all the tools you would ever need to do E2E testing and A/B testing. I recommend going through the documentation because not only is it very extensive, but it is also chock-full of advice and best practices when it comes to testing (which will also apply to any other library).
When discussing the very topic this week, a colleague said something which deeply resonated with me:
yeah, I saw the cypress a while back.. not sure as e2e seems too much work for me. haha
It struck a chord because that is precisely how I felt about E2E until recently. Sure it’s nice… sure it has a lot of upsides, but damn is it a pain. So this is why I wrote this article, to convey the fact that no, it’s not hard anymore. It’s very easy to set up, to write, to maintain, and to debug. And it is not reserved for front-end applications.
Cypress will work on any app, any stack, and in any environment. So if you’ve been putting it off for some time, my only advice to you is to give it a quick peek, take it for a run, and maybe like me you’ll realize we’ve come a long way and that E2E is not the behemoth it used to be — that it can be as easy to write with as a basic Jest unit test while providing ten times the value.
I often relied on snapshot testing, unit tests through RTL, and other approaches to test my React application. And even though I mostly managed to do what I set out for, I often had to make a lot of compromises because those were not replicas of the actual app but stubbed out versions with half the user path mocked away. It was also heavily discouraging because I often had the feeling that to test a more complex feature such as filling a field or clicking on a button would require hours of work. But that’s not the case anymore and I very much think I will keep using Cypress on any and all applications I write in the future. It has become so easy to add regression tests and ensure features work for the user that every other kind of test suddenly feels archaic and incomplete.
Conclusion? Is Cypress worth it?
So don’t fear the E2E monster anymore; don’t see it as this black hole of time that will eat away at your will to test. Give Cypress a shot and I’m convinced you won’t be disappointed.
Member discussion