Comparing different UI Testing frameworks (Selenium, Cypress and Microsoft Playwright)

Before Christmas holidays we had an internal tech meetup at Knowit with the focus on Testing and I had prepared a presentation on how to speed up your Automated Test process (which I’ll be sharing later in an upcoming blogpost).

A coworker of mine did a presentation on UI Testing with Selenium that was really nice and it dawned on me that for someone who enjoys testing as much as I do, I’ve payed very little interest on UI/Browser Testing up until now.

So I decided that 2022 would be the year when I will start to implement more UI Testing in my projects (which I’ve actually already have).

But, which framework should I choose? 
So the first thing I did (as always) was raising the question on Twitter just to get a sense of what frameworks are the most common that people seem to be using these days, and at least in my Twitter bubble it seems like these three are the most used:

According to general consensus Selenium seems to be considered “the old trusty” framework that has the best browser support and a lot of documentation, but not exactly the fastest player and not what “the cool kids” 😎 wants to be using these days (some even called it “a thing of the past” but I am not enough educated on the subject to have an opinion on that statement).

Javascript based UI Testing framework so if Javascript is you cup of tea (which is not the case for me) then this is probably a good fit for you. When you run you tests Cypress will generate a really nice Testing UI in your browser which gives you a great overview of all your tests and let’s you run them independently or grouped.

Cypress also runs all your tests in an emulator, meaning that you run all your tests from one single browser tab instead of having to watch a bunch of browsers spin up and close down over and over, which makes the test runs super fast!

Microsoft Playwright:
Built by the same team that built Puppeteer at Google, but now under Microsoft flag, we have Playwright. What’s nice about Playwright is that it supports a lot of different programming languages, such as Javascript, Typescript, Python, Java and .NET, the last of which is right up my alley. It also has a “headless first” approach that I really like (already covered in this blogpost). It also works really well with NUnit, which is another plus in my book.

But with all these different frameworks, how do you know which one is the “right one” for you?

Introducing the UI Testing framework comparison project
So what I did while experimenting with these different frameworks, without knowing which one I would eventually choose to work with (cliffhanger), was to write a bunch of basic UI tests against this blog and gather them all in one GitHub project.

All the different framework tests have the same tests so that it is easy to compare and find the one you think works best for you. There’s no advanced testing going on, just basic stuff like reading different heading, tags etc and making sure that different links takes you to the expected results.

The idea is that anyone like me who’s not sure on which framework they prefer can compare the same tests in different frameworks instead of the basic “Hello World” tests which are usually not enough to form an opinion.

There is no right or wrong framework!
We all come from different backgrounds and different things floats our boat. Some prefer their code to be typed, some don’t. Some prefer to be very explicit and define everything themselves and others prefer things to be handled “behind the scenes” by the framework itself.

Browse around and find the one that you think works best for you and your past experience. I know which one I prefer: Playwright! But I won’t elaborate more on why but instead let you form your own opinion.

Hope it can be useful to someone.
Take care of each other.

Cheers friends! ❤️