How to choose a test automation tool?

Ilya Filinin -

Today we will discuss how to choose a tool for test automation. Let's focus on E2E for now.

The list you will see below is what I use when selecting a tool for a new project.

1. Decide on the method of creating automated tests. Writing code or generating it (better not to). And if you choose to write code, you will need to choose the notation: classic or Gherkin (better not to).

Example:

  • Code generation - any recorder, such as Selenium IDE or a recorder built into Playwright.
  • Writing code in classic notation - Playwright, Selenide, PyTest.
  • Writing code in Gherkin notation - Cucumber, Robot Framework.

2. Try to combine the programming language for the product and the automated tests, but it is not necessary. This way, it will be easier to share expertise in writing automated tests with members of the development team and easier to integrate into the infrastructure.

Example:

  • Selenide - only Java or JVM-based languages
  • Playwright - Node js, Python, Java, .Net
  • Cypress - only JavaScript.

3. Understand which set of browsers and devices is important to include in the test suite.

For example, if you need to test your application in an old version of IE (just in case), then Playwright is not suitable for you, it's better to look towards Selenium.

4. Check the overall compatibility of the tool with your technology stack.

For example, if you use TMS, you need to make sure that the framework is compatible with it, can send reports, and so on.

5. Determine the size of the community and the availability of learning materials.

If you are taking a new tool, you should be prepared to solve many problems on your own, without relying on articles and videos. But sometimes such a risk is justified.

6. Understand which tool is best suited: paid or free.

It all depends on the size of the budget, but personally, I haven't found any reason to use paid tools. Free open source solutions meet all the requirements I have.