How to start with autotests?

In today's article, I will explain how to understand if you need automated testing and how to start doing it in a way that avoids embarrassing mistakes and pitfalls from the start. To do this, we will gradually answer a series of questions.

How to understand if you need automated tests? How to calculate their usefulness?

To determine if you need automated tests, consider the following factors:

• Complexity: if your application is highly complex, manual testing can be difficult. Automated tests can help you and can give you consistent testing of all features.
• Frequency of changes: if you frequently make changes to your software and want to understand that each change is of good quality and does not break anything, automated tests can help you identify errors. But, you should remember that supporting automated tests for frequent changes can be expensive. But continuous manual regression testing can be even more expensive.
• Time and resource constraints: if you have limited time or resources for testing, automated tests can help you be faster and more efficient. But you should remember that at the beginning, you will spend a lot of time writing automated tests.
• Project duration: if you plan to write a small MVP in a couple of months and after that stop working, writing automated tests can be a waste of money because there is not enough time for their profitability.
• Cost: automated tests can be expensive to develop and maintain, so you should consider the ratio of costs and benefits of their implementation.

To calculate the usefulness of implementing automated tests, consider the following:

• Saved time: calculate the time that will be saved by automating tests compared to manual execution. For example, you can calculate how much time you spend each month on conducting regression testing. After that, evaluate how long you plan to continue developing the product and how much time it may take to implement automated tests for the tested functionality. And then compare whether it is beneficial in your case or not.
• Importance of fast releases: evaluate how important it is for you to get testing results quickly. If this is important for you, calculate how much time is spent on manual regression testing in the context of the application release cycle. Automated testing will allow for much more frequent and faster releases.
• Money savings: calculate the savings that will be achieved by reducing the need for manual testing and potential errors. I can't suggest how to calculate it precisely since this point is quite individual and depends on the conditions you are in. But I would suggest considering how expensive it is for you to miss errors in production in terms of reputation and how expensive it is for you to release less frequently and not as quickly in terms of missed profits.

How to choose tools and programming language?

Choosing the right tools and programming language for automated testing largely depends on your situation, needs and requirements. I can give you some general recommendations that will help you make the right choice:

• Pay attention to the initial conditions. If you already have a product in some form and it's written in JavaScript + Php, don't bring Java into the project for autotests. Because you can save some money for implementing and maintaining infrastructure, and also share expertise in test writing to the development team. If you have chosen JavaScript for autotests, the choice of framework also becomes simpler, because not all frameworks support this language.
• Evaluate the capabilities of the tool. Consider the capabilities of the tools you are evaluating, such as support for web, mobile, or desktop applications, integration with your testing management and CI/CD systems, ease of use, and community support. If you have specific conditions, start by choosing a tool that will satisfy them, and then select the programming language based on the chosen tool.
• You can also pay attention to the cost of the tool, and evaluate whether it fits within your budget. But I would recommend using open source solutions and not burdening yourself with additional problems.

Why is architecture and design patterns important in software testing?

Architecture and design patterns in automated testing are important for several reasons:

• Maintainability: Your automated tests are easy to maintain and update. If your tests are poorly designed, it can be difficult to make changes, which can lead to increased maintenance costs and decreased efficiency.
• Reusability: Well-designed automated tests can be reused in different parts of your application, reducing the need to create new tests from scratch. You can also reuse parts of the tests you've already written.
• Scalability: As your application grows and changes, your automated tests should be ready to scale and adapt to new requirements. Good architecture and design patterns help ensure that your tests can handle increased complexity and remain reliable over time.
• Consistency: Your tests are structured in a way that makes them easy to understand and execute. This consistency can lead to improved collaboration among team members and reduce the risk of errors.

By investing time and effort into creating well-designed tests, you can save time and resources in the long run while also improving the overall quality of your software.

Let's stop here and go into more detail. Let's say we are beginner AQA and we are trying to write our first code. Let it be automation of logging into Google using Playwright.

This is a small script that's pretty easy to maintain. But if we continue to write automation tests for the login form, there will be more tests and some of them will use the same fields. This means that a lot of time will be spent maintaining such tests in the event of even minor changes to the form layout. How can we avoid this? By using the POM pattern and creating a separate class that will contain all the locators for the login page. This way, in the event of layout changes, it will not be necessary to change a dozen tests - it will be enough to change the locator in one place.

But we have another problem - not only can the layout change, but the way we interact with elements can also change. In such cases, we use actions. Let's take the fill function as an example, although actions are more often used to describe more complex things. For example, when several clicks are needed to perform an action.

What if the URL of the login page changes? Or what if we want to change the test user's credentials? Not to mention that storing credentials in the code is not a good idea. Here, constants and env variables come to the rescue, which we do not store in the code.

That is why architecture and design patterns for automated testing are crucial. They help to eliminate many problems before they even arise and become a concern.

Why considering parallelization is important?

Implementing parallelization is essentially one of the mandatory aspects because the main goal of automated testing is speed. And without parallelizing thread execution, achieving really high test execution speeds is impossible. However, it is important to do this correctly to avoid problems with parallelism or false positives/negative results in test results. For example, in Playwright, this is implemented like this.

Ensure thread safety: make sure that multiple threads can execute the same code without interfering with each other and without causing data corruption.

What additional testing tools do we need and how can they help?

There are cases in the testing process when it is impossible to fully execute it without using auxiliary tools. For example, when testing integrations.

Payment systems:

Testing payment systems is an integral part of the software development life cycle. When you encounter this in practice, you need to simulate various payment scenarios, and doing this with your own card and real money is not very good. Test cards come to the rescue. They are usually provided by payment gateway providers such as PayPal or Stripe. With their help, you can test various payment scenarios, such as valid and invalid card numbers, expired card expiration dates, or insufficient balance for the transaction.

Mail servers:

Registration is a critical component of many applications, and in the process of testing it, you may need to check the email in the mailbox. You can do this in several ways:

What is called "straight forward"

1. Create an account on one of the email services
2. Describe locators for pages of the email service to interact with them in code
3. Parse the list of emails to search for the necessary one, extract the link for authorization in the application
4. Follow the link to complete the verification of this test case

Using an SMTP server:

1. Install mailhog, mailcatcher or any other server you like
2. Redirect test environment emails to the installed mail server
3. Get the necessary link for the test via API
4. Follow the link to complete the verification of this test case

The advantage of the second approach is that such tests become less dependent on third-party systems, they are executed faster due to the ability to obtain data from the email via API, and this approach eliminates the need to have multiple email accounts for testing.

Of course, an SMTP server can be used not only for registration tests but also for any tests related to email.

Regarding data cleansing, what should one keep in mind?

Data cleansing is an important step in automated testing. It involves removing or modifying invalid, incomplete, or inconsistent data to ensure its accuracy and suitability for use in testing. Without this step, tests can become dependent on each other or start to fail.

Data can be cleansed in several ways. For example, created entities can be deleted using an API in the test. Or they can be designed so that no two tests interact with the same data, but this is very difficult and time-consuming. The correct solution is to automate data cleansing after each test, returning the data to the reference value.

About reporters and TMS, how to choose?

You've realized that you need automated tests, chosen a language and framework, laid out the architecture, and written the first tests. And then you realize that you need to somehow analyze the data. How many tests failed and when, how long the tests were executed, and so on. To do this, you need to choose a TMS and a reporter that will send the results of test runs to this TMS.

For me personally, the choice here is not difficult. I use what I do. Qase has all the necessary reporters, a pleasant interface, and I can customize it to my needs, which is undoubtedly very convenient.

Conclusions!

Automated testing is a very useful (often necessary in modern times) thing, but it is very important to use and configure this tool correctly. Errors in implementation can negate all the benefits and destroy trust in this type of testing among the team and management. But I hasten to reassure you - the basic rules are not that difficult, as you could see today.