Welcome to the second article in my series about improving the hiring and interviewing process for QA engineers — for both the companies and the candidates. The first article discussed job descriptions and CVs. The goal of this article is to provoke rational thought in the design of an interview process.
When considering hiring a QA engineer, it’s crucial to determine if hiring is the optimal decision. Hiring should be done for a reason. Common reasons include removing a constraint in knowledge and skills (e.g. the team doesn’t know how to do certain tasks) or to remove a constraint in productivity (e.g. the team can perform the tasks but the performance is suboptimal).
The interview process should reflect the specific needs of the company and open role
The symptom I often see is that many interviews are blindly copied from the “successful companies” or designed “intuitively.” In many cases I see:
- irrelevant questions copied from somewhere or even compiled from multiple sources
- too many steps in the interview process with the rationale “it’s better to be safe than sorry” or “Google does it this way, so we will too”
- enormous, irrelevant take-home exercises given out because the interviewers are “too busy”
I believe that this symptom signifies a problem: interviewers and managers rarely know how to verify if the person is a good fit for their role.
Companies, teams, and industries are unique, so their interview process should reflect that. Copying an interview process or even a list of questions from one company is far from ideal — and in some cases, it can be harmful. Interviewers could waste time checking for skills or experience that aren’t relevant to their specific role, which can lead to rejecting good candidates or hiring the wrong ones.
The interview process should be designed rationally to provide an optimal way to check the candidate against the actual requirements for the role.
I believe that the interview process is very similar to quality control: during the interview we need to quickly and efficiently verify if the candidate matches the requirements. Similarly, in software testing, we need to quickly and efficiently verify if the completed task matches the requirements.
When we don’t have prior working experience with a candidate, we can’t assure that they “match” the work we want them to do (the requirements), which is why we have to somehow control the process of verifying the match.
Note: If we could assure that the candidate “matches” the work, we wouldn’t need to interview them at all. This is one of the reasons that many companies have boot camps and internships: mentors teach interns to “match the work” (i.e. assure the candidates’ quality).
In order to control how the person matches the work, we have to design an interview. The interview process is our quality control procedure for the candidate.
How to design the QA engineer interview process
Interview process design, much like software testing, must start with understanding the requirements: what we want the person to do.
The process of defining the requirements for the job is called job analysis: we collect the tasks and activities which represent our candidate’s work and extract the most common and most important ones.
The requirements must be validated: do they cover all the work needed for this role? Do they include too many responsibilities? See our requirements testing article for inspiration.
Then we need to decide what checks we need to execute in order to see if the person is the right fit for our requirements. How do we design a test plan for the interview?
Note: This article is devoted only to interview design from the skills perspective. Make sure you understand the profile of the candidate you’re looking for and that you understand your company culture and how to verify if the person is a good fit for the culture. For instance, in some companies all QA tasks are just Jira tickets with predefined sets of activities while in others the desired tasks and activities also imply a big deal of proactivity.
Let’s explore a couple of scenarios:
1. Hiring a manual tester to join a team with an existing manual tester
As we already have one manual tester on the team, hiring another tester only makes sense when the constraint is performance — the current manual tester cannot cope with the workload. This also assumes there is no other way to reduce the workload for the current manual tester.
A job analysis reveals that our manual tester is most occupied with two kinds of activities: regular regression testing for the release every two weeks and testing new features.
To do regression testing, our manual tester goes through the whole project’s test documentation and reruns all the test plans manually.
To test the new features, our manual tester participates in two kinds of activities:
- requirements testing and creating test plans and test cases for the feature before the development phase
- testing the feature after the development according to the test plan
Essentially, we are hiring a person who would help our manual tester with handling these tasks.
As we already have a manual tester performing these job activities, they know how these activities should be performed and will be able to validate the candidate's answers.
The test plan for the interview is simple:
- To test the candidate’s ability to handle requirements testing, we can present them with a couple of user stories exactly as they would come to our manual tester. Then, we ask the candidate to prepare a test plan for the requirements.
- To test the candidate’s ability to test the feature according to the test plan, we ask them to test the feature according to the provided test plan.
- To verify their proficiency in regression testing, we ask the candidate to do the regression testing but limit the scope of testing to critical features only.
This way, we can check the candidate's abilities against the actual requirements devised from our real work tasks.
This interview process would be streamlined and relevant to the role’s actual expected duties — making it an optimal process. There is no reason to ask the candidate to complete a lengthy take-home exercise or split the interview into multiple steps — they can either perform the actual tasks needed for the role, or they cannot.
2. Hiring a manual tester for a team without any manual testers
Job analysis only works when we already have an employee performing the job or something very similar. If developers are doing the job of a manual QA, for example, we could follow the advice from the previous section.
If the team doesn’t perform this job due to the lack of knowledge and skills, we face the classic problem of information asymmetry: the team possesses less information than the candidates, it can neither define the requirements nor verify the candidates’ abilities to fulfill the requirements. Imagine a situation when a manual tester is presented with a task with no requirements and no way to test it. The tester would certainly say they cannot control the quality of such a task.
This is the classic problem of information asymmetry: the team possesses less information than the candidates.
In his seminal paper on information asymmetry George A. Akerlof suggested a few ways to deal with this problem:
- Signaling: the informed party sends a signal to the uninformed party about the quality of the product. In the job market, signals might include educational qualifications, certifications, or references that candidates present to potential employers. However, it’s doubtful that one can rely solely on any certification to verify a candidate’s abilities.
- Warranties and guarantees as a sign of quality: In a labor market, this might be analogous to a probation period or a performance-based contract, where the continuation of employment is contingent on demonstrated performance. However, this method suits products better than people, since in many cases the cost of rehiring is usually very high.
- Candidate reputation: A candidate’s professional reputation can be a powerful tool for reducing information asymmetry. If someone on our team has previously worked with the candidate and can verify that the candidate is capable of performing the necessary tasks, this could be a viable path.
I’d say that the third option is the most optimal one in many cases as we don’t need to control the quality of the candidate (interview them) — their quality is assured by our previous work with them.
Signaling might work well if we modify it slightly by inviting an external consultant we trust to do the job we need for a while. This would allow us to define the requirements for the candidate. Also, the consultant would be able to help us verify the candidate’s abilities during the interview.
Following this method should make designing interviews easier and more efficient.
References:
