3 paradigms of managing testing tasks: push, pull, and full autonomy

I've seen companies employ one of three paradigms of managing testing tasks: push, pull, or full autonomy. However, when I ask the engineers about the reasons for choosing one approach over another, I find out that it’s rarely a rational choice.

In this article, I’ll explore the pros and cons of each approach and, with the help of Qase QA Engineer Anastasiia Khabarova, share how we apply the principles from the full autonomy paradigm here at Qase.

1. Push

In a team using the push method, there is always someone who assigns tasks, such as a QA lead, team lead, or PM — this person is referred to as the “manager.”

In the push method, the manager assigns specific test cases to individuals on the team. This directive approach requires the manager to handle the complexity of overall testing management themselves.

The manager must review and understand all tasks, keep track of every team member’s knowledge, skills and motivation, and match tasks to team members in a way that ensures tasks are completed on time. The manager also has to consider the skills development path for each team member when assigning tasks. All of this means that the manager is responsible for planning and predicting how each team member will handle their assigned tasks, and track their workload.

Push method challenges

Managing and keeping track of all tasks for all team members is very difficult as the manager not only has to remain organized, but also must understand the technical nuances for every task.

Predicting when and what will be completed is also a challenge. Even if a manager succeeds in tracking tasks and skills, people are not machines, so there is no guarantee that a task that was previously completed in one day will always be completed in a single day. 

Lastly, the manager must ensure that every team member gets to test various parts of the application to avoid having a single point of failure. 

Essentially, the manager is mentally performing much of the work of all team members and is constantly trying to predict the future. This also makes the manager the single point of failure, therefore it’s quite hard for the manager to perform other duties, go on vacation, take a sick day, quit, or die (or “get hit by a bus” if we use the bus factor example). 

The push method usually works well in the following scenarios

• Onboarding and teaching new team members. The manager must understand the new person’s skills and the task complexity to guide, teach, and control them. The tasks assigned are usually not critical, so delays do not impact the overall team’s progress. Also, the onboarding process is finite, as soon as the new team member is performing well, the onboarding is completed.
• When testers are outstaffed (internally or externally) and are only responsible for executing assigned tasks. In this scenario, testers may not have an intrinsic interest in product quality and are more focused on only completing what’s assigned to them.
• When the company’s culture supports only this way of working. If team members are accustomed to it and resistant to change, this method may make sense.
• When project teams have a queue of tests is predetermined and set in the contract. If expectations are clearly defined and linked to tests, the push method can work well. 
• When there’s just one or two testers on the team and the overall complexity of testing tasks is relatively low.

How a good TMS can help support the push method of software testing

A test management system (TMS) can enable teams using the push method by: 

• Providing statistics on the average time taken for certain tasks, allowing the the manager to plan based on statistical data instead of estimations
• Offering an easy way to view the entire structure and prioritization of tasks
• Providing a traceability matrix to create new testing tasks when changes occur in the requirements, removing the need to manually track if tests still match the requirements
• Generating reports to help the manager see what each tester is doing, reducing the effort and the burden on gathering the statuses and updates from every team member
• Enabling a test review process, allowing managers to require certain tasks (e.g., those done by juniors) to be reviewed by others, which helps with onboarding and training

2. Pull

In the pull method, the team of testers, or a combined team of developers and testers, uses the queue of tasks and assigns the tasks to themselves. The manager’s role is to update this queue, adding more tasks as needed. Team members then pull tasks from the queue, self-assigning based on their skills and capacity.

Individual team members have the autonomy to choose tasks based on their understanding of their own skills and capacity. With the pull method, the team works towards the common goal of ensuring quick releases, promoting collaboration and shared responsibility.

Unlike with the push method, the manager does not need to track individual skills and capacities — team members manage this complexity themselves. This also means that the manager is rarely a constraint or a bus factor.

Highly motivated testers can accelerate their learning by selecting more complex tasks and if a task is too challenging, team members can return it to the queue or seek assistance from others. With the pull method, testers can take any task, as everyone shares some knowledge and usually chooses various tasks.

Pull method challenges

The pull method can be problematic if there are power dynamics within the team, leading to unfair task distribution where some members take on more interesting tasks, leaving others with less desirable ones.

If team members tend to overload themselves or if there are KPIs based on the number of tasks completed, this method can also lead to burnout and reduced quality of work. Individual performance reviews focused on task count can also undermine the effectiveness of this approach.

The pull method usually works well in the following scenarios

• In product teams where members are highly motivated and collaborative, willing to support each other and share the workload equitably.
• When team members are self-sufficient and possess enough skills (this method won’t work well for a team of juniors).

The pull method empowers team members by giving them control over tasks assignment, fostering a more flexible and dynamic work environment. While it offers several advantages over the push method, careful attention to team dynamics and workload management is essential to avoid potential pitfalls.

How a good TMS can help support the pull method of software testing:

A TMS provides a centralized task queue where all available tasks are visible to the team, making it an ideal complement to the pull method of testing. Some test management solutions can support better collaboration with key features such as Shared Steps, which enable team members to create groups of steps once and then share with the entire team to be reused across multiple test cases. 

A TMS with robust analytics will also allow managers to oversee their team’s work and help ensure tasks are fairly distributed within the team. 

3. Full autonomy

In the full autonomy method, both engineering and testing are usually integrated within a product team. The team collectively decides what needs to be tested and when, usually embracing some of the principles of shift-left testing. The team creates test tasks (test cases and test plans) as needed and either works on them collaboratively in an ensemble (mob) or individually, but always with a strong sense of team cohesion.

With this method, development and testing are performed by the same product team, which usually improves quality. The team collectively determines the testing tasks and their priorities, so the manager is never a constraint or a bus factor. And because testing is integrated early in the development process, issues are identified and addressed early on.

As an added benefit, autonomy and ownership can lead to higher job satisfaction and motivation, as team members feel more responsible for the end product. When working closely together, team members can learn from each other, sharing knowledge and skills more efficiently. Plus, the team can quickly adapt to changing requirements and priorities. 

Full autonomy method challenges

This method requires a high level of team maturity, trust, and communication. Teams must be well-educated in both development and testing practices.

The full autonomy method usually works well in the following scenarios

Full autonomy works best in mature agile teams where members have a high level of expertise and are comfortable with both development and testing tasks.

How a good TMS can help support the full autonomy method of software testing:

A TMS provides a centralized repository for test cases, plans, and results. The team decides how to use it, but the TMS provides everyone with access to the same information. Reporting and analytics can also help the team show management what they’re working on and demonstrate the value of that work, so there's less chance for trust erosion.

At Qase, we use the full autonomy method

Our QA folks test everything together with developers. From QA perspective, Qase SDLC looks like this:

Qase QA engineers are involved in many activities: from the proactive measures during the requirements stage to the usage stage monitoring (watching how real users work with our product).

During the testing phase, we have a group activity called “Team Testing.”

Before inviting the team for testing, the tester prepares the necessary test cases and plans. Sometimes the result is a list of fully detailed test cases, and sometimes it’s just a checklist. The format depends on the feature complexity. The goal is to make all the necessary checks understandable to all participants.

Once the tester has prepped, the entire development team gathers on a call (similar to an ensemble session), and distributes the tests to all the participants. As we use Qase to test Qase, we can distribute test tasks using single, bulk, or group assignment functions. 

During this call, every team member brings verification scenarios which are then described as test cases, leveraging the collective knowledge and experience of the team.  Additionally, we have live discussions about any defects that have been identified so we can find the fastest possible resolution. 

This group activity not only improves the test case coverage, but also allows developers to get a better understanding of the testers’ work or even perform some testing themselves, if it makes sense. 

Lessons we’ve learned through using the full autonomy approach

Before starting ensemble (mob) testing, we find it useful to have a shared testing checklist which everyone can review so that the collaborative testing is as efficient as possible. For the session to be productive, all team members need to understand and share the importance of the ensemble (mob) testing activity, and have enough time and motivation to participate. It’s impossible to force true collaboration.

We’ve also found that there’s little point in getting the whole team on an ensemble (mob) call for testing trivial bugs or issues. Long calls or calls without breaks never work well either. People tend to start losing focus after 30-40 minutes of intensive collaborative work. 

Lastly, we believe it’s important to ensure the team agrees on organizational nuances such as whether cameras will be on or off and how the sessions will be run. We find it helpful to discuss the results of the testing session during the retrospective to improve future ensemble testing sessions.

Testing methods should match the team you have today 

As with any other approach, method, or process, there’s no silver bullet, and choosing the optimal paradigm of managing testing tasks is no exception. The method (or combination of methods) that works best for your team will depend on your specific needs, the skills available on your team, and company culture.