As with any profession, a tester's work yields outcomes in the form of artifacts generated prior to, during, and after testing. Despite the popular interpretation of the "Working software over comprehensive documentation" principle from the Agile Manifesto, when people imply that documentation is dispensable, it is crucial to acknowledge that testing artifacts are still necessary to guarantee software quality.
Hence, in this article, I will not delve into the importance of testing artifacts, as well as the categorization and kinds of test documentation, as there is a wealth of information available on the internet. For instance, you may refer to this resource on Test Documentation in Software Testing for further insight.
My emphasis would be on elucidating the principles underlying the documentation of testers' work and the benefits accrued from adopting such methods.
1. Simplicity of interpretation
Over a decade ago, I coined the concept of the "granny from the street," which refers to an individual who lacks any prior knowledge or experience in testing processes and objects.
The fundamental idea behind this analogy is that any form of documentation, whether it be a bug report, a checklist, or a test report, should be drafted in a manner that enables this granny to easily navigate and understand it, conduct tests, identify errors, or comprehend the status of the tests.
Contemporary tools such as task trackers, knowledge bases, and test management systems are typically designed to facilitate the organization and structuring of testing artifacts. They often incorporate templates and user-friendly interfaces that make document usage intuitive for company staff.
Some people contend that the concept of a "granny from the street" is irrelevant, given that we operate in highly skilled teams.
Nevertheless, it is highly likely for professionals to succumb to the "curse of knowledge" cognitive bias, wherein they assume that if a concept is clear to them, it is equally lucid to others.
It is essential to recognize that all communication is fundamentally redundant, as in an ideal world, we would be able to comprehend each other's thoughts telepathically.
The world is far from the ideal, implying that the utilization of the "granny from the street" is, to some extent, a necessary measure. This approach curtails discrepancies and multiple interpretations at the expense of precision and clarity in presentation.
Adhering to this approach enables us to establish a comfortable environment for collaborative work with testing artifacts, rendering them comprehensible for all stakeholders, including new testers, developers, project managers, product owners, and so on. We set the stage for realizing the subsequent principle of openness.
2. Openness
"And let those who disagree express their disagreement now or remain silent until the end of time."
After crafting comprehensible documentation for your peers, it would be rather peculiar to conceal it from them.
However, testers often draft testing checklists and hide their notes on their personal computer or in Google Docs.
But if this artifact is attached to the task in the test management system, the information gets available to the whole team.
For instance, Qase TMS not only facilitates seamless integration with most widely used task trackers but also offers the opportunity to create free accounts with read-only access for non-testers.
Making these artifacts available to the whole team provides our colleagues with an opportunity to scrutinize and recommend various forms of revisions, ultimately leading to a positive impact on the quality of the delivered products.
3. Relevance
Documentation is inherently prone to obsolescence, which can have a deleterious effect on its primary objective: enhancing software quality.
Hence, it is of utmost importance to remember to keep documentation up to date.
Undoubtedly, periodic inventory checks and volunteer cleanups can alleviate the effects of document obsolescence, but these measures only address the symptoms and are short-lived in nature.
It is far more critical and beneficial to establish a documentation lifecycle process that involves regular updates and cleaning.
Let us consider a typical example: a testing checklist for a new feature.
Typically, the entire set of checks becomes redundant in the subsequent stages of regression testing. As a result, only a fraction of the checklist must be incorporated into the regression tests.
By implementing a consistent process for transferring test cases from the list of tests for a specific task to the regression set, we can substantially reduce the number of tests that need to be maintained.
For instance, in Qase, we established two projects for test cases: one for testing new tasks and the other for managing test coverage during regression testing.
For the first project, we created detailed checklists without nested test-case details. This allowed us to quickly outline the necessary checks, making them essentially temporary documentation for a short period of time. As an option, they could even be stored in a task tracker in this form.
For the second project, we transferred or cloned only the necessary test cases for smoke tests and regression tests.
Once transferred, the tests were marked as "Draft" to indicate that they needed to be expanded with playback steps, additional field values, and other attributes before being moved to the "Actual" status. This process was implemented to ensure that only relevant and up-to-date test cases were included in the regression testing.
During regular testing, if any obsolescence was detected, the test case was marked as "Invalid", transferred back to draft status, updated, and returned to operation.
This approach is essential because we often refer back to this documentation over a prolonged period, and as such, it should be more detailed and kept up-to-date. This is why having a test management system is crucial for the second type of tests, as it caters to the requirements of this approach.
By utilizing modern test management systems, it becomes feasible to implement effective test documentation lifecycle processes, which are crucial for ensuring software quality. Consequently, such tools become indispensable for testers, as they enable the comfortable and comprehensive implementation of the aforementioned principles. By adhering to these principles, it becomes possible to bring about systemic positive changes that enhance the quality of the products.
