Before I started working in software testing, I had never heard of heuristics. Over the years, I kept hearing the word “heuristics,” but I never really understood what they were or how to use them. The first time I deliberately researched the topic was for a workshop in 2019. Before this, I thought they were a fancy term used by a specific school of testing, but I learned that heuristics are a well-known and well-used concept in fields like psychology and mathematics, first introduced by Herbert Simon in the 1950s. The work of Amos Tversky and Daniel Kahneman in the 1970s built upon this and introduced the concept of cognitive bias and connected heuristics to problem-solving and decision-making.
I would explain heuristics as mental shortcuts. They take a pragmatic approach and choose to use some of the available information, to quickly find a satisfactory solution.
Or, if we dig into that a bit more, heuristics are:
- Approaches to problem-solving and making decisions
- Imperfect shortcuts, striving for good enough over perfect
- Tools to help reduce the cognitive load needed to make a decision
Heuristics can be especially useful when you are faced with complex problems and/or incomplete information. However, it’s important to remember that they are also experience-based and heavily affected by different biases.
Why are heuristics so tricky to understand?
One of the reasons I believe I previously struggled to understand and explain heuristics is that almost everything can fit into the term, depending on the context and individual person. They are also commonly confused or used interchangeably with mnemonics. This is particularly common with acronym mnemonics like PEMDAS (Order of mathematical operations: Parentheses, Exponents, Multiplication, Division, Addition, and Subtraction) or CRUD (Major functions for handling persistent storage applications: Create, Read, Update, Delete).
Mnemonics are memory aids that can support information retention. They come in many different forms and while acronym mnemonics can most definitely be used as heuristics, they are not the same.
And lastly, some heuristics seem to be generic and work the same way for most humans, while others are very contextual and personal and only useful for an individual or a group of people in a certain team or organization.
Everyday examples of heuristics
To give you a couple examples from our daily lives, let’s look at basic trial and error and the process of getting familiar with doors.
Trial and error is a method of problem-solving that we might use if it feels quicker to just try things out instead of first calculating the correct solution. Imagine having a box full of nuts and bolts in front of you and being asked to match them into pairs. There are tons of ways to solve that problem, including measuring each and every item. That would likely be a method that includes less false tries, but it will most likely be quicker and require less mental energy to just try it out.
Doors are another example I often use when I talk about heuristics. As children, the first time we encounter a door, we have absolutely no idea what it is, how it works, or what it can do. An opening and closing door could just as well be magic. But with each encounter of a door, we build a more complex mental model of doors, until we know enough about doors to understand different types of door handles, various locks, and doors that open in different directions. After a while, doors require little to no mental effort to use. The first time we encounter a new type of door we might have to pause and think. We may use the door incorrectly at first, but with each use, our door heuristic becomes richer and more useful.
You use heuristics every day whether you realize it or not — otherwise you’d be in a constant state of decision paralysis. The amount of decisions we have to make each day is almost endless, and considering every alternative’s cost and benefit in every decision would simply not be possible. We don’t have the time or access to all the necessary information — it simply would not be feasible for us to make purely logical and fully-informed choices all the time. We wouldn’t even be able to decide what to make for dinner or how to reply to questions.
Heuristics in software development in general and testing in particular
A lot of these heuristics carry over to software development — after all, we are all still humans, as far as I know — but we have also built our own toolbox of heuristics specifically for our trade. What is software development if not problem solving?
I mentioned the mnemonic CRUD above, which still covers much of the software being built today and is a useful way of thinking about which major functionality to build and test when working with any type of data object. If you can’t create, read, update, and delete your major data objects — customers, orders, records, documents or whatever is relevant to your business — how will your application be useful to your customers?
The Heuristic Evaluation method created by Jacob Nielsen’s work around Human-computer interaction in the 1990s is a rule-of-thumb method of usability assessment that remains important in UX design today.Design patterns are another set of heuristics that we use to build software. They are general, repeatable solutions to common problems. Some commonly used patterns in the industry are singleton, proxy, state, and factories. You probably have local design patterns that are used in your organization or team to make sure code pieces that are doing the same thing are written in the same way. This makes reading, debugging and adapting code take as little cognitive effort as possible while keeping speed and quality high. Similar to how we use templates for quickly creating artefacts like plans or reports.
In testing, we have a rich flora of our own heuristics and mnemonics. There are all the heuristics for Test Design, such as Goldilocks (Too big, too small, just right) and RCRCRC by Karen Johnson (Recent, Core, Risky, Configuration, Repaired and Chronic). There are also the heuristics used in test automation, like the Test Automation Pyramid, Amber Race’s POISED (Parameters, Output, Interoperability, Security, Error Handling and Data) or KISS (Keep it simple, stupid).
What can be achieved by getting deliberate about heuristics
While perfect is the enemy of good and functioning without these fast judgement calls throughout the day would be impossible, there are still dangers to avoid and benefits to be won by deliberate usage and awareness.
Awareness is important, because it helps us reduce the risk of being affected too much by bias. If we are aware of the fact that we, as an example, tend to look more favourably on things we are familiar with — things we can quickly think of or things we recently came across — we can try to pause and consider if there are other options available that would fit this particular problem better. If we are working on standard functionality (bread and butter solutions) the heuristic of familiarity is probably an excellent choice. As Jacob’s law of internet UX states: “Users spend most of their time on other websites, so they expect your site to work like all the other sites they already know.”
If, on the other hand, we are trying to innovate, work in a disruptive business, or there are other factors that are more important than familiarity in our current context, this heuristic might hinder us from thinking of other ways of solving our problem.
Awareness of our own, our team’s, or even our business’s biases can help us improve software and processes. Being aware of our typical ways of working can also help us apply heuristics in the right places at the right times.
Deliberate usage is the next step up from being aware because once we are aware, we can try to improve. As an example, I once realized that I leaned very heavily into testing corner cases of input data according to a set of heuristics that I had built over the years working as a developer, then tester, and then test lead.
Having a decade of coding meant that I was really good at knowing where it is easy to make mistakes while coding, and I used that to my advantage while testing. It meant that I found a lot of bugs very fast, which gave me a lot of positive reinforcement — making me spend more and more time on this. However, I became aware of the fact that this also meant that I had less time to spend on other types of testing, and didn’t grow or learn new things. It meant we had less of those types of bugs, but we also started risking a quality reduction in other areas. Once I became aware, I could start deliberately handing off those tasks to other people and pick up the slack in others — forcing me to learn, build, and use new heuristics.
Being deliberate may look like:
- Taking time to reflect on your work and looking for biases or gaps that should be addressed
- Using tools like Test Sphere or Would-heu-risk it to find new techniques, areas, or heuristics to focus on
- Learning about something new and trying to apply relevant heuristics from that area (For example, learning about OWASP and trying security testing, reading up on UX and practicing usability testing, or talking to developers about what patterns they use to come up with new tests based on that information.)
- Actively avoiding picking up your normal tasks and instead doing something you normally don’t so you can avoid the biases you’ve built
Heuristics should be paired with reflection
Heuristics are multi-faceted tools that we can use to our advantage — both to do better work but also to grow as professionals. However, like all models, they are imperfect. It is easy to forget that heuristics are fallible, especially once we become more senior and confident in our abilities and skills. I strongly encourage everyone to take the time for deliberate practices for reflection. Reflection is a great way of driving our own personal development and increasing the value of our work, but it also works as a way of practicing self-care and identifying all of the ways you have grown along the way.
