Exploratory testing: an introductory guide

I’m often asked which techniques are most important for testers to learn. While I typically respond to these kinds of questions with my standard “it depends,” when it comes to techniques, I have a simple answer: exploratory testing. Everyone, both automated and technical manual testers benefits from having a solid background and understanding of exploratory testing.  

Exploratory testing is unique and distinct from other test techniques. While other techniques, such as scripted testing, encourage software testers to stay on a defined path and focus on step-by-step instructions, exploratory testing encourages us to step off the path and look around. I often compare scripted testing to following the instructions of a GPS, while exploratory testing is more like hunting for long lost treasure. When searching for treasure the general area of the treasure is known, but the exact location has to be discovered.

Exploratory testing is a massive topic, so I’ll be covering it in a series of articles. To kick off the series, I’ll cover what it is, why it's used, and how to get started. 

What is exploratory testing?

There are many ways to describe or define exploratory testing. Personally, I like Elisabeth Hendrickson’s definition the most. In her book “Explore It!” Hendrickson explains “Exploratory testing is simultaneously designing and executing tests to learn about the system, using your insights from the last experiment to inform the next.” What I find most intriguing and accurate about this definition is the use of the word experiment. 

As testers, many of us have been conditioned to see testing as a linear activity: Feed a feature in → test feature → report bugs, wash/rinse/repeat. However, exploratory testing turns this on its head, making discovery and experimentation the primary goals and bugs a secondary byproduct of the process. 

Have you ever found something while testing an application that just felt “wrong” somehow? When you found this thing that felt wrong, what did you do with it? Most likely you designed some sort of experiment to learn more about it. If you did, then you have at least dipped a toe into the vast ocean that is exploratory testing. 

Exploratory testing is not ad hoc testing

Despite what you may have heard, exploratory testing and ad hoc testing are not the same thing. Both have their place and both serve very different purposes. Ad hoc testing is unstructured and often unfocused. This means that there are no defined outcomes or specific intentions when completing ad hoc testing.

Exploratory testing has a defined structure and is often time-boxed and session-based. When using the technique correctly, there is an identified focus and defined outcomes with a plan to achieve predefined objectives. Exploratory testing should be considered both a testing technique and a testing approach. Additionally, care should be taken to ensure that the kind of experimentation exploratory testing utilizes is considered in the overall methodology that the testing team is using.

Why use exploratory testing

In my article To automate or not to automate, I mention the difference between known unknowns vs unknown unknowns. Automation and scripted testing are great ways to find all of the things that I know can go wrong, also referred to as the “known unknowns.” Exploratory testing is how I find the unknown unknowns — all the things that can’t be anticipated. Often, these unknown unknowns are things like gaps between features or bugs that are only found on specific paths or under specific conditions.

I mentioned encountering something in the testing process that feels “wrong.” Exploratory testing helps turn that feeling into something actionable. Plus, it helps you develop your testing “spider senses,” making you more attune to your experiences and emotions while testing. 

Alex Schladebeck shared a story on her blog about watching two groups of students in an exploratory testing course that she taught. The students encountered and interacted with functionality in their respective applications that felt wrong. Schladebeck asked the first group why they experimented with this specific functionality and explained, “I asked them why they’d done those things. After getting through the initial ‘I don’t knows’ and ‘just because,’ we arrived at: ‘because we thought there might be a problem with this thing and we wanted to look at it more closely, and in different situations’.”

Exploratory testing shouldn’t be limited to just when something feels wrong, it can also be used to prove that functionality is working as it’s expected to. User acceptance testing (UAT), for instance, can be enhanced by using exploratory testing and pairing with other software testers and actual end users. 

You shouldn’t limit yourself to just using it for manual functional testing, as exploratory testing and experimental thinking can also be applied to APIs (examples here and here), security, and even automation. 

Getting started with exploratory testing

Exploratory testing is all about continuous learning, so this is a very brief introduction to what will become a career-long learning journey.

Exploratory testing charter template

Elizabeth Hendrickson shares what I consider to be the best charter template for getting started in exploratory testing in her book “Explore It!” Her charter consists of 3 focuses to get you started in your software testing. 

Explore: What are you looking to explore? Is it a specific feature or functionality? Make sure this is specific, focused, clear, and narrow. The clearer this is, the better your testing will be!

With: What resources do you need to do this testing? Do you need data created, a user with specific permissions, maybe a tool? Whatever it is, make sure you’ve identified it before you begin your exploratory testing session— this is your chance to pack your backpack for the exploration. 

To discover: What is the goal or defined outcome of this testing? The goal should be something more than “finding bugs.” Are you looking to learn more about this feature or gain information about a specific behavior? Whatever it is you’re looking to discover, make sure it’s clear and well defined.

Once you’ve written your charter template, it’s time to make some decisions.

Timebox and sessions

Sadly, testing can’t last forever on a feature and our time is often limited, so it’s important to be intentional with your exploratory test plans. Part of planning is determining how much time should be devoted to test execution. The time you allocate for testing is largely dependent on the importance and risk associated with what’s being tested.

For instance, if what’s being tested is low risk and only available to a small group of users, you’ll likely spend less time testing it compared to something that is super critical, high risk, and crucial to the user experience for a large group. To structure your testing, it’s best to timebox and use sessions. 

Timeboxing is the practice of setting a time limit to an activity. In this case, timebox refers to the total amount of time allocated to testing the charter.

Sessions break the timebox into smaller chunks of time. For exploratory testing you should plan multiple sessions that use the full amount of time you’ve timeboxed. For example, if you’ve timeboxed one hour, you might plan two thirty-minute sessions or four fifteen-minute sessions. Try a different experiment or approach to the charter in each session. 

In my opinion, two hours should be the maximum timebox for an exploratory testing charter. There are two main reasons I believe this. 

First, if the testing will take longer than two hours in total, what is being tested is too big and needs to be sliced to be more targeted. Second, thirty minutes is about how long the average adult can maintain extended focus. It’s critical that your mind be on just the task at hand. If your mind starts wandering during the session, valuable information will get lost.

Testing the charter

After the charter has been created and timebox and sessions defined, it’s time to do the fun stuff: testing! How and what you test is up to you, but no matter how you approach things, make sure to set a time for your session and document your testing adventure. I personally find that taking notes while testing is the hardest part of exploratory testing, so I’ve taken to recording my testing sessions on Zoom. This allows me to easily edit the recording to show any defects I may have discovered or rewatch the recording to find and recreate steps I may not remember. If you’re in a private space, consider narrating your session while you execute the testing. If you do, Zoom can provide you with a transcript and captions. 

As you test the charter, look for things that feel “wrong,” or confusing.

If you find something unrelated to the charter that just feels “wrong,” make a note of it to explore later. Exploratory testing should be focused on just the charter you’re currently working on, but that doesn’t mean you should ignore other things that pop up. 

Also keep in mind that it’s not unusual to start feeling confused, overwhelmed, or lost during a testing session. If you do, take a break! Pause the session, grab a cup of tea, go for a walk, or even talk to a peer about something else. Usually you can get back on track if you give your mind time to wander and work out what you’ve seen without stress or pressure.

And don’t be afraid to redefine the experiment. As Lindsay Weir reminded me recently, exploratory testing is equal parts art and science. In both art and science, we sometimes miss the mark and have to redefine the experiment. If the charter doesn’t feel right it’s okay to stop and go back to the beginning. If you do, redefine the charter and start again or debrief with a peer to get another perspective on the problem.

After all sessions have been completed, be sure to document your findings. How you do so is largely up to you and your team. Many teams will record their findings in their test management system for later use. This is especially true for teams who have made exploratory testing a part of their overall test strategy. 

Debrief and wrap up

After you’ve completed your testing efforts, you’ve got one step left: the debrief. As exploratory testing is in part about continuous learning, it’s important to share what you’ve learned with your team or peers. In a debrief, you will share what prompted you to explore the feature or functionality, share your charter, and talk through what was found during the testing sessions. Then, a peer will ask questions for clarification and to identify additional experiments to consider for future exploratory testing. Just as you documented findings from the testing sessions, you should also document anything noteworthy from the debrief including future test ideas identified in the debrief.

Putting it all together

Exploratory testing is not something that you’ll learn in a day. Even experienced testers are continuously learning new ways to use exploratory testing and pondering new aspects of the testing approach. 

This article is far from an exhaustive explainer of this type of testing. It should be treated as a quick start guide to help you begin exploring. In the next article in the series, I’ll help you dig a little deeper and share other ways to utilize exploratory testing, the advantages and disadvantages of the technique, and how to pair exploratory with automation.