Software defects cost the U.S. economy an estimated $59.5 billion every year, roughly 0.6% of GDP. That figure comes from a landmark study by the National Institute of Standards and Technology (NIST), "The Economic Impacts of Inadequate Infrastructure for Software Testing."[1] Over $22.2 billion of that cost could be eliminated by catching bugs earlier in the development lifecycle.

That study was published in 2002. Adjusted for the complexity of modern distributed systems, microservices, and continuous deployment pipelines, the real number today is almost certainly far larger.

And yet quality assurance professionals, the people who carry the heaviest share of that burden, are burning out at a rate of 75%. This is happening not because they are failing, but because organizations have made quality someone else's problem instead of everyone's responsibility.

The gap between what teams know and what they can cover

Every engineering organization has felt this tension. You know your test coverage is insufficient. Developers know it. Product managers know it. The people writing tests definitely know it. But there is no realistic path to closing the gap by adding QA headcount or asking the existing team to work harder.

Some industry observers have started calling this the "Anxiety Gap": the distance between the coverage an engineering organization knows it needs and the coverage it can realistically achieve through traditional methods.

The downstream effects hit the entire team. Escaped defects pull developers into firefighting. Regression cycles balloon sprint timelines for everyone. Release confidence drops, and the whole organization slows down. Meanwhile, the people with the deepest quality expertise spend their cognitive energy on repetitive maintenance instead of the strategic work that actually requires their judgment.

Why more features made the problem worse

For the better part of a decade, test management platforms competed on feature count. More dashboards. More configuration options. More integrations. The assumption was that teams needed more capability. What they actually needed was less overhead.

Every new feature that ships without reducing operational burden adds to the maintenance surface. The platforms designed to support quality ended up contributing to the very problem they were supposed to solve, and the burden fell disproportionately on the QA function while the rest of the engineering team moved on to the next sprint.

The correction is underway. Engineering teams are no longer asking "what can this tool do?" They are asking "how much of our team's time does this tool give back?"

The agentic shift is real, but unevenly distributed

The testing industry is repositioning rapidly around AI and autonomous agents. Major platforms are rebranding around the word "agentic." The underlying demand is genuine: engineering teams need testing capabilities that can operate with minimal human supervision on the repetitive, high-volume work that currently consumes disproportionate time; whether that work is done by QA professionals, developers writing tests, or SDETs maintaining automation frameworks.

But there is a meaningful difference between platforms positioning around the concept and platforms that have shipped working agentic capabilities into production.

What we built with AIDEN - and why it matters now

At Qase, we shipped AIDEN's Agentic Mode in January 2026; not as a response to the current positioning wave, but as the result of a deliberate product strategy focused on reducing cognitive load across the entire engineering workflow.

AIDEN is designed around a specific philosophy: quality expertise (wherever it lives in the team) should be the input, not the bottleneck. A QA engineer, a developer, or a product manager knows exactly how a feature should behave. The gap has always been the translation cost:- converting that thinking into selectors, wait statements, and boilerplate scripts that only a subset of the team can write.

AIDEN collapses that translation layer. Anyone on the team can describe what needs to be verified in plain language. AIDEN breaks it into actionable steps, navigates the application, captures visual evidence, and generates executable test code. When the application changes mid-run, AIDEN detects the failure, analyzes the change, and self-corrects.

This is not about replacing quality professionals. It is about making quality accessible to the entire engineering team while reclaiming the time currently spent on work that does not require deep expertise. QA specialists operate as strategic quality architects. Developers contribute to test coverage without context-switching into a different skill set. The whole team moves faster because quality stops being a “bottleneck” and starts being shared infrastructure.

What engineering teams should be evaluating

The "Anxiety Gap" will not be solved by any single tool. It will be solved by a shift in how organizations think about quality capacity; not as the output of one team, but as a capability the entire engineering organization owns.

Consolidate before you add. Every additional tool introduces integration overhead and maintenance burden. One platform that developers, QA, and product can all use is worth more than three best-of-breed tools that serve different audiences and do not talk to each other.

Automate the repetitive, not the strategic. The goal of autonomous agents is to remove humans from the parts of the loop that do not benefit from human judgment. Test generation, regression execution, and self-healing scripts are pattern-driven tasks. Exploratory testing, risk assessment, and coverage strategy are where human expertise creates irreplaceable value.

Measure time reclaimed, not features shipped. The metric that matters is how many hours per week your engineering team gets back for strategic work. Developer time lost to flaky tests matters just as much as QA time lost to manual regression.

The path forward: from bug-finding to reliability engineering

Better tooling alone will not close the gap. The deeper shift is organizational.

The quality crisis will not resolve until organizations stop treating quality assurance as tactical bug-finding owned by a single team and start recognizing it for what it actually is: strategic reliability engineering that the entire engineering organization is accountable for. Quality is not a phase at the end of the sprint. It is not a gate that one team owns. It is the shared discipline that prevents the disasters that never show up on a P&L statement: the outages that do not happen, the regressions that never reach production, the security gaps that never get exploited.

Until executives understand this (until quality is treated as a first-class engineering function rather than a team you staff up when things break) organizations will continue collapsing under impossible expectations while the $60 billion defect problem persists.

The 75% burnout rate among QA professionals is not inevitable. It is the predictable result of making quality one team's problem instead of everyone's responsibility, and then asking that team to do the work manually while undervaluing the strategic judgment that only humans can provide. The technology to change the first half of that equation exists today. The leadership to change the second half is overdue.

Sources:

• [1] National Institute of Standards and Technology (NIST), "The Economic Impacts of Inadequate Infrastructure for Software Testing," 2002. Total annual cost of software defects: ~$59.5 billion. Avoidable costs through earlier detection: $22.2 billion.

There's a defining moment in every engineering organization when the QA team's work stops being a checkpoint at the end of the sprint and starts being the infrastructure the whole company runs on.

You're not just catching bugs anymore. You're designing the quality systems that let your organization ship with confidence, maintain velocity as the codebase grows, and give leadership a clear picture of release health without someone having to manually compile a report. That is a different, harder, and far more valuable job than what most people outside QA understand.

We spend a lot of time talking with you, and one thing is abundantly clear: your expertise has never mattered more. But as your product scales, so does the operational weight: more test cases, more shared steps to maintain, more dashboards, more pipelines, more teams depending on the systems you built. The work compounds faster than the headcount does.

For Q1 2026, we focused entirely on clearing that weight. We spent the last three months building tools designed to amplify what you already do best, so you can spend your time on the strategic, high-leverage quality engineering that actually moves the needle, not on the maintenance and repetition that surrounds it.

Here is how we're supporting your growth.

Pillar 1: Test Automation, Minus the Boilerplate

The hardest part about scaling test automation isn't finding the time to write tests. It's the translation overhead. You know exactly how a feature should behave, that knowledge exists in your head in clear, plain language. The work is converting it into selectors, wait statements, and boilerplate scripts that actually run.

In Q1, we shipped a set of AIDEN capabilities designed to collapse that gap at every stage of your automation workflow.

Agentic Mode. You state the goal: "Verify a user can purchase shoes on our marketplace," and AIDEN figures out the path. It breaks your natural language into actionable steps, captures screenshots for visual feedback, and generates test code. You can stay high-level or get specific: "Click 'Get a Demo' and fill the email field as 'apidemo@qase.io'." AIDEN finds the field and handles the input.

Test Advisor in Folder View. Test Advisor has always helped you identify which test cases are ready for automation before you start converting. What's new this quarter is that it's now available in Folder view, and you can bulk-run it.

Select all your test cases from a folder, hit 'Run Advisor', and AIDEN analyses them in one go. Results come back color-coded: green cases are ready to automate with minimal oversight; yellow cases have good potential but are missing detail; red cases lack the steps or specificity that automation requires. Fix what needs fixing, re-analyse, then convert. No more discovering mid-generation that a test case wasn't ready.

The insight is yours, and it's also something you can share with any developer on the team who wants to contribute to test automation but isn't sure where to start.

Parameterized Test Conversion. If you've ever maintained a data-driven test suite, you know the problem: one test scenario, twelve input combinations, twelve separate test cases. Keeping them consistent when something changes is the kind of work that makes automation feel like more maintenance than it's worth.

AIDEN now handles this natively. When you convert a manual test that includes multiple input combinations, such as different usernames, passwords, and expected results, AIDEN identifies the parameterized structure, extracts the data, generates one reusable test flow, and creates individual execution instances for each combination automatically. One test. Multiple runs. All your combinations covered.

API Testing and Random Data. Tests aren't just front-ends. AIDEN can execute GET and POST API requests via CURL right in your flow, so a single test can validate both UI behaviour and the API calls underneath. And to keep your environments clean, the Random Data Generator creates unique emails, usernames, and passwords on the fly. No more hardcoded test_user_123 polluting your pre-production data.

Self-Correcting Generation. When your application changes mid-generation and a step breaks, AIDEN no longer stops and hands the problem back to you. It detects the failure, analyses what changed, and re-attempts with a corrected approach. The generation process keeps moving.

Canvas Support. Heavy <canvas> elements, tools like Miro or Figma, have always been automation dead zones. AIDEN now interprets visual elements within a canvas just like standard DOM elements. If your product includes a canvas-based UI, that coverage gap is closed.

Pillar 2: Architecting the Enterprise Test Library

When you build a mature test suite, shared steps transition from a convenience into a foundational dependency. Treat thousands of them as a flat list and you end up with the kind of codebase where nobody's quite sure what anything does, changes break things unexpectedly, and onboarding a new engineer takes two hours of tribal knowledge transfer.

We've fundamentally upgraded how you manage these assets.

Folder Structures for Shared Steps. Shared steps can now be organized into folders and subfolders by domain, like Billing, Auth, or Compliance, mirroring your actual product structure. Bulk move, clone, and rename entire directories. The shared step library now works the way a codebase should: navigable, structured, and maintainable as it grows.

Child Steps in Shared Steps. Complex systems demand modularity. You can now nest child steps directly inside shared steps, building sophisticated reusable workflows. A Checkout Flow shared step can contain the governed logic for payment, address validation, and inventory checks nested within it. Reuse at the macro level, visibility at the micro level.

Bulk Conversion and Safety. Select multiple local steps and promote them to Global Shared Steps in a single click. For cleanup, bulk delete with options to either cascade the deletion across all associated test cases or convert them back to local steps first, so you don't silently break historical data while refactoring.

Pillar 3: Data That Makes the Whole Team Look Good

Test data tells the story of your product's health. The challenge is that it usually tells that story in a way only the person who wrote the query can fully interpret. We want QA leads, engineering managers, developers, and product owners to be able to read it too.

Bar Charts and Donut Charts in QQL. You can now switch any QQL result directly to a Bar Chart or Donut Chart view, save it as the default, and add it as a widget on your dashboard. Bar charts for distribution and trends; donut charts for proportion: automation coverage by business unit, test pass rates by team, coverage gaps at a glance. The work you've done to build this data deserves to be understood by the people you're presenting it to. Now it will be.

Private Queries. When you're deep in an investigation, every exploratory query you save to the public list degrades the signal for everyone else. Queries are now private by default, visible only to you and promotable to public when they prove useful to the team, but not demotable once shared. A one-way door that keeps the shared workspace clean.

Pillar 4: Powering the Complex Enterprise

The modern engineering org doesn't map neatly to "one project, one repo." Monorepos, microservices, distributed systems, multiple teams working in multiple languages: your tooling needs to reflect that reality, not ask you to simplify down to it.

GitLab Requirement Traceability. For teams that need to demonstrate that specific requirements have been validated by specific tests, whether in regulated environments, compliance-heavy orgs, or anywhere auditors ask for evidence, we've extended full requirements traceability support to GitLab. Link test cases to GitLab requirements, track which requirements have test coverage, and see which open failures are blocking a requirement from being marked satisfied. GitHub teams have had this for a while. GitLab teams now have it too.

Multi-Project Routing. Our JavaScript and Python reporters now handle monorepos natively. A single CI run that touches three product lines routes those results to the correct Qase projects simultaneously, with no custom pipelines, no post-processing, no manual splitting after the fact.

Java Reporter Performance. Result processing time in our Java reporters has been cut by 40% on large projects. If you're running a significant Java suite, this shows up directly in your pipeline times.

Network Profiler in JS Reporters. JS reporters now capture network activity alongside test results. When a test fails because of a request that timed out or returned the wrong status code, that evidence is in the report, not something you have to reconstruct from server logs after the fact.

Interactive Test Reports. We published Qase Report, a CLI tool that generates a fully interactive, standalone HTML report from your test results in a single command. No server, no dashboard login required. Share it with a stakeholder, open it in a browser, done. Full write-up: Interactive Test Reports in a Single Command.

Expanded Framework Support. We added native reporters for MSTest and xUnit v3, and open-sourced a reporter for Pest (PHP). For mobile teams, NUnit and JUnit 4 for Android. Behave now supports step-level attachments; CucumberJS gets parameter and suite annotation support; Mocha, Vitest, and Jest support expected results and input data at the step level. Jira Data Center 10.x is now officially supported, whether you're on the latest version or holding stable.

The Finishing Touches

Alongside the major pillars, we squashed 10 bugs across the platform this quarter. Not every improvement ships as a feature announcement. Sometimes it's fixing what's been quietly bothering you for three months.

We also moved our community feedback and roadmap to roadmap.qase.io. Submit requests, vote on what you want built, and track ideas as they move from submitted to in progress to shipped. If you used Canny before, your data migrates automatically. The roadmap is public, the conversation is open, and what you vote for directly shapes what we prioritise.

We're proud of what shipped this quarter, and we're even more proud of the work you do. You are the architects of software quality, and our goal is to build a platform that feels like a true partner in that work. Everything in this post is live today.

We can't wait to see what you build next.

Happy testing!

Don't miss out on future updates and valuable content! Connect with us on LinkedIn to stay informed about all things Qase.

Testing is an economic investment, not a cost to minimize. The question "how much testing do we need?" is broken: it should be "which risks justify which investments?". This post introduces the Cost of Quality framework, portfolio thinking, and a four-step process for making testing decisions grounded in risk.

I read arguments and battles on "what's the proper coverage" on the internet a lot. This saddens me.

To me, this question is completely broken: it frames testing as a quantity problem. And quantity problems get quantity answers: "80% code coverage". Arbitrary numbers that feel responsible but aren't grounded in anything real.

To me, the right question is: which risks justify which investments?

A story about cutting coverage and improving quality

A few years ago I worked on a project sitting at 70% code coverage. In reports, this looked nice. In practice, the suite was full of flaky tests which failed intermittently for reasons unrelated to actual defects.

The costs were layered. The direct cost was obvious: the team was spending up to two man-days a week investigating failures that turned out to be noise. But the planning cost was worse: because the spikes were unpredictable, some weeks nothing, some weeks the whole team was blocked, sprint planning became guesswork. And then there was the trust cost, which was the most damaging: even when tests passed, nobody was confident they meant anything. The suite had lost its credibility. There was no way to tell "we're fine" from "the tests just aren't catching anything".

I ran the numbers: the effort going into maintaining and investigating those flaky tests was not producing expected value. So we cut them. Coverage dropped to 40%. We filled the gaps with manual testing while we slowly rebuilt, writing reliable tests for the things that actually mattered.

Quality improved, and not because we tested less, but because the evidence that remained was trustworthy. When the suite passed, it meant something. When it flagged a problem, we acted on it instead of adding it to the "probably flaky" pile.

"Less testing" was actually "better investment".

Same question, completely different answers

I can think of two very different systems. An insulin pump: a defect can kill a patient. The software is FDA-regulated, subject to formal verification, extensive documentation, and extensive testing at every level. A corporate internal CMS used by 50 people to publish blog posts: a defect means wrong formatting, or a post doesn't save on the first try.

Ask both teams "how much testing do you need?" and you should get completely different answers. Not because one team is more professional or more serious than the other, but because the risks are different. The stakes are different. The consequences of failure are different.

Testing strategy follows from risk, not from a generic idea of "the right amount".

What testing actually does

Testing is a learning activity; you explore a system, build understanding of how it behaves, and discover where it breaks. That learning produces evidence: about defects, failure modes, risks. In economic terms, this is appraisal: generating information that supports decisions.

It does not reduce risk on its own.

Risk is reduced when findings lead to action: a fix, a mitigation, a control, a rollback plan, a guardrail. "We tested it" does not mean "it's safe". "We tested it, found X, and did Y about it" — that's risk reduction.

This is what made the flaky test situation so corrosive. The tests were running, but the evidence they produced couldn't be trusted. So no risk was actually being reduced. The suite looked like an investment. It was a cost.

Testing as investment

Investment, in the economic sense: an asset obtained at cost, with an expectation that the future value it creates will exceed that cost.

This is exactly what testing is. You spend time, money, and attention today to reduce the probability and impact of costly failures tomorrow. The return isn't guaranteed — you're estimating expected value under uncertainty, not booking a fixed outcome. But that's true of any investment.

Same with any insurance, actually. You pay premiums expecting never to file a claim. If you never claim, that isn't waste — the protection was real. The decision to insure is based on risk exposure, not on the assumption you will definitely suffer a loss. Nobody buys scuba diving insurance for their daily commute. Construction sites invest heavily in safety equipment because the risks are real and the consequences of getting it wrong are severe.

Preventive healthcare works the same way. Catching a problem early is cheaper than treating it late. That's not a guarantee; it's a reason to invest.

But in my case, 70% coverage was a bad investment, more like a cost actually. The effort wasn't producing the future value we needed, namely reliable evidence about risk.

The Cost of Quality framework

There's a useful vocabulary for where quality-related money goes. The Cost of Quality framework breaks it into four categories.

Prevention: training, standards, design reviews, threat modeling — activities that stop defects from being introduced.

Appraisal: testing, code reviews, static analysis, audits — activities that detect defects that exist.

Internal failure: rework, debugging, re-testing, delayed releases — costs incurred when defects are found before they reach users.

External failure: incidents, support load, lost revenue, fines, reputation damage — costs incurred when defects reach users.

External failure is the most expensive category, because it combines remediation cost with customer impact, operational disruption, and often lasting reputational damage.

The economic goal is to invest in prevention and appraisal at a level that reduces the more expensive failure costs. The two man-days a week we were spending on flaky test investigation was internal failure cost, rework caused by unreliable appraisal. We were paying the failure tax without getting the appraisal benefit.

Diminishing returns and portfolio thinking

So the framework helps you see where your money goes. But even when you're investing in the right category — appraisal — not every dollar spent there delivers equal value.

The first tests you write tend to catch the most obvious, highest-impact problems. Each additional test catches progressively rarer or lower-impact issues. At some point, the cost of the next test exceeds the expected value of the risk it covers.

So you can't just "add more" — you need to choose what kind of investment you need for which risks. That's portfolio thinking: selecting a mix of complementary activities to achieve maximum risk reduction per resource spent. Static analysis catches things integration tests don't. Exploratory testing finds what scripted tests miss. The combination matters.

To build a portfolio, you need to know what you're investing against.

Start from risks

Investing against what, exactly? Risks to specific qualities your system needs to have. ISO 25010 gives a useful vocabulary for those: eight product quality characteristics that map to the categories of risk your system may carry. Functional suitability, performance efficiency, compatibility, usability, reliability, security, maintainability, portability.

For an insulin pump, the dominant risks sit in reliability, functional suitability, and security. For an internal CMS, they're mostly usability and maybe functional suitability. Different risk profiles mean different portfolios — different mixes of testing activity, different intensity, different coverage targets.

When investment decisions are grounded in risks, they become traceable and reviewable. You can explain why you're testing what you're testing. You can revisit the decision when conditions change.

What comes next

What we did on that project — cutting coverage from 70% to 40%, adding manual testing, rebuilding selectively — was an instinctive version of a more systematic process: calculate the cost, cut what isn't delivering, rebuild deliberately.

That process has four steps: identify and quantify risks, categorize and prioritize by exposure, decide where and how to invest in testing, then review and rebalance. It's a continuous cycle, not a one-time plan.

Next post in this series will cover the first step: how to identify and quantify the risks you're actually managing.

The full research behind this series is at BeyondQuality.

If you work in tech, or a tech adjacent industry, you’ll know that AI has accelerated everything, including software development. Engineers now write, refactor, and ship code faster than ever using tools like Claude, Cursor, and Copilot.

Development speed has increased, but testing workflows often struggle to keep up. Engineering teams face more features, more commits, and more releases than usual testing cycles were designed to handle.

AI is already helping close that gap. Teams increasingly use AI to generate test cases from feature descriptions, analyze code changes for regression scenarios, or explore edge cases that might otherwise be missed.

But generating tests is only part of the problem.

Organizations still need a way to manage those tests, track coverage, coordinate teams, and maintain visibility into quality over time.

This is where AIDEN and the Qase MCP Server work together.

AI can generate tests. Teams still need to manage them.

AI assistants are excellent at producing test ideas.

Developers can ask tools like Claude or Cursor to generate test cases from a specification. AI can analyze code changes and suggest additional coverage. Within seconds, teams can produce dozens of potential tests.

But AI tools alone don’t provide the structure organizations need to manage testing effectively.

Teams still need a system that provides:

• Traceability between requirements, tests, and defects
• Coverage visibility across features and releases
• Team coordination for QA workflows
• Audit trails and governance required by enterprises
• Human oversight to validate and evolve test strategies

Without a structured testing system, AI-generated tests remain scattered across conversations, documents, or temp files.

To fully benefit from AI-generated testing, those artifacts need to live inside a system designed to manage quality.

That system is Qase.

AIDEN: AI built specifically for testing

AIDEN is Qase’s AI testing expert, designed to help teams create and manage tests faster.

It allows teams to determine which tests are most suitable, as well as generate test cases directly inside Qase in multiple ways, including:

• Generating tests from requirements or feature descriptions
• Using natural language prompts to create or update test cases
• Expanding test coverage with AI-generated scenarios

AIDEN is an AI purpose-built for testing workflows inside Qase.

Instead of drafting tests in external tools and manually transferring them into a test management system, AIDEN generates tests directly where they will ultimately be stored, reviewed, and executed.

This helps engineering teams reduce backlog and accelerate test creation while keeping testing artifacts structured and visible to the entire team.

But modern engineering teams don’t rely on just one AI tool.

Developers already work with AI inside their IDEs, documentation systems, and collaboration tools. That’s where the Qase MCP Server comes in.

What MCP is and why it matters

Model Context Protocol (MCP) is an open standard that allows AI assistants to interact with external systems through structured operations.

Instead of AI tools working in isolation, MCP enables them to connect to real systems like version control platforms, documentation tools, and test management systems.

You can think of MCP as the bridge that allows AI assistants to interact with the tools teams already use.

For testing workflows, this means AI tools can do more than suggest tests, they can interact directly with the system where tests are managed.

The Qase MCP Server: connecting AI tools to your testing system

The Qase MCP Server allows external AI assistants to interact directly with Qase.

Teams can continue using AI tools like Claude or Cursor, but instead of generating test artifacts that remain in chats or notes, those tools can create and manage testing artifacts directly inside Qase.

Through MCP, AI assistants can:

• Retrieve structured testing data such as projects, suites, runs, and results
• Search testing data using QQL
• Create and update test cases
• Access execution history and environment data

This allows teams to combine the flexibility of AI assistants with the structure of a dedicated test management system.

In practice, it means AI tools can participate in real testing workflows rather than producing disconnected suggestions.

A practical example

Consider a typical development workflow.

After a developer opens a pull request, a QA engineer can use an AI assistant to analyze the changes and generate relevant test cases and regression scenarios. Through the Qase MCP Server, those tests can be created directly in Qase, where the QA team can review them, refine the scenarios if needed, and execute them as part of the testing workflow.

Instead of documenting suggested tests elsewhere or leaving them in AI chat threads, the assistant adds them directly to the appropriate test suites in Qase.

These workflows allow teams to keep the speed of AI-assisted development while ensuring that testing artifacts remain structured, visible, and manageable across the organization.

Why structured test management still matters

AI can accelerate test creation, but organizations still need a system that manages testing as part of the broader SDLC.

Testing systems provide the visibility teams rely on to answer important questions:

• What coverage exists for this feature?
• What failed in the last release?
• Where are defects clustering?

More importantly, they help teams answer the question that ultimately matters most: are we ready to ship?

They also provide the governance and traceability required by enterprise environments.

Qase acts as the system of record for quality, capturing the full lifecycle of testing activity across projects and releases.

By combining AIDEN’s AI-driven test creation with MCP’s ability to connect external AI tools, Qase helps teams ensure that AI-generated testing becomes part of a structured, collaborative process.

Bringing testing into the AI-augmented SDLC

The software development lifecycle is being reshaped by AI.

AI now participates in writing code, reviewing changes, generating documentation, and suggesting improvements. Testing is becoming part of that same AI-assisted workflow.

Instead of a linear process where testing happens at the end, teams are moving toward a continuous feedback loop: Code, testing, and analysis now happen simultaneously, feeding improvements directly back into the development process.

For this process to work effectively, testing data and workflows must be accessible to the AI tools teams already use

AIDEN accelerates test creation directly within Qase, while the Qase MCP Server connects external AI assistants to the same system.

Together, these tools make it possible for testing to operate as a continuous part of the development loop, not a separate phase that slows it down.

Takeaways 

AI is accelerating how software is built. Developers can generate code and tests faster than ever before using tools like Claude, Cursor, and other AI assistants. But speed alone isn’t enough. Without structure, visibility, and coordination, AI-generated tests remain fragmented and difficult to manage at scale.

That’s where Qase comes in. AIDEN enables teams to generate and expand test cases directly inside Qase, ensuring that test creation happens within a system designed for collaboration and execution. At the same time, the Qase MCP Server connects external AI tools to that same system, allowing teams to continue using their preferred AI workflows while ensuring the results are captured, organized, and actionable.

Together, AIDEN and MCP bridge the gap between AI-enabled speed and structured quality management. Teams don’t have to choose between moving fast and maintaining control because they can do both by ensuring AI-generated tests are created, managed, and tracked within Qase.

Our Berlin Quality Engineering community came together for another evening, this time at a new venue, W3.Hub, which turned out to be a fantastic spot for the event. Around 70 people showed up, and the energy was as good as ever: three very different talks, great pizza, and the kind of discussions that keep going well past the official end.

It is a true pleasure to have people coming up to me and saying "thank you for an amazing set of talks" or "you must run a conference". This means a lot, and maybe partly it works so well because I intentionally try to maintain a diversity of topics: a human-centric angle, a scientific one, and a day-to-day QA one. All about quality, of course, but from very different perspectives I believe we must always have.

Food QA: Testing Tasting

Mitesh Patel opened the evening with an awesome talk on how big food corporations test food.

Mitesh introduced the concept of "wetware QA": our brains are essentially running quality checks every time we taste food. When you bite into something and your brain decides whether it is good, it is running a testing algorithm — dissolution of chemicals in saliva, activation of taste receptors, conversion to nerve impulses, and recognition of flavours by the cortex.

So it is very easy for one person to "test" what they are eating and give a verdict: high or low quality. But everyone is different, and tasting preferences change even within one person depending on the time of day or their mood. So what would it take for a food producer to test the food they produce? Ask all their customers to taste it?

Human taste testing does not scale. The food industry tries to solve this with Sensory Evaluation Panels — groups of trained tasters — but these are expensive, slow, subject to biopsychosocial bias, and take months to set up. So maybe it would be possible to skip humans altogether? Teach a computer what food tastes?

Computers do not have taste receptors or saliva — they simply cannot run the same algorithm our brains do. But they can see flavours. Through near-infrared spectroscopy, food chemicals reveal unique light absorption signatures — what Mitesh called "seeing the invisible colours of food."

In software development, some systems are also too complex to fully analyse from the inside — large language models being the obvious example right now. So we do the same thing Mitesh described: find a proxy. Black-box testing, RAGAS evals, observability-based testing — all ways of seeing what we cannot directly taste.

Earn Trust, Keep Your Job

Asya Isakova gave the second talk about trust at work, grounded in organizational psychology research.

You got the job, now how do you keep it? Your colleagues and managers decide whether to trust you not by reading your CV, but by watching what you do in the first weeks and months. Trust is defined as "a psychological state comprising the intention to accept vulnerability based upon positive expectations of the intentions or behaviour of another". So how do you build it? How do you help people desire to be open or even vulnerable to you?

Asya identified three pillars: integrity (do the right thing consistently — be honest, fair, hold high standards), ability (show you can do the job well — skills plus good judgement), and benevolence (show you care about others' success, not just your own).

To show ability: make "done" explicit and testable, deliver something small early, map workflows, keep a blocker log, and communicate progress predictably — Done, Next, Blocked, Ask. To show integrity: make your work traceable, close loops, link your outputs to goals, respect team norms, escalate risks early, and own your mistakes. To show benevolence: ask stakeholders what success looks like from their side, show respect for people's time, share credit, offer small help early, avoid blame language (describe problems as system issues, not personal failures), and close the onboarding loop by sharing an "onboarding gaps and fixes" document at week four to six, so the next person has it easier.

This talk was just the tip of the iceberg. Trust has a much broader impact on how teams work — and how much waste a lack of it creates. Asya is currently doing research on the "trust tax" topic at BeyondQuality: how missing trust translates into extra process, slower decisions, and unnecessary controls. You can follow and contribute to the discussion on GitHub.

QA Myths Busting: A Practical Guide to Higher Quality

I gave the third talk about four QA myths that are unfortunately still prevalent in the industry. Each one has a detailed companion article:

• Quality is testers' responsibility
• More testing means better quality
• QA slows work down
• Quality can be measured

As always, a big thank you to the speakers for sharing their work, to W3 Hub for hosting us, and to everyone who came and stayed for the conversations afterwards.

If you have been thinking about giving a talk — whether it is your first one or your fiftieth — come talk to me. I am always happy to help you get there. See you at the next meetup!!!

Test results are more than just "green/red" in CI logs. Behind every run lies a story: which tests break most often, where performance degrades, which errors keep recurring. The problem is that standard terminal output doesn't answer these questions.

Qase Report is an open-source CLI tool that transforms test results into an interactive HTML report with a dashboard, analytics, screenshot gallery, and run history. A single file you can open in a browser without a server — and share with a colleague or attach to a ticket.

In this article, we'll walk through how to install and use Qase Report: from first launch to trend analysis across dozens of runs.

Installation

Qase Report is distributed as an npm package and requires Node.js 18+.

npm install -g qase-report

Verify the installation:

qase-report --help

You'll see two main commands: open and generate. More on each below.

Preparing Data

Qase Report works with data in the Qase Report Format. If you use Qase reporters for pytest, Playwright, Jest, Cypress, or other frameworks — the data is generated automatically.

The results directory structure looks like this:

results/
├── run.json              # Run metadata and statistics
├── results/
│   ├── {uuid-1}.json     # Individual test result
│   ├── {uuid-2}.json
│   └── ...
├── attachments/          # Screenshots, logs, files
│   └── ...
└── qase-report-history.json  # Optional: run history

The run.json file contains general run information: title, environment, execution time, and status statistics. Each file in results/ describes a single test case with its steps, attachments, and metadata.

Quick Start: Opening a Report

The simplest way to view a report is the open command. It starts a local server and opens the report in your browser:

qase-report open ./results

The browser will open automatically at http://localhost:3000. You'll immediately see the test list for the current run.

If port 3000 is already in use, specify a different one:

qase-report open ./results -p 8080

Generating Static HTML

For sharing or archiving, it's more convenient to generate a standalone HTML file:

qase-report generate ./results -o report.html

The resulting file contains everything: the application code, styles, and test data. You can open it with a double-click in any browser — no server required. This is useful for:

• Attaching to tickets in Jira or GitHub Issues
• Sending to colleagues via email or Slack
• Archiving as CI/CD artifacts

Interface Overview

The report consists of several sections, accessible via tabs at the top of the screen.

Test Cases — Test List

This is the default view that opens first. Tests are organized in a suite hierarchy that you can expand and collapse.

Each test displays its status, duration, and stability grade. Available statuses:

Above the list are status filters and a search field for finding tests by name.

Click on a test to open its details in a side panel: execution steps, error stacktrace, attachments, and parameters.

Analytics — Dashboard

The Analytics tab is an interactive bento-grid dashboard with key run metrics.

Here you'll find:

• Alerts — notifications about flaky tests and regressions
• Attention Required — tests that need attention: unstable, slow, new failures
• Quick Insights — top failing tests and slowest tests
• Test Health — stability grade for each test on an A+ to F scale
• Suite Health — pass rate by suite

When run history is connected (more on this in the next section), the dashboard is enriched with trends:

• Pass Rate Trends — pass rate trend chart across runs
• Duration Trends — execution time changes over time
• Recent Runs — cards for recent runs with summary statistics

Failure Clusters — Error Clustering

When multiple tests fail with the same error, the Failure Clusters section automatically groups them together.

This helps quickly understand the scope of a problem: if 15 tests failed with the same ConnectionRefusedError, the cause is likely infrastructure, not the tests themselves.

Attachments — Attachment Gallery

All screenshots, logs, and files attached to tests are gathered in one place.

Available features:

• Filtering by category (screenshots, logs, other files) and test status
• Sorting by name, date, size
• Two display modes: grid with adjustable tile sizes and list
• Full-screen screenshot viewing with zoom and navigation
• Search by file name

Timeline — Execution Visualization

The Timeline section shows how tests were executed over time. Each thread (worker) is displayed as a separate swimlane, and tests appear as colored bars with duration.

This is useful for:

• Parallelism analysis: is the load evenly distributed across workers?
• Finding bottlenecks: is there one long test holding up the entire run?
• Visually understanding test execution order

Zoom can be adjusted from 1x to 5x for detailed examination of specific segments.

Comparison — Run Comparison

When history is connected, the Comparison tab lets you compare two runs side by side.

You'll see:

• Which tests changed status (e.g., passed → failed)
• Which tests were added or removed
• How each test's duration changed
• Summary statistics: pass rate difference, change counts by status

Traces — Playwright Traces

If your tests use Playwright and generate trace files, an additional Traces tab appears in the report. It allows interactive playback of browser test recordings: actions, screenshots at each step, and network requests.

Run History and Trend Analytics

One of Qase Report's key capabilities is working with history. When a history file is provided, the tool accumulates run data and delivers powerful analytics.

Connecting History

When using the open command, history is saved automatically:

qase-report open ./results -H ./qase-report-history.json

Each time you open a new report, run data is appended to the history file. For the generate command, history must be passed separately:

qase-report generate ./results -H ./qase-report-history.json -o report.html

What History Provides

After accumulating data across several runs, the following become available:

Pass rate trends — how the test pass rate changed from run to run. Useful for tracking overall project health.

Flaky test detection — the tool analyzes status alternations (pass → fail → pass) and identifies unstable tests. Requires a minimum of 5 runs.

Stability Score — each test receives a grade from A+ to F based on three factors:

• Pass rate (50% weight)
• Flakiness (30% weight)
• Duration consistency (20% weight)

Requires a minimum of 10 runs to calculate.

Performance regression detection — if a test's duration exceeds the mean + 2 standard deviations, the system generates an alert.

CI/CD Integration

In a CI/CD pipeline, the history file is typically stored as an artifact passed between runs:

# GitHub Actions example
- name: Download history
  uses: actions/download-artifact@v4
  with:
    name: qase-report-history
    path: ./results
  continue-on-error: true  # First run — no history yet

- name: Run tests
  run: pytest --qase ./results

- name: Generate report
  run: qase-report generate ./results -H ./results/qase-report-history.json -o report.html

- name: Upload report
  uses: actions/upload-artifact@v4
  with:
    name: test-report
    path: report.html

- name: Save history
  uses: actions/upload-artifact@v4
  with:
    name: qase-report-history
    path: ./results/qase-report-history.json

Sending Results to Qase TMS

If you use Qase TMS for test management, results can be sent directly from the report — no additional scripts or configuration needed.

When using the qase-report open command, a Send to Qase button appears in the report header. Click it to open a dialog with three fields:

• Project Code — your project code in Qase (e.g., DEMO)
• API Token — API token from your Qase account settings
• Test Run Title — run title (automatically populated from report data)

After clicking Send Results, the tool:

1. Creates a new test run in Qase TMS
2. Uploads attachments (screenshots, logs, files)
3. Sends all test results with steps, errors, and parameters
4. Completes the run

Once finished, you'll receive a link to view the run in Qase TMS.

This feature is only available in server mode (qase-report open). The send button is not shown in static HTML files.

Additional Features

Dark and Light Themes

The report uses a dark theme by default. Switch to light theme via the icon in the top-right corner.

Loading a New Report

The "Load Report" button in the header lets you load a different results directory directly in the browser — without restarting the server.

Downloading the Report

The "Download" button in the header lets you download the current report as a standalone HTML file.

Summary

Qase Report solves three problems:

1. Visualization — transforms raw JSON data into a clear, interactive report with filters, search, and navigation.
2. Analytics — identifies flaky tests, performance regressions, and error patterns that are impossible to see from terminal output.
3. Easy sharing — a single HTML file that opens in any browser without installation, configuration, or servers.

Getting started is one command:

npm install -g qase-report && qase-report open ./results

Project repository: github.com/qase-tms/qase-report

There is a breaking point in every engineering organization. It usually happens when you cross a certain threshold of test cases or team members. Suddenly, the processes that worked for a team of ten become the bottlenecks for a team of a hundred.

Scale changes everything. It turns flat lists into endless scrolling, simple queries into noisy data, and single-project reporting into a logistical nightmare.

In February, we focused on that threshold. We shipped updates designed specifically for organizations operating at scale and features that turn your test repository from a simple storage unit into a governed, architected library.

Here is how we are supporting your growth.

Pillar 1: Architecting for Scale

From "Lists" to "Asset Management."

In a startup, a shared step is a convenience. In an enterprise, it is a dependency. When you have thousands of shared steps across dozens of projects, treating them like a flat list is a liability. It creates duplication, makes maintenance impossible, and slows down onboarding.

We have fundamentally upgraded how you manage these assets with Folder Structures for Shared Steps.

This isn't just about tidying up. This is about architectural hygiene. You can now structure your shared logic by domain—Billing, Auth, Compliance—creating a navigable hierarchy that acts as a single source of truth. We’ve added bulk actions to move, clone, and rename entire directories, giving you the controls you need to refactor your test library as easily as you refactor your code.

We also introduced Child Steps in Shared Steps.

Complex systems require modular testing. You can now nest child steps directly inside shared steps, allowing you to build sophisticated, multi-layer "meta-steps." A "Checkout Flow" shared step can now contain the specific, governed logic for payment, address validation, and inventory checks nested within it. It allows for high-level reuse without sacrificing low-level granularity.

Pillar 2: Workspace Governance

Signal over Noise with Private Queries.

In a large organization, a shared workspace can quickly become a tragedy of the commons. If every engineer saves every exploratory query to the public list, the team’s critical dashboards get buried under temporary data.

We are introducing Private Queries to solve this workspace pollution.

Now, engineers can save their specific, investigative queries without broadcasting them to the entire organization. It allows for personal productivity workflows that don't degrade the shared environment. If a query proves valuable for the wider team, it can be promoted to public status but we’ve designed the flow to default to hygiene first.

Note: To ensure strict permission governance, private queries are currently restricted from shared dashboards.

Pillar 3: Data-Driven Decision Making

Executive Visualization with Donut Charts.

At the executive level, you don't need rows of data; you need immediate insight into health and distribution. We’ve added Donut Chart Mode to QQL to support high-level reporting.

You can now toggle complex datasets from tabular views to visualizations instantly. Whether you are presenting a quarterly quality review or checking the distribution of automation coverage across business units, this view allows stakeholders to understand the "shape" of the data in seconds.

Pillar 4: The Monorepo Reality

Native Support for Complex Architectures.

The modern enterprise doesn't always fit neatly into "one project, one repo." You are likely running monorepos, microservices, or complex distributed systems where a single CI pipeline touches multiple business domains.

Your tooling needs to reflect that reality.

• Multi-Project Routing (JS & Python): We have engineered our reporters to handle the complexity of enterprise codebases. Our JavaScript and Python reporters now support Multi-Project routing. If you have a monorepo that triggers tests across three different product lines, our reporters can now intelligently route those results to the correct Qase projects in a single run. No distinct pipelines required.
• Pest Framework Support: We are committed to supporting the frameworks your teams use. We’ve released and open-sourced a native reporter for Pest, bringing first-class PHP testing into the Qase ecosystem. View the repository.
• Unified Reporting Standard: We’ve implemented a consolidated Qase Report schema to standardize how data flows from your CI/CD pipelines into our platform, ensuring reliability regardless of the environment scale. View the repository.

February was about ensuring that Qase grows with you. Whether you are managing a monorepo with hundreds of microservices or a shared step library with thousands of assets, our goal is to provide the governance and structure you need to keep moving fast.

Happy testing!

A well-written test case follows a clear, repeatable structure. Use this checklist to create consistent and traceable test cases.

1. Test Case ID – Assign a unique identifier for traceability.
2. Title / Description – Clearly define the objective and functionality under test.
3. Preconditions – Specify required system state, configurations, or dependencies before execution.
4. Test Steps – List precise, step-by-step actions the tester must perform.
5. Test Data – Provide exact input values required for execution.
6. Expected Result – Define the measurable outcome that should occur after the steps are executed.
7. Actual Result – Record what actually happened during testing.
8. Status – Mark the test as Pass, Fail, or Blocked.
9. Comments / Notes – Document observations, defects, or clarifications.

What Is a Test Case?

A test case is a structured document used in software testing to verify that a specific functionality behaves as expected. It defines precise test steps, required test data, preconditions, and expected results so a tester can validate system behavior in a controlled and repeatable way. Each test case focuses on a single objective and is written to produce a measurable outcome during test execution.

A standard test case includes a unique identifier (Test Case ID), a clear test case description, defined preconditions, step-by-step test steps, input data, expected results, and a recorded actual result. The tester executes the steps, compares the actual result with the expected outcome, and assigns a status. This structure supports traceability, validation, and consistent documentation across test plans and test suites.

Test cases are used across multiple types of testing, including:

• functional testing
• unit testing
• smoke testing
• regression testing
• acceptance testing
• end-to-end testing

They can be written for manual testing or converted into automated test cases for execution within automation frameworks. In both cases, well-written test cases ensure repeatable validation of application behavior.

Detailed guidance on organizing and managing test cases within Qase's TMS is available in the Qase documentation for Test Cases and the Getting Started guide.

Why Test Cases Are Important in Software Testing

Test cases play a central role in effective software testing because they define exactly how functionality should be validated. Without clearly written test cases, testing becomes inconsistent, difficult to reproduce, and harder to scale.

First, test cases establish structured test procedures. They define inputs, execution conditions, and expected results in a standardized format. This ensures that each tester follows the same steps and evaluates the same criteria, improving repeatability and reducing ambiguity.

Second, they improve test coverage. By mapping test cases to requirements, user stories, or acceptance criteria, teams can ensure that all critical functionality is validated. This is especially important in areas like functional testing and when comparing different approaches like black-box vs. white-box testing.

Test cases are also foundational for regression testing. As applications evolve, regression test suites help confirm that new changes do not break existing functionality. Well-structured test cases make it easier to build and maintain scalable regression suites, particularly when automation is involved. In fact, understanding the benefits and trade-offs of automation is essential when building long-term testing strategies, as discussed in this guide to QA automation benefits and best practices.

Beyond validation, test cases generate valuable data. During test execution, testers compare expected results to actual results, log defects, and contribute insights that help improve product quality and optimize workflows. Over time, this documentation supports traceability, compliance, and informed release decisions.

Ultimately, effective test cases build confidence. They provide stakeholders with evidence that features work correctly under defined conditions and that the product meets agreed-upon acceptance standards. In modern software development—whether Agile or traditional—test cases remain one of the most reliable tools for maintaining quality, consistency, and transparency throughout the testing lifecycle.

Types of Testing Covered by Test Cases (Functional, Regression, Unit & More)

Test cases are used across all major types of testing in software testing. In functional testing, a test case validates specific functionality against defined requirements and expected results. In unit testing, it verifies the behavior of individual modules or components in isolation. Within regression testing, structured test cases form reusable regression suites that detect defects introduced by code changes and protect previously validated features.

Test cases support a wide range of additional testing types, including:

• Performance testing, including load testing, stress testing, capacity testing, and recovery testing
• Compatibility testing across browsers, operating systems, and devices
• Portability testing to validate behavior in different environments
• Interoperability testing between integrated systems and APIs
• Security testing to validate protection against vulnerabilities
• Reliability testing, including chaos testing
• Usability testing and accessibility testing
• Localization and conversion testing
• Installability and disaster recovery testing
• Maintainability and procedure testing

Each of these areas relies on clearly defined preconditions, test steps, test data, expected results, and recorded actual results during test execution.

A comprehensive overview of testing approaches and execution strategies is available in the guide to software testing types, approaches, levels, and execution strategies. Regardless of testing type, well-written test cases enforce repeatable validation, structured documentation, and traceability within a test management process

Writing Test Cases for Manual vs. Automated Testing

Writing test cases for manual testing differs from writing automated test cases in structure and execution detail. Manual testing relies on human testers to execute step-by-step instructions, observe behavior, and compare actual results with expected results. Manual test cases emphasize clarity in test steps, detailed validation criteria, defined preconditions, and explicit test data so testers can accurately validate functionality.

The differences between manual and automated test cases appear most clearly in execution and maintenance:

• Execution method: Manual test cases are executed by a tester following documented test steps. Automated test cases are executed by tools or scripts that run predefined instructions without human intervention.
• Consistency: Manual testing can introduce variability depending on the tester. Automated tests execute the same steps consistently across environments.
• Speed and scalability: Automated tests run significantly faster and support repeated execution across builds. Manual testing requires more time as test suites expand.
• Use cases: Automated testing is commonly applied in regression testing, where maintaining scalable regression suites is critical. Manual testing is used for exploratory testing, usability validation, and rapidly changing features.

Automated testing supports continuous integration pipelines and repeated validation cycles. Strategic considerations around scalability and long-term efficiency are examined in QA automation benefits and best practices. Both approaches require structured test case writing to maintain traceability, validation accuracy, and maintainability within the software testing lifecycle.

Test Case Templates: Structure, Test Steps, Preconditions, and Expected Results

A structured test case template standardizes how testers document functionality, validation logic, and expected outcomes in software testing. Each test case begins with a clear title that defines the functionality under test, followed by a unique identifier (Test Case ID) that ensures traceability within test management systems and test plans. The description clarifies scope and intent, reducing ambiguity during test execution.

Preconditions define system state, dependencies, environment setup, and required configurations before execution begins. Test steps are written in a precise, sequential format so the tester can reproduce the validation process without interpretation. Each step references specific test data, including input values, configuration parameters, or API requests. Expected results describe the measurable outcome that should occur after executing the defined steps. During execution, the tester records the actual result, assigns a status, and documents relevant comments. Postconditions specify the required system state after completion, especially when tests affect shared environments or persistent data.

This structure supports consistent validation across functional testing, smoke testing, and regression testing. Test cases can be organized into reusable test suites and aligned with broader lifecycle practices described in the three pillars of managing testing artifacts and continuous testing.

How to Design Effective and Well-Written Test Cases

Effective test case design applies structured test design techniques to maximize coverage and reduce redundancy. Equivalence class testing groups input data into logical categories so a tester can validate representative values instead of testing every possible input. Boundary value testing targets minimum, maximum, and edge conditions where defects frequently occur. Decision table testing maps combinations of conditions to expected results, exposing logic errors in complex workflows. State transition testing verifies how a system behaves as it moves between defined states. Pairwise testing reduces combinations by validating parameter interactions in controlled pairs. Error guessing leverages prior defect patterns to expose hidden failure points.

Well-written test cases define explicit test steps, precise test data, and deterministic expected results. They isolate a single functionality, avoid overlapping scope, and remain reproducible across environments. Test cases should be prioritized based on risk, frequency of use, and business impact, then grouped into structured test suites. In regression cycles, these suites become the backbone of scalable regression suites. In lower-level validation, they align with unit testing. Broader testing strategies, including negative scenarios, are detailed in guides like how to use negative testing to create more resilient software and the overview of software testing types and execution strategies.

Design quality directly impacts maintainability. Clear structure, traceable identifiers, reusable steps, and accurate validation criteria reduce rework and debugging time. Effective test cases function as executable documentation within the software testing workflow.

Aligning Test Cases with Test Plans, Acceptance Criteria, and Workflow

Test cases must align directly with defined test plans, acceptance criteria, and the broader workflow of the software testing lifecycle. Each test case should map to a specific requirement, user story, or system component to ensure traceability and validation coverage. Identifying components and interfaces early ensures that interactions between modules, APIs, and integrated systems are validated through structured test steps and measurable expected results.

Stakeholder input shapes scope and validation logic. Product owners, developers, and QA leads define acceptance test criteria that determine whether functionality meets business and technical standards. Test cases must reflect these criteria explicitly, especially in contexts like functional testing, acceptance testing, and end-to-end testing. In integration-heavy environments, alignment with system integration testing (SIT) ensures that cross-system dependencies are validated.

Test plans define execution scope, test suites, dependencies, and timelines. Test cases should be prioritized according to risk, release impact, and regression exposure. In recurring release cycles, alignment with regression testing and smoke testing ensures that core functionality remains stable after changes. Workflow synchronization requires that updates to requirements trigger corresponding updates to test cases, preserving validation accuracy across iterations.

Acceptance criteria serve as the final validation checkpoint. During execution, actual results must be evaluated against defined acceptance conditions for functionality, performance, and compliance. Structured alignment between test cases, test plans, and workflow enforces traceability, measurable validation, and controlled release readiness in accordance with standards like ISO/IEC/IEEE 29119-3.

Test Execution and Test Environment Setup

Test execution is the controlled process of running predefined test steps within a stable test environment and comparing actual results against expected results. Execution begins only after confirming environment readiness, including validated hardware, software configurations, dependencies, and test data integrity. Unstable environments invalidate results and compromise traceability.

Execution can be performed manually or through automated test cases. In both cases, testers follow documented procedures, capture actual outcomes, and record status within a test management system. Discrepancies between expected and actual results are logged as defects, triggering debugging, correction, and retesting cycles. When defects are resolved, the affected test cases are re-executed to confirm validation.

Inputs to test execution include approved test plans, defined test procedures, validated test items, test basis documentation, and environment readiness reports. Execution supports multiple testing layers, including functional testing, performance testing, and component-level validation.

The test environment must simulate production conditions with controlled variability. Configuration management, dependency tracking, and environment updates must be documented before and after execution cycles. Repeatable execution within stable environments enforces reliability, traceability, and compliance with standards like ISO/IEC/IEEE 29119-1, ISO/IEC/IEEE 29119-2, and ISO/IEC/IEEE 29119-3.

Test Case Management Tools and Test Suites

Test case management tools centralize how teams create test cases, organize test suites, and control test execution across releases. These platforms provide structured repositories where each test case includes a unique identifier, defined test steps, test data, expected results, actual results, and execution history. Centralized storage improves traceability, enforces documentation standards, and reduces duplication across projects.

Test suites group related test cases based on functionality, feature area, regression scope, or release milestone. Well-structured test suites allow testers to execute targeted validation cycles like smoke testing, regression testing, or performance validation without rebuilding scope from scratch. In recurring release cycles, regression suites become reusable assets that protect core functionality and reduce execution risk. Continuous validation strategies are outlined in A Guide to Continuous Testing.

Modern test case management systems integrate with issue trackers, CI/CD pipelines, automation frameworks, and reporting tools. Integration capabilities allow automated test cases to push execution results directly into the platform, consolidating manual testing and automated test data in one environment. Automation scalability and integration strategy are further examined in QA automation benefits and best practices and How to Start with Autotests.

Core platform features include structured test plans, configurable workflows, defect management, reporting dashboards, and role-based access controls. Reporting tools aggregate execution status, pass/fail rates, and defect trends to support release decisions. Comparative evaluations of leading platforms are detailed in the Best Test Management Tools.

An integrated solution like Qase combines test case management, test suites, test plans, test runs, and defect tracking within a single system. Consolidated management enforces consistency, improves maintainability, and provides measurable visibility into software testing performance across the development lifecycle.

Frequently Asked Questions (FAQ)

What makes a good test case?

A good test case is clear, concise, and reproducible. It focuses on a single objective, defines precise test steps and test data, and includes measurable expected results. It should be traceable to a requirement or acceptance criterion and easy to execute without interpretation.

What is the difference between a test case and a test scenario?

A test scenario defines what needs to be tested at a high level. A test case defines how it will be tested, including detailed steps, data, and expected outcomes. Scenarios outline coverage; test cases provide executable validation.

How detailed should a test case be?

A test case should be detailed enough that any tester can execute it consistently without guessing. The level of detail depends on complexity, risk, and team experience. Critical or high-risk features require more explicit steps and validation criteria.

Are test cases still relevant in Agile development?

Yes. Even in Agile workflows, structured test cases ensure traceability, regression coverage, and alignment with acceptance criteria. While exploratory testing is common in Agile, documented test cases remain essential for repeatable validation and scalable regression testing.

Can test cases be reused for automation?

Yes. Well-structured manual test cases can serve as the foundation for automated tests. Clear test steps, defined inputs, and deterministic expected results make automation easier and reduce script maintenance over time.

How many test cases should I write for a feature?

The number depends on feature complexity, risk level, and acceptance criteria. Focus on covering core functionality, edge cases, negative scenarios, and regression impact rather than targeting a specific number. Prioritize based on business and technical risk.

January is usually the month of resolutions. In the software world, that often translates to lofty roadmaps and promises of what’s coming eventually.

But in engineering, we don't live in "eventually." We live in the current sprint. We live in the reality of the release cycle where the only thing that matters is shipped code. So for our first update of 2026, we didn't want to just promise the future. We wanted to ship it.

This month, we focused on the way you interact with your software quality. We’re moving from a world where you have to micromanage every click and selector to one where you simply state your intent.

Here is how we are setting the pace for the rest of the year.

Pillar 1: Intent-Based Engineering

Agentic Mode is Here.

We teased this back in 2025, and today it’s live.

For years, end-to-end testing has been a translation problem. You know what the feature is supposed to do ("Allow a user to buy shoes"), but you spend hours translating that intent into selectors, wait times, and boilerplate code. It’s a context switch that pulls you out of the creative work of building.

With AIDEN Agentic Mode, we are removing the translation layer.

You simply tell AIDEN your goal, "Verify I can purchase shoes on our marketplace," and it figures out the path. It builds the test for you.

AIDEN breaks your natural language instructions into actionable steps, provides visual feedback (screenshots) for every action, and generates the code in your preferred language. You can give simple commands, or you can get specific with compound instructions.

For example, instead of manually scripting input fields, you can just tell AIDEN: "Click on 'Get a Demo' and fill the email address as 'apidemo@qase.io'." AIDEN understands the context, finds the email field, and executes the specific input exactly as requested.

But we didn't stop at the UI.

API Testing and Random Data

Modern applications aren't just front-ends. AIDEN can now execute GET and POST API requests directly within your flow using CURL commands. This allows you to mix UI interactions with backend validations in a single, fluid test.

And to ensure your pre-production environment doesn't get clogged with duplicate data, we’ve added a Random Data Generator. Stop hardcoding static users. AIDEN can now generate unique usernames, emails, and passwords on the fly. Whether you are converting a manual test or building from scratch, just mark a field as "random," and we handle the entropy.

The real automation!

Pillar 2: Reliability in the "Untestable"

Self-Healing and Canvas Support.

There are two things that keep engineering teams up at night: flaky pipelines and black-box UIs. We tackled both this month.

Self-Correcting Tests

The most frustrating part of test generation is when AIDEN attempts to create a test but makes incorrect assumptions about the steps needed. AIDEN is now capable of self-correction during test generation. When a test fails to generate properly, AIDEN analyzes what went wrong, adjusts its approach, and re-attempts the generation with a corrected strategy. This means fewer failed test generations and more reliable test creation on the first try.

While full self-healing—where tests automatically update when your codebase changes—is on our roadmap, this self-correcting capability during generation is a significant step toward that goal. It keeps your test creation process smooth and reduces the manual intervention needed when working with dynamic applications.

Conquering the Canvas

Historically, tools like Miro, Figma, or any heavy <canvas> based applications were automation dead zones. Standard frameworks struggle to "see" inside those elements. AIDEN now supports Canvas-based UIs. It interprets visual elements within the canvas just like standard DOM elements, opening up a massive new frontier for test coverage for teams building complex, visual tools.

Pillar 3: Visualizing Your Velocity

Data is only useful if your brain can process it.

We noticed that many of you were running complex QQL (Qase Query Language) queries, then staring at rows of tabular data trying to spot trends. That creates cognitive friction when you're trying to make quick decisions about release health.

We’ve introduced QQL Bar Charts.

You can now instantly toggle your query results from a table view to a Bar Chart visualization. Want to see test case distribution by project? Or flaky tests grouped by status?

• SELECT (project, COUNT(*)) GROUP BY project
• SELECT (status, COUNT(*)) isFlaky = true GROUP BY status

Save these views to your dashboard. Spot the bottleneck in seconds.

Pillar 4: Repository Hygiene at Scale

Clean Code, Clean Tests.

As your product grows, "technical debt" isn't just in your codebase. It lives in your test repository too. Thousands of duplicate steps scattered across projects slow everyone down.

We’ve overhauled how you manage Shared Steps to make spring cleaning easier:

• Bulk Conversion: Select multiple local steps and promote them to Global Shared Steps in one click. Note that this puts them at the workspace level, making them accessible to your whole organization.
• Bulk Delete with Safety: Need to deprecate old logic? You can now bulk delete shared steps with options to either cascade the delete (be careful!) or convert them back to local steps within the test cases so you don't break history.

Enhanced Test Framework Support

Your test automation ecosystem is only as strong as the integrations that connect it. This month, we expanded our reporter coverage to meet you wherever you are in your testing stack:

We've added NUnit and Junit4 reporters for Android tests, giving mobile teams the same seamless reporting experience as web teams. The Behave reporter now supports attachments for steps, so your BDD scenarios can include screenshots and logs exactly where they matter. CucumberJS users can now leverage parameters and suite annotations for better test organization. And for JavaScript testing frameworks—Mocha, Vitest, and Jest—we've implemented support for expected results and input data at the step level, making test reports more informative and actionable.

Jira Data Center 10.x Support

Seamless integration isn't a luxury. It is a requirement. We are officially supporting Jira Data Center 10.x. Whether you are on the cutting edge of Jira versions or maintaining a stable enterprise environment, your data flows seamlessly between Qase and Jira.

Our New Feedback Tool

We moved our feedback and roadmap to a new tool at roadmap.qase.io. You can submit feature requests, vote on what you want to see next, and track what we're working on like you used to. We will continue to make sure that your requests are taken good care of.

If you used our old feedback platform, you can migrate your data. Sign in with your email to claim your posts.

We removed the barriers in January. Whether that is the barrier between natural language and code, or the barrier of "untestable" canvas elements.

We’re excited to see what you build when you don't have to sweat the small stuff.

Happy testing!

There's a specific kind of exhaustion that comes from knowing exactly what needs to be done, but the tools won't let you do it efficiently. You've got the vision—automated test coverage across three projects, a clean migration path, tests that actually reflect production—but between you and that vision sits a thousand small frictions.

Q4 was about eliminating them by paying attention to the places where your workflow stutters. The moments where you think "there has to be a better way" and then resign yourself to the manual approach because, well, that's just how it works.

Except now it doesn't have to.

The Automation Backlog Problem (And How We Actually Are Solving It)

Eliminating "Automation Paralysis" 

Let's talk about the elephant in the repository: your mountain of manual test cases that should be automated but aren't. Not because you don't value automation(you do), but because the conversion process is soul-crushing. One test at a time. Selectors. Wait times. Edge cases.

So the backlog grows. And every sprint planning meeting, someone mentions it. And everyone nods. And nothing happens.

AI-Powered Bulk Conversion changes the entire dynamic. Select a batch of manual tests and click 'Automate.' AIDEN runs a pre-flight check, flags the cases that need human attention (vague preconditions, overly complex flows), and converts while you grab coffee.

A UI Built for How You Work 

We rebuilt the entire AIDEN interface with a full-screen modal and unified flow whether you're converting or generating from scratch. Environment setup is baked into the process.

The JSON editor? Gone. AIDEN's stability reached the point where the safety net became unnecessary clutter.

You can now add steps on the fly, edit as you go, and export code from a dedicated tab in the right panel.

API Testing Because Your App Isn't Just a UI 

Here's something we heard a lot: "What about my API layer?" Fair question. AIDEN now generates autotests that validate APIs. 

Pre-warmed Environments for Speed 

We’ve re-engineered the backend of AIDEN so browser environments are ready the second you hit "Automate." No loading spinners.

Real-World Automation 

The 70% Truth: Most vendors will tell you their AI achieves 95% automation rates in a perfectly curated lab. We’d rather tell you the truth. Based on real-world usage in messy, edge-case-riddled production environments, AIDEN is now automating over 70% of manual boilerplate.

That might not look as "sexy" on a marketing slide, but think about the math: if your team spends 40 hours a week on manual regression, you just got 24 hours back. Every week. To make sure you actually use that time, we’ve doubled AIDEN credits across all plans and implemented hard expenditure caps, so you can scale without "Cloud Bill Heart Attacks."

Qase is Now "AI-Native" with MCP

We know that for many of you, AI is becoming the "connective tissue" of your entire engineering stack.

This quarter, we launched the Qase MCP Server. The Model Context Protocol (MCP) is a new open standard that allows AI models to safely interact with your data. By plugging Qase into your local AI assistant (like Claude Desktop), you can now ask high-level questions about your quality state without leaving your workspace.

AIDEN remains your expert for building and running tests, but the MCP server makes Qase a Knowledge Base for your entire engineering workflow.

Architecture That Scales With You

Shared Steps Go Global: Quick question: How many times have you copied the same "user login" shared step across different projects? 

If the answer is "more than once," you've been working too hard. 

Global Shared Steps live at the workspace level now. Build your authentication flows, payment scenarios, and teardown sequences once. Use them everywhere. Update in one place, deploy to infinity.

• Linked Shared Steps: Refactoring used to come with anxiety. Which tests use this? What will break if I change it? Now, when viewing a Shared Step, you instantly see all associated test cases. One click shows you the complete impact zone.

The Most Requested Features

• The Failed-Test Clone: Our user feedback board had one feature request that kept climbing: "Let me rerun just the failed and skipped tests." Close a test run with failed/skipped tests only, clone it with just those cases, and retest without wading through hundreds of passing tests.
• Clone Wars Sequel: Ever noticed how you avoid starting new test cycles because recreating the structure feels harder than it actually is? Test Plan cloning removes this worry entirely. Last sprint's plan becomes this sprint's foundation in three clicks. Keep the assignees or don't. Rename it inline. Done.

The activation energy to start drops to near-zero, so you start. Momentum builds from there.

• Webhooks: You can now push test results via Webhooks to third-party tools. This unblocked several enterprise customers who needed test data consolidated into platforms like Aha!.

It's the kind of integration that sounds mundane until you realize it's the glue holding your entire quality ops together.

• Data Retention: The Auditor Insurance Policy: Our default data retention works for most teams. But regulated industries, mature enterprises, and anyone who's ever heard the phrase "we need historical data for the compliance review"—they need more. The Data Retention Add-on extends test result history to 5 or 10 years. Transparent pricing, prorated per user. 

The Invisible Infrastructure Improvements

Some updates don't make headlines but change everything about how the platform feels:

• Async downloads means exporting 10,000 test cases doesn't turn into a staring contest with a progress bar.
• Full suite paths in QQL solved the "which Regression Tests suite are we talking about?" problem that plagued every large repository.
• Include children suites means querying a parent now returns everything nested beneath it. No more manual enumeration of every child suite.
• We also patched 20+ security vulnerabilities in our JS reporters, added Vitest support, updated Mocha to version 11, improved Cypress/Cucumber integration, and shipped quality-of-life fixes for Robot Framework, Pytest, and Playwright.

None of these will change your life individually. Collectively? They're the difference between a tool that fights you and a tool that flows.

What This Actually Means

Software releases usually follow a pattern: big promises, modest delivery, PR spin to bridge the gap.

We're not interested in that game.

Everything in this post is live. In production. Available now. The bulk conversion that eliminates your backlog anxiety? Shipped. The AIDEN UI that gives you room to work? Deployed. The global shared steps that scale across your workspace? Running right now.

QA is evolving from "the people who find bugs" to "the people who design quality systems." Q4 was about giving you the steering wheel for those systems.

We're not asking you to imagine what testing could be like. We're showing you what it is.

Happy testing.

Don't miss out on future updates and valuable content! Connect with us on LinkedIn to stay informed about all things Qase.

That's not a typo. That's the number of test results you sent to Qase in 2025.

If you shipped code this year, you know the grind. But as we look back at the Qase ecosystem, the sheer scale of what you all built (and protected) is staggering.

This wasn't just a big year; it was a half-billion-check-marks kind of year:

🚀 427.7 M Test Results Sent: Nearly half a billion potential fires extinguished before they ever hit production.

📈 ~15M Results Per Week: The current heartbeat of our ecosystem. Every seven days, 15 million moments of truth flow through Qase.

🤖 415.6M Auto Tests: Automation is the engine. Our cloud handled 415M+ checks, keeping your CI/CD pipelines screaming fast while you solved the hard problems.

🧠 12.1 Manual Results: For the edge cases, the UX nuances, and the "does this actually feel right?" moments. Nearly 10 million human-led tests prove that great software still needs great people.

🏃 6.6M Test Runs: Over 6 million distinct "go-to-market" moments.

The Power of the Ecosystem: Software doesn’t live in a vacuum. This year, you connected Qase to 5,354 different apps.

But here's what the numbers don't capture: The late-night debugging sessions. The "it works on my machine" moments. The relief when tests finally turn green. The quiet satisfaction of catching a bug before your users did.

To our ecosystem: You’re the reason we push updates at midnight. The reason we obsess over milliseconds. The reason we built AIDEN.

Behind every one of these results is a team that cares about their users. You didn't just ship features; you shipped quality. You didn't just meet deadlines; you met standards.

Happy New Year to the builders, the breakers, and the bug-hunters. 🥂

Thank you for trusting Qase to be the home for your quality data. Here's to shipping better software, together. 🚀

As covered in the articles on Network Profiler and Sleep Profiler, Qase provides powerful tools for monitoring and analyzing test scenarios. This article explains Database Profiler—a tool that automatically tracks and logs database operations.

The Problem with Databases in Tests

In integration and E2E tests, the database is critical. Without profiling it’s hard to see:

• Which queries ran and how long they took
• Why tests are slow: which specific query is the bottleneck
• Where to debug: the exact query text and error message
• How much data changed: number of affected rows

What is Database Profiler

Database Profiler intercepts SQL queries and sends each call to Qase as a separate step. You see the query text, execution time, affected rows, and connection info.

How Database Profiler Works

It relies on monkey patching and proxy classes: the profiler wraps query execution methods, measures their time, and sends the data to Qase TestOps as test steps.

Usage Example (PostgreSQL + pytest)

import psycopg2
from qase.pytest import qase


@qase.title("User Database Operations Test")
def test_user_operations():
    conn = psycopg2.connect(
        host="localhost",
        port=5432,
        database="testdb",
        user="testuser",
        password="testpass",
    )
    cursor = conn.cursor()

    cursor.execute("SELECT * FROM users WHERE id = %s", (1,))
    user = cursor.fetchone()

    cursor.execute(
        "UPDATE users SET email = %s WHERE id = %s",
        ("newemail@example.com", 1),
    )
    conn.commit()

    cursor.execute("SELECT COUNT(*) FROM users")
    count = cursor.fetchone()[0]

    assert count > 0
    conn.close()

Configuration

Via command line

# Enable only Database Profiler
pytest --qase-profilers=db

# Combine with other profilers
pytest --qase-profilers=db,network,sleep

Via qase.config.json

{
    "mode": "testops",
    "profilers": ["db"],
    "testops": {
        "api": {
            "host": "qase.io"
        }
    }
}

Conclusion

Database Profiler gives full visibility into SQL queries in tests: query text, timing, affected rows, and connection context. It accelerates bottleneck analysis, simplifies debugging, and helps keep tests stable and fast.

Inside Brandon DePesa’s journey from startup spreadsheets to unified, data-driven quality at SUSE 

SUSE is one of the most recognizable names in computing, from open source to enterprise. In its cloud-native business, Rancher Prime is the centerpiece: a platform for Kubernetes cluster provisioning and management.

For many customers, their first experience with SUSE begins not with a purchase order, but with open-source experimentation. 

“We’ve found that a lot of our customers start as open-source users,” explains Brandon, Director of Quality Engineering for SUSE’s cloud-native group. “They might be playing with Rancher at home or using it for a POC. We need something that works and works well, so they want to keep using it and recommending it.”

For Mike Latimer, Senior Director of Engineering overseeing QA, security, project management, and infrastructure, it boils down to trust:

“We as a business rely on trust between us and our customers. At the core of that trust is a reliable, functional product.”

That trust has to hold across open-source users and paying enterprise customers with complex hybrid environments. Quality is the foundation; QA is not “just another function,” it’s existential.

From three QA engineers to eighty: when startup systems needed to scale

SUSE’s cloud-native business went from scrappy, scattered QA artifacts to a single, data-driven quality hub that leadership and stakeholders across the company can see and trust.

When Rancher was a startup ten years ago, the QA team was tiny.

“When I started, QA was three people including myself,” Brandon recalls.

At that scale, using lightweight tools was fine. Test cases were tracked in spreadsheets, plans and notes were written in Confluence and internal wikis, and some teams were using GitHub issues or source control as makeshift test repositories.

“So even five years ago,” says Brandon, “with ten to fifteen people split across a few teams, it wasn’t ideal, but it wasn’t a huge problem.”

Then Rancher matured into an enterprise product, was acquired by SUSE, and everything changed. Today, Brandon’s QA organization is about eighty people, plus “tons of developers, project managers, and everyone else who is interested in the QA world.” That scale exposed cracks in the old way of working.

Different teams had different tools, different naming conventions, and different places where test plans and cases lived. If you weren’t on a given team, you might have no idea where their critical test artifacts were stored.

“It became very difficult to even understand where test artifacts were located,” Brandon says. “Test plans, test cases, automated tests, documentation—you could only reliably find them if you were on that team.”

For a business unit selling a suite of cloud-native products that have to work together, that was risky. You can’t easily validate cross-product scenarios if you can’t even see what’s being tested where.

Reporting that couldn’t keep up with leadership questions

The fragmentation problem got even worse when leadership wanted to see the big picture.

Questions like: “What’s our overall test coverage?”, “How many test cases do we have across Rancher Prime?”, “What percentage of those are automated?” were nearly impossible to answer at an aggregate level.

“We had no requirements traceability, no requirement coverage,” Brandon says. “At best, we could manually count test cases for each area. Things like automation velocity or manual vs. automated ratios across products, were just impossible to provide.”

To produce an answer, teams had to collect test assets from multiple spreadsheets, wikis, and repos, normalize them into a single format, and manually crunch numbers in yet another spreadsheet.

It was known to be so painful that leadership wouldn’t even ask for deeper QA metrics. Everyone knew the data either didn’t exist or wasn’t worth the effort to assemble.

The search: involving the people who’d use the tool

Brandon didn’t want to pick a platform from a slide deck. He wanted real engineers using real tools.

“You need a level of trust with your team,” he says. “If you just say, ‘Here’s the new tool, use it,’ even the best tool in the world will get resistance.”

So he assembled a group of engineers from multiple QA teams and ran a structured evaluation. They shortlisted four to five tools; a mix of open source and commercial products, including some of the biggest names in the space.

Each team got trial licenses. Engineers imported real test cases, ran test cycles, created defects, and tried to hook up automation. They documented what worked, what didn’t, and what felt painful day-to-day. That feedback loop drove the final recommendation.

“I’ve been in software and quality for almost twenty-five years,” Brandon says. “I still learned new things from the team through this process; especially what was painful for them, and what they actually wanted from a tool.”

Why Qase: the combination that finally checked all the boxes

In the end, Qase stood out not for one feature, but for a combination that matched SUSE’s reality.

UI that felt natural to engineers

“The UI was incredibly easy to use,” Brandon recalls. “Even without formal training or demos, very few people had issues. It’s just intuitive and it works.”

For an eighty-person QA org, this mattered. Every hour spent teaching people how to click around a tool is an hour not spent improving tests.

Flexible imports that made migration simple

One team had 2,500 existing test cases. They assumed migrating that volume would take at least several days. With Qase, they imported everything in a matter of hours.

“A lot of tools are very prescriptive,” Brandon says. “You have to fit their exact format or enter everything manually. Qase’s flexibility around how you import made it super simple to get going.”

That flexibility meant teams could move all their manual test assets into Qase first without derailing sprints.

Automation-friendly from day one

SUSE’s QA teams run a wide range of automated tests, from UI automation to backend and system tests, with different frameworks and reporting formats across teams.

Some teams plugged into Qase’s built-in integrations (like JUnit). Others had custom test result outputs, which they ran through a lightweight script.

Qase also helped close the loop between manual and automated worlds. If a result came in for a test that didn’t exist yet, Qase auto-created the test case. Teams could later link these auto-created cases back to existing manual tests if needed.

Once a few teams had working scripts and patterns, they shared them internally, and rollout to the rest of the org became very fast.

Read-only licenses that enabled real visibility

Cost was a factor for a large enterprise group. The QA team uses around 75 full Qase licenses. But there are as many as 200 other stakeholders—product managers, architects, security engineers, support, and leadership—who need to view test data without authoring or executing tests.

“If we had to pay [full price] for another 200 licenses just so people could view test data, that would have been a no-go from the beginning,” Brandon says.

Qase’s read-only licenses alleviated that burden. Now, anyone in the organization who needs to see QA data can get visibility for little additional budget.

“That visibility for people not working directly with tests was a huge point in Qase’s favor,” Brandon notes.

The impact: from invisible QA to a visible, strategic advantage

Before Qase, leadership’s view of QA was limited and sporadic. Now, they have aggregate dashboards showing QA activity and coverage across all cloud-native products, per-product dashboards for more detailed views, and real-time updates as teams add tests and execute runs.

Early on, there was a strong focus on the number of test cases added over time, and how many of those were automated. Those metrics were simple but powerful, showing continuous improvement and a growing investment in automation.

Over time, Brandon realized an even more important metric: test execution volume.

“We got to a point where the numbers for test cases and automation added were fairly consistent,” he says. “What that didn’t show was the overall level of testing actually happening.”

Now, SUSE’s dashboards clearly show how many tests are run, against how many versions, and how often those runs happen.

A simple example: suppose a team has ten test cases, five of them automated. On paper, that’s unremarkable. But when Qase reveals those tests are being run across three to five different versions, multiple times a week, you start seeing hundreds of executions for those same tests. That tells a much richer story about the team’s real-world effort.

“QA is difficult to measure,” Mike says. “If the product works, it just works. But if you can show you’re improving test coverage and doing a better job today than yesterday, that shows your focus is on quality and you’re not getting left behind.”

QA onboarding time cut from weeks to hours

Before Qase, a QA engineer moving to a new team or helping another product often needed a week or two just to understand where test cases lived and figure out what had been covered and what hadn’t.

Now, Brandon estimates that the same understanding can usually be achieved in a few hours.

With Qase, an engineer can browse or search test plans and suites in one place, filter by component, feature, or configuration, see which versions and environments are in scope, and understand the team’s existing coverage and known gaps.

This reduction in onboarding friction lets the QA org be far more fluid in how it deploys people across teams and initiatives.

Cross-product coverage: testing the suite, not just the pieces

SUSE doesn’t just sell a single product; it sells a suite that must work together: Rancher management, virtualization, observability, security, and more.

Qase has become central to understanding end-to-end coverage across these components.

“We’re able to look at test coverage for, say, Rancher management and our virtualization efforts,” Brandon explains. “We can see that one team has tested X, Y, and Z, but we’re missing A, B, and C on the virtualization side, or that we’ve never tested feature A with feature B enabled at the same time.”

That ability to quickly spot cross-product gaps is crucial for enterprise customers who run SUSE in heterogeneous, high-stakes environments.

QA made visible and interesting to the wider organization

Traditionally, testing is a black box. If things break, customers notice. If they don’t, QA has quietly done its job. Qase changed that dynamic inside SUSE.

“Testing is usually very opaque to anyone not in QA or development,” Brandon says. “The dashboards and metrics open a window into what teams are actually doing.”

Now architects browse Qase to understand test history and outcomes. Product managers dig into runs to reassure customers about specific OS or platform coverage. Project managers use Qase data to plan and track release readiness.

Brandon shares a story about an architect who saw a dashboard full of failing runs and got worried that releases were going out in that state. A quick walkthrough showed that he was looking at cumulative testing across all builds, including early, intentionally unstable ones. Filtering to only final release-candidate runs showed what actually went out to customers: fully passing test suites.

“You could see his confidence increase as we walked through it,” Brandon says. “He started asking deeper questions about how we test and why. Those are great conversations to have.”

And this broader engagement is only possible because read-only access is affordable. There’s far less financial friction to “just log in and poke around.”

Productivity: more time spent testing, less fighting tools

SUSE doesn’t try to measure productivity through micromanagement. No one is counting how many minutes a tester spends writing steps.

But the qualitative signals are strong. Engineers say it’s easier to write test cases and plans in Qase. Creating and assigning runs is simpler and faster than managing spreadsheets. Reporting both manual and automated results is far less tedious.

And quantitative indicators back that up:

“Our test cases and automation have been continuously increasing,” Brandon says. “That can’t happen if we’re getting bogged down by process and manual tools.”

By cutting away the friction of scattered documents and manual compilation, Qase lets Brandon’s org spend more time on what actually matters: finding issues early and ensuring releases are solid.

Conclusion: QA that meets the premium standard of SUSE’s cloud-native offering

SUSE’s cloud-native business operates in one of the most complex, demanding corners of enterprise infrastructure. Kubernetes, hybrid and edge deployments, an ever-growing matrix of providers and platforms—none of that leaves room for imprecise QA.

Brandon’s journey—from being one of three QA engineers at a startup, to leading an 80-person quality org using Qase as its backbone—is a blueprint for QA leaders facing the same transition.

From scattered artifacts to a single source of truth; from ad hoc reporting to real-time executive dashboards; from opaque QA work to a visible, respected discipline.

Qase didn’t create SUSE’s quality culture, but it gave Brandon and his teams the platform they needed to express that culture at enterprise scale, and to show executives, customers, and colleagues exactly what quality looks like in a cloud-native world.