When I joined Qase, the Qase Studies newsletter was one of the first projects I planned to revamp and relaunch. I sent the first revamped issue in October 2023, wrote about my re-launch strategy the following month, and I’ve been tracking various aspects of the Qase Studies newsletter ever since.
I’m excited to dig into both the quantitative and qualitative data. I’ll share key insights, go through our top-performing content, and wrap up with thoughts on what aspects of my strategy worked well, along with what I’d like to improve on in the next year.
Key takeaways
- Average open rate increased by 3.4%
- Click-through rate (CTR) increased by 2%
- We gained 8,115 subscribers in total
- Content related to QA methodologies, testing strategies, career advice, and QA-related humor resonate strongly with our readers
Top stories From the Qase Files
The From the Qase Files section, which features three pieces of content produced by the Qase team in each issue, was the section with the highest engagement. Stories by Qase received approximately 37% of all clicks over the past year.
Top 10 articles From the Qase Files
- Qase 2023 releases, 2024 roadmap, and an intro to our VP of product
- The QA role is changing, but QA will never die
- Embracing shift-left testing in software development
- A few of our favorite CrowdStrike outage memes
- Total Quality Management in software development and QA
- A complete guide to smoke testing
- How to design a software release process: advice and examples from a Head of QA & Release
- How tools can improve (or harm) relations between QAs and developers
- Building a continuous learning mindset as a QA professional
- The test pyramid: A complete guide
If you liked those articles, try these pieces from the Qase:
- How the sunk cost fallacy is sinking your tool stack by Nick Moore
- Getting intentional with heuristics in software testing by Lena Pejgan Nyström
- Understanding quality circles and how to adapt them for remote teams by Vitaly Sharovatov
Most intriguing Questions and Conversations
Each issue includes 4-5 questions or conversations from social media and forums. The questions and conversations section had the second-highest engagement rate, receiving approximately 24% of all clicks over the past year.
Top 10 Questions and Conversations
- Katja Obring said test coverage is not indicative of good testing, good test coverage is a side effect of good testing.
- The Qase team asked the community’s opinion on who was to blame for a quality assurance test that turned into a $340 million lawsuit.
- A Reddit user collected suggestions for Testing BS Bingo.
- A Quora user asked for stories about the best bugs people have found while testing.
- Richard Bradshaw said it’s OK (and inevitable) to have production issues, but it’s not OK to unfairly assign blame.
- Jenna Charlton used a real-life example to highlight why software teams must make DEI a central tenant.
- Santiago Valdarrama said there’s an epidemic of people building software that doesn’t work.
- A Reddit user asked which details should be included in a bug report.
- Melissa Fisher said a lot of testing misses the point of exploring unknowns.
- A Quora user asked for resources for learning about testing and QA
If you liked those conversations, check out some related conversations and resources:
- QA myth busting: quality is the testers’ responsibility
- Cassandra H. Leung asks, how do you document your impact at work?
- From capital to village: A QA engineer's bug investigation
- Software testing training: which option is best for you?
- Lee Marshall asks, what’s your favorite thing to test and why?
- Unpacking the CrowdStrike outage from a QA perspective
Top pieces and authors from What We’re Reading
The What We’re Reading section features 4-5 articles about QA, testing, and development in each issue. This section received approximately 23% of all clicks in the past year.
I couldn’t curate this section without the talented authors I follow, so I’m splitting this into the top 5 articles and top 5 most-featured authors so I can shout out even more creators.
Top 5 articles from What We’re Reading
- QA Horror Stories [Chapter 1]: Nightmare on Regression Street by Testify
- Testers — Don’t wait to be asked for info by Callum Akehurst-Ryan
- 9 Git Branching Strategies Every Developer Should Know by Saverio Mazza
- Laying Off Testers? Think Twice Before You Act! by Lidia Barkanova
- The Ultimate QA Engineer Roadmap: Your Guide to Success by Monish Correia
Top 5 featured authors
Congratulations and thank you to our most-featured authors! I highly recommend following these talented people.
- Kristin Jackvony
- Lena Pejgan Nyström
- Lisa Crispin and Janet Gregory (Paired together because they shared bylines)
- Mike Harris
- Maaret Pyhäjärvi
Author demographics
I strive to feature a diverse range of authors. In an industry that is not always known for its diversity, I feel this is very important. To curate content, I rely heavily on the network that my teammates and I create. The people we follow on social media, the forums we participate in, and the people we meet at events and meetups around the world are our main inspirations and sources of content.
47% of our featured authors are women/non-binary
Qase has talked about the importance of DEI in tech and product design many times. If you frequent QA events and conferences, you’re probably used to seeing a lot more men than women on the stages. But when you are intentional about seeking out creators who are sharing interesting and helpful content in this space, it’s clear that women deserve better representation at those conferences.
Featured authors reside in 24 different countries
Qase only shares content written in English, so it’s no surprise that many of the authors featured are based in the US and UK.
However, more than 50% of the featured writers reside in other countries, including Australia, Austria, Brazil, Canada, Croatia, Dominican Republic, Estonia, Finland, France, Germany, India, Italy, Malaysia, Netherlands, Poland, Portugal, Russia, Serbia, Spain, Sweden, Switzerland, and Thailand.
Thoughts on newsletter strategy
Before the relaunch, the Qase Studies newsletter was essentially a recap of recent Qase blogs and a couple of links sent only to existing Qase customers. The first thing I did was open the newsletter to anyone interested and start tracking key metrics, including open rate, click-through rate, top-clicked links in each issue, etc. After all, you can’t monitor (or optimize) what you don’t measure.
Wins
We increased subscribers by 202% even after intentionally removing more than 2,400 contacts.
About halfway through this year-long experiment, I noticed our open rate was on the decline so I decided to run a re-engagement campaign. With the help of my teammate Stephanie Armand, I segmented the unengaged contacts (anyone who had not interacted with the newsletter in the last 90 days) and sent them a series of three emails to determine if these unengaged subscribers were truly uninterested or if there was another explanation.
Email 1: Spam check - To make sure the newsletter wasn’t landing in the spam folder, we sent a short email to explain how to whitelist our newsletter.
Email 2: Did you know we’ve improved? - Much like an “under new management” sign, this email aimed to explain how we’ve enhanced the newsletter and included a link to a blog post with top content from the past 6 months to give the readers an idea of what to expect.
Email 3: Goodbye - At this point, everyone on the list was notified that they would be automatically removed unless they clicked to re-subscribe.
While 2,400 contacts were removed from the newsletter list as a result, we also saw a 1% boost in open rate and .37% increase in click-through rate.
Challenges
Gaining new subscribers is difficult without a dedicated newsletter landing page
My initial strategy document was full of optimistic ideas, including the plan to build a dedicated newsletter landing page where I could not only promote the newsletter in general but also publish previous issues so they would be available on demand.
I failed to consider the stage of the marketing team at the time. I was the very first content hire and quickly learned that our site was not set up to provide easy access for the marketing team. Once I realized that there were higher-priority landing pages and website improvements in the queue, I had to compromise. Instead of a dedicated landing page, I was able to get a subscription form in the footer of our website. This was a major improvement, but still left me without a direct link to share to encourage new subscribers. I addressed this by strategically adding a subscription form to key blog posts.
Engagement can be impacted by things outside of our control
Within a few months, the newsletter open and click-through rates were steadily rising. But as we entered the summer months, both metrics started to decline.
After some investigation and experimentation, I realized that this dip was most likely caused by our European subscribers taking their summer vacations and auto-deleting every email that hit their inbox during that time.
By late August, our open and click-through rates had recovered.
Improvements for the coming year
As I look back on the past year, I’m very proud of the growth and improvements, but there is still plenty of room to grow, so I’m setting a few goals for myself.
- Build a newsletter landing page: I’m not giving up on the dream! I can’t share too many details yet, but I will say that the marketing team and our resources have grown, and a newsletter landing page will be possible in 2025.
- Automate newsletter cleanup and re-engagement campaigns: I’m currently working with my teammate Remi Mortier on streamlining list management and re-engagement campaigns so we can maintain high engagement rates.
- More experiments: I admit that I was a little conservative with my experimentation. I really wanted this to be a year of growth, and experiments can be risky when you’re trying to keep your metrics moving up and to the right. But with a year of data and lessons, I’m ready to try some new things.
On that note, I’m inviting you all to be a part of this journey! If you’d like to help shape the Qase Studies newsletter, please fill out this survey to suggest resources and people in the QA space that I should follow and share what you’d like to see more of in Qase Studies.
And if you’ve made it this far and still aren’t subscribed, let’s fix that!
