🔶 The A/B test that made me question everything

Read time: 4 minutes

Hey! Sam here 👋 I’ve spent the last 10+ years leading marketing and growth for early-stage tech startups, helping them scale from 0 to 8 figures. Each week, I share actionable insights to help founders and marketing leaders like you drive sustainable growth.

Remember earlier this week when I confessed about my A/B testing failures?

Well, grab some tea, because today I'm sharing the full story of how I turned it around - and how you can too.

The wake-up call

It was 11 PM, and I was staring at my computer screen. Tired and frustrated.

We'd just finished our 24th A/B test of the quarter. Lots of green arrows. Lots of "winning" variations.

But our revenue? Flat as a tortilla.

Some movement, but nothing compelling given the number of tests we ran and their “success”.

Why?

That's when it hit me: We weren't really testing for impact.

Lots of movement, but little progress.

Here's how we turned it around, and how two companies I later worked with used these lessons to drive thousands in revenue.

The whiteboard revelation

I didn’t sleep well that night, going through what may be missing.

So, the next day I started mapping out our customer's journey on a whiteboard.

That's when I noticed something.

All our tests were focused on the bottom of the funnel.

But most of our visitors weren't ready to buy.

We were optimizing for hungry guests at a restaurant that was mostly full of people just browsing the menu.

So we changed our approach. Instead of asking, "How do we get more clicks?" we started asking, "What does the customer need at this stage of their journey?"

The e-commerce turnaround

This shift in thinking led to a big win. I was working with an e-commerce client drowning in cart abandonment.

They wanted to test discount offers.

Instead, we mapped their customer journey and started doing customer research.

We realized something: first-time visitors were abandoning because they didn't trust the site, while returning visitors were price-shopping.

We created two different sets of tests:

1. For new visitors: We tested adding trust signals and testimonials.

2. For returning visitors: We tested price-match guarantees and bundle deals.

The result? Cart abandonment dropped from 75% to 60%, driving extra thousands in revenue in just four months.

The SaaS onboarding breakthrough

Next, we tackled a SaaS company struggling with free trial conversions.

Their instinct was to test different call-to-action buttons.

But when we dug into the customer journey, we realized the problem wasn't the CTA - it was that users weren't experiencing the product's value during the trial.

We created a personalized onboarding sequence based on the user's role and goals, identified during sign-up. We tested different approaches:

- For marketers, we highlighted analytics features.

- For salespeople, we showcased the CRM integration.

Trial conversions jumped from 20% to 35%, translating to thousands in increase in ARR.

Here's the thing: A/B testing isn't about changing button colors. It's about understanding your customers.

When you align your tests with their journey, magic happens. And by magic, I mean money. Lots of it.

What's next?

Want to know how we turned this into a repeatable process? I'll share more next week.

🔍 Sam Pick’s - resources worth your time

• The Tenets of A/B Testing from Duolingo’s Master Growth Hacker by First Round

• The paradox of choice by The Decision Lab

• Example of a badly designed experiment can lead to poor results by Alan Klement

Here are more ways to connect:

• Follow me on Twitter/X and LinkedIn for content during the week.

• Work with me 1:1 to help you build a defensible growth engine to scale your business to $10MM ARR. Let’s chat.

  Ideal if you're a founder who reached $1M and now wants to scale.

Until next time 👋

Sam

P.S. Want personal advice? I'm doing free 30-minute calls with the first 5 people who respond.