How to A/B Test a Landing Page: What to Test First (and What to Ignore)

Most people A/B test in the wrong order — spending weeks tweaking button colors while the headline is losing them 80% of visitors. This guide gives you the exact testing priority order based on impact, the minimum traffic you actually need before testing, which free tools work, and a 30-day roadmap to find your first real conversion win.

1. What A/B Testing Actually Is (and What It Isn't)

An A/B test shows two versions of a page to two randomly split groups of visitors at the same time. Version A sees the current page. Version B sees the variant with one change. After enough visitors, you compare conversion rates and declare a winner.

That's it. The math is simple. What's hard is discipline:

• Only change one element per test. If you change the headline, CTA, and hero image at once and conversion goes up, you don't know which change caused it.
• Run both versions simultaneously. Sequential testing (A this week, B next week) is corrupted by traffic quality differences, seasonality, and external events.
• Reach statistical significance before calling a winner. Ending the test after 200 visits because "B looks like it's winning" is how you get fooled by noise.

2. The Testing Priority Order: High Impact First

Not all elements are equal. Test these in order — the top items have the highest potential lift and the clearest hypotheses:

3. How Much Traffic You Actually Need

This is where most people get confused. The minimum traffic depends on your current conversion rate and how big a lift you're trying to detect:

What If You Have Low Traffic?

If your page gets fewer than 100 visitors a day, traditional A/B testing is unreliable. Instead:

• Qualitative testing first — use Hotjar or Microsoft Clarity (free) to watch session recordings and find where visitors drop off or hesitate
• 5-second test — show your hero section to 5–10 people for exactly 5 seconds, then ask what the page does and who it's for. Confusion reveals headline problems instantly
• Implement the known winners — many copy changes (benefit headlines, specific CTA copy, risk removers) are so well-tested across thousands of sites that implementing them without A/B testing is still a net positive
• Test one big change at a time — run for a full month to accumulate enough data, and accept a wider confidence interval

4. Statistical Significance — Plain English

Statistical significance answers the question: "How likely is it that version B's results are real — not just random luck?"

In practice: You run a test, you get a result, your testing tool says "95% confidence." That means there's a 5% chance the result is random. For a button copy change that takes 10 minutes to implement, 90% is enough. For a full page redesign with 40 hours of work behind it, wait for 95%.

5. The Best A/B Testing Tools (Including Free)

For most freelancers and small teams: start with Microsoft Clarity (free) to identify where visitors are dropping off, then implement changes manually and track the result in GA4. You don't need a dedicated A/B testing tool to see your conversion rate change — you need consistent measurement.

6. The 30-Day A/B Testing Roadmap

7. What NOT to Test (Common Wasted Effort)

These are the tests that busy teams run for weeks and get nothing useful from:

• Button color — Unless your current button has no contrast with the background, button color tests rarely produce meaningful lifts. The copy on the button moves conversion rates; the hex code doesn't.
• Font choices — Between two readable, web-safe font pairs, conversion differences are statistically negligible in virtually all tests.
• Stock photo A vs stock photo B — Both are stock photos. Neither is your product. Test "stock photo vs product screenshot" — that's a meaningful variable.
• Number of bullet points (4 vs 5) — Micro-copy structure tests rarely reach significance without enormous traffic volumes. Test what those bullets say, not how many there are.
• Footer content — Footer A/B tests require massive traffic volumes to reach significance because so few visitors read it. The effort is rarely justified.
• Testing on mobile-only traffic — If you're separating desktop and mobile audiences to test separately, make sure each segment has enough traffic independently. Most small sites don't.

8. How to Read Your Results Without Getting Fooled

Novelty Effect

When you change an element on a page your existing audience has seen before, they engage with the new version simply because it's new. This creates an artificial short-term lift that fades. For returning visitor traffic, always run tests for at least 2 weeks to let novelty wear off.

Sample Ratio Mismatch

If your test was supposed to split 50/50 but your tool shows 48/52 or worse — investigate before reading the results. An unequal split can corrupt your data if the discrepancy is caused by a bug, caching issue, or bot traffic.

Segment Pollution

If your page gets traffic from wildly different sources (paid ads, organic, email, direct), a meaningful change for one audience might be noise for another. If your email audience already knows your brand and converts at 15% while your paid traffic converts at 1.5% — they shouldn't be in the same test.

External Events

A competitor going viral, a mention in a newsletter, a Product Hunt launch, a Twitter storm — any external event that changes your traffic quality mid-test corrupts the results. If something significant happened during your test window, note it and consider re-running if the event affected your traffic mix.

Continue Reading