A/B testing on websites with little traffic

This article explores the challenges of A/B testing on websites with low traffic, favourite tools to calculate sample size and reliability, and test tips for conducting successful experiments on sites with minimal visitor volume.

For conversion optimization, you need visitors. The more traffic you have, the more experiments you can conduct to learn from. Many start-ups, niche, or small businesses receive fewer than 100K visitors and 500 conversions per month, which is considered low traffic.

Why is low traffic a problem for A/B testing?

Reliable test results depend on two factors:

1) Sample size (traffic levels)

2) Uplift (the difference between conversion rates of A and B)

Consider testing two calls to action with 5000 visitors per variation:

• Version A: 100 goals/5000 visitors = 2% conversion rate
• Version B: 250 goals/5000 visitors = 5% conversion rate

Version B shows 150% improvement. Higher traffic leads to quicker results—100K+ monthly visitors can yield results in 1-2 weeks, while low traffic requires more time to achieve sufficient sample size.

With significant conversion rate differences, you can obtain results with smaller sample sizes. However, small sample sizes combined with low uplift make it statistically challenging to draw conclusions.

How can you calculate the significance of a test?

A test result is significant when you can statistically determine the difference in conversion rate wasn’t due to chance.

Use test duration calculators like Abtestguide or VWO’s calculator to determine if your intended test can yield statistically significant results within a certain timeframe and how many visitors you’ll need.

How many visitors are enough?

There’s no magic number—just mathematics. A good rule of thumb is approximately 1000 visitors per week on the page you want to test, or about 50 conversions per week. Use tools like this one to calculate required visitors for each specific A/B test.

Tips for A/B testing on sites with few visitors

1. Test on pages with the most traffic

Focus on your top 3 most visited pages and analyse click-through rates to the next funnel step. Look for sitewide changes in navigation, such as highlighting menu items or changing labels to see which resonates better with visitors.

2. Focus on micro-conversions

Test incrementally rather than immediately checking revenue impact. Micro-conversions that lead to primary goals help you achieve results faster. Examples include filling out forms, creating accounts, clicking to product detail pages, or adding items to cart.

3. Limit the number of variations

More variations require more visitors and time for reliable results. Run a few simultaneous A/B tests until you have sufficient traffic. If simultaneously adjusting headlines, calls to action, descriptions, and images, you won’t identify which change drove results.

4. Extend your durations if needed

Don’t stop tests after a few days. Let them run 1-2 weeks minimum for good understanding. Use VWO’s duration calculator to determine optimal test length.

5. Combine quantitative and qualitative testing

Quantitative data answers “How many people use this?” or “How often does this occur?” Qualitative data suits both high and low-traffic sites and complements quantitative findings. User tests, session recordings, and surveys identify UX problems and answer “Why are users exhibiting this behaviour?”

Always focus your objectives and ensure they guide your method selection.

To conclude

Whether your hypothesis is proven or disproven, you’ll learn something valuable about your customers. That knowledge is invaluable for improving your digital presence.