How long should an A/B test run?

Until the sample size you calculated up front is reached – and across at least one full weekly cycle so weekdays and weekend are covered. A fixed end date without a sample calculation regularly leads to wrong calls.

How many visitors do I need for an A/B test?

It depends on your current conversion rate and the effect you want to detect. The smaller the expected difference, the larger the sample. Calculate it before launch; at very low volume, heatmaps, session replays and funnel analysis are the faster alternative.

What's the difference between A/B and multivariate testing?

An A/B test compares two variants. A multivariate test evaluates combinations of several elements at once, needs considerably more traffic and in return reveals interactions between elements.

Can I stop a test as soon as it's significant?

Better not. Repeated checking and early stopping (peeking) increases the chance of reading noise as a win. Stick to the planned sample size or use an evaluation method designed for continuous monitoring.

What do I do with a test that shows no difference?

Document it and rule the hypothesis out. A neutral result usually means the element you changed wasn't the bottleneck – so look at funnel steps with higher drop-off next.

Is A/B testing GDPR-compliant?

Assigning a visitor to a variant is processing, so the same rules apply as for tracking: establish a legal basis, access device information only with consent, and keep data minimal. BigHoot delivers the accompanying analytics GDPR-compliant with primary EU hosting.

Do I still need a testing tool if I use BigHoot?

Yes, for serving the variants. BigHoot provides the hypotheses beforehand and the explanation afterwards: heatmaps, session replays, Smart Funnels and Attribution Tracking show where the problem sits and whether the effect really comes from the tested step.

A/B Testing: Guide to Valid Tests

Set up A/B tests properly: hypothesis, sample size, runtime and significance explained clearly – plus the most common mistakes and alternatives when traffic is low.