Decide whether your A/B test result is real signal or noise — with the statistics done correctly.
Two test types.
- Proportion / conversion rate. Enter successes + visitors for control and treatment. The tool computes conversion rates, relative lift, a pooled Z-test, and the 95% / 99% confidence interval for the difference.
- Continuous metric (mean & SD). Enter mean, standard deviation and sample size for each group. The tool runs a Welch t-test (which does not assume equal variance) with Welch–Satterthwaite degrees of freedom, and reports the difference, relative change and confidence interval.
Beyond the p-value. A single p-value is not enough to act on. This tool also gives you:
- Lift — the relative change, the number stakeholders actually care about.
- Confidence interval — the plausible range of the true effect; if it crosses zero the result is not significant.
- Sample size — how many users per variant you would need to reliably detect an effect this size at the chosen power (80% / 90%).
- Recommended experiment days — based on your daily traffic and split.
- Bonferroni correction — when you run multiple comparisons, the effective α shrinks; the tool tells you if the result survives the correction.
Sequential-testing warning. Peeking at the p-value mid-experiment and stopping early inflates false positives. The tool warns against this. If you must look repeatedly, use an always-valid sequential method (e.g. mSPRT) or pre-register your sample size and stick to it.
Reading the verdict. "Significant" means the data is incompatible with there being no difference at α = 0.05 — it does not mean the lift is big or important. Always pair significance with effect size (the lift and CI) before shipping.