# A/B Experiment Design, Sample Size, and Statistical Power

Plan an A/B experiment, size its evidence, select the right test, and review effect size and uncertainty before treating a result as actionable.

> Canonical page: https://elysiatools.com/en/hubs/ab-testing-experiment-design-and-statistical-power

- **Keywords:** A/B experiment design, A/B test sample size, statistical power, minimum detectable effect, confidence interval, effect size, experiment acceptance

## Frequently asked questions

### Can these tools run the experiment?

No. They calculate statistics from supplied inputs. Randomization, exposure logging, event definitions, monitoring, and business review belong in the experiment system.

### What does power mean?

It is the chance of detecting a specified real effect under a stated design. It depends on sample size, variability, alpha, and effect size; it is not business value or the probability that a winning result is true.

### When is a small p value not enough to ship?

When the lift is below the useful threshold, the interval includes harmful or trivial effects, a guardrail worsens, the sample is biased, or the analysis changed after repeated looks.

## Related content

- [Statistical Analysis, Tests, and Distribution Tools](https://elysiatools.com/en/hubs/statistical-analysis-tools): Summarize numeric data, measure spread, compare groups, calculate probability and confidence, and model relationships with focused statistics tools.
- [Data Quality and Anomaly Investigation Workflows](https://elysiatools.com/en/hubs/data-quality-anomaly-workflows): Profile tabular data, isolate duplicates, missingness, referential breaks, outliers, and time-series anomalies, and turn findings into defensible quality evidence.