A/B Test Significance Calculator
Check whether an A/B test result is statistically significant with a two-proportion z-test, p-value, uplift and confidence interval.
—
- Relative uplift
- —
- Absolute difference
- —
- P-value
- —
- Confidence
- —
- Z-score
- —
- Difference interval
- —
How it works
Enter the visitors and conversions for your control (A) and your challenger (B) and the calculator runs a two-proportion z-test as you type. It pools both conversion rates to get the standard error, turns the gap between them into a z-score, and converts that z-score into a p-value with the normal distribution. The p-value is the probability of seeing a gap this large if the two variants were really identical, so a p-value below your alpha (0.05 at 95% confidence) means the result is statistically significant.
Relative uplift is the percentage change from A to B, while the absolute difference is measured in percentage points; the difference interval is the confidence interval around that absolute difference, so if it crosses zero the test has not settled yet. Pick a two-tailed test unless you decided up front that only one direction counts. Everything is computed in your browser with plain JavaScript — no accounts, no uploads and no test data leaves your device.
Frequently asked questions
How do I know if my A/B test is statistically significant?
Enter the visitors and conversions for both variants. The calculator runs a two-proportion z-test and returns a p-value: if it is at or below your alpha (0.05 at 95% confidence) the difference is statistically significant and the verdict names the winning variant. If the p-value is higher, or the difference interval still crosses zero, the test has not settled and you should keep collecting data.
What is the difference between relative uplift and percentage points?
Relative uplift is the percentage change from the control to the challenger. Going from a 10% to a 13% conversion rate is a +30% uplift. The absolute difference is the same gap measured in percentage points, so it is +3.00 pp. Reporting only the relative number makes small tests sound bigger than they are, which is why this calculator shows both plus a confidence interval around the absolute difference.
Should I use a one-tailed or two-tailed test?
Use a two-tailed test in almost every case: it asks whether the variants differ at all, so it catches a challenger that performs worse as well as one that wins. A one-tailed test is only valid when you decided before running the experiment that a difference in the opposite direction would be ignored. One-tailed p-values are half the size, so switching after seeing the data is a classic way to declare a win that is not there.