False Negatives

Result of a statistical test that fails to detect a real effect, wrongly concluding that there is no significant difference between two variants, when there actually is one. In the context of CRO and A/B testing, a false negative can lead to rejecting a variation that is actually performing well, simply because the test lacked power (e.g., sample too small, insufficient duration, MDE too low).

False negatives are often contrasted with false positives, which wrongly indicate that an effect is significant when it is not. The balance between these two errors depends on the chosen statistical parameters (confidence level, power), and must be carefully calibrated to avoid erroneous decisions in optimization tests.