False Positives
Results wrongly indicating that an effect is present when it is not.
Result of a statistical test that wrongly indicates that an effect is significant, when it is not actually the case. In the context of CRO and A/B testing, a false positive can lead to validating an ineffective variation, which risks resulting in counterproductive decisions (e.g., rolling out a version that brings no real gain).
False positives are linked to the confidence level chosen for a test (often 95%): there is always a probability (e.g., 5%) of concluding a difference that is simply due to chance. This phenomenon is even more critical when running many tests simultaneously, without controlling the false positive rate (e.g., via Bonferroni correction or false discovery rate testing).
False positives are the opposite of false negatives, which, on the other hand, miss an effect that is actually present. Finding the right balance between these two types of errors is essential for reliable experiments and a solid CRO strategy.
Autres termes : Statistiques
View the full glossary- Frequentist Approach (A/B test)A statistical approach that estimates model parameters based solely on observed data, without using prior information.Statistiques
- Bayesian (A/B test)A statistical approach that uses probability to represent uncertainty about model parameters.Statistiques
- Statistical confidenceLevel of certainty one has in the results of a statistical test.Statistiques
- False NegativesResults that fail to detect an effect that is actually present.Statistiques
- Mann-Whitney U TestA non-parametric test used to compare two independent samples to determine whether they come from the same distribution.Statistiques
- MDE (Minimum Detectable Effect)The smallest difference that, if it exists, can be reliably detected in a statistical test.Statistiques