AA Test

A type of test in which two groups receive exactly the same experience (same page, same content, same conditions) in order to verify the reliability of the testing system. In a CRO (Conversion Rate Optimization) context, an A/A test helps ensure that:

• traffic distribution between variants is well balanced,
• conversion tracking is working correctly,
• there is no technical or behavioral bias in the testing tool,
• the differences observed between variants under real conditions are not due to chance or incorrect setup.

This type of test is often used before launching a critical A/B test, particularly when:

• a new testing tool has been installed (e.g., Dynamic Yield, AB Tasty, Optimizely),
• a new segmentation or targeting logic is in place,
• server-side variations are being used and need to be validated.

💡 Good to know: It is normal for small statistical differences to appear in an A/A test, but none should be significant. If one variant outperforms the other with a high confidence level, this may indicate a bias in the configuration, a tracking bug, or a randomization issue.