Mann-Whitney U Test

Non-parametric statistical test used to compare two independent samples, in order to determine whether they come from the same distribution. Unlike parametric tests (such as Student's t-test), it assumes no normality of the data, which makes it particularly useful in contexts where data is skewed, noisy, or influenced by extreme values.

Application in CRO / A/B testing:

In a Conversion Rate Optimization approach, this test is ideal for analyzing non-binary metrics (e.g., average order value (AOV), revenue per session, time spent on a page) when:

• the distribution is heavily skewed,
• outliers are frequent (e.g., very large carts),
• or variances are unequal between groups.

Example:

An A/B test aims to compare the average cart value between two variants. The data shows a highly skewed distribution with a few very large carts that distort the mean.
➡️ Rather than using a t-test, the Mann-Whitney test makes it possible to compare the actual central tendency (relative position of values) without being biased by these extremes.

Advantages:

• Robust to non-normal distributions,
• Less sensitive to outliers,
• Suitable for continuous or ordinal metrics that cannot be converted into rates.

Note:

It tests distribution differences, not necessarily means. It is therefore useful to interpret it as part of a comprehensive analysis, alongside other metrics and visualizations.