The Sample Ratio Mismatch (SRM) test can be used to detect a wide variety of data quality issues that may affect online experiments (aka A/B tests). Only expected proportions and observed sample counts are required as input for this procedure, so this test can be used even in cases where experimenters only have access to summary statistics; such as when using third-party tools.
Expected ratios will be multiplied by {{ 1/sum_props }} so they sum to 1.
Warning. Possible sample ratio mismatch detected with {{ display_srm_p }}.
The observed sample size in treatment does not match the expected proportion of the total sample size.
This is an indicator for randomisation failure or missing data issues.
Comparative statistics may be invalid as a result.
No indication of sample ratio mismatch with {{ display_srm_p }}.