One neat trick to run better experiments
This Chrome Extension automatically flagged potential data quality issues on supported experimentation platforms.
The SRM Checker Chrome extension is no longer maintained and has been retired.
Over the years, the extension has helped hundreds of experimenters detect Sample Ratio Mismatches (SRMs) and improve the quality of their A/B testing data. However, a few things have changed:
- Google introduced changes to how Chrome extensions work, breaking the current implementation
- I no longer have the time to maintain or update the extension
- Most major experimentation platforms now include built-in SRM checks, making the extension less necessary
Because of this, I’ve decided to retire the extension.
Thank you to everyone who used, supported, and contributed to the SRM Checker over the years!
The website and related documentation will remain available for reference, but the extension itself will no longer be updated, supported, or guaranteed to work.
What is Sample Ratio Mismatch? Why should we care?!
A Sample Ratio Mismatch (SRM) test can be used to detect a wide variety of data quality issues (paper) (presentation) that may affect online experiments (aka A/B tests). Only expected proportions and observed sample counts are required as input for this procedure, so this test can be used even in cases where experimenters only have access to summary statistics; such as when using third-party tools.
For more information about Sample Ratio Mismatch please read the Frequently Asked Questions.
About the project
Sample Ratio Mismatch Checker was created by Lukas Vermeer, Heinrich Mahr and Georgi Georgiev.
Platforms supported by this Chrome Extension
- Convert.com
- Omniconvert
- Optimizely
- SiteGainer/Symplify
- Visual Website Optimizer (VWO)
- Zoho PageSense
Several experimentation platforms natively perform Sample Ratio Mismatch checks for their users. These platforms will not be supported by our Chrome Extension, because that would be redundant. An incomplete overview of these platforms can be found here.
License
Sample Ratio Mismatch Checker is distributed by an MIT license.
Contributing
You want to contribute to this project? That’s amazing! We’d love that! <3
Feel free to fork the repository on GitHub, make the changes you want to see, and issues a pull request. If you are not sure what to work on, we maintain a list of outstanding issues on the GitHub project. (If you don’t want to code, adding to that list of issues is also a great way to contribute.)
Thank you for making this project more awesome.
