We just launched a new version of our open source A/B testing platform, Growth Book. We are excited to share the new features and changes.
Don’t forget to follow us on Twitter and give us a star on GitHub.
PrestoDB & TrinoDB support
You can now select PrestoDB and TrinoDB amongst the ever growing lists of supported data sources. If you have any requests, just let us know.
Improved display of ‘risk’
We improved the user experience for showing the Bayesian risk in results. You can learn more about the risk metric in our blog article. Risk shows you what impact you choosing a variation will have on a metric given a worst case scenario. In the above example, if you chose the Google Login variation, the risk to the signup metric is .1%, even though this metric is not significant yet. You might be okay to take that risk and call the test without having to wait for significance on the metric. This allows you to move faster and test more.
Metric improvements — groups and guardrails
We made some major improvements to the handling of metrics. You can now tag metrics which helps you organize and search for them. This is particularly helpful when you start adding a lot of metrics. Tagged metrics can be used as groups, and be easily added to experiments. For example you could add tags to metrics that you want all sell page experiments to measure, under ‘sell page’ and easily add these metrics to all sell page experiments.
We also added support for guardrail metrics, which are metrics you want to keep an eye on and make sure they don’t degrade, but not specifically trying to improve. For example, many companies use page load time, crashes, error rate, or support requests as a guardrail metric to make sure they don’t increase.
Other improvements and bug fixes
- Better logging and error messages
- Allow customizing snowflake role
- Bug fix: variationIdFormat select box not working
- Bug fix: switch from UNION to UNION ALL in queries for better support
- Bug fix: use redshift default for sql formatting to properly handle table names with dashes
We’d love to hear from you and get your thoughts and ideas. Drop us a line via email, Slack, Twitter, or just reply to this email.
