This May we released a new version of GrowthBook. In Version 2.1 we’ve added sequential testing support, multiple testing correction, element reordering in the visual editor, SDK improvements, and more. Read about all the new developments we’ve been up to.
Sequential Analysis and Multiple Testing Correction
With experimentation, the more ways you look at a test, the more likely you are to see a significant result when there really isn’t one (a false positive). This is especially an issue for Frequentist statistics and p-values. We added 2 optional settings to help combat this. For the Multiple Testing Problem (adding lots of metrics and dimensions to an experiment), we let you specify a p-value correction method — either Bonferroni or Benjamini-Hochberg. For the “peeking problem” (looking at results too early), we let you enable Sequential Analysis to ensure p-values are always valid.
Visual Editor Improvements: Multi-page experiments and element reordering
GrowthBook’s visual editor continues to improve in 2.1. It now supports running experiments across multiple pages (e.g. changing both a landing page and the next page in the funnel). We also added the ability to reorder elements on the page, expanding the types of experiments that non-technical users can run without engineers.
Major SDK updates for PHP, Python, Ruby, Kotlin, and Java
We’ve been hard at work updating our many SDKs to support the latest GrowthBook features, including encrypted feature flags and built-in fetching and caching. We’ve also greatly improved the code quality and developer experience. For example, adding typed Ruby support (with RBS), snake case APIs for Python, better debug logging in PHP, and more.
Saved Groups API
Saved Groups let you easily target feature flags to groups of users. Until now, the only way to update this list of users was through the GrowthBook UI. Now, there are new REST API endpoints so you can update the lists programmatically (e.g. add a company id to your “paid users” group when they start a subscription).
Project-scoped Stats Engine
In some larger organizations, there may be some teams that prefer Bayesian statistics and others that are more comfortable with Frequentist. Now, you can configure the stats engine on a per-project basis. We plan to add lots of other project-scoped settings in the future so stay tuned.
Exportable audit logs
For larger organizations, auditing events that happen on the GrowthBook system is important for accountability and compliance. With this new Enterprise feature, you can now easily download a complete list of all the changes that have been made on the GrowthBook platform.
Other Features and Improvements
- Stats engine adjustable per project
- Data source schema browser in SQL editor
- Redis support within the GrowthBook Proxy
And many more changes and bug fixes which you can read about here: https://github.com/growthbook/growthbook/releases
