GrowthBook Version 2.1

Sequential testing support, multiple testing correction, visual editor improvements, and more

· 2 min read
How sequential analysis can avoid false positives caused by peeking
How sequential analysis can avoid false positive results caused by peeking

This May we released a new version of GrowthBook. In Version 2.1 we’ve added sequential testing support, multiple testing correction, element reordering in the visual editor, SDK improvements, and more. Read about all the new developments we’ve been up to.

Sequential Analysis and Multiple Testing Correction

With experimentation, the more ways you look at a test, the more likely you are to see a significant result when there really isn’t one (a false positive). This is especially an issue for Frequentist statistics and p-values. We added 2 optional settings to help combat this. For the Multiple Testing Problem (adding lots of metrics and dimensions to an experiment), we let you specify a p-value correction method — either Bonferroni or Benjamini-Hochberg. For the “peeking problem” (looking at results too early), we let you enable Sequential Analysis to ensure p-values are always valid.

Visual Editor Improvements: Multi-page experiments and element reordering

Visual editor reordering of elements
Showing how the visual editor can reorder elements

GrowthBook’s visual editor continues to improve in 2.1. It now supports running experiments across multiple pages (e.g. changing both a landing page and the next page in the funnel). We also added the ability to reorder elements on the page, expanding the types of experiments that non-technical users can run without engineers.

Major SDK updates for PHP, Python, Ruby, Kotlin, and Java

We’ve been hard at work updating our many SDKs to support the latest GrowthBook features, including encrypted feature flags and built-in fetching and caching. We’ve also greatly improved the code quality and developer experience. For example, adding typed Ruby support (with RBS), snake case APIs for Python, better debug logging in PHP, and more.

Saved Groups API

Saved Groups let you easily target feature flags to groups of users. Until now, the only way to update this list of users was through the GrowthBook UI. Now, there are new REST API endpoints so you can update the lists programmatically (e.g. add a company id to your “paid users” group when they start a subscription).

Project-scoped Stats Engine

In some larger organizations, there may be some teams that prefer Bayesian statistics and others that are more comfortable with Frequentist. Now, you can configure the stats engine on a per-project basis. We plan to add lots of other project-scoped settings in the future so stay tuned.

Exportable audit logs

For larger organizations, auditing events that happen on the GrowthBook system is important for accountability and compliance. With this new Enterprise feature, you can now easily download a complete list of all the changes that have been made on the GrowthBook platform.

Other Features and Improvements

And many more changes and bug fixes which you can read about here:

Read next

Want to give GrowthBook a try?

In under two minutes, GrowthBook can be set up and ready for feature flagging and A/B testing, whether you use our cloud or self-host.