Optimizely Blog

For Killer Response Rates in Large Campaigns, Try Experimental Design

How experimental design can increase response rates to promotional offers, adding hundreds of millions of dollars of value.

Von John Senior

First published on März 25, 2015
Min. Lesezeit


This article originally appeared on the Optimizely blog. Your customer base is dynamic and complex. If you’re trying to optimize across your entire population of users, traditional A/B experiments may fall short. In this case, you should consider a technique we call experimental design because it allows for massive increase in the amount of variance in your digital marketing campaigns. Experimental design is a form of multivariate testing. This technique allows marketers to massively increase the possible combinations of individual variables in digital marketing campaigns as opposed to binary changes in A/B tests. Marketers can test and project the impact of many variables (such as web page formats and designs, product offers, messages, incentives, interstitials, and so on) by testing just a few combinations of them. We have seen multivariate experiments increase response rates to promotional offers for products, subscriptions, and other marketing campaigns by 3 to 5 times, adding hundreds of millions of dollars of value. In this post, we’ll discuss how to make large-scale experimental design campaigns actionable. Read more on Optimizely's blog.

This article originally appeared on the Optimizely blog.

Your customer base is dynamic and complex. If you’re trying to optimize across your entire population of users, traditional A/B experiments may fall short. In this case, you should consider a technique we call experimental design because it allows for massive increase in the amount of variance in your digital marketing campaigns.

Experimental design is a form of multivariate testing. This technique allows marketers to massively increase the possible combinations of individual variables in digital marketing campaigns as opposed to binary changes in A/B tests. Marketers can test and project the impact of many variables (such as web page formats and designs, product offers, messages, incentives, interstitials, and so on) by testing just a few combinations of them.

We have seen multivariate experiments increase response rates to promotional offers for products, subscriptions, and other marketing campaigns by 3 to 5 times, adding hundreds of millions of dollars of value.

In this post, we’ll discuss how to make large-scale experimental design campaigns actionable.

Read more on Optimizely's blog.