How we can help
The best way to discover what customers want is to let them tell you—but you have to know how to listen. Experimentation at Scale lets you test hundreds of messaging, pricing, and product variables to find the winning combination that resonates with your audience.
Experimentation at Scale is a powerful AI/ML-driven approach that helps companies continuously improve their digital marketing, retail and e-commerce engines, owned media (direct mail, email, websites), pricing, and other key business activities.
We examine every step of the customer journey to determine which content, experiences, and product features resonate with customers. Our data and analytics experts and digital designers work with you to understand your unique priorities, then apply design-thinking principles to develop a pipeline of tests to optimize these variables. We then leverage a cutting-edge toolkit that includes fractional factorial multivariate testing (MVT), geographic matched market A/B/N testing, and multiarmed-bandit testing (MAB) to generate results faster and help you get the most insight from your audiences.
The result? Bigger, bolder ideas that translate to extraordinary impact and an experimentation engine that continues to drive results.
Proven test methodology that allows you to quickly analyze results, identify the best combination of attributes, and compare them across customer segments and other criteria
Our approach ensures that successful results are rolled out quickly and achieve the desired outcomes
Develop a roadmap for long-term testing that produces consistent, impressive ROI
Build the muscles to consistently design high-impact tests that produce customer-resonant content
Cash payback delivered after our engagements
Increase in likelihood that digital leaders use experimentation to optimize campaigns
Higher conversion rates driving incremental revenue through our MVT campaigns
Dissatisfied with the growth trajectory of its flagship consumer service, McAfee sought to understand why it wasn’t converting more new customers through its 30-day trial program. We helped them analyze how and when content was served, how users reacted to it, and the reasons why potential customers were not converting. This generated a series of hypotheses on how to improve the end-to-end customer experience, which McAfee tested via prototypes. The data-driven, test-and-learn mindset has become an engine for the company’s long-term growth, propelling industry-leading success across customer acquisition, retention, and product development.
Results:
CableCo was facing increased competitive pressures from satellite TV and telecommunications providers that enjoyed significant growth in the pay TV and broadband markets. When we were enlisted to help them win back and retain high-value customers, we relied on experimental design techniques. Fractional factorial analysis allowed our media consultants to launch a subset of potential in-market offer combinations to test different variables. A second analytical tool, a net present value (NPV) model, helped CableCo project the financial impact of all offer combinations. Through experimental design, the response rate grew to three to four times from the existing offer.
Result:
Coca-Cola wanted to increase spending on digital platforms, but needed to better understand which platforms were most effective. To help quickly move from opinions to facts, it ran experiments on connected television (CTV), the fastest-growing video ad platform. The company believed that increasing the CTV share within online video would lift sales and generate a higher return, but italso wanted to identify the maximum spending before hitting diminishing returns. In one example market, the experiment showed that raising the share of CTV in the digital video mix beyond roughly 25% proved to be more effective until the share exceeded roughly 75%. For the first time, Coca-Cola had connected CTV ad investment to sales and not just media impressions.
Result:
When InsuranceCo struggled to improve marketing performance, we were engaged to assess opportunities to drive incremental performance and faster speed to market. After an in-depth analysis of $350 million in marketing spending, we launched six tests to identify about a dozen additional unique opportunities for near-term activation and testing. This work helped InsuranceCo develop a true testing muscle, supported by a holistic framework for evaluating and evolving marketing capabilities centered on measurement, activation, and ways of working to move opportunities to market faster; a process to size the magnitude of optimizations; as well as a rubric for determining when testing is necessary.
Result:
Dissatisfied with the growth trajectory of its flagship consumer service, McAfee sought to understand why it wasn’t converting more new customers through its 30-day trial program. We helped them analyze how and when content was served, how users reacted to it, and the reasons why potential customers were not converting. This generated a series of hypotheses on how to improve the end-to-end customer experience, which McAfee tested via prototypes. The data-driven, test-and-learn mindset has become an engine for the company’s long-term growth, propelling industry-leading success across customer acquisition, retention, and product development.
Results:
CableCo was facing increased competitive pressures from satellite TV and telecommunications providers that enjoyed significant growth in the pay TV and broadband markets. When we were enlisted to help them win back and retain high-value customers, we relied on experimental design techniques. Fractional factorial analysis allowed our media consultants to launch a subset of potential in-market offer combinations to test different variables. A second analytical tool, a net present value (NPV) model, helped CableCo project the financial impact of all offer combinations. Through experimental design, the response rate grew to three to four times from the existing offer.
Result:
Coca-Cola wanted to increase spending on digital platforms, but needed to better understand which platforms were most effective. To help quickly move from opinions to facts, it ran experiments on connected television (CTV), the fastest-growing video ad platform. The company believed that increasing the CTV share within online video would lift sales and generate a higher return, but italso wanted to identify the maximum spending before hitting diminishing returns. In one example market, the experiment showed that raising the share of CTV in the digital video mix beyond roughly 25% proved to be more effective until the share exceeded roughly 75%. For the first time, Coca-Cola had connected CTV ad investment to sales and not just media impressions.
Result:
When InsuranceCo struggled to improve marketing performance, we were engaged to assess opportunities to drive incremental performance and faster speed to market. After an in-depth analysis of $350 million in marketing spending, we launched six tests to identify about a dozen additional unique opportunities for near-term activation and testing. This work helped InsuranceCo develop a true testing muscle, supported by a holistic framework for evaluating and evolving marketing capabilities centered on measurement, activation, and ways of working to move opportunities to market faster; a process to size the magnitude of optimizations; as well as a rubric for determining when testing is necessary.
Result:
