An Airline Transforms with Test-and-Learn

When a major global airline set out to increase margins through enhanced upselling and increased ancillary revenue, it sought inspiration well outside the industry: namely, from companies such as e-commerce retailers, streaming media services, and online booking sites. These digital natives excel at customer personalization powered by robust test-and-learn programs, and AirlineCo* knew that those capabilities were core to its new focus on offering the right products in the right way at the right price.

Over the course of a two-year, multi-phase program, we worked closely with the airline to make that happen. We focused on two critical components: holistic ancillary product management and digital experimentation.

A new organizational mindset

One advantage that digital natives have over airline companies is close collaboration between their digital and commercial teams. In our initial diagnostic of AirlineCo’s nascent test-and-learn capability, we identified an absence of teaming as a major impediment to its testing ambition, along with having too few dedicated full-time employees. 

We moved quickly to demonstrate the impact of scaling experimentation, increasing the volume of  tests by 15x, from 20 in 2023 to 300 in 2025. The impact on EBITDA was equivalent to 2–3% of the airline’s ecommerce revenue.

Not every test will be a winner; in fact, most (~75%) fail. That’s why scale, speed, and a diversified portfolio of tests are so important. When AirlineCo’s senior leadership saw what happens when a test succeeds, they experienced a mindset shift: Investing in the talent needed to conceptualize, launch, and analyze hundreds of tests a year would be money well spent if even a fraction of the experiments paid off.

Making it happen

Having proven the value of experimentation through successful tests on AirlineCo’s website—and having won senior leadership commitment as a result—we worked with the company to develop more organizational muscle. Critical components included the creation of a robust governance structure and the launch of an “experimentation cockpit” that would coordinate testing and share the results across all relevant teams.

We then expanded the scope of testing from AirlineCo’s website to address all touchpoints, from the booking and reservation pages of its website to check-in and its mobile app. With display testing up and running at scale across AirlineCo’s website and app, we extended experimentation to two additional, critical areas: personalization and ancillary pricing.

To facilitate personalization, we created and tested a series of recommendation engines that determine which offers (and in what form) are made to customers during the booking flow (branded fares, bundles) and post-booking (ancillaries, upgrades), both on the website and in the app. These engines use reinforcement learning to achieve next-level effectiveness in the direct channel. Three initial use cases yielded impressive results: an immediate lift in EBITDA in the first quarter and a projected 10% increase in net profit by year three. 

On pricing, we shifted the company from traditional (i.e., manual) approaches to a system that deployed experimentation software to increase velocity and precision. For example, in a traditional approach, a company might apply different prices for different routes or at different time periods. But since no two routes or time periods are exactly the same, the results may not yield sufficient clarity. By deploying an experimentation platform, a company can randomize pricing on all routes at the same time and instantly detect which offers generate a revenue uplift. Crucially, our ancillary pricing team conducted price-change tests before rolling out the changes and found that most changes did not boost revenue, and a few trended negative. This underscores the reality of testing: Many experiments don’t work, but when you find and replicate the ones that do, the impact is significant. As a next step, we implemented dynamic pricing algorithms for select ancillary products that continuously change prices based on demand and inventory.

These successes across three connected experimentation tracks (display/merchandising, recommendation engine/personalization, and pricing) provide a solid foundation for AirlineCo’s 2030 vision, which strives to make every digital touchpoint (offer, price, copy, and placement) dynamically personalized for each traveler. The airline’s embrace of test-and-learn will extend to bundles, journey triggers, and copy design, becoming more sophisticated and impactful year after year. Having taken a page from the playbook of digital natives, AirlineCo is now well on the way to achieving its full ambition: becoming a modern retailer.