Big Data: The Organizational Challenge

Big Data: The Organizational Challenge

If you don't know who (and where) your chief analytics officer is, you may already be behind the curve.

  • min read


Big Data: The Organizational Challenge

Samsung uses it to power the content recommendation engine on its newest smart TVs. Progressive Insurance relies on it to capture driving behavior, determine customer risk profiles and decide on competitive pricing. LexisNexis Risk Solutions uses it to identify individuals, including family relationships, thus helping financial institutions and other clients reduce fraud.

It, of course, is Big Data—the mining and processing of petabytes’ worth of information to gain insights into customer behavior, supply chain efficiency and many other aspects of business performance. We say of course, because Big Data is hard to miss these days. Industry analysts and media observers hype it as the next big thing for every enterprise, and many companies have been rushing to climb on board. But is building an advanced analytics capability really worth the investment? Until now, data to answer that question has been scarce.

A recent Bain & Company study, however, should put the question to rest. Early adopters of Big Data analytics have gained a significant lead over the rest of the corporate world. Examining more than 400 large companies, we found that those with the most advanced analytics capabilities are outperforming competitors by wide margins (see Figure 1). The leaders are:

  • Twice as likely to be in the top quartile of financial performance within their industries
  • Five times as likely to make decisions much faster than market peers
  • Three times as likely to execute decisions as intended
  • Twice as likely to use data very frequently when making decisions


This helps to explain why so many companies are now asking where they stand on Big Data vis-à-vis their rivals— and whether they’re missing out on a new and essential competitive tool.

To get in the Big Data game, a company needs three kinds of table stakes. The first is the data itself: large quantities of information in a format allowing for easy access and analysis. Most large companies already have this—in fact, they generally have more than they can use. The second is advanced analytical tools, such as Hadoop and NoSQL. Both proprietary and open-source tools and platforms are widely available these days— all you need are people capable of putting them to work. That brings us to the third, and usually the most challenging, set of table stakes: expertise. Advanced analytics requires staff with state-of-the-art skills in everything from data science to worldwide privacy laws, along with an understanding of the business and the relevant sources of value.

But table stakes alone won’t help you win, because Big Data isn’t just one more technology initiative. In fact, it isn’t a technology initiative at all; it’s a business program that requires technical savvy. So you can’t just add more capacity and expertise, and expect your IT or marketing functions to begin generating data-based insights. Even if they did, the rest of the company would be unlikely to act on those insights.

As the analytics leaders have discovered, succeeding with Big Data requires a different approach: You need to embed Big Data deeply into your organization. It’s the only way to ensure that information and insights are shared across business units and functions. This also guarantees the entire company recognizes the synergies and scale benefits that a well-conceived analytics capability can provide.

Let’s look at what’s involved.


Leading companies begin the embedding process by spelling out their ambition. We will embrace Big Data as a new way of doing business. We will incorporate advanced analytics and insights as key elements of all critical decisions. A declaration like this from the senior leadership team is an essential precondition for the kind of behavior change this article will discuss. But the senior team must also answer the question: To what end? How is Big Data going to improve our performance as a business? What will the company focus on?

There are four areas where analytics can be relevant: improving existing products and services, improving internal processes, building new product or service offerings, and transforming business models. These objectives often overlap. Progressive’s new “Snapshot” device, which monitors driving behavior, helps the company determine whether a given driver is the right customer for the company. Intuit’s acquisition of Mint.com has helped expand its business beyond purchased software to ad-supported software. Humana, the insurance provider, is using Big Data to transform its business: Using claims data, the company can determine who is likely to end up in a hospital for preventable reasons and then intervene early. Humana and other health insurance carriers are also mining data to help improve patient outcomes and to reward healthy behaviors.

Most companies are opportunity-rich when it comes to analytics, and large enterprises can pursue multiple avenues, either simultaneously or sequentially. Still, nearly every company can improve its trajectory by determining priorities and picking the right angle of entry.

To succeed with Big Data, executives need to embed analytics deeply into their organizations, says Rasmus Wegener, a partner in Bain's Advanced Analytics practice.

Horizontal analytics capability

With ambition defined, Big Data leaders work on developing a horizontal analytics capability. They learn how to overcome internal resistance, and create both the will and the skill to use data throughout the organization.

This is a big job. Organizations don’t change easily and the value of analytics may not be apparent to everyone, so senior leaders may have to make the case for Big Data in one venue after another. They may need to help people change their everyday behaviors and then continue along the new path without backsliding. As with any major initiative, executives and managers have a variety of tools at their disposal. Leading companies typically define clear owners and sponsors for analytics initiatives. They provide incentives for analytics-driven behavior, thereby ensuring that data is incorporated into processes for making key decisions. They create targets for operational or financial improvements. They work hard to trace the causal impact of Big Data on the achievement of these targets.

For example, Nordstrom elevated responsibility for analytics to a higher management level in its organization, pushed to make analytical tools and insights more widely available and embedded analytics-driven goals into its most important strategic initiatives. Another global consumer electronics company selected high-impact analytics projects for additional support, creating positive business results stories and additional demand for Big Data solutions. The company added incentives for senior executives to tap Big Data capabilities, and the firm’s leadership has reinforced this approach with a steady drumbeat of references to the importance of analytics in delivering business results.


The Who, Why and How of Big Data

If you don't know who (and where) your chief analytics officer is, you may already be behind the curve.

An organizational home

The Big Data leaders then create an organizational home for their advanced analytics capability, often a Center of Excellence (CoE) overseen by a chief analytics officer.

Creating an organizational home involves several key design decisions. A company has to set its strategy for Big Data deployment. It has to assign collection and ownership of data across business functions, plan how to generate insights, and prioritize opportunities and allocation of data scientists’ time. It must host and maintain the technological infrastructure, set privacy policy and access rights, and determine accountability for compliance with local laws and data security. All of that is a tall order. To get it done, companies typically pursue one of four models:

  • Business unit led. When business units have distinct data sets and scale isn’t an issue, each business unit can make its own Big Data decisions with limited coordination. AT&T and Zynga are among the companies that rely on this model.
  • Business unit led with central support. Business units make their own decisions but collaborate on selected initiatives. Google and Progressive are examples of this approach.
  • Center of Excellence. An independent center oversees the company’s Big Data. Units pursue initiatives under the CoE’s guidance and coordination. Amazon and LinkedIn rely on CoEs.
  • Fully centralized. The corporate center takes direct responsibility for identifying and prioritizing initiatives. Netflix is an example of a company that pursues this route.

Note that in none of these models does IT own Big Data. While IT often plays a critical role in providing and maintaining the infrastructure and tools required to run Big Data analytics, most companies find that it’s a mistake to have IT own or manage the business adoption capability.

A company’s choice of model obviously depends on its ambition and its existing operating model. For example, companies with deep analytics capabilities and an emphasis on experimentation and innovation, such as Google and Progressive, can rely on a generally decentralized approach. But many analytics leaders have found that a CoE has the most advantages and the fewest limitations (see Figure 2). A well-functioning CoE enables cross-business-unit access and sharing of data. It takes responsibility for supporting and coordinating every initiative from a business unit, thus providing synergies and scale benefits. On the corporate level, the CoE serves as the go-to organization for analytics strategy and insight support. It sets the road map, and it establishes and maintains privacy policies. A leading European telecommunications company, for example, is in the process of deploying Big Data for a range of purposes, including analyzing customer data to provide better offers and services, and using network trsffic data to optimize network management and investments. It will house these capabilities in a variety of settings, but all will be coordinated by a CoE.


Building a solid CoE from scratch can take time. The center needs experienced leadership and a clear plan for staying connected to the business. It should have a strategy designed to ensure continuous learning, so that it maintains state-of-the-art capabilities. Staffing can be a particular challenge. A CoE requires not only skilled PhD-level data scientists, but also analytics engineers, business managers to identify and prioritize opportunities, and legal talent for advice on standards for data privacy and security. Finding team leaders and identifying partners to fill out the center’s staffing may take between six and 12 months, with scaling up requiring another 12 to 18 months.

Getting started

Many companies are already dipping their toes into Big Data waters. But given the complexities we have discussed—in particular the need to anchor analytics capabilities in the organization—toe-dipping isn’t likely to produce significant insights. That’s why only a select few, so far, have made substantial progress. Right now, many of these leaders are pulling even farther ahead of competitors, so others are playing the necessary game of catch-up.

But it isn’t too late. A good first step is to benchmark your industry and determine your company’s current position in Big Data analytics and capabilities, compared with that of your chief rivals. This exercise will help you determine the investment necessary to improve your relative position. If you are significantly behind the competition, you will have the kind of burning platform that is often required to create and sustain change. You can then begin experimenting, testing hypotheses to learn where and how advanced analytics is most likely to help your business. This type of review will help you determine your Big Data ambition, embed a culture of analytics and decide where Big Data’s organizational home should be.

Travis Pearson is a partner with Bain & Company and based in the firm’s San Francisco office. Rasmus Wegener is a Bain partner based in Atlanta.