Big Data solution providers make big promises. Just plug your data into our solution, they say. We’ll deliver a stream of insights that enable dramatic improvements in marketing productivity, customer experience quality and service operations efficiency. It’ll be a snap for you and your team; our technology and your data scientists will do all of the heavy lifting.

Feel like you’ve seen this movie before? If you were caught up in the initial euphoria of the customer relationship management (CRM) revolution, then you did. Starting in the early 1990s, many companies bought into the hype and the technology, only to wind up with unusable databases, rebellious sales teams and depleted capital budgets.

The CRM industry has since matured, and there is no doubt that CRM solutions can now deliver real value to many organizations. As evidence, CRM was the sixth most popular business tool in Bain’s 2015 Management Tools & Trends survey. And global CRM spending totaled $20.4 billion in 2014, up from $18 billion the previous year, according to Gartner research.

Yet CRM failure rates remain high. A 2014 report from C5 Insight found that more than 30% of all CRM implementations fail—and second and third CRM implementations at the same companies had only slightly lower failure rates. And this is 20 years into the “revolution”!

We see Big Data going down a similar path, making big promises about customer impact and value creation predicated on large investments in technology and expertise. In a recent report, Gartner predicted that “through 2017, 60% of Big Data projects will fail to go beyond piloting and experimentation and will be abandoned.” Why is history repeating itself? It’s not for lack of interest, effort or investment. Instead, it reflects the difficulty of generating value from existing customer, operational and service data—let alone the reams of unstructured, internal and external data generated from social media, mobile devices and online activity.

Companies are under increasing pressure to harness Big Data and advanced analytics. Customers demand more from the organizations with which they do business. Competition is intensifying, especially in mature industries such as financial services, retail, telecoms and media. Data-driven businesses continue to disrupt the status quo. Disruptors old and new—including Progressive, Capital One, Amazon, Google, Uber and Zappos, to name a few—have created data-driven business models that apply deep insights to deliver tailored products and services that win in the marketplace.

US auto insurer Progressive, for example, uses plug-in devices to track driver behavior. Progressive mines the data to micro-target its customer base and determine premium pricing. Capital One, an American financial services company, relies heavily on advanced data analytics to shape its customer-risk scoring and loyalty programs. To this end, Capital One exploits multiple types of customer data, including advanced text and voice analytics. Meanwhile, US retail giant Amazon mines customer data intensively to create personalized online shopping experiences. Amazon uses purchase histories and click streams to create a sophisticated recommendation engine that it presents on customized Web pages. On the logistics front, Amazon has also been a leader in applying data analytics to optimize inventory distribution and reduce shipment times.

Leading users of Big Data set a high bar for success. They have assembled deep benches of analytical talent and created processes that allow their organizations to glean useful insights from advanced analytics. They have built technology platforms that deliver timely data and insights when and where they are needed in the organization. Many have also created cultures of continuous innovation based on rigorous “test and learn” methodologies.

So how can your company profit from Big Data? The first step is learning how to distinguish the actual potential from the extravagant claims. Much of the ongoing hype rests on three flawed promises: The first is that Big Data technology will identify business opportunities all by itself. The second is that harvesting more data will automatically generate more value. The third is that expert data scientists can help any company profit from Big Data, no matter how that company happens to be organized.

Below we identify perils associated with each of these three promises, and present examples of companies that have overcome each on the way to creating real value from advanced customer analytics.

Promise: The technology will identify business opportunities all by itself.

Peril: Limited return on investment despite large expenditures of money and time.

Failed technology deployments often start with the assumption that the shiny new tool will generate value all by itself. Companies that successfully harness the power of Big Data solutions tend to start by applying advanced analytics to solve a small number of high-value business problems with in-house data before investing in technology. In the process they learn how to implement solutions organizationally. They also gain insight into operational challenges and come to understand the limitations of their data and technology. They can then define the requirements for their Big Data technology solution based on an understanding of their actual needs (see Figure 1).

For example, one large insurance company recently focused its data analytics program on fraud. The company was seeing a spike in fraudulent claims, and was incurring significant costs to investigate these claims. The program aimed to reduce fraudulent behavior at lower cost. To this end, the company built a text-mining algorithm that generated fraud propensity scores. This algorithm helped the company achieve a 20% increase in the number of fraudulent scores that it detected. The upshot was fewer cases under investigation and about $30 million in savings. Having proven the value of advanced analytics, the company is now increasing its technology and capability investments.

Promise: Harvesting more data will automatically generate more value.

Peril: Overinvestment in unproven data sources and inattention to valuable data sources closer to home.

The temptation to acquire and mine new data sets has intensified with the explosion of social media and mobile devices. And yet many large organizations are already drowning in data, much of it held in silos where it cannot easily be accessed, organized, linked or interrogated. We’ve found that successful Big Data journeys tend to start by fully exploiting the organization’s existing data.

From an analytic perspective, it is generally easier to work with data that has some history than it is to attack brand-new data sets. One large US telecom company took just this approach. The company faced increasing competition and wanted to create a program to systematically increase the value of its existing customer base. To achieve this goal, it combined more than 200 data elements from 15 marketing, service and operations databases to create “high definition” portraits of all its customers. The company used these portraits to develop targeted onboarding, cross-selling and customer engagement programs.

One of its new onboarding programs focused on customers who showed signs of low engagement with the company’s products. The data showed that low engagement was linked to higher customer churn. Instead of sending sales-focused marketing messages to these customers, the company began sending them product awareness and engagement messages that were designed to stimulate product usage. The result: product usage increased, early stage churn declined and more of these customers upgraded their services. In parallel, the company increased its cross-selling marketing to more engaged customers because the data showed that these customers were more likely to upgrade. This resulted in a 2.5-fold increase in cross sales and a far higher return on marketing investment. In total these programs generated many millions in incremental annual revenue.

This company is now incorporating new data sets that will further enhance its rich customer portraits. To supplement the insights generated by historical data, it is designing experimental marketing campaigns that inject forward-looking variance (e.g., new prices, promotions and offers) into their system.

Promise: Good data scientists will find value for you.

Peril: Your existing organization is not ready to realize the value from the data.

In order to profit consistently from Big Data, you need to create an operating model that harnesses the power of the data and advanced analytics in a repeatable manner. Successful data-driven businesses align their organization, processes, systems and capabilities to make better business decisions based on the insights from their data and analytics teams (see Figure 2).

One telecom service provider created a partnership model that encompassed its data and analytics teams, its technology division and its frontline functions (including sales, marketing, customer operations and product development). In this model, the business intelligence team, which includes data scientists, statisticians and data miners, partners closely with the business units to solve specific issues by applying advanced analytics to their large internal data sets.

The business units inject business experience and frontline knowledge into the insights from the data scientists, increasing the odds that their solutions will be pragmatic and scalable. The IT division, which owns the data architecture, figures out how to incorporate new technology such as data lakes, manages the ever-growing data sets and defines the policies and rules that govern them.

One of the first challenges the telecom company tackled with this new partnership model focused on improving the economics of value-destroying customers. In this instance, the sales and marketing team defined the specific issue for the business intelligence team, who then worked with the IT team to consolidate and merge two years’ worth of customer data from marketing and operational databases to identify the root causes of the value-destroying behaviors. Working together, the three teams defined a set of targeted customer strategies that could turn these value-destroying customers into profitable customers. The result: millions of dollars in incremental revenue.

Conclusion

The Big Data revolution has already disrupted many industries. Certain data-driven businesses have captured significant value from this revolution, but many traditional companies are playing catch-up. Technology alone cannot close this gap. Companies that realize the promise of customer data analytics tend to follow three rules:

1. Prove your organization can apply advanced analytics to solve a few high-value business problems before investing in Big Data technology solutions.
2. Create value from your in-house data before expanding to new data sources. Then use test-and-learn approaches to inject forward-looking data sets into your historical data.
3. Align your operating model to enable your organization, particularly the front line, to act quickly and with confidence on the insights from your advanced analytics teams.

Companies that follow these rules will be better positioned for success in the age of Big Data.

Eric Almquist is a partner in Bain’s Boston office. He is a leader of the Advanced Analytics practice and a member of the Customer Strategy and Marketing practice. John Senior is a partner in the Sydney office and a leader in the Technology, Telecommunications and Media as well as Customer Strategy and Marketing practices. Tom Springer is a partner in the Boston office and co-leader of the Advanced Analytics practice.