Companies everywhere have been enticed by the promise of digital transformation. CEOs are once again turning to technology as a force that can give them a competitive edge by ushering in exciting new capabilities, rapid innovation and efficiencies. Enterprises now expect to have regular and easy access to pay-as-you-go computing, where nimble, cloud-based tools replace plodding, capital-intensive infrastructure.

But a funny thing has happened on the way to the digital revolution. The process of getting there has turned out to be complex, risky and expensive. Many chief information officers (CIOs) have found their initial enthusiasm tempered by the frustration of not being able to deliver digital transformation to their companies in a cost-effective way.

Featured topic

More Digital Transformation Insights

Digital transformation is a topic of rich and vital discussion in boardrooms and among executive teams around the world. Here are some insights on what it takes to lead and deliver a digital transformation.

Nobody is arguing about the need to go digital. In today’s competitive marketplace, digital transformation has become a requirement, not an option. The problem is not the destination; it’s the journey. To become fully digital, enterprises have to increase dramatically the scope, scale and speed of their IT—all while staying in sync with the evolving needs of the business. That can be a challenge. Instead of making a smooth transition to digital, CIOs are struggling with a complex environment, higher costs, vexing compliance and security issues, and a perpetual need for faster implementation and sharper insights.

Superior performance in the digital age calls for an adaptable technology infrastructure that manages the complexities of a multicloud environment, embedded security and compliance policies, and deep business alignment. Best-in-class IT operations and the software vendors that support them are adopting a playbook based on six core rules for IT design (see Figure 1).

• Break boundaries across IT stacks. Given that companies are unlikely to achieve complete migration to the public cloud anytime soon, CIOs need monitoring, discovery and configuration tools that function in hybrid, multicloud environments as well as up and down the stack, from legacy systems to consumer-facing apps.
• Embrace DevOps. As firms increase the cadence of their digital offerings, they have no choice but to integrate software development and IT operations. Already, as many as 60% of enterprises are using or planning to use a DevOps approach to building and installing software, according to a survey by Gartner. Modern IT organizations require software that works across the production chain and that’s designed for rapid testing and validation.
• Be open. No modern solution can be an island. As designers produce focused, best-in-class solutions instead of massive monolithic systems, openness becomes critical. Companies need modular, open-source and application-program-interface–friendly software that is designed for easy extensibility and integration with other apps. CIOs expect to be able to combine the capabilities of their disparate systems to serve new needs.
• Incorporate policy engines. Cost pressures have driven CIOs to seek to automate their IT operations. They want to escape the massive manual efforts that they currently rely on to monitor policies, including compliance, data governance and security rules. They need solutions that have built-in logic to identify and remediate against rules in order to enable policy management across a hybrid infrastructure.
• Induce insights. As digital apps proliferate, companies are becoming flooded with an abundance of data—some of it useful, some of it not. CIOs need analytical tools that use techniques such as machine learning to glean insights from disparate sources.
• Insist on user-friendly experiences and tools. In a complex world, IT professionals are demanding intuitive, easy-to-use software. They are no longer satisfied with hard-to-master, second-rate applications; they want a consumer-level user experience. They need solutions that are software-as-a-service (SaaS) capable, simple to install and have immediate, out-of-the-box functionality.

Infographic

An IT Playbook for Digital Transformation

For many CIOs, the journey to digital is hampered by many obstacles. Here's a playbook for modern IT design centered around six core rules.

Managing the complexities of multicloud infrastructure

Companies moving to the cloud have had to relearn a difficult lesson: When it comes to advances in technology, the new does not replace the old, at least not immediately. Instead of using the cloud to build an entirely new infrastructure, CIOs have to add to their existing structure. Of the $77 billion that companies spent on the public cloud in 2015, nearly half (about 45%) went not toward migrating existing workloads but toward developing new software for existing business or toward creating apps for new digital operations. IT decision makers surveyed by Bain & Company expect that trend to continue (see Figure 2).

Out of necessity, CIOs are creating a new form of infrastructure—one that is complex, modular and hybrid. The new structure includes public clouds, private clouds and SaaS, as well as on-premises components that encompass many legacy systems and services. In their quest for simplicity, CIOs have ended up with complexity, and many have realized that these hybrid infrastructures are more than a transitory phase; they are the new reality. To that end, new tools and designs are needed to break boundaries across the IT stacks (rule No. 1).

CIOs who once dreamed of moving 100% of their computing to the cloud have dialed back their expectations. Companies now use the cloud and other new technologies for 40% of their computing and storage, up from 31% in 2013, according to the Bain survey of IT decision makers. Those same firms expect the cloud to account for 45% of their computing and storage by 2019—an indication that the hybrid infrastructure model is here to stay (see Figure 3).

At the same time, companies are slowly breaking the barriers between developers and operations, although most are still finding their way along this path. That is why embracing DevOps (rule No. 2) is critical. Many firms are adopting Agile, creating action-oriented developer teams and using DevOps models to reduce cycle times, but they still rely on a traditional, compartmentalized and sequential waterfall approach for infrastructure. These waterfall processes are slow, onerous and expensive. For a developer who’s joined an Agile team, the experience can be similar to speeding through TSA PreCheck at the airport only to find that the flight has been delayed indefinitely by air traffic control.

In these new multicloud and multitool environments, much of the digital innovation in companies is happening at the business unit level. Many of these projects are often not readily scalable to meet the quality and compliance requirements of enterprise-grade solutions or the standards of external service providers. Therefore, infrastructure must be open (rule No. 3) so that it’s easy to integrate the disparate components and tools that originate from different places within the organization as well as from external sources.

Addressing cost, security and compliance

While managing multicloud environments, enterprises are under constant pressure from the challenges of cost, security and compliance—all of which are exacerbated by the complexity and opacity of working in a hybrid, distributed structure.

No longer does the corporate IT department have a monopoly on the development and implementation of software. Business unit leaders are increasingly buying ready-made, off-the-shelf SaaS solutions on their own, bypassing the IT department. With SaaS solutions, business units can get rapid installation, a simple user experience and variable pricing. Instead of enduring lengthy, centralized procurement and testing processes, business unit managers can now purchase software using nothing more than an Internet browser and a corporate credit card, and it can be up and running in mere hours.

This decentralized approach to development makes it hard to control costs. Many CIOs expected pay-as-you-go cloud computing to be cheaper, reasoning that they’d no longer have to bear the high fixed expenses of on-premises computing. What they often get instead is cloud “bill shock.” That’s because they have little visibility into how business units are using the cloud to develop, test and produce new apps.

One CEO at a large U.S. financial company received a nasty surprise when a cloud that had been hosting a small test environment experienced a huge volume spike, which was only discovered after the end of the month when the bill that came in from the public cloud vendor was more than 20 times the normal amount.

The decentralized and distributed nature of modern IT environments makes it very hard for CIOs to apply, monitor and automate the rules and policies that govern where the workloads run and where the data resides. CIOs have to optimize performance while contending with security threats and complying with an ever-increasing number of specific geographic and industry regulations.

CIOs need an integrated, end-to-end framework for identifying and remediating threats across the organization. In a complex, distributed environment, major vulnerabilities can emerge in the gaps between targeted security solutions. CIOs face the same challenges in ensuring that all the decentralized elements of their structure are set up to comply with internal corporate rules and external regulations.

In an attempt to control these kinds of situations, CIOs have been relying on manual processes, workarounds and best guesses. Some companies are so worried about compliance that they avoid moving workloads to the cloud. Nearly 75% of CIOs interviewed by Bain do not believe that today’s compliance solutions cover their needs. CIOs are looking for enterprise-wide solutions that incorporate policy engines (rule No. 4), using built-in logic to automatically identify and enforce rules across a hybrid infrastructure.

Accelerating the digital agenda

CIOs will need to draw on solutions that induce insights (rule No. 5) and that insist on user-friendly experiences and tools (rule No. 6) to deal with one of the biggest headaches associated with digitalization: the glut of data.

IT organizations have more data than ever, and they struggle to produce meaningful insights from it. Typically, they segregate data into silos by type, including logs, events, tickets and performance metrics. And the insights they are able to glean are narrow and limited; they are mostly backward-looking snapshots of recent history. A Bain survey of IT leaders on the analytics needs for operations management found that more than half of IT organizations use at least three different analytics providers to generate performance reports. CIOs urgently want to be able to integrate and synthesize separate data sources into one analytics engine that can reach across the entire infrastructure.

Software developers are starting to create programs that use advanced analytics (AA) and machine learning to detect recurring and predictive patterns from a stream of data that is wide, deep, diverse and evolving. By deploying these automatically adjusting algorithms, companies have the potential to generate insights that are truly forward looking, such as whether a customer is at risk of defecting to a competitor.

Advanced analytics, similar to many of the tools that CIOs will need to underpin the six IT design rules, is a work in progress. In the Bain analytics survey, IT operations managers ranked AA as the capability that they would most like to have, yet only a fifth say they have access to the technology now.

The digital revolution has arrived, although it hasn’t happened quite the way many people had expected. On the road to realizing the promise of digital, CIOs are learning to address the challenges of hybrid management, cost, compliance, security and insight generation. The digital revolution is rich with promise and replete with complexity. CIOs need a guidebook, and so do developers. By following the six core rules of IT design, they’ll have the tools they need to help make digital transformation a reality.

The authors would like to acknowledge the contributions of Hamish Nairn, a consultant with Bain in San Francisco.

Jordi Moncada is a partner with Bain & Company in Silicon Valley in the Telecom, Media and Technology (TMT) practice, Vishy Padmanabhan is a Bain partner in New York in the Information Technology practice and Velu Sinha is a Bain partner in Silicon Valley in the TMT practice.