Audio

How to Feed AI’s Hunger for Power

The explosive growth of artificial intelligence will need a lot of electricity from a utility industry that hasn’t seen demand grow for 20 years.

Dunigan O'Keeffe, Grant Dougans, David Crawford, Aaron Denman, Anne Hoecker, Jue Wang, Matthew Crupi, and Jim Wininger

First published on мая 10, 2024
min read


At a Glance The explosive growth of artificial intelligence comes with soaring demand for electricity—as much as 5% per year in the next few years—to power data centers Utilities in the developed world haven’t seen electricity demand grow in more than a decade; many have been shuttering coal-powered plants to meet decarbonization goals Meeting data center demand—expected to grow 10-30% over the next five years—poses a quandary for utilities and technology firms Tech companies and utilities are at the forefront of a balancing act between technological advancement and environmental sustainability. The urgent push to decarbonize intersects with unprecedented demand for energy, accelerated by the AI sector's surging need for data center power. The resulting scenario presents a complex challenge: how to fulfill the burgeoning energy requirements of the tech industry without derailing efforts towards sustainability. What follows is a “Future Sensing” panel discussion that brings together leaders from Bain & Company's technology, energy, and strategy practices. The discussion sheds light on some of the intricacies of satisfying the dual demands of sustainability and energy in the era of AI, offering insights into possible paths forward. Read a transcript below. DUNIGAN O'KEEFFE: The impetus for this conversation was the dual challenge we're facing between sustainability and energy of how do we both decarbonize, but also provide the energy we need for our world and for everything our world needs. These two are now coming together, and that's the topic of today's discussion. I think what we're going to do is, we have a set of panelists who we've selected to kick us off. And I think with Grant to start, and then I will navigate through the other panelists. GRANT DOUGANS: Sure. I'm happy to do that. I'll share a few thoughts and then I might kick it over to Jue to pick it up. You know, one of the things, as I've gotten involved with the future sensing effort, is just reflecting on the value of conversations across practices and listening to what our clients are saying. So I'll start with the utility view. you know, for the longest time, I think it's important to remember over the last decade, growth has been flat. We have not seen an increase in electricity sales really in the developed world for more than a decade. And so, you know, utility systems and the way the utilities operate is really geared for that world. And frankly, that's a good thing because building power plants is quite challenging to do. Getting gas pipelines done is very challenging to do. LEARN MORE Future Sensing Future Sensing provides the insight to not only navigate turbulence, maximize opportunities, and reduce risks, but understand how the interplay of these dynamic forces will affect your strategy. But when we think about the conversation with our utility clients, I think two things to highlight that are relevant for this. The first is, utilities are now starting to see the impact of both data centers for AI, but also IRA related commercial industrial load really starting to come in. And organizations, which historically have seen no low growth, are now being asked to increase the size of their generation fleets by 10, 20, 30%. And, you know, when you think about, you know, power plants, these are pieces of physical infrastructure. They take three, four, five, six years to build. For the case of AI, I think it's really important for all of us to recognize that the type of load that data centers are requesting will only be served by fossil fuel generation resources today. That is the only way to speed this load. Full stop. We do not have a viable nuclear industry in this country. We do not have-- renewables are not going to cut it to run data centers. And so, you're kind of just sitting there as a utility, you've just been through the stakeholder wringer. And now we have, you know, the tech companies coming and asking us to build like seven gas plants. And that is a very challenging mission, both politically but also from a kind of feasibility perspective, even physically. And so the conversation that we're starting to have with our clients is, a) maybe we don't want all the data center load; and b) if we have data center load, how do we extract more from the data centers for the right to build in our service territory. We have a lot of people that want to do a lot of stuff with our systems. You know, data centers are just one of many. And AI is going to be very challenging politically, and I imagine there's going to be some real throttling of growth that's going to be sort of downstream of that. Organizations, which historically have seen low to no growth, are now being asked to increase the size of their generation fleets by 10, 20, 30%. Grant Dougans, Partner and Leader, Energy & Natural Resources Practice IP Council DAVID CRAWFORD: Thanks, Grant. What are the implications of that, in terms of, let's say for a second that you don't get to say, go away, to the data centers. If they're going to grow radically, what's the implications for growth behind the meter, powering, that kind of stuff? Will they end up all being built in areas where there's large natural gas supply? Are we going to see data centers built in Alaska? What about cooling? What are the implications? DOUGANS: Yeah. I mean, if gas is the only way to power data centers, which I think is a valid assumption for the next five to 10 years in this country, then, you know, somebody, regardless of whether the utility builds it or whether, you know, Microsoft builds it, somebody is going to need to be building large numbers of very large gas turbines and will also need to be getting the gas from the wellhead to those turbines. In our experience in the utility sector, we spent the last decade, trying to build even one gas pipeline is a absolutely miserable process that involves us being in court for a decade. And so I think, I imagine, Aaron, I invite you to comment Like proximity to gas is going to-- if you need to power this stuff with gas, proximity to gas is going to be important where you need to put this stuff. CRAWFORD: And you start by saying, it is gas. So there's no oil based option or anything like that. DOUGANS: Oil will be very expensive. And the air permitting issues with oil, are expensive. There's no way you're going to be building oil powered units to power this stuff. AARON DENMAN: But I do think, you know, to your point, Grant, you have to look at where the pinch points and where there are alternatives. And so if you could build these in the Permian, to your point of building them in Alaska, where you're not having to cross state lines, you can avoid some of the regulatory challenges, when it comes to permitting a gas pipeline. And so it doesn't mean the pressure won't be there, but you're sort of taking out some of the tension in the ability to do that. The role of alternative energy O'KEEFFE: And Aaron or Grant, just for completeness, what's the role of battery technology in solar, just it doesn't scale, doesn't fly here? What's the constraint? Why doesn't that work? AARON DENMAN: Yeah. So the data center load profile is relatively constant. And there's some flex that you can do with that, right. And so-- and let's just call it a renewable world. You're only going to be able to power with the sun, when the sun is shining in the middle of the day. So you're going to have to massively overbuild for solar for that facility. You're going to have to put in a large number of batteries, and you're likely also going to need to build wind, if you want to power some of these things in the night. And so just the amount of physical infrastructure that would be required, and even in that setup, it doesn't guarantee you, if you have a relatively cloudy, less windy day for 24 hours, just the sheer amount of batteries that would be required would just be extraordinary. And not to mention that many of those to be able to find a site where you could plow a data center of the scale we're talking about with all of this infrastructure, that also means a lot of transmission infrastructure and distribution infrastructure. And so just the complexity and the cost in a renewable plus storage world is incredibly challenging and costly to do that, relative to the gas alternative. DOUGANS: Right. what's changed is, you know, five, 10 years ago, the system had a lot of extra capacity. The power system was not tight. It was quite loose. So we actually had a lot of extra gas plants and power plants around because we, as a utility sector, have been closing coal plants over the last five to 10 years for climate reasons, all of the excess capacity in the system has been sort of necked down for environmental reasons. So there just isn't a lot of extra gas plants around you can contract with. There just isn't any-- there's no slack in the electricity system anywhere at this point. So anything you're going to-- if you need the power, you're going to find a way to build it de novo, And then on solar, for a one gigawatt data center, like run the math on how many square miles of land you're going to have to pave over with solar panels to power a gigawatt data center. It's a lot. O'KEEFFE: It's not small. Regional variations ANNE HOECKER: And then, Grant, have you guys looked at how the US compares to Europe or Asia for this challenge? Right, because these AI data centers are going to be popping up everywhere. Have we looked at this outside the US and if the problem is worse? DENMAN: You have similar challenges in many parts of the world, maybe not in the Middle East. But you do have some of the similar challenges in parts of Europe. I mean, the ability to have land access is already challenging for renewables. And then, you have some of the same interconnection and permitting challenges. Europe is also structurally much more expensive, relative to the US. And so as you factor in alternatives, it's just the operation maintenance power costs of that are going to be significantly more expensive in Europe. I think you'll see some explore Europe. But if you have to build gas in Europe, the environmental dynamics are even more challenging in Europe. And so we think that will be quite, quite challenging. Now Canada, I think if you're the Canadian government-- there's quite a bit of, you know, can we build additional nukes? There's some, you know, demand or encouragement for that. So Canada could be a choice. And then when you get into the Middle East, I think the solar option, just because of land availability and just the amount of sun and things like that, I think could be a viable choice in parts of the Middle East. Maybe the other thing that's, I think, relatively unique, and I just put a finer point on Grant's intro comments, we've not had load growth for 20 years. Just it's been flat for 20 years. And you see, we're talking about the AI tech issue, which is both important. But it's happening at the same time that we're asking really large industrial loads to decarbonize, and that is electrification mostly. You have, because of your structurally high energy cost, you have other industries, which are trying to relocate manufacturing capacity into the US because of low power prices, which is also driving load. You've got EV penetration, which I know is slower here recently. But-- so you just have-- you have the tech happening at exactly the same time that everything else is wanting to electrify. And so you just have this-- it's not only the AI tech challenge-- but it's also how do we compete for power in a world where we've sort of been tight for, or been getting tighter for 20 plus years. And so it's a unique challenge, just the confluence of all those things coming together at the same time. [The AI tech demand] is happening at the same time that we're asking really large industrial loads to decarbonize, and that is electrification mostly. Aaron Denman, Partner with Bain's Energy & Natural Resources Practice CRAWFORD: And how important are EVs, Aaron? Maybe it just depends on their adoption pattern or are they also another load expander? DENMAN: They are. I think the passenger vehicle is less of a concern because of charging habits and sort of when they'll happen. I think when you look at heavy duty loads, what you have, similar to the data center issue, is a concentrated amount of load that's happening at a location in a very rapid time frame. And so what that drives is a significant uptick in T and D infrastructure around those particular sites. And again, I think this speaks to, in a-- utilities who are trying to raise capital and finances and keep it affordable, it's just another demand for capital that they're going to have to navigate. The tech industry energy demand profile O'KEEFFE: Jue, do you want to jump in and just get the other side of the ledger a little bit, and for those who aren't spending all their time thinking about data centers, share a little bit the dynamics that you're seeing with our tech clients, as they address this exciting AI opportunity. JUE WANG: Yeah, sure.. I'll start by saying, you know, for those of us who spend too much time in tech, we thought adding semiconductor manufacturing capacity was hard and slow. And talking to Grant and our utility friends, we realized, oh, there's this industry that has never added capacity and had zero slack for the 10 years. So, you know, now that we have gotten through the little bit of the shortage of semiconductor and these GPUs are actually in data center, they're going to start to consume power. And this is not an industry that brings that up very quickly. So that was, I thought that was interesting start of this conversation, an important one. I will say, though, you already see from a lot of the questions in the chat that we're asking, from the tech side, I feel like there's no dispute about, we're seeing a fundamental shift in demand, the explosive demand driven by generative AI and other things, right. Now, I think we definitely would value a lot more baseline on the supply side. Like how much do we really have left? How far out are these alternative energy solutions? On the demand side, I will say, and I would invite my others to chime in, I will say, though, we have this sentiment a little bit in companies we talk to is that, you know, it is a problem. But there are levers we can pull on the technology side if this becomes a much severe problem-- things ranging from how can we squeeze more compute into a fixed, you know, chip power envelope, how do we link up the different chips in the data center, the interconnect, the cooling to make it more efficient, to how we optimize the software stack to reduce the energy consumption and compute needed. You know, it's like everyone is staring at building these new data centers and how much energy they're going to consume, now realizing how much existing data centers we have and how inefficient they are, and can we swap some of those out. So I would say, I think the general sentiment there is, it is a problem, but we have levers that we have not fully utilized. So we'd love to compare notes, you know, on the supply and demand balance side of how much of an issue are we staring into the next 12, 24 months versus the five to 10 years. But David, Anne, Arjun, Matt, please jump in. CRAWFORD: No, I think that's right. It's like we thought we were gapped by compute silicon. And now, it's actually power. And I do think it comes pretty quickly to nuclear is the long term option. Different nations have better access to that or variable access to that. And the short term option is gas. And gas may end up just dictating the location of these things, whereas historically, you've also cared about access and proximity to the user or proximity to cooling. I think now it's going to be in the short term, gas. But the other is the need for lobbying and just elevating our awareness. Depending on what you believe about the growth and use of these, you could arrive at a conclusion that you're already in a crisis or soon to be in a crisis or, you know, the world will, with the right demand signals, the world will adjust and get this going. HOECKER: Yeah. I just think it's super interesting because we've been looking at this for a while to figure out what's the rate limiting move. Because these forecasts for how much silicon and how much AI demand is going to be, there's a lot of variability. And I think that, especially coming off the chip shortage, people were asking, you know, is it semi fab capacity. It's going to be your rate limiting move. Right now, it's packaging capacity with co ops. But that is an actually, a pretty simple thing to fix in the spectrum of this. Then there's the affordability. Like are CSPs and everyone to just be able to spend as much as these forecasts are dictating. And then power I think is the one that is the longest term need to fix and probably actually the most rate limiting move out of all of them, even though at the beginning I think it was more focused on fab capacity. Backing capacity is the current rate limiter and then just overall affordability. We've been looking at this for a while to figure out what's the rate limiting move. Anne Hoecker, Leader of Bain's Americas Technology & Cloud Services Practice Data center efficiencies MATTHEW CRUPI: The one thing I would add on to Jue's point, and I think it's just we're early in things in this. And so a lot of the effort has basically been on, how do I get a GPU in someone's hands as fast as possible. And it hasn't been around like, are we designing for minimal power consumption around how we're allocating everything. And I think back to, hyperscaler data centers look completely different than enterprise data center. They're cooled differently. They're very thoughtful around that. And so I think there's going to be a world where we have to think differently around server design, around cooling design. We're going to make advances in liquid cooling. It's not to say that that's going to offset all of this. I think, you guys are right. Like it's the rate limiting factor. But I do think that there is efficiency to be squeezed out on the technology side. I was talking to one of my clients the other day. And he said that basically, they don't have enough space in their data centers to do the GPU build out that they want to. And so, they're splitting it over a couple of data centers. And they were basically discussing one, you know, if they were doing this from scratch, they would build a completely different data center. It would look very different, and that would help them get a lot more efficiency out of the same space. The second one is, they have a whole bunch of old servers that are in their data centers that are really inefficient. And if space and power and cooling become the constraints, they might actually hot-swap a bunch of stuff out to newer servers that have a much, much better energy profile than some of the stuff that's sort of still in their data center now. I think that that's actually unlikely to happen in the short term. But what's going to happen, though, is we're going to see the cycling of the installed base into these servers that are more energy efficient. And sort of, you'll see design changes and everything else. Again, does that give you 5% or 10% kind of efficiencies? Maybe, right. And then that doesn't help you with the long term, given the growth trajectory. But there is probably some to squeeze out there on the technology design side, the data center design side. We're going to see the cycling of the installed base into these servers that are more energy efficient. Matthew Crupi, Global Leader of Bain's Corporate Strategy & Finance solution Edge computing and power demand O'KEEFFE: Matt, to that question, there was an interesting question that Myles put in the chat. I'd be curious maybe to build off of that, which is that this push pull between centralization and more edge compute, where centralization might pull you more toward-- for energy supply, whereas for latency and other needs, you might want to push out more towards the edge. But like, how do these two things interplay with one another? CRUPI: Yeah. I was talking to a client last night, who told me something to the effect of, there are four customers in the industry right now that are buying about 80% of the GPUs. But there are 2000 enterprises that are buying one z, 2z servers to test this out. And so I think right now, you're seeing a lot of centralization As other folks ramp up capacity, you're going to see a little bit of difference and you'll see enterprise catch up. I think it'll be reasonably centralized, but there will be a little bit of both. I don't know. David, you look like you're about to say something. CRAWFORD: Nope. I was just listening to your deep insight on it. I was actually thinking through all the dynamics testing location are pretty important because you have this whole access to gas. And then if you think gas is a bridge technology to nuclear, then you're basically saying, I'm going to build this giant data center in a gas location. Then I'm going to subsequently come back and build a nuclear reactor in that location. Not everybody wants a nuclear reactor in their location, just because they haven't had a gas pipeline now. And so that's complex. And then I actually think it also depends a lot on the workload. Like we've been saying in the tech practice that the killer app of edge computing is going to be generative AI because we've been waiting-- you know, edge has been growing and we've been hoping to find the killer app that really makes it grow rapidly. Well, this is it. It's latency sensitive. It's data rich. It's, you know, it's computationally heavy. It fits. It fits all the criteria that we care about for an edge workload. But if we're talking about building-- if we're talking about doing a lot more training, a lot more training of models, that training activity can be highly centralized. It's not an edge workload. And it has huge implications for how big the data centers need to be, where they need to be located, et cetera. There's also this national security issue of like, you don't really want one giant data center they can hit with a nuclear weapon. You actually want five of these things with some redundancy and failover effects and stuff. HOECKER: And every country is going to want their own. The killer app of edge computing is going to be generative AI. David Crawford, Leader of Bain's Global Technology & Cloud Services practice The next three to five years O'KEEFFE: Maybe I'll take that. Grant and Aaron, I'd love to get your perspective, in terms of just how bad this could get over in the near term. I mean, a little bit where the conversation is going is, we now appreciate there's a bottleneck. That bottleneck is going to take time to play itself out because even if it's just gas, it's going to take some time and then nuclear and more time. But like, are we-- is this going to get really bad in the next three to five years? Like, are we going to hit a wall or because we're all appreciating this, like people will find a way to manage their way through. And there's optimization levers we can pull, both on the supply side, but also probably on the demand side, as tech companies think differently about their buildout. But how acute is this? DOUGANS: I mean, maybe a few thoughts for me. I mean, I'm excited to hear that there's efficiency opportunities in data centers. I don't know about those data centers at all. But, you know, if you think about other industries that have been through this journey, especially who've been growing rapidly, you know, efficiency has been a huge part of what has kept load down over a long period of time. So I think if that is an actual lever where there's opportunities, it could be really exciting. And you know, just one thing to think about maybe for the tech folks is, if you can take a hundred megawatts of demand out of a system, for example, with efficiency measures, you can get paid by the utility for doing that. And say, look, instead of-- we'll have to build milliwatts of new steel. If you can provide milliwatts by consuming less energy, we could actually potentially pay you for some of that. So I think there could be some really interesting plays around the efficiency side of things to help with this. Efficiency has always been the first lever that most load sources have gone to help mitigate these issues. The power equipment supply chain DOUGANS: I feel compelled to ask one more question. I'm going to pass it to my friend, Jim Wininger. Jim, can you talk about the turbine supply side of this? Do you have any concerns on the turbine side of things? WININGER: Yeah. I may have been living in the world of flat load growth for the last 20 years. And they're trying to peg exactly how big of a tailwind is this. And how the folks who are going to be building it and how they think about the timeline, because that's driving a lot of the pace and the nature of investments. But I think we're going to see shortages, frankly, because gas and round numbers is moving from a 30 gigawatt global market to a 40 gigawatt global market. And the players don't have the capacity to produce that, particularly given that supply chains haven't fully recovered from COVID. And so I think turbines are going to be a constraint. I actually think the bigger constraint is going to be transformers, which are already constrained. And you have a two year lead time today. And I hear anecdotes of hyperscalers going ahead and buying up big stocks of those to be ready to build whatever is in their plans. Good for them. It causes a lot of constraints on some of the other things we're talking about, in terms of building the right interconnect capability and building the right most effective grid for all. So I think this is a place where, collectively, there just need to be a lot more dialogues between really the suppliers and the users in the space, so that they all understand that the needs on the other side and the capabilities on the other side, so we can get to a little bit more efficient solutions right now. O'KEEFFE: All right. We'll leave it at that. Thank you all for joining. I got a ton of energy around this. Thank you.

At a Glance

The explosive growth of artificial intelligence comes with soaring demand for electricity—as much as 5% per year in the next few years—to power data centers
Utilities in the developed world haven’t seen electricity demand grow in more than a decade; many have been shuttering coal-powered plants to meet decarbonization goals
Meeting data center demand—expected to grow 10-30% over the next five years—poses a quandary for utilities and technology firms

Tech companies and utilities are at the forefront of a balancing act between technological advancement and environmental sustainability. The urgent push to decarbonize intersects with unprecedented demand for energy, accelerated by the AI sector's surging need for data center power. The resulting scenario presents a complex challenge: how to fulfill the burgeoning energy requirements of the tech industry without derailing efforts towards sustainability.

What follows is a “Future Sensing” panel discussion that brings together leaders from Bain & Company's technology, energy, and strategy practices. The discussion sheds light on some of the intricacies of satisfying the dual demands of sustainability and energy in the era of AI, offering insights into possible paths forward.

Read a transcript below.

DUNIGAN O'KEEFFE: The impetus for this conversation was the dual challenge we're facing between sustainability and energy of how do we both decarbonize, but also provide the energy we need for our world and for everything our world needs. These two are now coming together, and that's the topic of today's discussion.

I think what we're going to do is, we have a set of panelists who we've selected to kick us off. And I think with Grant to start, and then I will navigate through the other panelists.

GRANT DOUGANS: Sure. I'm happy to do that. I'll share a few thoughts and then I might kick it over to Jue to pick it up. You know, one of the things, as I've gotten involved with the future sensing effort, is just reflecting on the value of conversations across practices and listening to what our clients are saying.

So I'll start with the utility view. you know, for the longest time, I think it's important to remember over the last decade, growth has been flat.

We have not seen an increase in electricity sales really in the developed world for more than a decade. And so, you know, utility systems and the way the utilities operate is really geared for that world. And frankly, that's a good thing because building power plants is quite challenging to do. Getting gas pipelines done is very challenging to do.

LEARN MORE

Future Sensing

Future Sensing provides the insight to not only navigate turbulence, maximize opportunities, and reduce risks, but understand how the interplay of these dynamic forces will affect your strategy.

But when we think about the conversation with our utility clients, I think two things to highlight that are relevant for this. The first is, utilities are now starting to see the impact of both data centers for AI, but also IRA related commercial industrial load really starting to come in. And organizations, which historically have seen no low growth, are now being asked to increase the size of their generation fleets by 10, 20, 30%.

And, you know, when you think about, you know, power plants, these are pieces of physical infrastructure. They take three, four, five, six years to build. For the case of AI, I think it's really important for all of us to recognize that the type of load that data centers are requesting will only be served by fossil fuel generation resources today. That is the only way to speed this load.

Full stop. We do not have a viable nuclear industry in this country. We do not have-- renewables are not going to cut it to run data centers. And so, you're kind of just sitting there as a utility, you've just been through the stakeholder wringer.

And now we have, you know, the tech companies coming and asking us to build like seven gas plants. And that is a very challenging mission, both politically but also from a kind of feasibility perspective, even physically. And so the conversation that we're starting to have with our clients is, a) maybe we don't want all the data center load; and b) if we have data center load, how do we extract more from the data centers for the right to build in our service territory.

We have a lot of people that want to do a lot of stuff with our systems. You know, data centers are just one of many. And AI is going to be very challenging politically, and I imagine there's going to be some real throttling of growth that's going to be sort of downstream of that.

Organizations, which historically have seen low to no growth, are now being asked to increase the size of their generation fleets by 10, 20, 30%.

Grant Dougans, Partner and Leader, Energy & Natural Resources Practice IP Council

DAVID CRAWFORD: Thanks, Grant. What are the implications of that, in terms of, let's say for a second that you don't get to say, go away, to the data centers. If they're going to grow radically, what's the implications for growth behind the meter, powering, that kind of stuff? Will they end up all being built in areas where there's large natural gas supply? Are we going to see data centers built in Alaska? What about cooling? What are the implications?

DOUGANS: Yeah. I mean, if gas is the only way to power data centers, which I think is a valid assumption for the next five to 10 years in this country, then, you know, somebody, regardless of whether the utility builds it or whether, you know, Microsoft builds it, somebody is going to need to be building large numbers of very large gas turbines and will also need to be getting the gas from the wellhead to those turbines.

In our experience in the utility sector, we spent the last decade, trying to build even one gas pipeline is a absolutely miserable process that involves us being in court for a decade. And so I think, I imagine, Aaron, I invite you to comment Like proximity to gas is going to-- if you need to power this stuff with gas, proximity to gas is going to be important where you need to put this stuff.

CRAWFORD: And you start by saying, it is gas. So there's no oil based option or anything like that.

DOUGANS: Oil will be very expensive. And the air permitting issues with oil, are expensive. There's no way you're going to be building oil powered units to power this stuff.

AARON DENMAN: But I do think, you know, to your point, Grant, you have to look at where the pinch points and where there are alternatives. And so if you could build these in the Permian, to your point of building them in Alaska, where you're not having to cross state lines, you can avoid some of the regulatory challenges, when it comes to permitting a gas pipeline. And so it doesn't mean the pressure won't be there, but you're sort of taking out some of the tension in the ability to do that.

The role of alternative energy

O'KEEFFE: And Aaron or Grant, just for completeness, what's the role of battery technology in solar, just it doesn't scale, doesn't fly here? What's the constraint? Why doesn't that work?

AARON DENMAN: Yeah. So the data center load profile is relatively constant. And there's some flex that you can do with that, right. And so-- and let's just call it a renewable world. You're only going to be able to power with the sun, when the sun is shining in the middle of the day.

So you're going to have to massively overbuild for solar for that facility. You're going to have to put in a large number of batteries, and you're likely also going to need to build wind, if you want to power some of these things in the night. And so just the amount of physical infrastructure that would be required, and even in that setup, it doesn't guarantee you, if you have a relatively cloudy, less windy day for 24 hours, just the sheer amount of batteries that would be required would just be extraordinary.

And not to mention that many of those to be able to find a site where you could plow a data center of the scale we're talking about with all of this infrastructure, that also means a lot of transmission infrastructure and distribution infrastructure. And so just the complexity and the cost in a renewable plus storage world is incredibly challenging and costly to do that, relative to the gas alternative.

DOUGANS: Right. what's changed is, you know, five, 10 years ago, the system had a lot of extra capacity. The power system was not tight. It was quite loose. So we actually had a lot of extra gas plants and power plants around because we, as a utility sector, have been closing coal plants over the last five to 10 years for climate reasons, all of the excess capacity in the system has been sort of necked down for environmental reasons.

So there just isn't a lot of extra gas plants around you can contract with. There just isn't any-- there's no slack in the electricity system anywhere at this point. So anything you're going to-- if you need the power, you're going to find a way to build it de novo, And then on solar, for a one gigawatt data center, like run the math on how many square miles of land you're going to have to pave over with solar panels to power a gigawatt data center. It's a lot.

O'KEEFFE: It's not small.

Regional variations

ANNE HOECKER: And then, Grant, have you guys looked at how the US compares to Europe or Asia for this challenge? Right, because these AI data centers are going to be popping up everywhere. Have we looked at this outside the US and if the problem is worse?

DENMAN: You have similar challenges in many parts of the world, maybe not in the Middle East. But you do have some of the similar challenges in parts of Europe. I mean, the ability to have land access is already challenging for renewables. And then, you have some of the same interconnection and permitting challenges.

Europe is also structurally much more expensive, relative to the US. And so as you factor in alternatives, it's just the operation maintenance power costs of that are going to be significantly more expensive in Europe. I think you'll see some explore Europe. But if you have to build gas in Europe, the environmental dynamics are even more challenging in Europe. And so we think that will be quite, quite challenging.

Now Canada, I think if you're the Canadian government-- there's quite a bit of, you know, can we build additional nukes? There's some, you know, demand or encouragement for that. So Canada could be a choice. And then when you get into the Middle East, I think the solar option, just because of land availability and just the amount of sun and things like that, I think could be a viable choice in parts of the Middle East.

Maybe the other thing that's, I think, relatively unique, and I just put a finer point on Grant's intro comments, we've not had load growth for 20 years. Just it's been flat for 20 years. And you see, we're talking about the AI tech issue, which is both important. But it's happening at the same time that we're asking really large industrial loads to decarbonize, and that is electrification mostly.

You have, because of your structurally high energy cost, you have other industries, which are trying to relocate manufacturing capacity into the US because of low power prices, which is also driving load. You've got EV penetration, which I know is slower here recently. But-- so you just have-- you have the tech happening at exactly the same time that everything else is wanting to electrify.

And so you just have this-- it's not only the AI tech challenge-- but it's also how do we compete for power in a world where we've sort of been tight for, or been getting tighter for 20 plus years. And so it's a unique challenge, just the confluence of all those things coming together at the same time.

[The AI tech demand] is happening at the same time that we're asking really large industrial loads to decarbonize, and that is electrification mostly.

Aaron Denman, Partner with Bain's Energy & Natural Resources Practice

CRAWFORD: And how important are EVs, Aaron? Maybe it just depends on their adoption pattern or are they also another load expander?

DENMAN: They are. I think the passenger vehicle is less of a concern because of charging habits and sort of when they'll happen. I think when you look at heavy duty loads, what you have, similar to the data center issue, is a concentrated amount of load that's happening at a location in a very rapid time frame. And so what that drives is a significant uptick in T and D infrastructure around those particular sites. And again, I think this speaks to, in a-- utilities who are trying to raise capital and finances and keep it affordable, it's just another demand for capital that they're going to have to navigate.

The tech industry energy demand profile

O'KEEFFE: Jue, do you want to jump in and just get the other side of the ledger a little bit, and for those who aren't spending all their time thinking about data centers, share a little bit the dynamics that you're seeing with our tech clients, as they address this exciting AI opportunity.

JUE WANG: Yeah, sure.. I'll start by saying, you know, for those of us who spend too much time in tech, we thought adding semiconductor manufacturing capacity was hard and slow. And talking to Grant and our utility friends, we realized, oh, there's this industry that has never added capacity and had zero slack for the 10 years.

So, you know, now that we have gotten through the little bit of the shortage of semiconductor and these GPUs are actually in data center, they're going to start to consume power. And this is not an industry that brings that up very quickly. So that was, I thought that was interesting start of this conversation, an important one.

I will say, though, you already see from a lot of the questions in the chat that we're asking, from the tech side, I feel like there's no dispute about, we're seeing a fundamental shift in demand, the explosive demand driven by generative AI and other things, right. Now, I think we definitely would value a lot more baseline on the supply side. Like how much do we really have left? How far out are these alternative energy solutions?

On the demand side, I will say, and I would invite my others to chime in, I will say, though, we have this sentiment a little bit in companies we talk to is that, you know, it is a problem. But there are levers we can pull on the technology side if this becomes a much severe problem-- things ranging from how can we squeeze more compute into a fixed, you know, chip power envelope, how do we link up the different chips in the data center, the interconnect, the cooling to make it more efficient, to how we optimize the software stack to reduce the energy consumption and compute needed.

You know, it's like everyone is staring at building these new data centers and how much energy they're going to consume, now realizing how much existing data centers we have and how inefficient they are, and can we swap some of those out. So I would say, I think the general sentiment there is, it is a problem, but we have levers that we have not fully utilized.

So we'd love to compare notes, you know, on the supply and demand balance side of how much of an issue are we staring into the next 12, 24 months versus the five to 10 years. But David, Anne, Arjun, Matt, please jump in.

CRAWFORD: No, I think that's right. It's like we thought we were gapped by compute silicon. And now, it's actually power. And I do think it comes pretty quickly to nuclear is the long term option. Different nations have better access to that or variable access to that.

And the short term option is gas. And gas may end up just dictating the location of these things, whereas historically, you've also cared about access and proximity to the user or proximity to cooling. I think now it's going to be in the short term, gas.

But the other is the need for lobbying and just elevating our awareness. Depending on what you believe about the growth and use of these, you could arrive at a conclusion that you're already in a crisis or soon to be in a crisis or, you know, the world will, with the right demand signals, the world will adjust and get this going.

HOECKER: Yeah. I just think it's super interesting because we've been looking at this for a while to figure out what's the rate limiting move. Because these forecasts for how much silicon and how much AI demand is going to be, there's a lot of variability.

And I think that, especially coming off the chip shortage, people were asking, you know, is it semi fab capacity. It's going to be your rate limiting move. Right now, it's packaging capacity with co ops. But that is an actually, a pretty simple thing to fix in the spectrum of this.

Then there's the affordability. Like are CSPs and everyone to just be able to spend as much as these forecasts are dictating. And then power I think is the one that is the longest term need to fix and probably actually the most rate limiting move out of all of them, even though at the beginning I think it was more focused on fab capacity. Backing capacity is the current rate limiter and then just overall affordability.

We've been looking at this for a while to figure out what's the rate limiting move.

Anne Hoecker, Leader of Bain's Americas Technology & Cloud Services Practice

Data center efficiencies

MATTHEW CRUPI: The one thing I would add on to Jue's point, and I think it's just we're early in things in this. And so a lot of the effort has basically been on, how do I get a GPU in someone's hands as fast as possible. And it hasn't been around like, are we designing for minimal power consumption around how we're allocating everything.

And I think back to, hyperscaler data centers look completely different than enterprise data center. They're cooled differently. They're very thoughtful around that. And so I think there's going to be a world where we have to think differently around server design, around cooling design. We're going to make advances in liquid cooling.

It's not to say that that's going to offset all of this. I think, you guys are right. Like it's the rate limiting factor. But I do think that there is efficiency to be squeezed out on the technology side.

I was talking to one of my clients the other day. And he said that basically, they don't have enough space in their data centers to do the GPU build out that they want to. And so, they're splitting it over a couple of data centers. And they were basically discussing one, you know, if they were doing this from scratch, they would build a completely different data center. It would look very different, and that would help them get a lot more efficiency out of the same space.

The second one is, they have a whole bunch of old servers that are in their data centers that are really inefficient. And if space and power and cooling become the constraints, they might actually hot-swap a bunch of stuff out to newer servers that have a much, much better energy profile than some of the stuff that's sort of still in their data center now.

I think that that's actually unlikely to happen in the short term. But what's going to happen, though, is we're going to see the cycling of the installed base into these servers that are more energy efficient. And sort of, you'll see design changes and everything else.

Again, does that give you 5% or 10% kind of efficiencies? Maybe, right. And then that doesn't help you with the long term, given the growth trajectory. But there is probably some to squeeze out there on the technology design side, the data center design side.

We're going to see the cycling of the installed base into these servers that are more energy efficient.

Matthew Crupi, Global Leader of Bain's Corporate Strategy & Finance solution

Edge computing and power demand

O'KEEFFE: Matt, to that question, there was an interesting question that Myles put in the chat. I'd be curious maybe to build off of that, which is that this push pull between centralization and more edge compute, where centralization might pull you more toward-- for energy supply, whereas for latency and other needs, you might want to push out more towards the edge. But like, how do these two things interplay with one another?

CRUPI: Yeah. I was talking to a client last night, who told me something to the effect of, there are four customers in the industry right now that are buying about 80% of the GPUs. But there are 2000 enterprises that are buying one z, 2z servers to test this out.

And so I think right now, you're seeing a lot of centralization As other folks ramp up capacity, you're going to see a little bit of difference and you'll see enterprise catch up. I think it'll be reasonably centralized, but there will be a little bit of both. I don't know. David, you look like you're about to say something.

CRAWFORD: Nope. I was just listening to your deep insight on it. I was actually thinking through all the dynamics testing location are pretty important because you have this whole access to gas. And then if you think gas is a bridge technology to nuclear, then you're basically saying, I'm going to build this giant data center in a gas location. Then I'm going to subsequently come back and build a nuclear reactor in that location.

Not everybody wants a nuclear reactor in their location, just because they haven't had a gas pipeline now. And so that's complex. And then I actually think it also depends a lot on the workload. Like we've been saying in the tech practice that the killer app of edge computing is going to be generative AI because we've been waiting-- you know, edge has been growing and we've been hoping to find the killer app that really makes it grow rapidly.

Well, this is it. It's latency sensitive. It's data rich. It's, you know, it's computationally heavy. It fits. It fits all the criteria that we care about for an edge workload. But if we're talking about building-- if we're talking about doing a lot more training, a lot more training of models, that training activity can be highly centralized. It's not an edge workload.

And it has huge implications for how big the data centers need to be, where they need to be located, et cetera.

There's also this national security issue of like, you don't really want one giant data center they can hit with a nuclear weapon. You actually want five of these things with some redundancy and failover effects and stuff.

HOECKER: And every country is going to want their own.

The killer app of edge computing is going to be generative AI.

David Crawford, Leader of Bain's Global Technology & Cloud Services practice

The next three to five years

O'KEEFFE: Maybe I'll take that. Grant and Aaron, I'd love to get your perspective, in terms of just how bad this could get over in the near term. I mean, a little bit where the conversation is going is, we now appreciate there's a bottleneck. That bottleneck is going to take time to play itself out because even if it's just gas, it's going to take some time and then nuclear and more time.

But like, are we-- is this going to get really bad in the next three to five years? Like, are we going to hit a wall or because we're all appreciating this, like people will find a way to manage their way through. And there's optimization levers we can pull, both on the supply side, but also probably on the demand side, as tech companies think differently about their buildout. But how acute is this?

DOUGANS: I mean, maybe a few thoughts for me. I mean, I'm excited to hear that there's efficiency opportunities in data centers. I don't know about those data centers at all. But, you know, if you think about other industries that have been through this journey, especially who've been growing rapidly, you know, efficiency has been a huge part of what has kept load down over a long period of time.

So I think if that is an actual lever where there's opportunities, it could be really exciting. And you know, just one thing to think about maybe for the tech folks is, if you can take a hundred megawatts of demand out of a system, for example, with efficiency measures, you can get paid by the utility for doing that. And say, look, instead of-- we'll have to build milliwatts of new steel. If you can provide milliwatts by consuming less energy, we could actually potentially pay you for some of that.

So I think there could be some really interesting plays around the efficiency side of things to help with this. Efficiency has always been the first lever that most load sources have gone to help mitigate these issues.

The power equipment supply chain

DOUGANS: I feel compelled to ask one more question. I'm going to pass it to my friend, Jim Wininger. Jim, can you talk about the turbine supply side of this? Do you have any concerns on the turbine side of things?

WININGER: Yeah. I may have been living in the world of flat load growth for the last 20 years. And they're trying to peg exactly how big of a tailwind is this. And how the folks who are going to be building it and how they think about the timeline, because that's driving a lot of the pace and the nature of investments.

But I think we're going to see shortages, frankly, because gas and round numbers is moving from a 30 gigawatt global market to a 40 gigawatt global market. And the players don't have the capacity to produce that, particularly given that supply chains haven't fully recovered from COVID.

And so I think turbines are going to be a constraint. I actually think the bigger constraint is going to be transformers, which are already constrained. And you have a two year lead time today. And I hear anecdotes of hyperscalers going ahead and buying up big stocks of those to be ready to build whatever is in their plans.

Good for them. It causes a lot of constraints on some of the other things we're talking about, in terms of building the right interconnect capability and building the right most effective grid for all. So I think this is a place where, collectively, there just need to be a lot more dialogues between really the suppliers and the users in the space, so that they all understand that the needs on the other side and the capabilities on the other side, so we can get to a little bit more efficient solutions right now.

O'KEEFFE: All right. We'll leave it at that. Thank you all for joining. I got a ton of energy around this. Thank you.