Winning in the Agentic Era: A Conversation with Andrew Ng

Full Transcript

Part 1: How is agentic AI different from generative AI?

CHUCK WHITTEN: I'm incredibly excited to be having a conversation with Andrew Ng, our strategic partner and my friend, about the topic of agentic AI. It is hard to overstate the impact that agents are going to have on the trajectory of AI.

Over the last few years, we've all become accustomed to AI as a co-pilot, making all of us and our businesses more productive. But now we have AI systems capable of taking autonomous action and theoretically running entire parts of our organization in the future. Now, we're early in the curve, but the potential for businesses is enormous.

Let's start with something that I think is a fun fact for everyone. You coined the term "agentic." Maybe why agentic, and how did that come to be?

ANDREW NG: Yeah, a few years ago, I saw a trend coming where we would not just be using large language models to generate an output, but we would instead be building these things into complex, multistep agentic workflows or multistep workflows. And I felt that there would be things that are slightly autonomous, things that will be highly autonomous, and that using the term "agentic" to describe this range of things we would want to build in businesses and elsewhere would be an important new concept.

And I think since then, the term "agentic AI" has certainly taken off much faster than my expectations. But the good news is the business value from building agentic AI is also rapidly rising.

CHUCK WHITTEN: I think the distinction between generative AI as a copilot and agents is really, really important. So maybe we should start a bit basic. As companies start to think about evolving their application of AI from generative to agentic, what are the things they have to think about? What's fundamentally different about an agent from the way we've been applying AI over the last couple of years?

ANDREW NG: Over the last year or two, AI has become more autonomous and able to take more steps of action by itself, or more steps of reasoning and/or action. And that's the heart of what an agentic workflow is.

It's not just something that is a copilot, although that has a nice role to play too. Instead, businesses are looking at very complex business process workflows and figuring out what steps can you automate. What steps can augment humans, and designing these complex processes with a mix of human and AI, they can deliver much more value than what we saw during the copilots era of generative AI.

CHUCK WHITTEN: It sounds like it's as much a business problem to solve as it is a technology problem. Would you agree with that?

ANDREW NG: Frankly, it feels more like a business problem than a technical problem. Although I think it is, as you point out, absolutely both. I feel like businesses still underestimate both the value, but also the complexity and the work that lies ahead of us to look at what our businesses do.
But the practical business reality is, there's a lot of work needed to do the data plumbing, to make the systems reliable, figure out when to put human in the loop, to make these things build evaluations so you can trust these systems. I wish we'd be done in a few years. But I think I should take us many years to build these agentic workflows. But the good news is, for many businesses, including quite a few Bain clients, [they’re] starting to see very strong growth in value, and we're just at the beginning of that.

Part 2: What’s needed to realize business value with agentic AI?

CHUCK WHITTEN: So, you know, what we see inside of clients is agents are pretty simple today. It's sort of a specific task, sort of heavy human intervention to make it work. If we're honest, the technology sometimes frequently struggles to perform. So when you see agents struggling in applications, what are some of the inhibitors? What makes applying agents difficult inside a company?

ANDREW NG: I think the top one is reliability, that stereotype that the proof of concept can be thrown together in a week, but then it takes months to make it reliable and enterprise grade. That stereotype is totally true. But what I'm seeing is a lot of the time, not all the time, but very often, if you put in the work to evaluate agentic workflow, many of them can be made robust, but it is more work to do so than maybe is widely appreciated.

And then the other thing I'm seeing is there's been a lot of "let 1,000 flowers bloom" type of strategy. And that's great. There's a lot of value to that. But between pure bottom-up innovation and top-down innovation, I find it often takes that top-down view to see the entire business process workflow.

So you're not taking a small piece of it and optimizing it to get that 3% efficiency gain, which is nice. But it takes that broader view to understand how the business actually operates, so that maybe you do take one piece out of many pieces and employ an agent to do it for you. But the business value is not from that 3% or 5% efficiency gain. It's from rethinking that entire workflow that then gives you the massive productivity gains.

CHUCK WHITTEN: We talk to clients a lot about that, which is that one of the challenges of this technology is you can't apply it everywhere. And so you have to make choices, and choices usually come back to strategy. Where are the handful of places inside my business I should be sort of applying agents?

ANDREW NG: I think a lot about bottom-up versus top-down innovation. So take the example that I know you're familiar with, with AI for underwriting. Underwriting alone, there are multiple steps. And you've got to get the initial loan application, maybe approve it or not. And then you have due diligence, do the final docs, manage, service loan, whatever. But so to create value, there are multiple steps.

So say someone builds an AI agentic workflow to accelerate the initial approval or denial. What some businesses will do is look at that and say, oh, we can take out some costs of this long process. And that's somewhat interesting. Let's do that.

But what is much more interesting is not this point improvement. It is if the business says, you know what? We cannot get back to the customer in 10 minutes instead of a week. So this is a very different product, and this drives growth.

And one of the challenges with AI value creation, and why the "let 1,000 flowers bloom" strategy mostly hasn't worked out is a lot of bottom-up innovation takes one step in the process, makes it more efficient, and that is not transformational. But instead, the more transformative stuff is when you do take one step and use AI for it. But you decide either let's do this 100 times faster, or the other pattern I see is, let's do this 100 times more. So you're not just getting cost savings, but driving real business growth, and that's where the value will be.

CHUCK WHITTEN: How do you think about that balance between top-down and bottom-up?

ANDREW NG: Unless either the bottom-up innovation-- say that engineer or product manager or whatever, that person has a broader view to understand how this transforms not one step, but a broader piece of business, and sometimes that takes a top-down executive to spot to identify that this is a flower, to really nurture this one, this would sprout to cover the entire field or something. But that's what makes this hard.

I feel like it is very promising. I know when Bain's gone in, and when AI Aspire has gone in, we often spot these opportunities. But spotting them really takes that broader view, which a bottom-up or a top-down motion could succeed in. But it takes someone that understands how the business actually creates value to spot and allocate resources to execute on these bigger opportunities.

Part 3: How does agentic rewire the organization?

CHUCK WHITTEN: I have never met an executive that has said, my data is in good shape. Everyone is frustrated with fragmented, siloed concern about security, privacy. So how do you think about-- you mentioned data warehousing and data lakes as the sort of old way of thinking about addressing data. How do we think about getting data fit for purpose for these applications in the AI world?

ANDREW NG: I usually recommend to businesses to not preemptively try to build the perfect data because it never happens. Even leading AI companies in the world where I've worked in or I know a lot of friends think their data is messy, siloed, not good enough. So everyone thinks that. So if someone's listening to this, they think the data is messy, it's OK. I think everyone else does.

And so cleaning up data is a never-ending journey. In fact, I've also seen businesses that thought they would get to good data in raw. But guess what? You acquire another business, and that business has different data and a different cloud, and it's messy again, so it never ends.

But I find that if we have a point of view on something really valuable to build, then focusing on building that allows us to pull the data in from forward, and we incrementally improve the data to support the specific applications. And having a team that knows how to have a North Star for where to get the data, even though we will never get there, while letting concrete applications pull us forward in that direction, seems to be the best recipe for making incremental gains, rather than preemptively spending two years to make the data perfect before we build valuable applications. This doesn't work.

CHUCK WHITTEN: So start with the business value, the application, and then decide what data you need. Don't start the data project to nowhere.

ANDREW NG: Yeah. And as you work on the data project, if you have a point of view on the role-based access controls, the data governance to framework, then that lets you move the data forward in support of one application. But that sets it up to be useful for other applications as well.

CHUCK WHITTEN: And I think both of us I would categorize as technology optimists and believers in human exceptionalism. And so the application of agents always are going to have humans on the loop, to some extent. Maybe two levels of question-- one tactical, one bigger picture. Tactical, how do you think about human control of agents in the application today? And then maybe bigger picture question, how do you think about the future of work? How do you think about what humans are only going to be able to do as we go from simple agentic to much more complex agentic?

ANDREW NG: So I think humans will always control AI. The lack of control of AI is one of the things that has been overhyped. It turns out that, well, there's almost nothing in the world that we control perfectly. So, frankly, when I get into an airplane, the pilot doesn't perfectly control the airplane because winds will buffet it around. But we can control airplanes well enough, but not perfectly. But we control airplanes well enough that we get an airplane without really worrying about landing safely.

So too it is with AI. We can't control AI exactly. And it will sometimes be buffeted around by random factors. But with the right engineering, we can control them well enough for most applications. So I feel like a number of things in AI have been overhyped, but that we can't control AI is one of the things that has been overhyped. I'm not that worried about that, but we do need good engineering.

And for humans, the nature of work-- I am really excited, as humans get better tools through AI, to have this massive improvement in productivity. Because it turns out that if every human can have an army of interns working for us, boy, how much more productive and effective will people be? And maybe how much more fun will all of our jobs be? But my worry, too, is our ability to train and upskill people fast enough for the age of AI.

So that pithy saying that AI won't replace a human, but someone who uses AI will replace someone that doesn't, that's largely true. And if we're unable to help businesses upskill their workforces fast enough, that will be challenging for a lot of people.

And just for transparency, there's one asterisk to that. So it turns out that if you look at most jobs, my economist friends, and I know Bain's done analysis too, if you look at task-based analysis of jobs-- take a job, break it down into tasks, figure out what AI could do--often, AI could do 30%, 40% of most jobs. And that means you still need a person to do 60%, 70%. And people that know AI will be much more productive. So it'd be great for people.

CHUCK WHITTEN: I would even say it maybe a little more provocatively, which is I think over the last few years, we've sort of ignored the talent agenda. We've taken this technology, and we've been heavily productivity-focused. Can we get it to work? And then we've dealt with the human implications.

You're sort of challenging, we need to upskill. We also need to think about a future I think that's coming where, as you said, well, humans and agents and maybe robots are all working sort of as collaborators side by side. That's a big leadership challenge.

ANDREW NG: I know we've often seen in our collaborations, leadership comes from the top. And it's when the CEO or C-suite embraces AI, takes the time to think through strategy, that positions the company leadership to bring the team along.

One thing I've actually seen, a challenge to the COs, frankly, is many COs talk about AI in earnings calls, in public, and so on. In addition to that, I think it's worthwhile for the CO to spend at least a few hours to really learn about the technical aspects of AI.

Because you and I are both technical people, we see that basic understanding of technology is critical for driving strategy because what has changed in the world is technology. And understanding, spending just hours to really understand that, allows leaders to make technical decisions.

Part 4: What does it take to lead in the agentic era?

CHUCK WHITTEN: What should companies be looking at, as they look at horizon two and three of where AI is headed?

ANDREW NG: I think people still underestimate the importance of the voice software stack. If we look at the sci-fi movies of the future, almost none envisioned humans sitting at keyboards typing anything. You want to talk to computers. Very natural. We evolve. Humans speak much more naturally than we write, and I think the voice tech is just getting there. I think it will take off soon.

People talk about physical AI. Progress is rapid. I think it will take a little bit longer than most people think for humanoid robots to become very widespread, but progress is rapid. I think there's a lot to do still. And I wouldn't underestimate just that maybe now widely known, but not yet boring, work ahead to just build a lot more single-agent and multi-agent systems to partner alongside humans.

CHUCK WHITTEN: So we're in the agentic era for the foreseeable future, you would say?

ANDREW NG: I think we'll be in the agentic era. And I wish we could be done in one or two years. I think 10 years from now, we'll still be figuring out new agentic workflows to build. Oh, but the good news is, because AI models are becoming more intelligent, it's actually becoming easier over time to build agentic workflows.

I find that the code we need to write to build a lot of agents a year ago was much more than what we need to do now. And in fact, to be technical, the stuff we built a year ago still works. But if we re-implement it now, we could do it with less code. We can rip out a lot of the scaffolding and let AI do more by itself without as much careful human steering. So that's been great. The cost of building AI agents is falling.

CHUCK WHITTEN: Andrew, if I go back a few years, it was there was this lightning bolt that came down from boards at CEOs, which is what are you doing about generative AI? And it sort of started the, as you've described, the "1,000 flowers blooming," lots of action, lots of proof of concepts. Over the last 18 months, the dominant discussion has been, are we getting ROI from this technology? Why do companies get stuck?

ANDREW NG: So it turns out that it is true that the vast majority of businesses have not yet seen massive ROI from AI yet. At the same time, when we talk to AI teams, we see this. The AI teams are all really busy and are seeing really rapid growth and massive demand.

And the reality is, AI is in the very early stages, and so the penetration is very low. But if you look at how the AI teams are scaling and just overwhelmed with work, we've seen this story before many times in tech, which is maybe it's a small percentage impact now. But when the growth is rapid, give it a few years. You will see what it compounds to. And the right time to do it is now, not after it's already obvious and dominating the market, maybe because the competitor embraced it earlier.

In terms of what I think businesses should do, I would say two things. One is it is really difficult for businesses to identify, maybe in a top-down fashion, the really valuable use cases to allocate significant resources behind to really make it work. It is true that we can run pilots and proof of concepts really inexpensively.

But beyond a certain point, you can't do everything in a tiny experimental budget. At some point, the experiments have to graduate, or initiatives have to be identified where we make a decision to bet significantly on them. And that process of finding the meaningful bets is difficult because it takes marrying the business knowledge with a good understanding of the technology to select that. So I think many businesses have not yet done that well.

And the other thing is still the people change management. Upskill the team, bring the team along, and people change management process. I know some people don't like the term. I've actually been-- it's been amazing to see, every business that I know that just did the standard people change management playbook that you learn about in MBA programs, it mostly works with some patience, a lot of work. But doing that well is easy on paper, really hard in practice.

CHUCK WHITTEN: Yeah. We start talking about bilingual teams and the importance of partnership, Andrew, between the technical side and the business side. Because one thing that strikes us about the technology is maybe unlike prior technology disruptions, this technology requires you to reimagine a business process or change a customer value proposition to get value from it, and that's a team sport.

ANDREW NG: Yeah. I mean, even though I'm a technologist, when we work with businesses, I really prefer the process selection to be business led, not technology led. Because as much as I love the technology, ultimately to create value, it has to be a business transformation that's executed through AI.