How to Win with AI
Decision 4: Technology Architecture
Decision 4: Technology Architecture
Own enterprise orchestration.
By Sarah Elk, Chuck Whitten, Hernan Saenz, Gene Rapoport, Nicolas Bloch,
Pascal Gautheron, and Anne Hoecker
How to Win with AI
Own enterprise orchestration.
By Sarah Elk, Chuck Whitten, Hernan Saenz, Gene Rapoport, Nicolas Bloch,
Pascal Gautheron, and Anne Hoecker
The most important technology decision you will make in your AI transformation is not which model to use. Models are changing so rapidly that any specific choice will be obsolete within months. Most of the stack you can buy. But a few layers are where your competitive advantage is encoded and where your risks are controlled, and those risks you have to own yourself. We call this enterprise orchestration: The layers where your company manages its agents, tools, and skills as first-class enterprise assets, governs what those agents are allowed to do, and turns its proprietary data into reusable agent-ready assets, rather than ceding to a scattered collection of vendor-supplied tools you cannot change, connect, or hold accountable.
Owning your enterprise orchestration means you do two things extremely well:
If you cede that layer to a single vendor, you are handing them architectural control of your AI estate. Your agents will be limited to what that vendor’s platform allows. Your ability to switch models, add new capabilities, or integrate with new data sources will depend on their roadmap, not yours. Owning the enterprise orchestration layer is the technology equivalent of owning your supply chain; it is the strategic control point that determines how much flexibility and advantage you can extract from everything else.
Asset management, governance, and security aspects of good enterprise orchestration are ongoing throughout the software and agent life cycle process inside a company. It is not something that is built once and left; it is living and breathing as builds progress and capabilities improve.
Ramp, the comprehensive financial operations platform for businesses, shows what proprietary intelligence looks like on the ground. After reaching 99% adoption of AI tools across the company, the team realized most employees had plateaued—not because the models were inadequate, but because there was no shared infrastructure to connect tools, propagate workflows, or carry context across sessions. The team built Glass, an internal AI productivity layer where one person’s breakthrough rapidly becomes the company’s baseline, and the system accumulates persistent memory across the organization. As the company’s leaders put it, internal productivity is a moat, and an organization does not hand its moat to a vendor.
At this point, a reasonable CEO will ask why anyone should build infrastructure that will be obsolete within months. The specific orchestration code, the agent designs, the integration patterns, and even today’s architectural conventions are all moving targets. The objection is fair—and it is the wrong reason to wait. The artifacts of an AI build are partly disposable by design, but the assets the build produces are not. Going through an actual orchestration effort is how a company builds out its semantic layer, hardens its registries and governed gateways, develops the engineering muscle to operate probabilistic software at scale, accumulates change-management capability, and earns the experience curve that no shortcut replaces. None of those efforts is the code itself. All survive every rewrite, and each one makes the next iteration faster. The companies that wait for the technology to stabilize before they invest are not protecting themselves from obsolescence; they are allowing the durable assets of an early build to accrue to their competitors.
Agentic AI is a fundamentally different class of software, and the differences carry serious operational consequences. Traditional enterprise software is deterministic: same input, same output, every time. Agents are probabilistic, stateful, and continuous: Behavior can drift as data and underlying models evolve. The most critical distinction is that traditional software fails safely; agents don’t. In multi-agent systems, an orchestration-level error propagates through every downstream agent before anyone detects a problem. Cascade risk is among the most underappreciated challenges in enterprise AI.
Enterprises must also distinguish functional correctness from adversarial resilience. Golden test suites and shadow-mode testing confirm output quality on anticipated inputs. Still, they don’t reveal how an agent behaves under active manipulation by techniques such as injected prompts, tool abuse, data exfiltration, or privilege escalation across agent handoffs. Both assessments are required before any agent with meaningful system access reaches production, and threat model results should inform the trust policy that the runtime enforces.
Governance must operate at the boundary—every tool call, data source, external connection, and agent delegation—because the LLM’s internal reasoning is inaccessible.
Three mechanisms make this work:
The economic argument for a shared platform is as important as the technical one. Without shared infrastructure, every agent team duplicates the same foundational work, data preparation, system integrations, governance mechanics, evaluation pipelines, and deployment automation. The third team spends the same months as the first team, solving problems the first team already solved, producing results that the fourth team cannot reuse. With a governed platform, data models built for one domain are reusable across adjacent domains. Tool integrations built for one use case are available to every subsequent use case in that domain. Skills are the encoded procedural expertise that tells an agent how your organization actually operates, and they improve with every calibration run and benefit every agent that uses them. The organization accumulates operational knowledge as a compounding asset rather than as a series of disposable projects.
What is required to do enterprise orchestration well?