Article

Innovation with Impact: Redefining Upstream R&D

Three principles can help companies deploy at speed and realize far more value.

By Dave Rennard, Brian West, and Jim Claunch

min read


Even as they grapple with investor pressure for cash returns, energy transition uncertainty, digital disruption, and geopolitical volatility, oil and gas companies that engage in upstream exploration and production face an additional dilemma: how to allocate capital across the competing priorities of dividends, M&A, and project capex. That puts R&D funding—long forced to earn its place in the capital stack—at risk. Add to that the propensity for IT to reach into the R&D budget as it looks to build out new digital and AI capabilities, and suddenly innovation programs that can’t demonstrate a direct link to current asset performance are being cut. It’s worth remembering that, for decades, upstream R&D helped international oil companies (IOCs) secure access, build credibility, and prove the feasibility of large-scale projects. It also supported immediate (but challenging) field-focused solutions. But the innovation cycle was slow, often spanning 30 years from lab to full-scale deployment. And, ironically, the same firms that invested the most heavily in R&D were often the most risk-averse in deploying new technology at scale. That model no longer fits. Technology is now global and democratized. Digital platforms, ecosystem partners, and AI-enabled tools are increasingly available to many entities beyond traditional upstream energy firms. The differentiator is no longer who invented something but who can apply and scale it at speed. That’s why we believe the role of upstream R&D must be reframed for a new era, one in which value delivery is paramount. The mandate has shifted from prestige and portfolio enhancement to the disciplined pursuit of near-term impact. Effective R&D organizations will start by aligning with the business need, establish ways of working that focus on value creation (including a clear path to commercial deployment), and apply R&D consistently to inform a continuous improvement cycle. In short, the role of the oil and gas operator is changing from inventor to integrator. This shift demands a new operating model that facilitates: Ecosystem management and integration. Access external innovation and apply it to fit-for-purpose internal applications. Portfolio discipline. Direct capital and talent toward scalable, high-value opportunities. Build/partner/buy clarity. Optimize value and speed over control or legacy advantage. The changing role of upstream R&D Historically, R&D was a core IOC capability that signaled strength to host governments and helped de-risk frontier projects. Much of it was bespoke, focused on specific assets, and executed in isolation from the business. As a signal of capability, the value of innovation was intrinsic. Research projects often had only a loose connection to business needs and no real path to value. Researchers fell prey to common traps: “If we invent this, we can cheaply acquire the assets since no one else is bidding on them today.” But that stance ignores the fact that new technologies often fail to unlock an advantage in acquiring and developing reserves, dramatically underestimates practical oil and gas development schedules, and risks tying up significant capital today for leases that can’t be developed until the technology matures years down the road. Many technologies stalled due to a difference in risk tolerance between R&D and the businesses: Companies invented a new technology but did not have the risk profile to commercialize it until a competitor profited from it first. Many R&D portfolios still drift—detached from core business strategy, built around pet projects, or scaled too slowly to matter. As innovation shifted from lab-based experimentation to simulation and systems integration, barriers to participation dropped and the model began to fracture. External innovation ecosystems matured as service companies scaled technology for the industry, academic and public-sector partners coinvested, and digital and AI advanced. As a result, oil companies that could adapt and apply the latest innovations gained an advantage over those that remained focused on invention. IOC contribution to overall energy innovation has steadily declined despite rising industry patent activity Notes: Analysis includes select companies across IOCs, NOCs, and OFS players, spanning patents in upstream, midstream, downstream, and equipment services segments; change in share of total patents for each category is displayed at two-year intervals rather than annually Sources: GlobalData; Bain analysis Simultaneously, efforts to better quantify the value of the R&D portfolio drove researchers to link their work to business impact, reinforcing the focus on applications in the field. Access to proprietary data sets ensures that operating companies will always have a role in the front end of innovation—in fact, it’s worth noting that IOCs have not scaled back investment in R&D, which remains at about 0.35% of spend. But the last mile of innovation, applying the technology to the real conditions of an operation, is where value is ultimately realized. That’s why specific areas of investment have shifted—from invention to implementation, from the lab to the field, and, increasingly, to software development. This does not diminish the importance of science. Subsurface modeling, completions design, flow assurance, and materials science all still matter. But the winners will be those that tie R&D directly to business outcomes, reduce time to impact, and scale proven solutions enterprise-wide. For IOCs, especially in the upstream, that means reducing costs across the portfolio (not just for the marginal barrel), expanding economically recoverable reserves, and de-risking development. This value derives from impacts to assets and often has little to do with licensing revenue or even IP exclusion. For national oil companies (NOCs), the mandate further includes local capability building, technology transfer, long-term strategic technology independence, and national development goals. The challenge is to serve those objectives while still enhancing competitive advantage. For oilfield services companies (OFSs), innovation must be product-centric: modular, repeatable, and scalable. Changing R&D needs across the exploration ecosystem Source: Bain & Company Three principles for deploying at speed These shifts are visible across the ecosystem, affecting IOCs, OFSs, and NOCs alike. Each group is rethinking how it defines success, interfaces with others, and manages innovation. One common thread is that, as innovation cycles shorten, deploying at speed becomes a top priority. Three guiding principles can help all of these players achieve that. 1. Align R&D with business outcomes. Across archetypes, one challenge is universal: ensuring that technology solves real problems and generates real value. Historically, upstream R&D has suffered from weak linkage between problem definition and solution development. Central groups, particularly in large IOCs, often generated innovations that lacked a clear connection to operations and might even be disconnected from the portfolio. But leading companies now derive research focus directly from business strategy, and some even run R&D as an organizational child of strategic planning. Key elements of their approach include: codeveloping portfolios with the business, grounded in real asset plans and mutual accountability between the business and the R&D lead; aligning on practical deployment and critical technical and nontechnical risks; ensuring each effort addresses clear pain points or future commercial needs; and balancing incremental improvement with bold, strategic bets. Tighter alignment isn’t just best practice—it’s the clearest path to improving the ROI of R&D investment. 2. Focus on ways of working and portfolio management. Leading companies have reconfigured their R&D operating models around value creation, with effective governance structures ensuring that technologies are stewarded on a value basis. KPIs are aligned to business outcomes, and incentives and career progression are linked to value rather than project technical milestones. Technical champions are identified or embedded within assets, and the organization pulls together to achieve a new technology’s value potential. Scaling is not simply an exercise in handing off from a research lead to a project manager or a pilot to a business development team. It’s now a core capability enabled by talent and tools and—critically important—supported by ways of working that focus on value. Technologies are stewarded with the entire scale-up process in mind at each step; customers, interfaces, and applications are defined, tested, and retested as new learnings are discovered. Almost all projects organizations have discovered that a faster path to value is technology qualification. Many high-impact innovations come from recombining existing tools, and these solutions are often far faster to deploy than brand-new technologies, given internal risk management systems. The essence of rigorous portfolio management is clarity: what a technology is worth, what the risks are, what are the next best alternatives, how it will be deployed, and when to walk away. R&D portfolios should be governed like exploration pipelines. Each initiative should have: a defined value hypothesis; a scale-up plan and targeted deployment roadmap; a quantified business case tied to specific assets; clear off-ramps if progress stalls; a plan for value assessment after implementation or post-mortem analysis after wind-down; appointed business-side “product owners” to steer solutions from pilot to rollout; and early integration of commercial and change enablers (training, standards, support models). Ways of working must also enable cycling of the portfolio. Ending low-potential projects isn’t failure. It’s progress—and a learning opportunity. 3. Measure, analyze, and readjust as needed. Every technology option should have a clear value hypothesis, a quantified business case, and a credible pathway to deployment. Prospective valuation (including risked expected value, time to impact, and dependency mapping) helps prioritize early investments, while retrospective valuation ensures that actual business outcomes inform future decisions. This dual lens enables organizations to invest boldly in high‑potential opportunities while terminating projects that no longer merit capital. Future decisions must be made with prospective valuation techniques, but these techniques should be refined through rigorous retrospective analysis, sometimes years after decisions are made, to inform adjustments to process, decision making, and prioritization. The discipline of portfolio governance is particularly important in the digital era, where shiny objects proliferate and the temptation to chase novelty can overwhelm operational focus. But the same principles apply across subsurface science, drilling hardware, completions design, production chemistry, and subsea engineering. Innovation beyond digital and AI Although digital technologies often dominate conversations around innovation, upstream value continues to be driven by a broad range of advances in other domains. Breakthroughs in reservoir characterization, drilling automation, completions design, flow assurance, integrity management, and materials science often deliver equal or greater returns than digital solutions, and these innovations are harder to source outside the sector. That said, embedding digital and AI solutions within these highly technical, sector-specific areas can produce considerable value. This requires combining deep sector knowledge with cutting-edge digital tools and adjusting the talent mix. Digital fluency is now a requirement, not a specialization, although success will continue to demand technical depth in traditional engineering fields. Getting there from here To deliver impact, innovation must be embedded in a company’s operating model. High-performing firms combine scientific, commercial, and organizational strength by: aligning R&D with business outcomes; establishing ways of working focused on value creation; and measuring value and connecting back to principles 1 and 2 through feedback cycles. R&D still matters—but only when guided by practical application and strategy. Differentiation through R&D no longer comes from the resulting innovations alone; it comes from deployment and business impact. Leaders will identify the problems that matter; access or adapt the right technologies; and deploy them with speed, consistency, and measurable value. That’s the new mandate for upstream R&D. The companies that embrace it will define the next era of performance in oil and gas. The goal is not invention. It’s value.

Even as they grapple with investor pressure for cash returns, energy transition uncertainty, digital disruption, and geopolitical volatility, oil and gas companies that engage in upstream exploration and production face an additional dilemma: how to allocate capital across the competing priorities of dividends, M&A, and project capex. That puts R&D funding—long forced to earn its place in the capital stack—at risk. Add to that the propensity for IT to reach into the R&D budget as it looks to build out new digital and AI capabilities, and suddenly innovation programs that can’t demonstrate a direct link to current asset performance are being cut.

It’s worth remembering that, for decades, upstream R&D helped international oil companies (IOCs) secure access, build credibility, and prove the feasibility of large-scale projects. It also supported immediate (but challenging) field-focused solutions. But the innovation cycle was slow, often spanning 30 years from lab to full-scale deployment. And, ironically, the same firms that invested the most heavily in R&D were often the most risk-averse in deploying new technology at scale.

That model no longer fits. Technology is now global and democratized. Digital platforms, ecosystem partners, and AI-enabled tools are increasingly available to many entities beyond traditional upstream energy firms. The differentiator is no longer who invented something but who can apply and scale it at speed.

That’s why we believe the role of upstream R&D must be reframed for a new era, one in which value delivery is paramount. The mandate has shifted from prestige and portfolio enhancement to the disciplined pursuit of near-term impact.

Effective R&D organizations will start by aligning with the business need, establish ways of working that focus on value creation (including a clear path to commercial deployment), and apply R&D consistently to inform a continuous improvement cycle. In short, the role of the oil and gas operator is changing from inventor to integrator.

This shift demands a new operating model that facilitates:

Ecosystem management and integration. Access external innovation and apply it to fit-for-purpose internal applications.
Portfolio discipline. Direct capital and talent toward scalable, high-value opportunities.
Build/partner/buy clarity. Optimize value and speed over control or legacy advantage.

The changing role of upstream R&D

Historically, R&D was a core IOC capability that signaled strength to host governments and helped de-risk frontier projects. Much of it was bespoke, focused on specific assets, and executed in isolation from the business. As a signal of capability, the value of innovation was intrinsic. Research projects often had only a loose connection to business needs and no real path to value.

Researchers fell prey to common traps: “If we invent this, we can cheaply acquire the assets since no one else is bidding on them today.” But that stance ignores the fact that new technologies often fail to unlock an advantage in acquiring and developing reserves, dramatically underestimates practical oil and gas development schedules, and risks tying up significant capital today for leases that can’t be developed until the technology matures years down the road.

Many technologies stalled due to a difference in risk tolerance between R&D and the businesses: Companies invented a new technology but did not have the risk profile to commercialize it until a competitor profited from it first. Many R&D portfolios still drift—detached from core business strategy, built around pet projects, or scaled too slowly to matter.

As innovation shifted from lab-based experimentation to simulation and systems integration, barriers to participation dropped and the model began to fracture. External innovation ecosystems matured as service companies scaled technology for the industry, academic and public-sector partners coinvested, and digital and AI advanced. As a result, oil companies that could adapt and apply the latest innovations gained an advantage over those that remained focused on invention.

Line and bar chart of patent shares and totals, 1980–2025: IOC falls from 50% to 3%, NOC rises to 23%, total patents peak near 25k in 2021. — IOC contribution to overall energy innovation has steadily declined despite rising industry patent activity

Simultaneously, efforts to better quantify the value of the R&D portfolio drove researchers to link their work to business impact, reinforcing the focus on applications in the field. Access to proprietary data sets ensures that operating companies will always have a role in the front end of innovation—in fact, it’s worth noting that IOCs have not scaled back investment in R&D, which remains at about 0.35% of spend.

But the last mile of innovation, applying the technology to the real conditions of an operation, is where value is ultimately realized. That’s why specific areas of investment have shifted—from invention to implementation, from the lab to the field, and, increasingly, to software development.

This does not diminish the importance of science. Subsurface modeling, completions design, flow assurance, and materials science all still matter. But the winners will be those that tie R&D directly to business outcomes, reduce time to impact, and scale proven solutions enterprise-wide.

For IOCs, especially in the upstream, that means reducing costs across the portfolio (not just for the marginal barrel), expanding economically recoverable reserves, and de-risking development. This value derives from impacts to assets and often has little to do with licensing revenue or even IP exclusion.
For national oil companies (NOCs), the mandate further includes local capability building, technology transfer, long-term strategic technology independence, and national development goals. The challenge is to serve those objectives while still enhancing competitive advantage.
For oilfield services companies (OFSs), innovation must be product-centric: modular, repeatable, and scalable.

Table comparing IOCs, OFSs, and NOCs by mandate, scope, time horizon, value capture, org interface, ecosystem role, and challenges. — Changing R&D needs across the exploration ecosystem

Three principles for deploying at speed

These shifts are visible across the ecosystem, affecting IOCs, OFSs, and NOCs alike. Each group is rethinking how it defines success, interfaces with others, and manages innovation. One common thread is that, as innovation cycles shorten, deploying at speed becomes a top priority. Three guiding principles can help all of these players achieve that.

1. Align R&D with business outcomes.

Across archetypes, one challenge is universal: ensuring that technology solves real problems and generates real value. Historically, upstream R&D has suffered from weak linkage between problem definition and solution development. Central groups, particularly in large IOCs, often generated innovations that lacked a clear connection to operations and might even be disconnected from the portfolio. But leading companies now derive research focus directly from business strategy, and some even run R&D as an organizational child of strategic planning.

Key elements of their approach include:

codeveloping portfolios with the business, grounded in real asset plans and mutual accountability between the business and the R&D lead;
aligning on practical deployment and critical technical and nontechnical risks;
ensuring each effort addresses clear pain points or future commercial needs; and
balancing incremental improvement with bold, strategic bets.

Tighter alignment isn’t just best practice—it’s the clearest path to improving the ROI of R&D investment.

2. Focus on ways of working and portfolio management.

Leading companies have reconfigured their R&D operating models around value creation, with effective governance structures ensuring that technologies are stewarded on a value basis. KPIs are aligned to business outcomes, and incentives and career progression are linked to value rather than project technical milestones.

Technical champions are identified or embedded within assets, and the organization pulls together to achieve a new technology’s value potential. Scaling is not simply an exercise in handing off from a research lead to a project manager or a pilot to a business development team. It’s now a core capability enabled by talent and tools and—critically important—supported by ways of working that focus on value. Technologies are stewarded with the entire scale-up process in mind at each step; customers, interfaces, and applications are defined, tested, and retested as new learnings are discovered.

Almost all projects organizations have discovered that a faster path to value is technology qualification. Many high-impact innovations come from recombining existing tools, and these solutions are often far faster to deploy than brand-new technologies, given internal risk management systems.

The essence of rigorous portfolio management is clarity: what a technology is worth, what the risks are, what are the next best alternatives, how it will be deployed, and when to walk away. R&D portfolios should be governed like exploration pipelines. Each initiative should have:

a defined value hypothesis;
a scale-up plan and targeted deployment roadmap;
a quantified business case tied to specific assets;
clear off-ramps if progress stalls;
a plan for value assessment after implementation or post-mortem analysis after wind-down;
appointed business-side “product owners” to steer solutions from pilot to rollout; and
early integration of commercial and change enablers (training, standards, support models).

Ways of working must also enable cycling of the portfolio. Ending low-potential projects isn’t failure. It’s progress—and a learning opportunity.

3. Measure, analyze, and readjust as needed.

Every technology option should have a clear value hypothesis, a quantified business case, and a credible pathway to deployment. Prospective valuation (including risked expected value, time to impact, and dependency mapping) helps prioritize early investments, while retrospective valuation ensures that actual business outcomes inform future decisions.

This dual lens enables organizations to invest boldly in high‑potential opportunities while terminating projects that no longer merit capital. Future decisions must be made with prospective valuation techniques, but these techniques should be refined through rigorous retrospective analysis, sometimes years after decisions are made, to inform adjustments to process, decision making, and prioritization.

The discipline of portfolio governance is particularly important in the digital era, where shiny objects proliferate and the temptation to chase novelty can overwhelm operational focus. But the same principles apply across subsurface science, drilling hardware, completions design, production chemistry, and subsea engineering.

Innovation beyond digital and AI

Although digital technologies often dominate conversations around innovation, upstream value continues to be driven by a broad range of advances in other domains. Breakthroughs in reservoir characterization, drilling automation, completions design, flow assurance, integrity management, and materials science often deliver equal or greater returns than digital solutions, and these innovations are harder to source outside the sector.

That said, embedding digital and AI solutions within these highly technical, sector-specific areas can produce considerable value. This requires combining deep sector knowledge with cutting-edge digital tools and adjusting the talent mix. Digital fluency is now a requirement, not a specialization, although success will continue to demand technical depth in traditional engineering fields.

Getting there from here

To deliver impact, innovation must be embedded in a company’s operating model. High-performing firms combine scientific, commercial, and organizational strength by:

aligning R&D with business outcomes;
establishing ways of working focused on value creation; and
measuring value and connecting back to principles 1 and 2 through feedback cycles.

R&D still matters—but only when guided by practical application and strategy. Differentiation through R&D no longer comes from the resulting innovations alone; it comes from deployment and business impact. Leaders will identify the problems that matter; access or adapt the right technologies; and deploy them with speed, consistency, and measurable value.

That’s the new mandate for upstream R&D. The companies that embrace it will define the next era of performance in oil and gas. The goal is not invention. It’s value.