While all eyes have been on the shortage of semiconductors this year, a fundamental transformation with even bigger implications for the future of silicon has picked up momentum. Fast-rising demand for specialized processors is shifting the equilibrium between special- and general-purpose silicon that has existed for decades (see Figure 1).

Figure 1

Demand for specialized processors is increasing rapidly

Special-purpose chips, known as application-specific integrated circuits (ASICs), have historically occupied an important segment in the market. Each ASIC is designed to perform a limited set of repeatable functions, such as video transcoding or speech processing. Because they don’t have to be a Swiss-army knife, end customers know they can substantially outperform general-purpose (GP) processors, such as the industry workhorse, the CPU, for certain functions. However, ASIC customers are also well aware that these specialized chips have expensive up-front design and component costs, such as for masks, and their limited programmability makes them inflexible.

On the other end of the spectrum, GP processors have the benefit of being programmable for a wide range of workloads. Plus, silicon architectural compatibility gives chip buyers peace of mind that software development investments are portable to future generations of GP processors. This compatibility allows software developers to significantly upgrade application performance as the silicon industry makes advances in manufacturing and chip design.

Historically, this general-purpose computing proposition has fueled a roughly $60 billion compute microprocessor market. Four structural forces have served as catalysts for GP processors’ dominance. First, Moore’s Law has typically allowed GP processors to promise a steady performance-per-watt improvement every two to three years. Second, computing workload diversity in data centers, networks, and devices has persisted, favoring programmable platforms that can flexibly handle a range of workloads. Third, the scale of any individual workload is often insufficient to justify the high fixed costs of designing and manufacturing special-purpose processors. Lastly, GP processors and software tools from GP silicon vendors have reduced software development costs and time to market, enabling developers to bring new services and applications online much more quickly and efficiently.

The shifting semiconductor equilibrium

Today’s computing environments are shifting this equilibrium more toward specialized silicon and blurring the line between special- and general-purpose processors.

As Moore’s Law slows, top semiconductor buyers—the “hyperscaler” cloud service providers (CSPs)—have the intent and scale to control their destiny by designing more custom processors in-house. Workloads are consolidating to a limited number of enormous, global “computers” or “computing clusters” located in buildings of several hundred thousand square feet and often owned by the large CSPs. These warehouse-scale computers can consolidate specific workloads into more homogeneous server clusters and cloud domains that are focused on a small number of tasks. As such, data center operators are more capable of tailoring computing system and silicon design for this narrow range of tasks, and they can do it more economically than was possible before.

At the same time, the largest semiconductor buyers are increasingly realizing that the proliferation of video, images, speech, and other high-data-volume workloads such as AI training are better suited to chips tuned for the particular processing required (such as vector, matrix, and graph processing). These data workloads aren’t served as well by the predominant GP compute platform, CPUs. Engineers are therefore faced with a choice: Use a specialized ASIC chip or a newer type of general-purpose processor, either a graphics processing unit (GPU) or field programmable gate array (FPGA). The ease of developing software using GPUs and FPGAs has improved in recent years, but still lags CPU maturity. Meanwhile, top CSPs have the deep pockets and expertise on staff to develop applications on custom ASICs, and they’re planning to do more of this. As one example, look at how dramatically the largest CSPs intend to increase their use of special-purpose chips for internal AI workloads over the next decade (see Figure 2).

Figure 2

Hyperscaler cloud service providers plan to dramatically increase their use of special-purpose processors

How to adapt?

General-purpose silicon vendors might be hearing alarm bells, but there’s no need to panic; GP computing has a bright future. That’s partly because special-purpose processors are hard to program, and the GP model’s flexibility still makes it desirable in most cases. That said, the blurring line between the two calls for GP silicon incumbents to adapt in three areas.

Product portfolio. Watch for GP silicon providers to develop more processors that can serve diverse scalar, vector, matrix, and graph workloads at various power-performance thresholds. The holy grail may be to offer a family of compatible GP processors to cover the diversity of workloads. This inevitably presents a software challenge as companies try to unify development tools under a common software application programming interface (API) substrate. GP vendors are already adjusting their product portfolios accordingly; see Nvidia’s interest in acquiring ARM and Intel’s introduction of GPUs as well as its acquisitions of Nervana Systems and Habana Labs for AI processors.

Accelerator blocks. The industry’s evolving dynamics are making it necessary to tightly integrate companion accelerator processors and intellectual property (IP) “blocks” with GP processors, in order to offload critical high-volume workloads such as AI and media processing. The industry is in a race; the x86 CPU ecosystem is working to add vector and matrix processing to its chips, while the GPU ecosystem is working to better accommodate scalar processing and general software developers. Ultimately, the trick is hitting the right mix of accelerated special-purpose and GP compute engines, ensuring tight coupling and low latency, and making it easy for programmers to use.

Customer engagement model. GP silicon vendors’ largest customers, the hyperscaler CSPs, will increasingly require customization. This involves trimming the unused “kitchen sink” features in GP silicon, calibrating features to large customers’ needs, and realigning the organization and resources to enable a higher-touch, nimbler customer engagement model. The industry is moving in this direction, at varying paces. AMD appears to have been prioritizing this with its Semi-Custom business unit.

Meanwhile, all signs point to GP silicon remaining the majority of semiconductor manufacturers’ production volumes of leading-edge chips. Winning a fair share of these crucial customers remains at the top of the agenda for foundry executives. Nevertheless, the new silicon equilibrium creates the possibility of enough demand for large-scale production of leading-edge ASICs. A prime example may be cryptocurrency processors, where demand has spiked in recent years. Leading manufacturers will go after these new special-purpose opportunities, particularly in artificial intelligence (AI) and media processing.

As for hyperscale CSPs, evolving data center economics will require them to take more control of their silicon destiny, in order to differentiate their services and lower their total cost of data center ownership. CSPs recognize, of course, that only they have the necessary insight into the telemetry of their server fleet and most critical computing workloads. Identifying the next growing workloads that aren’t well-suited to GP processors, then creating customized system and silicon solutions, will become necessary to stay ahead of the competition. But silicon design requires scale to succeed, so getting into this business introduces critical make-vs.-buy choices.  

There are a few trends to watch here. First, within GP silicon, GPUs will likely continue to take share from CPUs. An example of this is Nvidia’s data center GPU computing business, whose annual revenue has grown by 16 times in the last six years, to $5 billion in fiscal 2021. Second, there’s a strong argument that value-managed relationships will emerge between traditional GP silicon vendors and their CSP customers, which will closely collaborate with the silicon vendors as they increasingly customize their GP processors and add accelerators and other enabling IP, such as memory capabilities. As this introduces the risk of IP leakage for the hyperscalers, many of these value-managed relationships will likely include varying degrees of exclusivity and firewalling—a new behavior for GP silicon vendors. Lastly, for select ultra-high-volume computing workloads, hyperscalers will likely build custom ASICs in order to gain computation efficiency advantages over competing CSPs. We see this already in Google Cloud’s tensor processing unit and Amazon’s data processing unit investments.

What does all of this mean for the special-purpose end of the market? Specialized silicon vendors have seen a burst of venture funding and the growth of ASIC providers, especially in edge and AI processing. However, there’s an open question about the longevity of small-scale, standalone ASIC vendors. The ones that make it will most likely be acquired by a hyperscaler or GP silicon vendor. We see this in acquisitions such as Intel’s Habana deal and Amazon purchasing Annapurna Labs. Don’t be surprised to see ASIC vendors pursue more partnerships with their customers, both to learn from them and find a potential exit path.

Meanwhile, the few independent ASIC-design companies with a large-scale business, a proven IP portfolio, and strong foundry relationships are becoming more valuable than ever. Just look at Broadcom’s cloud ASIC business, which has an estimated annual revenue run rate exceeding $1 billion.

Ultimately, general-purpose silicon is here to stay, increasingly blending with special-purpose silicon as it becomes more prevalent in the years to come. That spells opportunity for companies across the semiconductor ecosystem to thrive, as long as they nimbly adapt to find their place in this shifting equilibrium.