On August 25, 2026, OpenAI announced the first public benchmark results for its self-developed AI inference chip, Jalapeño, at the Hot Chips annual chip technology conference: in the InferenceX benchmark published by SemiAnalysis, Jalapeño completed 1.5 to 1.9 times more AI inference work per watt than Nvidia's GB200 and GB300 systems, with end-to-end latency reduced to 28% to 59% of the comparison systems. The chip, with a rated power consumption of 700 watts, comprehensively outclassed Nvidia's flagship rated at 1400 watts in efficiency.
OpenAI's hardware chief, Richard Ho, told media after the event: "Jalapeño can handle more AI work per unit of energy while responding faster. It can serve large numbers of users efficiently, and it can also deliver extremely low latency." Sam Altman put it even more succinctly: "We made a chip. It's fast."
What exactly does this chip do
Jalapeño is a custom AI accelerator co-developed by OpenAI and Broadcom, designed from the outset solely for large-model inference. This stands in stark contrast to Nvidia GPUs' "handle both training and inference" approach. Specialization means architectural trade-offs can be made that general-purpose GPUs would not dare to attempt.
The tests covered three industry-representative open-source models: GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T. With DeepSeek R1 as a reference, an inference task with 8K input and 1K output dropped from 5.99 seconds to 1.65 seconds; the minimum token generation interval compressed from 5.90 milliseconds to 1.43 milliseconds. According to SemiAnalysis test data, in interactive low-latency scenarios, Jalapeño's advantage expanded further to 2.1 to 4.1 times.
At the full rack level, 128 Jalapeño accelerators form one system, delivering 1.7 exaFLOPS of 4-bit compute, equipped with 27.5TB of HBM4 memory and nearly 2 petabytes per second of memory bandwidth.
Why it's fast: breaking through the "memory wall"
Modern AI chips' compute speed has long outstripped data movement speed — the chip computes quickly, but the rate at which weights and KV Cache can be moved from memory to compute cores lags behind, leaving compute units waiting for data a large fraction of the time. This is the long-standing "memory wall" problem in large-model inference.
Jalapeño's solution is to design compute, memory, and inter-chip networking as a unified architecture, rather than treating them as independent modules to be stitched together. According to SemiAnalysis, each Jalapeño chip is equipped with 216GiB of HBM4 memory, reaching 15.4TB/s of memory bandwidth — roughly twice that of Microsoft's Maia 200 (HBM3e). Inside the chip, compute cores and HBM are divided into multiple corresponding local regions; model weights and KV Cache preferentially stay in the local memory responsible for processing them, and only inter-region communication traverses the dedicated high-speed network.
Power consumption figures are the most direct reflection of this architecture's advantage: Jalapeño has a rated power of 700 watts, with measured sustained power below 550 watts; Nvidia's GB200 is rated at 1200 watts, and the GB300 at 1400 watts. Data center electricity costs are a real operating expense, and the gap in energy efficiency means the electricity, cooling, and rack space required for comparable compute scale will shrink dramatically.
An AI chip built in nine months
Jalapeño's development cycle is equally noteworthy. According to OpenAI's official blog, from formal design start to tape-out took only nine months. According to Semiconductor Engineering's industry estimates, formal design cycles for AI accelerators of this complexity typically require 18 to 24 months.
The timeline is as follows: the project entered formal design around February 2025, with the initial team drawn largely from the former Google TPU project; tape-out was completed in November 2025; OpenAI received its first finished chip in May 2026; debugging was completed and results were publicly presented at Hot Chips in August 2026. A key point is that OpenAI's own AI models were deeply involved throughout: models contributed to design space exploration, design verification, and arithmetic circuit optimization. Once the chip was in hand, OpenAI used Codex and internal models to complete support within two months for models not originally in the adaptation plan, including DeepSeek R1 and Kimi K2.5.
Where Nvidia's moat lies
Jalapeño is a pure inference chip, not involved in training tasks. Nvidia's dominance in AI training — built on the CUDA ecosystem and H-series/B-series GPUs — is currently unchallenged by any chip.
According to estimates from multiple institutions, Nvidia currently holds approximately 70 to 75 percent of the AI chip market (measured by data center AI accelerator revenue). And according to Tom's Hardware's industry analysis, custom AI server ASIC shipments are growing at roughly 44.6 percent annually in 2026, far outpacing the 16.1 percent growth rate of commercial GPUs. The two currently coexist rather than substitute for one another.
OpenAI itself has made clear that it will continue purchasing Nvidia GPUs. The hardware reality is this: large-scale training of new models still depends on Nvidia's ecosystem, while the scale and frequency of everyday inference services are another order of magnitude entirely — for operating a product with hundreds of millions of users globally, inference cost is the largest ongoing expenditure. At this level, Jalapeño addresses a genuine economy-of-scale problem.
Broadcom's hidden footprint
Jalapeño is not an isolated event but part of a larger trend. Broadcom currently partners on chip design for Google's TPU, Meta's MTIA, Microsoft's Maia, and OpenAI's Jalapeño, holding roughly 60 percent of the design collaboration market in custom AI server ASICs. Almost every top-tier AI lab or hyperscale cloud provider pursuing a "self-developed chip" path has Broadcom behind it.
This landscape reveals a structural industry shift: AI compute is moving from "procuring Nvidia standard products" toward "vertically integrated custom silicon," and Broadcom has become the most important infrastructure provider along this path. Whoever controls the model and bears the inference cost has the incentive to build custom chips. This is not just OpenAI's logic — it is the road that Google, Amazon, Microsoft, and Meta have each taken.
Conclusion
This is the first time OpenAI has produced publicly comparable hardware benchmark data, rather than PPTs or concept demonstrations. The figure of a 700-watt chip accomplishing twice the work of Nvidia's 1400-watt flagship system means the cost structure of AI inference will be fundamentally rewritten.
Nvidia's moat in training cannot be shaken in the short term, but the inference market is precisely where AI service providers spend the most money today. The core logic of this custom chip race has never been "defeat Nvidia," but rather "can inference costs be brought down enough to support AI applications at even greater scale?" The answer Jalapeño provides is the most compelling one so far.
© 2026 Winzheng.com 赢政天下 | 转载请注明来源并附原文链接