OpenAI Built Its Own Chip. The First Benchmarks Just Landed — And They're Not What Nvidia Wanted to See

Building a competitive AI chip from scratch usually takes years, and even then, first-generation attempts rarely hold their own against Nvidia's finely-tuned hardware. OpenAI seems to have skipped that step entirely.

At the Hot Chips conference in Silicon Valley on Tuesday, OpenAI presented the first independent benchmark results for Jalapeño, its custom inference chip developed in partnership with Broadcom and system integrator Celestica. The numbers, measured using semiconductor analysis firm SemiAnalysis's public InferenceX benchmark, showed Jalapeño delivering 1.5 to 1.9 times more computing work per watt than an equivalent Nvidia Blackwell system, alongside 1.7 to 3.6 times lower response latency. On the most interactive, chat-style workloads — essentially the kind of traffic ChatGPT generates constantly — the gap widened even further, to 2.1 to 4.1 times faster.

Richard Ho, OpenAI's head of hardware, put it bluntly during the presentation: "The bottom line is that the results show a very, very significant performance advance over state of the art."

SemiAnalysis, an independent firm that verified portions of the testing on-site, didn't hold back either. "In general, first-generation chips are not competitive," the firm wrote in its analysis, "but OpenAI bucks the trend by being industry-leading and beating every Nvidia, AMD, and Google chip we have been able to test on multiple top open source models." SemiAnalysis founder Dylan Patel echoed the sentiment more casually elsewhere, noting that first-gen chips typically aren't competitive at all — making OpenAI's results against Blackwell, and even Nvidia's newer Vera Rubin platform, genuinely unusual.

It's worth understanding exactly what Jalapeño does and doesn't do. This is strictly an inference chip — meaning it runs already-trained AI models to generate responses, rather than training new models from scratch. Training remains Nvidia's uncontested territory for now. But inference is where the actual, ongoing cost of running an AI product lives. Every time someone asks ChatGPT a question, that request gets processed through inference hardware, and at OpenAI's scale — hundreds of millions of users — even small efficiency gains compound into enormous savings.

The development timeline is almost as notable as the results themselves. Design work on Jalapeño reportedly began in mid-2024, and the chip went from initial team formation to manufacturing tape-out in roughly nine to sixteen months, depending on which stage is being measured — an unusually fast pace for custom silicon development. OpenAI has said it used its own generative AI models to help accelerate portions of the hardware engineering process itself, a detail that adds a slightly recursive flavor to the announcement.

Analysts aren't ready to declare Nvidia's dominance over, though. Several important caveats temper the headline numbers. SemiAnalysis pointed out that a fairer, more like-for-like comparison would be against Nvidia's newer Vera Rubin platform rather than Blackwell, since both Jalapeño and Vera Rubin use next-generation HBM4 memory. Even against Vera Rubin, Jalapeño reportedly still edges ahead on output tokens per megawatt — despite Nvidia's chip using a multi-token prediction optimization that Jalapeño hasn't yet adopted. On total cost of ownership per token, though, the two land roughly even. Nvidia and AMD have also published benchmark results on newer, larger models that haven't been tested on Jalapeño yet, leaving some comparisons incomplete.

Deployment, for now, remains modest. OpenAI plans only a small-scale rollout of Jalapeño within its own infrastructure by the end of 2026, with broader scaling expected sometime in 2027 — meaning this remains, for the moment, more of a proof-of-concept milestone than an operational shift.

The timing carries its own weight, too. The announcement landed just one day before Nvidia was scheduled to report its fiscal quarterly earnings — a coincidence that CNBC and other outlets have been quick to note. Analysts told CNBC the development adds real pressure to Nvidia's near-monopoly position in advanced AI chips, especially with Google, Amazon, and Meta all separately developing their own custom silicon as well. None of that means Nvidia's position is suddenly at risk — the company still commands the overwhelming majority of AI training workloads globally — but the message from this week's benchmarks is fairly clear: for the specific, high-volume task of running AI models rather than training them, the era of Nvidia facing zero credible competition may be quietly coming to an end.

Got a tip or breaking story for us? Reach out at tips@kanilprwire.com.