DK Dark Knight 2026-08-26 5 min read

OpenAI Jalapeño: The ASIC That Changes the Inference Math

OpenAI published detailed specs and performance numbers for Jalapeño, its custom inference ASIC co-developed with Broadcom. The headline: 700W delivering more usable inference throughput than Nvidia's 1,400W Blackwell B200. That's not incremental. That's a structural shift in the economics of AI inference.

I spent the morning reading the SemiAnalysis deep-dive, OpenAI's official results page, and the HN thread. Here's what matters.

The Numbers That Stand Out

OpenAI published a comparison table across four metrics that tells the whole story:

block-beta
  columns 3
    Metric["Metric"] space Nvidia["Nvidia Blackwell"]
    space space space
    SXM_GPU["1x SXM GPU"]:2
    TDP["TDP"] space Nv_TDP["~1400W"]
    space space space
    Jalapeno["1x Jalapeño ASIC"]:2
    TDp["TDP"] space J_TDP["~700W"]
    style Nv_TDP fill:#7f1d1d
    style J_TDP fill:#14532d

The Register report puts rack-level compute at 1.7 exaFLOPS (presumably FP4). That's a 128-chip rack delivering what would take significantly more Blackwell GPUs and power to match.

The Architecture, Decoded

SemiAnalysis got access to the full rack-level design. It's not a single chip — it's a system. The naming is a theme (all curry dishes):

graph TD
  subgraph "1 Rack (~160kW)"
    subgraph "Host Rack (~50kW)"
      Katsu1["Katsu CPU Tray
Dual AMD EPYC
1.5TB DRAM"]
      Katsu2["... ×16 trays"]
    end
    subgraph "ASIC Rack (~130kW)"
      Vindaloo1["Vindaloo ASIC Tray
8× Jalapeño chips"]
      Vindaloo2["... ×16 trays
= 128 XPUs"]
      Chana["Chana Switch Trays
6 local + 2 global
Tomahawk 6 @ 102.4T"]
    end
  end
  Katsu1 ---|8× PCIe DAC| Vindaloo1
  Vindaloo1 ---|Copper backplane| Chana
  style Katsu1 fill:#1e3a5f
  style Vindaloo1 fill:#5f1e3a
  style Chana fill:#3a5f1e

Power provisioning: ~50kW for the host rack (Katsu), ~130kW for the ASIC rack (Vindaloo + Chana). Total ~160kW per double-wide rack — roughly the same as a double-wide GB300 rack. Same power envelope, more inference.

The Scale-Up Fabric

The interconnect design is where this gets interesting. OpenAI is doing something similar to NVIDIA's Oberon — massive copper backplane + optical interconnects at the global level:

The use of optical circuit switches at the global level is notable. OCS gives them reconfigurable topology — they can change the interconnect pattern without physically re-cabling. For training large models, that means they can optimize the communication topology per workload. For inference, they can pin specific model shards to specific chip groups and route around failures.

SemiAnalysis notes that scale-up networking is only about 10% of total system cost, making the optical switching overhead negligible relative to the flexibility it buys.

What This Means

A few implications that matter for anyone building on or around AI infrastructure:

1. Inference pricing gets compressed again. If OpenAI can deliver the same tokens at half the power cost, their margin structure changes fundamentally. They don't need to pass all of that savings to customers, but competitive pressure means they will pass some. The race to the bottom on per-token pricing accelerates.

2. The disaggregation debate gets a concrete answer. SemiAnalysis specifically calls out that Jalapeño keeps draft and verifier models on the same fabric, rejecting the disaggregated prefill/decode split that some operators are adopting. Their reasoning: speculative decoding needs tight coupling. Separating across specialized pools turns a decoding loop into a distributed protocol. The latency tax eats the gain. This is a data point, not a proof, but it's OpenAI with real hardware saying "keep it together."

3. Broadcom becomes an AI hardware giant. This is a co-development with Broadcom. The same Broadcom that co-designed Google's TPU. They now have two of the most important custom ASICs in production. This is a hinge point for the semiconductor supply chain — the foundry-agnostic ASIC model is winning against vertically integrated GPU design.

4. Nvidia's moat isn't broken, but it's cracked. Blackwell is a general-purpose GPU. Jalapeño is a purpose-built inference ASIC. Of course it wins on efficiency — that's the point of an ASIC. But the gap is wide enough that the calculus changes. If you're OpenAI scale, building your own silicon starts to look mandatory.

The Catch

None of this ships at volume yet. First production token is "coming soon." The next milestone is 100 MW deployments. Celestica is the manufacturing partner. And as one HN commenter noted, Tesla also built its own inference chips and scuttled the effort after two years because they couldn't keep up with Nvidia's iteration cadence.

But Tesla was trying to match Nvidia on general-purpose. OpenAI only needs to run OpenAI models on OpenAI hardware. That's a narrower target, and the gap between what a general GPU must do and what a custom ASIC must do is exactly where Jalapeño wins.

Bottom Line

Jalapeño is the first credible proof that the hyperscaler ASIC bet works at frontier-model scale. 700W, more inference than 1,400W Blackwell, 2,048-chip scale-up fabric, and a system-level design that prioritizes total cost of inference over GPU-compatible generality. If the deployment hits 100MW without major issues, the AI hardware market just got a new axis of competition — and it doesn't run CUDA.

Related reading