OpenAI's Jalapeño ASIC beats Nvidia on inference efficiency

OpenAI's Jalapeño ASIC beats Nvidia Blackwell on inference efficiency with unified architecture and AI-assisted software bring-up

OpenAI achieved a 16-month tape-out-to-silicon cycle on Jalapeño, an inference ASIC developed with Broadcom, demonstrating 1.5–1.9x better efficiency (tokens per megawatt) and 1.7–3.6x lower latency than Nvidia's GB200/GB300 across DeepSeek R1, Kimi K2.5, and GPT-OSS models—all without speculative decoding or prefill-decode disaggregation. The chip uses a unified homogenous architecture with out-of-order cores and local HBM4 slices per core, trading theoretical peak efficiency for practical performance across variable workload shapes (low-latency and high-throughput scenarios alike). SemiAnalysis notes Jalapeño's 700W TDP competes favorably with Vera Rubin's 900–1,150W, and on perf/TCO metrics the chips are head-to-head, though Jalapeño's advantage will expand once speculative decoding is implemented. The software story is equally significant: OpenAI used Codex-powered kernel generation to write functional MLA kernels without human intervention, scaling from TP8 to TP32 in 8 days with 2x performance gains in under 2 weeks. The team built a custom Gluon programming language with Linear Layouts abstraction for provably correct memory swizzling, enabling rapid bring-up that invalidates industry assumptions about universal compilers. OpenAI plans small-volume deployment by end of 2026 and ramp-up into 2027, signaling a credible competitive threat to Nvidia's inference dominance through hardware-software codesign—though SemiAnalysis cautions that AgentX benchmarks (reflecting real production cache behavior) remain untested, and first-generation chips often underperform at scale.

Anthropic's Q2 revenue hit $11.6bn annualized while OpenAI did $6.7bn; Nvidia deploys $105bn in guarantees and $7bn acquihire to lock in ecosystem

Anthropic's Q2 revenue reached $11.6bn annualized (up from $4.8bn in Q1), with run-rate exceeding $65bn by July—outpacing OpenAI's $6.7bn Q2 revenue and signaling where enterprise AI adoption is concentrating fastest, particularly in agentic coding. Concurrently, Nvidia is recycling hyperscaler cash back into the ecosystem through $105bn in guarantees for OpenAI/SoftBank's Ohio data center, $1.5bn in SB Energy, $3bn in Lacium power infrastructure, and a $7bn acquihire of Poolside for open models—a strategy designed to strengthen GPU lock-in, undercut competitors' custom chips, and hedge against closed model labs by backing open alternatives.

Ramp's Inspect platform powers 75% of merged PRs and spawned 200+ internal agents, demonstrating custom AI infrastructure outpaces frontier lab offerings

Ramp built Inspect, an internal remote-sandbox coding agent, which now powers 75% of merged PRs and reached 1M sessions in under a year—outpacing third-party tools like Claude Code and Cursor by enabling unlimited concurrent sessions, deep internal integrations (telemetry, feature flags, production databases), and autonomous verification of backend and frontend changes. Rather than a standalone tool, Ramp extended Inspect into a shared platform that abstracts cloud infrastructure (using Cloudflare Durable Objects, Modal sandboxes, and OpenCode as the harness), enabling 150+ engineers to rapidly build 200+ specialized agents (ReviewBuddy, Oncall Assistant, Ramp Research) with access to production systems and observability data. The team spins up fully provisioned remote dev environments in under 5 seconds, with 80% of Inspect itself written in Inspect sessions. This demonstrates that custom AI infrastructure tailored to organizational context can outpace frontier lab offerings when given access to internal data and verification loops—a pattern replicated at Block (Goose), Stripe (Minions), and Shopify (River).

San Francisco's Market Street remains devastated despite crime recovery, with foot traffic down 50% from 2019

Market Street's collapse stems from pandemic-driven remote work, early 2020s crime waves, and a 2019 car ban that removed vehicles without adding pedestrian infrastructure or public space—creating a transit corridor rather than a destination. This illustrates how policy decisions compound economic shocks and is relevant to SF urban policy dynamics.

AgentX 1.0: Open-Source Agentic Coding Inference Benchmark at 1M Context

SemiAnalysis published a comprehensive open-source agentic inference benchmark report covering realistic multi-turn, long-context workloads. Main takeaways:

• DeepSeek V4 Pro 0813 achieves 88k p50 / 272k p90 input sequence length and 413 p50 / 2.2k p90 output sequence length across 1.6M+ completed requests, establishing production agentic baseline metrics
• B200 vLLM and B300 vLLM now outperform MI355X on performance-per-dollar after August 21 optimizations, though AMD's SGLang development remains competitive on specific configurations
• AgentX generated 70+ upstream PRs across vLLM, SGLang, TensorRT-LLM, and other frameworks, with optimizations directly transferable to production traffic
• Benchmark reveals agentic workloads are fundamentally systems problems requiring KV cache efficiency, prefix reuse optimization, and multi-node routing—not just raw chip performance
• Open infrastructure includes public REST API consumed by tier-1 AI lab capacity planning teams, GitHub Actions CI with provenance, and anonymized Claude Code traces for realistic replay

Ways to think about token pricing

Ben Evans published an essay on the long-term equilibrium of AI token pricing and value capture. Main takeaways:

• Current 40-50% inference gross margins are unsustainable given that training costs dwarf revenue; profitability depends on unknown future training cost trajectories and demand scaling
• Four critical unknowns will determine whether frontier models capture pricing power or become low-margin commodity infrastructure: (1) how many use cases justify frontier model costs with positive ROI, (2) whether frontier capabilities continue improving faster than efficiency gains, (3) whether competition remains among equivalent frontier models, and (4) how much value the model itself captures versus tooling/data/go-to-market
• Mobile data networks provide the most relevant historical comparison: despite orders-of-magnitude traffic growth and trillion-dollar industry revenue, carriers captured minimal value as it flowed to companies further up the stack
• Semiconductor manufacturing (TSMC's de facto monopoly) and fiber overbuild (fixed vs. marginal cost dynamics) offer partial but imperfect parallels to AI's infrastructure trajectory

How Claude Code is Built: Inside Anthropic's AI-First Engineering Team

The Pragmatic Engineer published an in-depth interview with Claude Code's founding engineers on how the product was built and operates. Main takeaways:

• Claude Code generates $500M+ annual run-rate revenue with 10x usage growth in three months since May 2026 release; 80%+ of Anthropic engineers use it daily
• Tech stack (TypeScript, React, Ink, Yoga, Bun) chosen to be "on distribution" with Claude's capabilities; 90% of Claude Code's own codebase written by Claude Code itself
• Anthropic's engineering team doubled in size while PR throughput increased 67% (not decreased as typical), attributed directly to Claude Code adoption
• Product overhang thesis: Claude model could already traverse filesystems and explore codebases, but no product existed to capture this capability until Claude Code
• Team operates with minimal business logic; model does almost all work via exposed tools and UI hooks, keeping the product as a "lightweight shell" on top of Claude

OpenAI Jalapeño: Better Than Nvidia

SemiAnalysis published a detailed technical report on OpenAI's Jalapeño inference chip, benchmarked against competing systems. Main takeaways:

• Jalapeño achieves 1.5-1.9x higher throughput per watt and 1.7-3.6x lower latency than Nvidia GB200 and AMD alternatives across GPT-OSS, DeepSeek R1, and Kimi K2.5 models, without speculative decoding
• Chip went from design to tape-out in 16 months using AI-accelerated design; B0 stepping delivers ~25% perf-per-watt improvement over A0 silicon currently in testing
• Architecture prioritizes homogeneous compute pools over prefill-decode disaggregation to adapt to shifting workload mixes across knowledge, reasoning, and agentic eras
• At concurrency 1 (interactive workloads), Jalapeño reaches 700 tok/sec/user on Kimi K2.5 and 1,400 tok/sec/user on GPT-OSS—9-50x better than GB200
• Benchmarks use InferenceX suite on 8k1k workloads; AgentX (long-context, multi-turn) results pending; comparison to Vera Rubin (HBM4, shipping now) more relevant than Blackwell

You Probably Forget How Cheap Memory Used To Be – And Is Not So Now

The Next Platform published an essay on memory and storage pricing dynamics in the AI infrastructure buildout. Main takeaways:

• DRAM, HBM, and NAND flash spot prices have increased ~7x due to hyperscaler demand colliding with constrained supply from five major manufacturers (Micron, Samsung, SK Hynix, Kioxia, Intel).
• AWS, Google, Microsoft, and Meta account for the vast majority of AI-driven capex since 2021; Alibaba barely registers despite being China's largest hyperscaler.
• Nvidia faces 15% price increases on Grace-Blackwell and Vera-Rubin systems in 2027 due to rising HBM costs; component allocations are locked through end of 2027.
• Memory and flash production share the same manufacturing lines, creating commodity-like dynamics where suppliers prioritize highest bidders during shortages, disadvantaging smaller buyers.
• Industry is shifting toward Processing-In-Memory (PIM) and externalized flash-based storage tiers to reduce memory bandwidth bottlenecks and accelerate AI workloads.