Rant
So much happened this month. I had run through many possible futures for the U.S.–China AI race, but never this one: the U.S. taking a knife to its own thigh and forcing out Jensen Huang, the godfather of the AI industry — a cut straight into the artery. As an AI researcher, it feels like the sky is falling on AI. From here on, the industry enters a new era — the post-Huang era. The question is: without Jensen Huang, does the AI industry still have glory days ahead?
NVIDIA’s Secret Weapons
CUDA, NVIDIA’s crown jewel, was Huang’s contrarian bet starting in 2006: making gaming cards — dismissed at the time as a distraction from serious business — support scientific computing. As AI took off, demand for parallel matrix multiplication exploded, lifting NVIDIA’s GPUs onto a pedestal overnight. How important is CUDA to AI? AI might have reached today’s level without it — but humanity would have had to wait a lot longer. That is Huang’s technical foresight on full display: a bet placed a full ten years early. CUDA and AI grew up deeply intertwined, each making the other.
As AI’s appetite for compute outgrew a single card, NVIDIA introduced NVLink to fuse multiple GPUs seamlessly, and then a whole series of technologies optimizing racks and data centers. NVIDIA’s accumulated expertise in GPU cluster scheduling is far ahead of the industry; its cluster solutions come close to the theoretical limits on energy efficiency and scalability, making them the first choice for deploying AI inference. That lead in system integration, in turn, rests on NVIDIA’s supply-chain integration — which rests, in the end, on Huang himself and his grip on the upstream and downstream supply chain. “Integration” doesn’t even do his contribution justice; “leadership” is the better word. He drove the entire industry chain to iterate and improve fast, and became the de facto standard.
Without Huang, Are the Secret Weapons Still Secret Weapons?
CUDA is, at this point, Huang’s technical legacy from twenty years ago. As I argued in my analysis, AI coding capabilities have advanced to the point where the human effort needed to port CUDA to other chips has shrunk dramatically. Within the next few years, CUDA’s advantage will most likely be wiped out entirely — it will no longer be the only choice for research and industry, and it may not even be the first choice. The CUDA ecosystem moat, for practical purposes, has no future. As for where the next software ecosystem begins — well, the brilliant minds on Capitol Hill can slap their own dumb asses and come up with something.
What about the system-integration capability? As everyone knows, Huang commands enormous respect among Asian chip-packaging and HBM makers; a large share of supply agreements are “handshake deals,” resting purely on Huang’s personal word, with no contract behind them. Whether those “agreements” still count once Huang is gone is very much an open question. Until now, every vendor in the supply chain has followed Huang’s lead; he guided the whole industry’s technological iteration and innovation. Without him, the pace of innovation across the chain will most likely slow, and NVIDIA will lose its position as the supply chain’s most important customer. Even if NVIDIA comes up with an innovation internally, without active cooperation from the supply chain, its time to market will be hard to guarantee.
With no CUDA in the future and its leadership of the industry chain gone, NVIDIA is unlikely to produce the kind of stunning innovation it did over the past twenty years — and it will struggle to sustain the technology premium behind a 75% gross margin.
Conclusion
Huang = extraordinary personal charisma + first-rate technical foresight + supply-chain relationships + superhuman stamina (facing a 60-person executive team at once). Where is the second person like him?! I saw NVIDIA’s recent announcement that it has hired a third-party headhunter to find a suitable CEO, and I had to laugh. Who could they possibly find? Most likely a professional manager answerable to financial targets — and the role itself decides what he can see: this year’s bonus and next year’s boasts. An empty head (at least where technology is concerned) can only understand financial metrics. NVIDIA is destined for mediocrity.
That is the worst of it: the world no longer expects NVIDIA to get better. What it needs now is something new to replace it!