Jensen Huang Is Out; Are Inference Providers Screwed?

What a week. Jensen Huang, the CEO and cofounder of the world’s second-most-valuable company, resigned, Nvidia said Thursday morning, after reports of extramarital affairs both with a woman alleged to have spied for Beijing as well as a woman who reported to him at Nvidia. Nvidia chief financial officer Colette Kress will become interim CEO, the company said, while it searches for a permanent successor.Much remains uncertain, including implications for Nvidia’s business of selling advanced AI chips and for Nvidia’s aggressive investments in AI model developers and infrastructure startups that are holding up the AI economy. I’m sure we’ll be getting into all of it in the upcoming days.Meanwhile…It’s not only a tough week to be Huang, but it’s also a tough week to be an inference provider, a well-funded category of startups that help developers use open-source models at high speeds across Nvidia servers.Most recently, Fireworks, one of the most heavily-funded startups in this category, suffered a security breach that exposed customer data to hackers and has also prompted some large customers to consider ditching the service. (In a note to customers, Notion, one of Fireworks’ customers, suggested that, although the inference provider promised it was using zero data retention, meaning that it wouldn’t store any data from Notion after processing a request, Notion later learned that data was actually being retained.)