Every now and then something happens in AI that is genuinely good news for the little guy, and this is one of them. A Paris startup called ZML has released a free, open-source tool called LLMD, an inference server that runs open large language models fast across whatever chips you happen to have: Nvidia, AMD, Google TPU, Apple Metal, Intel Arc, the lot.
If that sounds boring, stay with me, because it is aimed squarely at the soft underbelly of the most valuable company on earth. Nvidia’s real moat has never been the raw silicon. It is CUDA, the software layer that nearly all of AI is built on, which turns switching to a cheaper chip into a nightmare and keeps the entire industry politely hostage. A free tool that lets you run your models efficiently on any chip is a small, precise kick at exactly that.
What it actually does
LLMD ships with the features that make an inference server genuinely useful rather than a toy: OpenAI-compatible API endpoints (so you can point existing code straight at it), continuous batching, paged attention and sharding, and it supports the popular open model families including Gemma, Llama, Mistral, Ministral and Qwen. It is a technical preview, it is free, and the code is open, which means anyone can read it, run it and improve it. The entire pitch is cutting the cost of serving models by refusing to be married to any single chip vendor.
The AI tool stack actually worth paying for
One email a week. The models, tools and moves that matter, stripped of hype and filtered so you don’t have to drink from the firehose. Free, and you can bail anytime.
The people behind it
ZML was founded in 2023, has a team of about twenty, and just raised $20 million in July from a clutch of venture firms including 20VC, Kima Ventures and LocalGlobe. The detail that raises an eyebrow: it is backed by Yann LeCun, the Turing Award winner and one of the godfathers of modern AI, who has spent a good while now arguing that the industry is dangerously over-reliant on a narrow slice of approaches and hardware. Him putting his name on a chip-agnostic, give-it-away-free inference tool is not an accident.
The read
One free tool is not going to topple Nvidia, and nobody serious is claiming it will. But this is how monopolies actually get worn down: not with a single giant blow, but with a steady drip of good, free, open alternatives that make the lock-in a little less total every quarter. A small French startup just gave away the exact thing Nvidia would much rather you did not have. Good.