Luminal Built an ML Compiler That Makes vLLM Look Slow
Everyone is fighting over which model to run. Luminal is fighting over how fast you can run any of them. Their ahead-of-time compiler turns AI models into optimized GPU code and is already beating vLLM and TensorRT-LLM on throughput benchmarks. Three people, $5.3 million, and a very different theory of how inference should work.