vLLM adds day-zero support for Recursive_SI’s Flint coding model

What happened

vLLM announces day-zero support for Recursive_SI’s Flint, with NVFP4 and BF16 ready. Flint has 40B parameters with 2.5B active per token. vLLM reports that its custom NVFP4 kernels deliver 500 TPS on a single GB200, even at 100K context without speculative decoding.

The story so far

Ollama made Recursive_SI’s Flint coding model available to run locally, while vLLM announced day-zero support with NVFP4 and BF16 ready. Flint is an open mixture-of-experts model with roughly 40 billion parameters; vLLM says 2.5 billion are active per token. Ollama describes it as designed for on-device agentic coding. The platforms reported performance under different hardware and context conditions. Ollama reported more than 125 tokens per second on an M4 Pro MacBook Pro and more than 150 tokens per second on a 64GB NVIDIA RTX Spark, both with short contexts. vLLM reported that its custom NVFP4 kernels delivered 500 tokens per second on a single GB200, even at a 100K-token context, without speculative decoding.

Keep reading

Trending now

1. 1Mistral and NVIDIA Announce Industrial Engineering AI Collaboration and Open Benchmarks
2. 2Mistral Releases Voxtral 3 Speech Models, Voice Agents API and LiveKit-Based SDK
3. 3PHP releases security updates after OpenAI says its models helped researchers find critical flaws
4. 4DeepSeek Open-Sources DeepInfer and API Components, With Model and Hardware Limits