vLLM is a library designed for high-throughput and memory-efficient inference and serving of large language models (LLMs). It supports a wide range of model architectures and offers various features for efficient model execution and integration with popular frameworks.
Collecting history — the radar snapshots this repo daily. The trend line appears after 3 days of data (1 so far).
What it is
vLLM is a high-throughput and memory-efficient inference and serving engine for large language models (LLMs). It supports over 200 model architectures from Hugging Face, including decoder-only LLMs, mixture-of-expert LLMs, hybrid attention models, and multi-modal models.
How it works
vLLM provides efficient management of attention key and value memory through PagedAttention and features continuous batching of incoming requests, chunked prefill, and prefix caching. It supports various quantization methods including FP8, INT4, and GPTQ/AWQ. The library also includes optimized attention kernels and supports speculative decoding techniques.
Getting started
Install vLLM with the following command:
uv pip install vllm
Alternatively, users can build from source for development purposes.
Recent releases
- v0.25.0 (2026-07-11): Features 558 commits from 232 contributors, with Model Runner V2 as the default for all dense models.
- v0.24.0 (2026-06-29): Introduces support for the MiniMax-M3 model.
- v0.23.0 (2026-06-15): Notes that Minimax M3 is not supported in this version.
- v0.22.1 (2026-06-05): A patch release with targeted bug fixes and new model support.
- v0.22.0 (2026-05-29): Includes 459 commits from 230 contributors and a major hardening pass for DeepSeek V4.
Traction
- Stars: 85992
- Forks: 19281
- Open issues: 5702
Caveats
- License: Apache-2.0
- Open issues: 5702
- Age: Created on 2023-02-09, with the last push on 2026-07-12.