vllm
vLLM is an open-source engine that runs large language models fast and cheaply in production, used widely to self-host models like Llama, Qwen and DeepSeek.
View vllm-project/vllm on GitHub- GitHub stars
- 96,870
- Forks
- —
- Language
- Python
Project details
- Repository
- vllm-project/vllm
- Licence
- Apache-2.0
- Primary language
- Python
- Catalog status
- Available
- Catalog check
- 2026-11-13
Repository description
A high-throughput and memory-efficient inference and serving engine for LLMs
Explore by topic
More tools in Models & training
- transformers
huggingface · Apache-2.0
The open-source library that provides ready-to-use implementations of virtually every major AI model for text, vision, audio, and multimodal use, letting developers run…
- unsloth
unslothai · Apache-2.0
A free local app for running and training your own AI language and image models on your own machine.
- tokenspeed
lightseekorg · MIT
A high-speed engine for running large language model inference.
- freetoken
flashml-org · Apache-2.0
FreeToken lets you run massive, datacenter-scale open-weight AI models directly on your own gaming PC or laptop, with a desktop app and CLI that manage GPU/CPU memory…