airllm
Lets you run massive open-source LLMs, including ones with hundreds of billions of parameters, on a single consumer GPU with as little as 4-12GB of VRAM, no quantization needed.
View lyogavin/airllm on GitHub- GitHub stars
- 42,287
- Forks
- —
- Language
- Jupyter Notebook
Project details
- Repository
- lyogavin/airllm
- Licence
- Apache-2.0
- Primary language
- Jupyter Notebook
- Catalog status
- Available
- Catalog check
- 2026-11-13
Repository description
AirLLM 2.8T LLM inference with single 4GB GPU
Explore by topic
More tools in Models & training
- soup
makazhanalpamys · Apache-2.0
Soup lets you fine-tune an 8-billion-parameter AI model on a laptop with just 4GB of GPU memory, using one YAML config file instead of wrestling with cloud setup.
- labllm
greninja9257 · MIT
LabLLM is a free native macOS app that lets you design a GPT-style model, train it from scratch on your own data using Apple Silicon, then fine-tune and chat with the…
- vllm
vllm-project · Apache-2.0
vLLM is an open-source engine that runs large language models fast and cheaply in production, used widely to self-host models like Llama, Qwen and DeepSeek.
- unsloth
unslothai · Apache-2.0
A free local app for running and training your own AI language and image models on your own machine.