llama-modchip
Lets you run a huge 284-billion-parameter AI model on an ordinary 64GB computer using only the CPU, no GPU required.
View synchopate/llama-modchip on GitHub- GitHub stars
- 1,183
- Forks
- —
- Language
- Python
Project details
- Repository
- synchopate/llama-modchip
- Licence
- Apache-2.0
- Primary language
- Python
- Catalog status
- Available
- Catalog check
- 2026-11-13
Repository description
Run 284B DeepSeek-V4-Flash (<59GiB RAM: 41.1 cold pp, 9.1 N=1 tg, up to 25.3 spec tok/s) & 30B MoEs (22.4 N=1 / 111.6 spec tok/s) on 64GB CPU (0 GPUs).
Explore by topic
More tools in Models & training
- project-maya
mw00 · MIT
Lets anyone run a very large open-weight AI model privately on their own GPUs instead of relying on a cloud provider.
- slotstream
carloslfu · MIT
Runs a 125-billion-parameter language model on a Mac with far less RAM than it needs by streaming parts of the model from the SSD.
- kaggle-tpu-lab
arahim3 · MIT
A recipe for serving the 27B-parameter Qwen3.8 model on Kaggle free TPUs with a public OpenAI-compatible endpoint and a 262k-token context window, no paid GPU.
- kimi-k3-in-c
fareedkhan-dev · Apache-2.0
Lets a 2.78-trillion-parameter AI model run inference on a regular laptop with no GPU and just 8GB of RAM, by rewriting the whole inference engine in dependency-free C.