taskdistill
A tool that watches how you actually use an expensive AI model on one task, then trains a small local model to handle most of those calls itself, falling back to the big model only when unsure.
View b0yko/taskdistill on GitHub- GitHub stars
- 3
- Forks
- —
- Language
- Python
Project details
- Repository
- b0yko/taskdistill
- Licence
- Apache-2.0
- Primary language
- Python
- Catalog status
- Available
- Catalog check
- 2026-11-13
Repository description
Distil an expensive LLM API call on a narrow task into a small local model: capture traffic, curate, LoRA fine-tune with MLX on Apple Silicon, evaluate against the teacher, and serve an OpenAI-compatible cascade that escalates low-confidence requests.
Explore by topic
More tools in Models & training
- labllm
greninja9257 · MIT
LabLLM is a free native macOS app that lets you design a GPT-style model, train it from scratch on your own data using Apple Silicon, then fine-tune and chat with the…
- soup
makazhanalpamys · Apache-2.0
Soup lets you fine-tune an 8-billion-parameter AI model on a laptop with just 4GB of GPU memory, using one YAML config file instead of wrestling with cloud setup.
- slotstream
carloslfu · MIT
Runs a 125-billion-parameter language model on a Mac with far less RAM than it needs by streaming parts of the model from the SSD.
- unsloth
unslothai · Apache-2.0
A free local app for running and training your own AI language and image models on your own machine.