patchy631 / time-to-first-token

time-to-first-token

A free 10-week roadmap for learning how LLM inference serving actually works under the hood.

Project details

A 10-week, 30-minutes-a-day roadmap for LLM inference serving and optimization. vLLM, SGLang, quantization, speculative decoding, benchmarking.

ai-engineering-lab zorost · MIT
Zorost Intelligence AI Lab open-sourced a free, self-paced 24-week AI engineering course that takes a beginner from Python to production LLM/RAG/agent systems, with 43…
kimi-k3-in-c fareedkhan-dev · Apache-2.0
Lets a 2.78-trillion-parameter AI model run inference on a regular laptop with no GPU and just 8GB of RAM, by rewriting the whole inference engine in dependency-free C.
project-maya mw00 · MIT
Lets anyone run a very large open-weight AI model privately on their own GPUs instead of relying on a cloud provider.
kaggle-tpu-lab arahim3 · MIT
A recipe for serving the 27B-parameter Qwen3.8 model on Kaggle free TPUs with a public OpenAI-compatible endpoint and a 262k-token context window, no paid GPU.