trnrt
An open-source engine that runs large language models efficiently on AWS Trainium chips, with an OpenAI-compatible API and support for 1-million-token context windows.
View runanywhereai/trnrt on GitHub- GitHub stars
- 1,570
- Forks
- —
- Language
- Python
Project details
- Repository
- runanywhereai/trnrt
- Licence
- Apache-2.0
- Primary language
- Python
- Catalog status
- Available
- Catalog check
- 2026-11-13
Repository description
The open LLM inference engine for AWS Trainium: NKI megakernels, 1M-token context, OpenAI-compatible API. By RunAnywhere.
Explore by topic
More tools in Models & training
- vllm
vllm-project · Apache-2.0
vLLM is an open-source engine that runs large language models fast and cheaply in production, used widely to self-host models like Llama, Qwen and DeepSeek.
- project-maya
mw00 · MIT
Lets anyone run a very large open-weight AI model privately on their own GPUs instead of relying on a cloud provider.
- kaggle-tpu-lab
arahim3 · MIT
A recipe for serving the 27B-parameter Qwen3.8 model on Kaggle free TPUs with a public OpenAI-compatible endpoint and a 262k-token context window, no paid GPU.
- transformers
huggingface · Apache-2.0
The open-source library that provides ready-to-use implementations of virtually every major AI model for text, vision, audio, and multimodal use, letting developers run…