Popular repositories Loading

1. control-layer control-layer Public

   The world’s fastest AI model gateway (450x less overhead than LiteLLM). Unified access to LLMs across endpoints (openAI, self-hosted, etc.) behind a single authentication layer - with API key gener…

   Rust 43 3

2. deepseek-reddit-agent deepseek-reddit-agent Public

   An example notebook which shows how you can build a LLM agent that scrapes information from Reddit and summarize key bullets using a self-hosted DeepSeek-R1-Distill-Llama-8B deployed with Titan Tak…

   Jupyter Notebook 11 2

3. inference-stack inference-stack Public

   The Doubleword Inference Stack is the easiest & most performant way to run genAI infrastructure in your private environment.

   Smarty 2

4. outlet outlet Public

   A high-performance Axum middleware for capturing and correlating HTTP requests and responses with full streaming support.

   Rust 2

5. onwards onwards Public

   A router for openAI compatible endpoints

   Rust 1

6. fusillade fusillade Public

   Batched LLM request processing daemon with efficient request coalescing and per-model concurrency control

   Rust 1