FlashInfer: Kernel Library for LLM Serving

Grok open release

Training and serving large-scale neural networks with auto parallelization.

A high-throughput and memory-efficient inference and serving engine for LLMs

AICI: Prompts as (Wasm) Programs

MSCCL++: A GPU-driven communication stack for scalable AI applications

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. Tensor…

Updated C version of the Test Suite for Vectorising Compilers

NCCL Profiling Kit

TACCL: Guiding Collective Algorithm Synthesis using Communication Sketches

Microsoft Collective Communication Library

Graph-structured Indices for Scalable, Fast, Fresh and Filtered Approximate Nearest Neighbor Search

Development repository for the Triton language and compiler

Synthesizer for optimal collective communication algorithms