Pinned Loading

1. llm-d/llm-d llm-d/llm-d Public

   Achieve state of the art inference performance with modern accelerators on Kubernetes

   Shell 3.2k 468

2. kubernetes-sigs/gateway-api-inference-extension kubernetes-sigs/gateway-api-inference-extension Public

   Gateway API Inference Extension

   Go 667 289

3. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 79.8k 16.7k

4. deepseek-ai/DeepEP deepseek-ai/DeepEP Public

   DeepEP: an efficient expert-parallel communication library

   Cuda 9.6k 1.2k

5. llm-d/llm-d-latency-predictor llm-d/llm-d-latency-predictor Public

   Latency prediction service for ML-model based scoring with llm-d-inference-scheduler

   Python 2 3

6. llm-d-inference-scheduler llm-d-inference-scheduler Public

   Forked from llm-d/llm-d-inference-scheduler

   Inference scheduler for llm-d

   Go