Ada-MK: Adaptive MegaKernel Optimization via Automated DAG-based Search for LLM Inference
GPUHBMMLIRTensorRT-LLMmemoryvLLM
100 重要
65 热度
91 综合
AI Infra
推理系统、Serving、GPU/Kernel、部署架构和成本优化。
Blink: CPU-Free LLM Inference by Delegating the Serving Stack to GPU and SmartNIC
GPUNPURDMASGLangTensorRT-LLMmemory
100 重要
65 热度
91 综合
Comparative Analysis of Large Language Model Inference Serving Systems: A Performance Study of vLLM and HuggingFace TGI
GPUPagedAttentionTGImemoryvLLM推理系统
100 重要
65 热度
91 综合
TokenWeave: Efficient Compute-Communication Overlap for Distributed LLM Inference
GPUNVLinkSGLangTensorRT-LLMvLLM
100 重要
65 热度
91 综合
FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving
DiTFlashInferGPUSGLangmemoryvLLM
100 重要
65 热度
91 综合
COMET: Towards Partical W4A4KV4 LLMs Serving
DiTGPUKV cacheTensorRT-LLMmemoryquantization
100 重要
65 热度
91 综合
SCOOT: SLO-Oriented Performance Tuning for LLM Inference Engines
TensorRT-LLMvLLM
100 重要
65 热度
91 综合
QServe: W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
DiTGPUKV cacheTensorRT-LLMmemoryquantization
100 重要
65 热度
91 综合
SGLang: Efficient Execution of Structured Language Model Programs
KV cacheNPUSGLang
100 重要
65 热度
91 综合