Deepseek Inference 5 1

DeepSeek Inference 5.1 Released

The latest report released by LMSYS Org shows that DeepSeek Inference 5.1 stands out in the MLCommons September 2025 inference benchmark. This engine is specifically designed for large language models (LLMs), focusing on low-latency and high-throughput inference optimization.

Core Features

• Multi-Framework Support: Seamless integration with SGLang, vLLM, and TensorRT-LLM, enhancing compatibility.
• Memory Optimization: Utilizes PagedAttention and KV cache quantization to reduce memory usage by up to 40%.
• Distributed Inference: Supports tensor parallelism and pipeline parallelism, suitable for multi-GPU clusters.

Benchmark Results

In the Chatbot Arena Elo Rating evaluation, models powered by DeepSeek Inference 5.1 achieved an average score of 1285, leading competitors by 50+ points. Specific data from the MLPerf Inference v5.1 test:

• Llama 3.1 405B: Throughput 2.5x vLLM, latency reduced by 30%.
• GPT-4o scale: QPS increased to 450, supporting 8x H100 cluster.
• Energy efficiency: Tokens per watt increased by 1.8x.

Comparison chart (based on original data):

Practical Applications and Recommendations

DeepSeek Inference 5.1 is suitable for production-grade chatbots and enterprise-level deployment. Developers can quickly get started with pip install deepseek-inference. Future versions are expected to integrate more MoE model support.

Overall, this version marks a new milestone for open-source inference engines, further driving down AI inference costs.