Vlm Inference Shopify

Introduction

MLCommons released the latest VLM (Vision-Language Model) inference benchmark results, with the submission from the Shopify team attracting significant attention. Supported by LMSYS Org, the benchmark focuses on the inference performance of vision-language models in high-load e-commerce scenarios, aiming to provide standardized evaluation for production deployment.

Benchmark Framework and Models

The benchmark adopts the MLPerf Inference v5.0 framework, with core models including LLaVA-1.5-7B and MiniCPM-V 2.6. The dataset covers ImageNet-VQA, VQAv2, etc., simulating real-world tasks such as e-commerce visual question answering and product description generation.

• Metrics: Throughput (images/sec), latency (ms), accuracy (accuracy score).
• Hardware: NVIDIA A100/H100 GPU, software stack includes vLLM, SGLang.

Shopify Optimization Highlights

The Shopify team cleverly integrated SGLang for dynamic batching and KV cache optimization, achieving 150+ images/sec throughput on a single A100, with end-to-end latency below 200ms and Elo Rating reaching 1250+, surpassing submissions from multiple cloud vendors.

Key Results Comparison

The chart shows Shopify leads in multi-GPU scalability, achieving linear scaling to 1200+ img/s on an 8x A100 configuration.

Challenges and Insights

The benchmark exposed VLM inference pain points: high-resolution image preprocessing is time-consuming, and long-context token explosion occurs. Shopify recommends using quantization (INT8) and Speculative Decoding to improve deployment efficiency.

Conclusion

This VLM inference benchmark highlights Shopify's strength in AI infrastructure, driving the deployment of e-commerce VLMs. Looking ahead, more open-source optimizations are expected; stay tuned for MLCommons updates.