MLCommons Releases New MLPerf Storage v3.0 Benchmark Results

MLPerf Storage v3.0 Benchmark Officially Released

MLCommons recently announced the release of results for the industry-standard MLPerf Storage v3.0 benchmark suite. The benchmark measures storage system performance under machine learning workloads in an architecture-neutral, representative, and reproducible way.

Two New Benchmarks Added

Version 3.0 introduces the KV Cache test to evaluate storage performance for cache read and write operations during LLM inference. The KV cache is a key technology for improving the inference speed of transformer models, especially for autoregressive LLMs.

It also adds the Vector Database (VDB) test, which measures storage performance for vector indexing and query workloads. Vector databases are widely used to store high-dimensional unstructured data and are an important component of AI applications.

Support for S3 Object Storage Access Layer

The new version also adds support for the S3 object storage access layer as an alternative to the POSIX layer. This improvement makes it possible to compare more managed storage options under the same workload. About one-sixth of the submissions used the S3 access layer.

11 New Organizations Submit for the First Time

A total of 19 organizations submitted results in this round, including 11 first-time submitters such as Azure, NVIDIA, and Nebius. Submitters included cloud and on-premises solution providers as well as storage system developers.

The results show the current state of power efficiency in the storage industry: for the checkpointing write test, the median for on-premises submissions was 14 GB/s per watt, with a maximum of 201 GB/s; for the UNet3D read test, the median was 34 GB/s per watt, with a maximum of 277 GB/s.