Mlperf Tiny V1 3 Results

The MLCommons organization recently released the MLPerf Tiny v1.3 benchmark results, a significant milestone in the edge AI field. The benchmark focuses on resource-constrained microcontrollers and embedded devices, evaluating model performance in real-world deployments.

Test Scenarios Overview

This test covers four core tasks:

• Image Classification: MobileNet V2 model using the ImageNet dataset.
• Keyword Spotting: Based on the Google Speech Commands dataset.
• Anomaly Detection: For industrial sensor data.
• Visual Wake Words: Visual wake word detection.

The tests emphasize low power consumption, high throughput, and low latency, simulating real edge scenarios.

Top Results Overview

In the Image Classification Offline category, Arm's Cortex-M85 + Ethos-U85 NPU took the top spot with a score of 1895.1, far exceeding the previous record. NXP i.MX RT700 came second with 1623.4, followed closely by STMicroelectronics STM32N6 series with 1512.8.

Intel performed well on the Server side, demonstrating the edge optimization potential of the x86 platform.

Contribution from LMSYS Org

As one of the submitters, LMSYS Org demonstrated its strength in specific optimization tasks, leveraging efficient inference frameworks to improve model deployment efficiency. The organization is well known for projects such as Chatbot Arena and SGLang, and this participation further expands its footprint in the edge AI field.

Technical Highlights and Progress

• Added support for RISC-V architecture, expanding ecosystem compatibility.
• Introduced quantization optimization and INT8/INT4 precision testing to reduce memory footprint.
• Significant performance improvements: Image Classification saw an average 25% increase over v1.2.

These results highlight the importance of hardware-software co-optimization, driving AI penetration into more compact devices.

Future Outlook

MLPerf Tiny will continue to iterate, with v1.4 expected to incorporate more multimodal tasks. Developers can download the full dataset and submission details via the official link to accelerate edge AI innovation.