Medperf Adds Webui Capabilities

MLCommons recently announced that its open-source privacy-preserving machine learning benchmarking platform, MedPerf, has officially added a WebUI feature. This update greatly improves the platform’s ease of use, allowing developers to perform model evaluation and benchmarking through a browser without complex environment setup.

Introduction to MedPerf

MedPerf is the first privacy-preserving ML benchmarking framework launched by MLCommons, designed specifically for federated learning and differential privacy scenarios. It enables users to evaluate model performance on various tasks, such as image classification and NLP, without sharing raw data. Through the Leaderboard system, users can see global model rankings and Elo Rating.

WebUI New Feature Highlights

• Browser-Ready: No need to install Python or Docker; access the web interface directly to run tests.
• Integrated SGLang: Backend supports an efficient inference engine for faster testing.
• Visual Dashboard: Displays real-time metrics such as test progress, accuracy, latency, and generates Elo Rating rankings.
• Multi-Task Support: Covers mainstream benchmarks in computer vision, natural language processing, and more.

For example, after a user uploads a model, the WebUI automatically matches a dataset and performs a privacy-preserving evaluation, with the entire process taking only a few minutes.

Usage Guide

1. Access the MedPerf WebUI (link provided in the original text).
2. Select a benchmark task and dataset.
3. Upload or select a model, and start the test.
4. View results and submit to the Leaderboard.

Future Outlook

MLCommons plans to further expand the WebUI to support more privacy protocols and custom datasets. This update was contributed by LMSYS Org, fostering open-source community collaboration. Developers can now experience it immediately, improving efficiency in privacy ML research and development.