Get Started
Home
Topics
Search
Library
Research questionHow can Bayes-optimal BER and AUC be estimated from corrupted soft labels with unknown class priors?Class imbalance and noisy annotations can make accuracy a misleading summary of binary-classification performance. The difficulty is inferring the best achievable BER or AUC when soft labels are distorted or noisy and the true class prior is unavailable.
AI
Evaluation & Benchmarks
Machine Learning
Statistical Machine Learning
Latest papersRecent research connected to this question, newest first.Bayes-Optimal BER and AUC: Estimation and Evaluation of EstimatorsThe evidence covers settings with known true soft labels and class priors, as well as settings where soft labels undergo an unknown order-preserving transformation and possibly additive noise, the class prior is unknown, and auxiliary hard labels are available. It includes estimators, finite-sample error bounds, and an evaluation procedure for optimal BER or AUC estimators without requiring the true optimum, validated on synthetic and real-world datasets.research paper · Sep 2, 2026
Related questions
How can we measure label-preserving adversarial vulnerability separately from ordinary misclassification in high-dimensional classifiers?Can one squared-loss estimator achieve both minimax and universal exponential rates for finite versus countably infinite hypothesis classes?How can teams estimate whether multiclass accuracy gains reduce optimization regret before building a model?How can we interpret multimodal video model performance when benchmark label reliability is unknown?