Boxplots of the empirical distribution of distances of the lower 95% confidence bounds to the true AU C values (y-axis) for the bootstrap, and Learn2Evaluate without (L2E -BC) and with bias correction (L2E + BC) in the N = 100, ν = 1000 setting. The learning curve is fitted by an inverse power law and we determine n opt by MSE minimization. Outliers are not depicted.

Boxplots of the empirical distribution of distances of the lower 95% confidence bounds to the true AU C values (y-axis) for the bootstrap, and Learn2Evaluate without (L2E -BC) and with bias correction (L2E + BC) in the N = 100, ν = 1000 setting. The learning curve is fitted by an inverse power law and we determine n opt by MSE minimization. Outliers are not depicted.

Source publication
Preprint
Full-text available
In high-dimensional prediction settings, it remains challenging to reliably estimate the test performance. To address this challenge, a novel performance estimation framework is presented. This framework, called Learn2Evaluate, is based on learning curves by fitting a smooth monotone curve depicting test performance as a function of the sample size...

Similar publications

Preprint
Full-text available
Translating machine learning algorithms into clinical applications requires addressing challenges related to interpretability, such as accounting for the effect of confounding variables (or metadata). Confounding variables affect the relationship between input training data and target outputs. When we train a model on such data, confounding variabl...