Computer Vision News - November 2022

56 Biomedical Image Analysis Validation performed in a way that’s not entirely correct, ” Minu tells us. “ A big part of this is the inappropriate use of validation metrics, the measures against which algorithm performance is assessed. People wrongly apply or use metrics that aren’t mathematically suited to the underlying problem. That was the starting point for us to investigate. How can we help people choose the right metrics? ” The group found that rankings used in popular community challenges are sometimes computed in ways that are not Validation is critical to automated AI- basedbiomedical imageanalysis .However, while research focuses on developing new models and algorithms, less attention is paid to whether thesealgorithms arebeing appropriately validated . If an algorithm is not validated according to the relevant outcome measures, it is not possible to make a reliable statement about whether it has the potential to be used in a practical setting in the future or if, for all intents and purposes, it just looks good on paper. “ Our group found that validation is often METRICS RELOADED Paul Jäger is the Principal Investigator of the Interactive Machine Learning Research Group at the German Cancer Research Center (DKFZ) in Heidelberg, where Minu Tizabi is a Scientist in Lena Maier-Hein’ s Intelligent Medical Systems Research Group. They speak to us about their follow-up work to Metrics Reloaded, a recommendation framework for biomedical image analysis validation. Paul Jäger Photography Melanie Maerz Minu Tizabi