Computer Vision News - October 2022

63 Reuben Dorent Learning from partially annotated datasets. Since there is no large annotated dataset for joint brain structure and tumour segmentation, we proposed to exploit annotated databases that are partially annotated and hetero-modal. Starting from a variational formulation of the joint problem, we leveraged the disjoint nature of the label sets to propose a practical decomposition of the joint loss. We then minimised the expected risk under the constraint of missing modalities via a tractable upper bound. The proposed approach achieved higher accuracy than well-established atlas-based approaches while not requiring manual tumour delineation. Handling missing imaging modalities. We proposed a principled formulation using probabilistic graphical modelling to handle missing imaging modalities at inference time. Specifically, all imaging modalities (including segmentation) are assumed to be conditionally independent via a multi-scale latent representation. As a result, the proposed framework successfully performs image segmentation and image reconstruction with incomplete sets of input images. Improving robustness using weak or missing annotations. We explored weakly-supervised and unsupervised approaches to ensure that a network trained on a data distribution can successfully generalise on another one. This led to the creation of the first medical benchmark for cross-modality domain adaptation (crossMoDA). The level of performance reached by the top-performing teams from all over the world is strikingly high and close to full supervision. Nextsteps: Exploitingthedevelopedalgorithmsforpre- tointra-operativeimageregistration! Goal: Robust segmentation models for joint (multi-class) problems FLAIR Lesion ... missing Anatomy Anatomy T1 ... missing Example 1 Example 2 Lesion Partial annotations Different imaging sequences Different acquisition settings T1 Joint segmentation missing FLAIR missing missing Existing datasets FLAIR Lesion ... missing Anatomy Example 3 T1 missing Weakly annotations Domain Adaptation Hetero-Modality Weakly-Supervised Learning Multi-Task Learning