Deep label fusion: A generalizable hybrid multi-atlas and deep convolutional neural network for medical image segmentation


doi: 10.1016/j.media.2022.102683.


Online ahead of print.

Affiliations

Item in Clipboard

Long Xie et al.


Med Image Anal.


.

Abstract

Deep convolutional neural networks (DCNN) achieve very high accuracy in segmenting various anatomical structures in medical images but often suffer from relatively poor generalizability. Multi-atlas segmentation (MAS), while less accurate than DCNN in many applications, tends to generalize well to unseen datasets with different characteristics from the training dataset. Several groups have attempted to integrate the power of DCNN to learn complex data representations and the robustness of MAS to changes in image characteristics. However, these studies primarily focused on replacing individual components of MAS with DCNN models and reported marginal improvements in accuracy. In this study we describe and evaluate a 3D end-to-end hybrid MAS and DCNN segmentation pipeline, called Deep Label Fusion (DLF). The DLF pipeline consists of two main components with learnable weights, including a weighted voting subnet that mimics the MAS algorithm and a fine-tuning subnet that corrects residual segmentation errors to improve final segmentation accuracy. We evaluate DLF on five datasets that represent a diversity of anatomical structures (medial temporal lobe subregions and lumbar vertebrae) and imaging modalities (multi-modality, multi-field-strength MRI and Computational Tomography). These experiments show that DLF achieves comparable segmentation accuracy to nnU-Net (Isensee et al., 2020), the state-of-the-art DCNN pipeline, when evaluated on a dataset with similar characteristics to the training datasets, while outperforming nnU-Net on tasks that involve generalization to datasets with different characteristics (different MRI field strength or different patient population). DLF is also shown to consistently improve upon conventional MAS methods. In addition, a modality augmentation strategy tailored for multimodal imaging is proposed and demonstrated to be beneficial in improving the segmentation accuracy of learning-based methods, including DLF and DCNN, in missing data scenarios in test time as well as increasing the interpretability of the contribution of each individual modality.


Keywords:

Deep learning; Generalization; Multi-atlas segmentation; Multimodal image analysis.

Conflict of interest statement

Declaration of Competing Interest Dr. Xie became an employee of Siemens Healthineers in May 2022, but the current study was conducted during his employment at the University of Pennsylvania. The authors declare that no other known competing financial interests or personal relationships could have appeared to influence the work reported in this paper.

Share on facebook
Facebook
Share on twitter
Twitter
Share on linkedin
LinkedIn
Share on vk
VK
Share on pinterest
Pinterest
Close Menu