doi: 10.1016/j.media.2022.102683.
Online ahead of print.
Affiliations
Affiliations
- 1 Penn Image Computing and Science Laboratory (PICSL), Department of Radiology, University of Pennsylvania, Philadelphia, USA. Electronic address: [email protected].
- 2 Department of Diagnostic Radiology, Lund University, Lund, Sweden.
- 3 Penn Image Computing and Science Laboratory (PICSL), Department of Radiology, University of Pennsylvania, Philadelphia, USA.
- 4 Penn Memory Center, University of Pennsylvania, Philadelphia, USA; Department of Neurology, University of Pennsylvania, Philadelphia, USA.
Item in Clipboard
Long Xie et al.
Med Image Anal.
.
Display options
Format
doi: 10.1016/j.media.2022.102683.
Online ahead of print.
Affiliations
- 1 Penn Image Computing and Science Laboratory (PICSL), Department of Radiology, University of Pennsylvania, Philadelphia, USA. Electronic address: [email protected].
- 2 Department of Diagnostic Radiology, Lund University, Lund, Sweden.
- 3 Penn Image Computing and Science Laboratory (PICSL), Department of Radiology, University of Pennsylvania, Philadelphia, USA.
- 4 Penn Memory Center, University of Pennsylvania, Philadelphia, USA; Department of Neurology, University of Pennsylvania, Philadelphia, USA.
Item in Clipboard
Display options
Format
Abstract
Deep convolutional neural networks (DCNN) achieve very high accuracy in segmenting various anatomical structures in medical images but often suffer from relatively poor generalizability. Multi-atlas segmentation (MAS), while less accurate than DCNN in many applications, tends to generalize well to unseen datasets with different characteristics from the training dataset. Several groups have attempted to integrate the power of DCNN to learn complex data representations and the robustness of MAS to changes in image characteristics. However, these studies primarily focused on replacing individual components of MAS with DCNN models and reported marginal improvements in accuracy. In this study we describe and evaluate a 3D end-to-end hybrid MAS and DCNN segmentation pipeline, called Deep Label Fusion (DLF). The DLF pipeline consists of two main components with learnable weights, including a weighted voting subnet that mimics the MAS algorithm and a fine-tuning subnet that corrects residual segmentation errors to improve final segmentation accuracy. We evaluate DLF on five datasets that represent a diversity of anatomical structures (medial temporal lobe subregions and lumbar vertebrae) and imaging modalities (multi-modality, multi-field-strength MRI and Computational Tomography). These experiments show that DLF achieves comparable segmentation accuracy to nnU-Net (Isensee et al., 2020), the state-of-the-art DCNN pipeline, when evaluated on a dataset with similar characteristics to the training datasets, while outperforming nnU-Net on tasks that involve generalization to datasets with different characteristics (different MRI field strength or different patient population). DLF is also shown to consistently improve upon conventional MAS methods. In addition, a modality augmentation strategy tailored for multimodal imaging is proposed and demonstrated to be beneficial in improving the segmentation accuracy of learning-based methods, including DLF and DCNN, in missing data scenarios in test time as well as increasing the interpretability of the contribution of each individual modality.
Keywords:
Deep learning; Generalization; Multi-atlas segmentation; Multimodal image analysis.
Copyright © 2022 Elsevier B.V. All rights reserved.
Conflict of interest statement
Declaration of Competing Interest Dr. Xie became an employee of Siemens Healthineers in May 2022, but the current study was conducted during his employment at the University of Pennsylvania. The authors declare that no other known competing financial interests or personal relationships could have appeared to influence the work reported in this paper.
Cite