KEYWORDS: Computed tomography, Magnetic resonance imaging, 3D image processing, Deep learning, Image guided radiation therapy, Deep convolutional neural networks
CT image synthesis from MR images is necessary for MR-only treatment planning, MRI-based quality assurance (QA), and treatment assessment in radiation therapy (RT). For pediatric cancer patients, reducing ionizing radiation from CT scans is preferred for which MRI-based RT planning and assessment are truly beneficial. Recently, deep learning-based synthetic CT (sCT) generation have demonstrated promising results on adult data. Generally, it is challenging to develop a pediatric sCT generation model due to significant anatomical variability and relatively smaller number of available pediatric data compared to adult. In this study, we investigated a 3D conditional generative adversarial network (cGAN)-based transfer learning approach for accurate pediatric sCT generation. Our model was first trained using adult data with augmentation by scaling to simulate pediatric data, followed by fine-tuning on pediatric data. We compared three different training scenarios; (1) training on 50 adult patient data with scaling augmentation, (2) training on combined 50 adult and 50 pediatric patient data, and (3) fine-tuning on 50 pediatric data using the pre-trained model on 50 adult data. 3D cGAN with transfer learning showed significantly better synthesis performance than the other models with average mean absolute error (MAE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM) index of 51.99 HU, 24.74, and 0.80, respectively. The proposed 3D cGAN-based transfer learning was able to accurately synthesize pediatric CT images from MRI, allowing us to realize pediatric MR-only RT planning, QA, and treatment assessment.
Radiation therapy (RT) planning for pediatric brain cancer is a challenging task. Manual contouring of organs-at-risk (OARs) is particularly difficult due to the small size of brain structures, time-consuming, and shows inter-observer variability. Furthermore, RT plans are typically optimized using CT, thus exposing patients to additional ionizing radiation. MR-only RT planning has recently been actively explored due to its potential to overcome these challenges. While numerous methods have been proposed to solve MR to CT image synthesis or OAR segmentation separately, there exist only a handful of methods tackling both problems jointly, even less specifically developed for pediatric brain cancer RT. We propose a multi-task convolutional neural network to jointly synthesize CT from MRI and segment OARs (eyes, optic nerves, optic chiasm, temporal lobes, hippocampi, and brainstem) for pediatric brain RT planning. The proposed network consists of a modified 3D U-Net architecture with a common encoder for both the synthesis and segmentation tasks combined with two task-specific decoders. The proposed model was trained, validated, and tested on 50, 5, and 15 pediatric brain RT cases, respectively, and achieved a mean±SD structural similarity index of 0.82±0.03 between the synthetic CT and ground truth CT, and dice score for the autosegmentation of 0.92±0.03 (eyes), 0.78±0.06 (optic nerves), 0.690.13 (optic chiasm), 0.91±0.02 (temporal lobes), 0.75±0.09 (hippocampi), and 0.91±0.06 (brainstem) compared to the expert’s manual segmentation. Our proposed multi-task joint synthesis and segmentation network achieves state-of-the-art performance for both tasks for MR-only RT planning.
Proton therapy planning requires Hounsfield unit (HU) data from CT images to calculate dose and, in the case of pelvic sarcomas, accurate registration of the CT to MRI to delineate tumor. MR-only proton therapy planning would eliminate the uncertainty associated with CT/MR image registration and the need for CT, reducing exposure to radiation and anesthesia in children. We determined whether MR-only proton therapy planning is feasible by introducing a transfer learning-based cycleGAN (TL-cycleGAN) method to convert pelvic MRI to synthetic CT (sCT) for dose calculation and to specifically address the challenge of a small training dataset, commonly associated with pediatric studies. The TLcycleGAN was designed to transfer knowledge gained from converting a large number (n=125) of pediatric brain MRI studies to sCT and finetune the well-trained model on pelvic data. Sixteen patients (aged 1.1–21.3 years, 7 females) who received proton therapy to the pelvis were randomly divided into training (n=11) and testing (n=5) groups. sCT generated from T1- T2-weighted fat suppression MR images was compared to the real CT in terms of peak signal-tonoise ratio (PSNR), structural similarity (SSIM) index, mean error (ME), and mean absolute error (MAE) in HU. The mean ± standard deviation of PSNR, SSIM, ME and MAE were 30.6±3.0, 0.93±0.04 -3.4±10.2 HU, and 52.4±17.6 HU, respectively, for T1W MRI; 29.2±1.5, 0.93±0.02, -6.6±24.8 HU, and 85.4±18.8 HU, respectively, for T2W MRI. Transfer learning facilitates MR-only pediatric pelvic proton therapy planning by generating highly accurate sCT for a small training dataset and a large variation.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.