The purpose of this study was to develop a robust, automated multi-organ segmentation model for clinical adult and pediatric CT and implement the model as part of a patient-specific safety and quality monitoring system. 3D convolutional neural network (Unet) models were setup to segment 30 different organs and structures at the diagnostic image resolution. For each organ, 200 manually-labeled cases were used to train the network, fitting it to different clinical imaging resolutions and contrast enhancement stages. The dataset was randomly shuffled, and divided with 6/2/2 train/validation/test set split. The model was deployed to automatically segment 1200 clinical CT images as a demonstration of the utility of the method. Each case was made into a patient-specific phantom based on the segmentation masks, with unsegmented organs and structures filled in by deforming a template XCAT phantom of similar anatomy. The organ doses were then estimated using a validated scanner-specific MC-GPU package using the actual scan information. The segmented organ information was likewise used to assess contrast, noise, and detectability index within each organ. The neural network segmentation model showed dice similarity coefficients (DSC) above 0.85 for the majority of organs. Notably, the lungs and liver showed a DSC of 0.95 and 0.94, respectively. The segmentation results produced patient-specific dose and quality values across the tested 1200 patients with representative the histogram distributions. The measurements were compared in global-to-organ (e.g. CTDvol vs. liver dose) and organ-to-organ (e.g. liver dose vs. spleen dose) manner. The global-to-organ measurements (liver dose vs. CTDIvol: 𝑅 = 0.62; liver vs. global d’: 𝑅 = 0.78; liver vs. global noise: 𝑅 = 0.55) were less correlated compared to the organ-to-organ measurements (liver vs. spleen dose: 𝑅 = 0.93; liver vs. spleen d’: 𝑅 = 0.82; liver vs. spleen noise: 𝑅 = 0.78). This variation of measurement is more prominent for iterative reconstruction kernel compared to the filtered back projection kernel (liver vs. global noise: 𝑅𝐼𝑅 = 0.47 vs. 𝑅𝐹𝐵𝑃 = 0.75; liver vs. global d’: 𝑅𝐼𝑅 = 0.74 vs. 𝑅𝐹𝐵𝑃 = 0.83). The results can help derive meaningful relationships between image quality, organ doses, and patient attributes.