The incidence of esophageal squamous cell carcinoma (ESCC) in China is high and the prognosis is poor. Aimed to evaluate the ESCC, we investigated computerized quantitative analyses on diagnostic computed tomography (CT) and digital histopathological slices. A retrospective study with IRB approval was conducted to assess the prognosis of ESCCs in 158 patients who underwent esophagectomy. Every of them has 3D CT image and pathological hematoxylin-eosin staining tissue slide. We performed quantitative analysis on digital histology slices and diagnostic CT volumes. A total of 125 computerized quantitative features were extracted including 20 pathological image clustering (PIC) features and 105 CT features. Then we selected 45 CT features by correlation analysis and 5 PIC features by univariate survival analysis for preventing overfitting. Cox hazard model with L1 penalization was used for prognostic indexing. We compared three different prognostic indices in order to build a robust model: Model A: using CT features alone; Model B: using PIC features alone; Model C: using both CT and PIC features. For testing the model, we performed leave-one-out cross-validation for testing and we used the index of concordance(C-index) to access the accuracy of the prognostic model. The testing C-index values of Model A, B, C were 0.667, 0.601 and 0.711. The improvement of the final Model C is significant and it’s able to provide effective stratification. Quantitative prognostic modeling with joint information from clinical, histopathology, and diagnostic CT has the ability to stratify the patient well. Our prognostic model has potential to impact the perioperative care in the future clinical practice and improve the quality of individual life.