A major clinical challenge in rectal cancer currently is non-invasive identification of tumor regression to standard- of-care neoadjuvant chemoradiation (CRT). Multi-parametric MRI is routinely acquired after CRT, but expert radiologists find it highly challenging to assess the degree of tumor regression on both T2-weighted (T2w) and Gadolinium contrast-enhanced (CE) MRI; resulting in poor agreement with gold-standard pathologic evaluation. In this study, we present initial results for integrating quantitative image appearance (radiomic) features from post-CRT T2w and CE MRI towards in vivo assessment of pathologic rectal tumor response to chemoradiation. 29 rectal cancer patients with post-CRT multi-parametric 3 T MRI (with T2w, initial and delayed CE phases) were included in this study. Through spatial co-registration, the treated region of the rectal wall was identified and annotated on T2w and all CE phases (as well as correcting for motion artifacts in CE MRI). 165 radiomic features (including Haralick, Gabor, Laws, Sobel/Kirsch) were separately extracted from each of T2w and 2 CE phases; within the entire rectal wall. The top 2 response-associated radiomic features for each of (a) T2w, (b) 2 CE phases, (c) combined T2w+CE phases were identified via feature selection and evaluated in a leave- one-patient-out cross validation setting. Integrating T2w and CE radiomic features was found to be markedly more accurate (AUC=0.93) for assessing post-CRT pathologic tumor stage, compared to T2w radiomic features (AUC=0.80) and CE radiomic features (AUC=0.63) individually. Top-ranked features captured heterogeneity of gradient responses on T2w MRI and macro-scale Gabor wavelet responses of contrast enhancement on CE MRI. Combining radiomic features from post-CRT T2w and CE MRI may hence enable more comprehensive evaluation of response to neoadjuvant therapy in rectal cancers, which can be used to better guide follow-up interventions.