This study aimed to develop a simulation framework to synthesize accurate and scanner-specific Computed Tomography (CT) images of voxel-based computational phantoms. Two phantoms were used in the simulations, a geometry-based Mercury phantom and a “textured” anthropomorphic XCAT phantom, both with an isotropic voxel size of 0.25 mm. The simulator geometry and physics were based on a clinical scanner. The projection images were calculated by computing each detector’s signal using the Beer-Lambert law. To avoid aliasing artifacts, the focal spot and detectors were subsampled four and nine times, respectively. The simulator was designed to function both axially and helically, and account for “Z” and in-plane flying focal spots and various bowtie filters. Quantum and electronic noise were added to the detector signals as a function of the tube current using experimental measurements. The resulting projection images were calibrated to suppress the beam hardening artifact using a 4th-order polynomial water correction. The simulation procedure was accelerated using multi-threading and graphics processing unit (GPU) computing. The projection images were reconstructed using clinical reconstruction software. To evaluate the accuracy of the simulator, the reconstructed images of the computational Mercury phantom were compared against experimental CT scans of its physical counterpart in terms of resolution, noise, and HU values. Results showed that our proposed simulator can generate CT images with image quality attributes close to real clinical data. The new CT simulator, combined with anthropomorphic “textured” phantoms, provides a new way to generate clinically realistic CT data and has the potential to enable virtual clinical studies in advance or in lieu of costly clinical trials.