Although photon counting systems have shown strong clinical potential, this technology has not yet been fully evaluated or optimized for specific clinical applications. The purpose of this study was to develop a framework for realistic virtual clinical trials (VCTs) in photon counting CT (PCCT) imaging. We developed a photon counting CT simulator based on the geometry and physics of an existing research prototype scanner. The developed simulator models primary, scatter, and noise signals, detector responses, vendor-specific bowtie filters and X-ray spectra, axial/helical trajectories, vendor-specific acquisition modes, and multiple energy thresholds per detector pixel. The simulation procedure is accelerated by parallel processing using multiple GPUs. The generated projection images can be reconstructed using generic reconstruction algorithms as well as a commercial reconstruction software (ReconCT Siemens). A computational model of a physical Mercury phantom was imaged at multiple energy thresholds (25 and 75 keV) and dose levels (36, 72, 144, and 216 mAs). Noise magnitude was measured in the simulated images and compared against noise measurements in a real scan acquired with a research prototype photon counting scanner (Siemens Healthcare). The results showed that our simulator was capable of synthesizing realistic photon counting CT data. The simulator can be combined with realistic 4D high-resolution XCAT phantoms with intra-organ heterogeneities to conduct VCTs for specific clinical applications. This framework can greatly facilitate the evaluation, optimization, and eventual clinical use of PCCT.