Virtual Clinical Trials (VCTs) of breast imaging have been used as a tool for the evaluation and optimization of novel imaging systems through computer simulations of breast anatomy, image acquisition, and interpretation. VCTs offer significant advantages over clinical trials in terms of cost, duration, and radiation risk. The performance of VCTs depends on the selection of simulated breasts to represent the population of interest. We have developed a method for selecting populations of software breast phantoms to match the clinical distribution of compressed breast thickness and breast percent density. We extracted the compressed thickness information from anonymized DICOM headers of mammography images from 10,705 women who had their breast screening exams within a year (09/2010-08/2011). Percent density was estimated using an open source software tool. Characteristic clinical sub-populations were identified by performing k-means clustering, and represented by separate sets of phantoms. The corresponding thickness of uncompressed phantoms was selected assuming 50% thickness reduction during mammographic compression. The phantom volumetric density was selected based upon a relationship between mammographic (2D) percent density and volumetric (3D) density, estimated from clinical images. Using a set of 24 representative phantoms, we were able to match the analyzed clinical population completely for the compressed breast thickness, and within two percentage points of the volumetric breast density. Representative phantoms can be used to generate the full population of virtual patients, of a size determined by the power-analysis of the specific VCT, by random variations of the internal phantom composition.