Current digital mammography systems primarily employ one of two types of detectors: indirect conversion, typically using a cesium-iodine scintillator integrated with an amorphous silicon photodiode matrix, or direct conversion, using a photoconductive layer of amorphous selenium (a-Se) combined with thin-film transistor array. The goal of this study was to evaluate a methodology for objectively assessing image quality to compare human observer task performance in detecting microcalcification clusters and extended mass-like lesions achieved with different detector types. The proposed assessment methodology uses a novel anthropomorphic breast phantom fabricated with ink-jet printing. In addition to human observer detection performance, standard linear metrics such as modulation transfer function, noise power spectrum, and detective quantum efficiency (DQE) were also measured to assess image quality. An Analogic Anrad AXS-2430 a-Se detector used in a commercial FFDM/DBT system and a Teledyne Dalsa Xineos-2329 with CMOS pixel readout were evaluated and compared. The DQE of each detector was similar over a range of exposures. Similar task performance in detecting microcalcifications and masses was observed between the two detectors over a range of clinically applicable dose levels, with some perplexing differences in the detection of microcalcifications at the lowest dose measurement. The evaluation approach presented seems promising as a new technique for objective assessment of breast imaging technology.