The current standard of care for early stages of breast cancer is breast-conserving surgery (BCS). BCS involves a lumpectomy procedure, during which the tumor is removed with a rim of normal tissue-if cancer cells found in that rim of tissue, it is called a positive margin and means part of the tumor remains in the breast. Currently there is no method to determine if cancer cells exist at the margins of lumpectomy specimens aside from time-intensive histology methods that result in reoperations in up to 38% of cases. We used fluorescence lifetime imaging (FLIm) to measure time-resolved autofluorescence from N=13 ex vivo human breast cancer specimens (N=10 patients undergoing lumpectomy or mastectomy) and compared our results to histology. Tumor (both invasive and ductal carcinoma in situ), fibrous tissue, fat and fat necrosis have unique fluorescence signatures. For instance, between 500-580 nm, fluorescence lifetime of tumor was shortest (4.7 ± 0.4 ns) compared to fibrous tissue (5.5 ± 0.7 ns) and fat (7.0 ± 0.1 ns), P<0.05 (ANOVA). These differences are due to the biochemical properties of lipid, nicotineamide adenine dinucleotide (NADH) and collagen fibers in the fat, tumor and fibrous tissue, respectively. Additionally, the FLIm data is augmented to video of the breast tissue with image processing algorithms that track a blue (450 nm) aiming beam used in parallel with the 355 nm excitation beam. This allows for accurate histologic co-registration and in the future will allow for three-dimensional lumpectomy surfaces to be imaged for cancer margin delineation.