Time-domain diffuse optical tomography (TD-DOT) is interested to many scholars due to its unique features such as rich information in each measured pulse, elimination of cross-talk in the reconstruction, and no need for the measurement calibration, although TD-DOT imaging system requires very expensive equipment. Usually, the generalized pulse spectrum technique (GPST) is applied to transform the real domain optical diffusion forward model into its Laplacian domain. However, the application of TD-DOT has been limited by the low spatial resolution due to strong optical scattering in tissues. In this study, we introduced the anatomical guidance from computed tomography (CT) and positron emission tomography (PET) into TD-DOT to improve the quality of the reconstructed TD-DOT images with the soft prior method. Furthermore, we have investigated to maximize the utilization of the rich information contained in the measured pulses by increasing the frequency number in the Laplacian transformation. To validate the proposed method, we have performed numerical simulation studies by using the CT/PET breast images of a breast cancer patient, in which we generated the breast mesh from CT images and obtained the breast tumor position and size from the PET images. The soft prior guidance based TD-DOT reconstruction was performed in the Laplacian domain. Our simulation results indicate that the spatial resolution and the accuracy of the reconstructed TD-DOT images have been improved substantially after applying the anatomical guidance.