This paper describes our "virtual" Doppler ultrasound (DUS) system, in which colour DUS (CDUS) images and DUS spectrograms are generated on-the-fly and displayed in real-time in response to position and orientation cues provided by a magnetically tracked handheld probe. As the presence of complex flow often confounds the interpretation of Doppler ultrasound data, this system will serve to be a fundamental tool for training sonographers and gaining insight into the relationship between ambiguous DUS images and complex blood flow dynamics. Recently, we demonstrated that DUS spectra could be realistically simulated in real-time, by coupling a semi-empirical model of the DUS physics to a 3-D computational fluid dynamics (CFD) model of a clinically relevant flow field. Our system is an evolution of this approach where a motion-tracking device is used to continuously update the origin and orientation of a slice passing through a CFD model of a stenosed carotid bifurcation. After calibrating our CFD model onto a physical representation of a human neck, virtual CDUS images from an instantaneous slice are then displayed at a rate of approximately 15 Hz by simulating, on-the-fly, an array of DUS spectra and colour coding the resulting spectral mean velocity using a traditional Doppler colour scale. Mimicking a clinical examination, the operator can freeze the CDUS image on-screen, and a spectrogram corresponding to the selected sample volume location is rendered at a higher frame rate of at least 30 Hz. All this is achieved using an inexpensive desktop workstation and commodity graphics card.