In this paper it is shown that the reference beam for true time delay operation of a heterodyne system must be carefully chosen to achieve proper signal time delay behavior. True time delay is defined here as an equivalent delay of the envelope and carrier frequency of a carrier-modulated waveform, resulting in no apparent phase shift of the envelope. For acousto-optic (AO) tapped delay lines, or more complex AO systems utilized for tapped delay line filters, signal excisors, or beamformers, the reference beam must be equivalent to the undiffracted beam exiting the AO device in order to achieve true time delay. Two examples are used to demonstrate the application of the time delay concepts described in the paper. The first example is a heterodyne transform architecture that uses an external reference beam to select a tap position within an AO cell. The second example is an AO tapped delay line filter that employs the undiffracted beam from the AO interaction as the reference beam.