A stereo video coding algorithm based on H.264 is presented in this work. The main difference between the multi-view and the single-view video coding algorithm is whether a disparity vector (DV) is used or not. For stereo video, we use binocular sequences that consist of left- and right-view sequences. The left-view sequence is coded by the conventional H.264 algorithm and used as a reference image for the prediction of the right-view sequence. The right-view sequence is coded with motion vectors (MVs) and DV. To code a forward prediction (P) frame or a bi-directional prediction (B) frame of the right-view sequence, we employ 2 or 3 prediction vectors. They are forward MV, backward MV and DV. For the coding of stereo sequences with DV and forward/backward MVs, we propose new macroblock (MB) type tables and a temporal prediction structure. Experimental results show that the proposed algorithm offers a higher peak signal to noise ratio (PSNR) than the H.264 simulcast method.