Electromagnetic band-gap structures (EBG) have received considerable attention in the microwave regime, due to their tremendous potential for different applications. In this communication, guided wave propagation in parallel-plate, rectangular and circular waveguides with Kronig-Penney morphology is considered. Propagation modes in these waveguides are classified as either transverse electric (TE) or transverse magnetic (TM) with respect to the propagation direction. At frequencies above the modal cut-off frequencies in the considered waveguides, band gaps exists wherein propagation is forbidden. The allowed and forbidden bands are obtained for the TE and TM propagation modes, after invoking the Bloch theorem. From the study of the ideal EBG structures, the dependencies of the locations and the widths of the band gaps -- on the modal order, the waveguide geometry and dimensions, the permittivity contrast and the relative volumetric proportion of the two materials constituting the unit cell of the EBG structure -- are deduced. Also, propagation in the corresponding real EBG structures, with finite numbers of unit cells, is studied using the scattering matrix technique. As the number of unit cells in a real structure increases, its transmission characteristics converge to those of its ideal EBG analog in the band gaps.