Traditional polarization gratings (PGs) have been studied with increasing intensity since 2005, in part because they can manifest 100% single-order diffraction efficiency and strong sensitivity to input polarization, in both theory and practice. They can be made using patterned anisotropic materials (e.g., liquid crystals) or nanostructures (e.g., metasurfaces). Nearly every prior work on traditional PGs has implemented a linear spatial phase-shift that is either continuous or which samples the 2π phase period with multiple (≥ 4) discrete phase levels. As far as we know, only two prior works (Bhandari et al, Phys. Rep. 281 (1997); and Wang et al, Appl. Phys. Lett. 108 (2016)) have considered the circumstance when the phase is sampled with exactly two phase levels, with π radians between them. We call this a Binary PG (Bin-PG). In this work, we apply Jones calculus and the small angle (i.e., paraxial) approximation to derive the fundamental optical behavior of Bin-PGs: far-field efficiencies, input polarization sensitivity, and output polarizations. We show that Bin-PGs manifest properties that are a compelling and unique mixture of both traditional (non-binary) PGs and standard diffraction gratings (e.g., surface-relief-gratings (SRGs)). Like non-binary PGs, their output polarization is often different than the input and diffraction efficiencies are dependent on the effective retardation of the film or surface. However, like SRGs, they show a maximum of 80% total first-order efficiency and are insensitive to input polarization.