The dispersion relation equation of hybrid modes in hollow optical fiber is derived in the closed form by modeling the fiber in a three-layered structure and by rigorous hybrid mode analysis. The effective refractive indices of hybrid modes as a function of wavelength are accurately calculated. Modal characteristics and transverse field distributions of hollow optical fiber are precisely analyzed by use of the derived hybrid mode dispersion relation equation and the compact 2-D finite difference time domain method. In addition, the effects of the fiber dimension on propagating properties are presented. The number of hybrid modes that can exist in hollow optical fiber is calculated as a function of fiber dimension. We believe that this rigorous analysis is very accurate and provides precise computing methods to design and characterize hollow optical fiber based optical devices.