We present the design concept of the wavelength calibration system for the Habitable-zone Planet Finder instrument (HPF), a precision radial velocity (RV) spectrograph designed to detect terrestrial-mass planets around M-dwarfs. HPF is a stabilized, fiber-fed, R~50,000 spectrograph operating in the near-infrared (NIR) z/Y/J bands from 0.84 to 1.3 microns. For HPF to achieve 1 m s-1 or better measurement precision, a unique calibration system, stable to several times better precision, will be needed to accurately remove instrumental effects at an unprecedented level in the NIR. The primary wavelength calibration source is a laser frequency comb (LFC), currently in development at NIST Boulder, discussed separately in these proceedings. The LFC will be supplemented by a stabilized single-mode fiber Fabry-Perot interferometer reference source and Uranium-Neon lamp. The HPF calibration system will combine several other new technologies developed by the Penn State Optical-Infrared instrumentation group to improve RV measurement precision including a dynamic optical coupling system that significantly reduces modal noise effects. Each component has been thoroughly tested in the laboratory and has demonstrated significant performance gains over previous NIR calibration systems.