Due to many experimental data required and a lot of calculations involved, it is very complex and cumbersome to model prism-based liquid-refractive-index-measuring methods. By use of the feature of TE-polarized wave and TM-polarized wave and differential measurement principle, we developed a new method of mathematical modeling for measuring refractive index of a liquid based upon Fresnel formula and prism internal reflection at incident angle less than critical angle. With this method only two different concentrations measurements for a kind of solution can lead to the determination of computational model. It introduces the principle of an optic-fiber sensor system based on prism-coupler for measuring refractive index of a liquid, and it contains the configuration picture of the sensing optical path, the spectrogram of the semiconductor laser and the structure block diagram of measuring system, the system is mainly made up of the semiconductor laser with 1654.14nm in wavelength, 1×2 optical switch, Y-shaped photo-coupler with coupled rate 50:50, the detector based on isosceles prism-coupler, the data process and control system based on AT89C51 and photoelectric transformer. For TM-polarized wave and TE-polarized wave, theoretical simulations show that the ratio of sensitivity is 1.11, therefore, the beam that the component of TM-polarized wave is more than the one of TE-polarized wave is advantageous to heightening the systemۥs measurement sensitivity. Measurements are performed to examine the validity of the theoretical model and four theoretical models are given, and these results indicate the feasibility of four theoretical models with an error of 3%. In this study, a beam of light is broken down into two beams in the coupler of Y-shaped coupler, the one acts as the reference optical path, the other is known as the sensing optical path, consequently the method can limit well the fluctuation of the light source, the variation of the photodiodeۥ s dark-current of photodiode and many other interfering factors and can be used for real-time detection and online analysis of liquid refractive-index. The method is also capable of measuring even smaller changes in the optical refractive index of the material on a metal surface by the surface plasma resonance sensing techniques.