The Minimum Resolvable Temperature Difference (MRTD or MRT) is the most widely accepted and inclusive figure of merit for describing a thermal imaging system's performance. It is the product of analytic mathematical models and traditional man-in-loop system hardware performance measurements that describe IR systems. MRT is a basis for thermal field performance model predictions and is commonly used in specification of thermal imagers. The MRT test is subjective because it requires human observers to just discern increasingly smaller 4-bar patterns as a function of temperature differences between bars and the background. When performed by trained observers, the MRT test is an accurate measure of sensitivity as a function of spatial resolution. The ability to resolve 4-bar patterns varies between observers. Furthermore, MRT is a psychophysical task, for which biases are unavoidable. In this paper, uncertainties in MRT measurements are reported for individual trained observers and between observers as functions of some biases, such as random and fixed pattern noise. For this paper, virtual MRTs were performed on a new, custom visual acuity test simulator, developed for NVESD, that allows precise control over significant sensor and display parameters, and these results are compared. Through a process of eliminating sources of MRT variability, we have been able to quantify the observer variability.