Infrared astrometry at the 10-milliarcsecond (mas) level is applicable to experiments in stellar evolution astronomy, solar system dynamics, relativistic gravitation, and deep space laser tracking. We are pursing astrometry with the U. C. Berkeley Infrared Spatial Interferometer (ISI) on Mt. Wilson to demonstrate a 10-mas capability for tracking stellar and solar system objects. Astrometric data from the ISI, taken and analyzed over the last 5 years, have shown that instrumental and atmospheric effects limit current demonstrations. The ISI data show that point-to-point interferometric phase fluctuations due to tropospheric and quantum noise, for optimal integration times of 0.2 seconds, are approaching the 0.1-cycle level needed to reliably connect the phase. Modeling the ISI data suggests that atmospheric fluctuations on Mt. Wilson, during the best seeing, are dominated by a low-lying component, within the first 25 meters above the ISI, which, in the future, may be minimized with in situ calibration. A calculation of atmosphere-limited astrometric accuracy shows that the ISI will soon be able to achieve 10-mas astrometry, on a 13-m baseline in a single observing session, employing current ground-based laser distance interferometer calibrations to minimize atmospheric effects.