Viscosity is often a critical characteristic of biological fluids such as blood and mucus. However, traditional rheology is often inadequate when only small quantities of sample are available. A robust method to measure viscosity of microquantities of biological samples could lead to a better understanding and diagnosis of diseases. Here, we present a method to measure viscosity by observing particle Brownian motion within a sample. M-mode optical coherence tomography (OCT) imaging, obtained with a phase-sensitive 47 kHz spectral domain system, yields a viscosity measurement from multiple 200-1000 microsecond frames. This very short period of continuous acquisition, as compared to laser speckle decorrelation, decreases sensitivity to bulk motion, thereby potentially enabling in vivo and in situ applications. The theory linking g(1) first-order image autocorrelation to viscosity is derived from first principles of Brownian motion and the Stokes-Einstein relation. To improve precision, multiple windows acquired over 500 milliseconds are analyzed and the resulting linear fit parameters are averaged. Verification experiments were performed with 200 µL samples of glycerol and water with polystyrene microbeads. Lateral bulk motion up to 2 mm/s was tolerated and accurate viscosity measurements were obtained to a depth of 400 µm or more. Additionally, the method measured a significant decrease of the apparent diffusion constant of soft tissue after formalin fixation, suggesting potential for mapping tissue stiffness over a volume.