We consider the problem of automatically tracking the mitral valve in cardiac ultrasound time series and present an unsupervised method for decomposing and segmenting the mitral valve from noisy ultrasound videos. To do so we propose a Robust Nonnegative Matrix Factorization (RNMF) method that naturally decomposes the time series into three separate parts, highlighting the cardiac cycle, mitral valve, and ultrasound noise. The low rank component of RNMF captures the simple motions of the cardiac cycle effectively aside from the sporadic motion of the mitral valve tissue that is captured innately in our RNMF sparse signal term. Using the RNMF representation, we introduce a simple valve object detection algorithm. Our method performs especially well in noisy time series when existing methods fail, differentiating general noise from the subtle and complex motions of the mitral valve. The valve is then segmented using simple thresholding and diffusion. The method presented is highly robust to low quality ultrasound video, and does not require manual preprocessing, prior labeling, or any training data.