In several studies, intraplaque neovascularization (IPN) has been linked with plaque vulnerability. The recent development of contrast enhanced ultrasound enables IPN detection, but an accurate quantification of IPN is a big challenge due to noise, motion, subtle contrast response, blooming of contrast and artifacts. We present an algorithm that automatically estimates the location and amount of contrast within the plaque over time. Plaque pixels are initially labeled through an iterative expectation-maximization (EM) algorithm. The used algorithm avoids several drawbacks of standard EM. It is capable of selecting the best number of components in an unsupervised way, based on a minimum message length criterion. Next, neighborhood information using a 5×5 kernel and spatiotemporal behavior are combined with the known characteristics of contrast spots in order to group components, identify artifacts and finalize the classification. Image sequences are divided into 3-seconds subgroups. A pixel is relabeled as an artifact if it is labeled as contrast for more than 1.5 seconds in at least two subgroups. For 10 plaques, automated segmentation results were validated with manual segmentation of contrast in 10 frames per clip. Average Dice index and area ratio were 0.73±0.1 (mean±SD) and 98.5±29.6 (%) respectively. Next, 45 atherosclerotic plaques were analyzed. Time integrated IPN surface area was calculated. Average area of IPN was 3.73±3.51 mm2. Average area of 45 plaques was 11.6±8.6 mm2. This method based on EM contrast segmentation provides a new way of IPN quantification.