In this paper, a 3D semantic object motion tracking method based on Kalman filtering is proposed. First, we use a specially designed Color Image Segmentation Editor (CISE) to devise shapes that more accurately describe the object to be tracked. CISE is an integration of edge and region detection, which is based on edge-linking, split-and-merge and the energy minimization for active contour detection. An ROI is further segmented into single motion blobs by considering the constancy of the motion parameters in each blob. Over short time intervals, each blob can be tracked separately and, over longer times, the blobs can be allowed to fragment and coalesce into new blobs as motion evolves. The tracking of each blob is based on a Kalman filter derived from linearization of a constraint equation satisfied by the pinhole model of a camera. The Kalman filter allows the tracker to project the uncertainties associated with a blob center (or with the coordinates of any other features) into the next frame. This projected uncertainty region can then be searched rot eh pixels belonging to the blob. Future work includes investigation of the effects of illumination changes and simultaneous tracking of multiple targets.