The conventional color edge detection methods generate color edge maps by calculating the vector gradient at each pixel. These methods may cause false edge detection for color stripe images and other special color images and they are sensitivity to noise. A new vector template matching method based on pyramid structure is presented. It can generate more precise edge points. The pyramid image structure has been used to reduce the computational cost in the multiresolution analysis. The effect of Gaussian noise can be reduced by selecting a suitable weighting function in the color pyramid generation. The edge interpolation, adjustment and smoothing are carried out to obtain edge map of a lower pyramid level from the edge map immediately above by the top-down strategy. The vector average median filtering is applied in this procedure to reduce the effect of impulsive noise. The method is applied to color palm images and another special images with both Gaussian and impulsive noise. Good edge maps are obtained from experiments.