This paper presents a robust object tracking method under pose variation. In practical environment, illumination and pose of objects are changed dynamically. Therefore, the robustness to them is required for practical applications. However, it is difficult to be robust to various changes by only one tracking model. Therefore, the robustness to slight variations and the easiness of model update are required. For this purpose, Kernel Principal Component Analysis (KPCA) of local parts is used. KPCA of local parts is proposed for the purpose of pose independent object recognition. Training of this method is performed by using local parts cropped from only one or two object images. This is good property for tracking because only one target image is given in practical applications. In addition, the model (subspace) of this method can be updated easily by solving an eigen value problem. However, simple update rule that only the tracked region is used to update the model for next frame may propagate the error to the following frames. Therefore, the first given image which is a unique supervised sample should be used effectively. To reduce the influence of error propagation, the first given image and tracked region in t-th frame are used for constructing the subspace. Performance of the proposed method is evaluated by using the test face sequence captured under pose, scaling and illumination variations. Effectiveness of the proposed method is shown by the comparison with template matching with update. In addition, adaptive update rule using similarity with current subspace is also proposed. Effectiveness of adaptive update rule is shown by experiment.