Heart rate (HR) measurement from facial videos makes physiologic detection more convenient, which achieves much attention of many researchers. However, the existing methods are not very robust against motion disturbance caused by nonrigid movements of the face (e.g., smiling, yawning, and talking). This is due to the illumination variation on the local skin region, which results in the fluctuation of color intensity values in the region of interests. Different from previous approaches using fixed regions, we propose a self-adaptive region selection algorithm and then learn a 1D convolution neural network (CNN) by fusing these selected regions for HR estimation. Concretely, the observation blood volume pulse (BVP) matrix with sub-BVP is constructed from different separated regions and denoised by the rank minimization. To explore the sub-BVP of different regions, a clustering-based method is designed to generate the regional weights. In addition, a self-adaptive region selection is constructed to dynamically filter some useful face regions for robust HR estimation. Eventually, the regions and weights are separately embedded into a 1D CNN by a multiregion-fusion module. Experiments on the MAHNOB-HCI database show that the proposed approach is effective and reaches the best performance over five metrics.