Utilizing an implicit nonparametric learning framework, a neighbor-embedding-based method is proposed to solve the remote-sensing pan-sharpening problem. First, the original high-resolution (HR) and downsampled panchromatic (Pan) images are used to train the high/low-resolution (LR) patch pair dictionaries. Based on the perspective of locally linear embedding, patches in LR and HR images form manifolds with similar local intrinsic structure in the corresponding feature space. Every patch in each multispectral (MS) image band is modeled by its K nearest neighbors in the patch set generated from the LR Pan image, and this model can be generalized to the HR condition. Then, the desired HR MS patch is reconstructed from the corresponding neighbors in the HR Pan patch set. Finally, HR MS images are recovered by stitching these patches together. Recognizing that the K nearest neighbors should have local geometric structures similar to the input query patch based on clustering, we employ a dominant orientation algorithm to perform such clustering. The K nearest neighbors of each input LR MS patch are adaptively chosen from the associate subdictionary. Four datasets of images acquired by QuickBird and IKONOS satellites are used to test the performance of the proposed method. Experimental results show that the proposed method performs well in preserving spectral information as well as spatial details.