Transfer learning provides a useful solution to learn a new conceptual domain from few examples, which exploits prior knowledge from a related domain. We proposed a simple and yet effective transfer learning method for image classification that constructs an activation ensemble generative adversarial net (AE-GAN) to transfer knowledge from one dataset to another. The AE-GAN is mainly composed of three convolutional layers and adopts an ensemble of multiple activation functions. Experimental results on five benchmark datasets show that when only a few samples are available for training a target task, leveraging datasets from other related datasets by AE-GAN can significantly improve the performance for image classification with a small set of samples.