Deep learning is applied in the Cognitive Radio field, such as anti-jamming communication based on spectrum waterfall image. However, the anti-jamming communication is a decision problem essentially, which interacts with the environment dynamically and is quite different from the problems of classification and detection. The training for decision problem often requires sampling a large number of labeled dataset from true environment, which undoubtedly suffers from the extremely high time consumption problem. In this paper, an offline fast model training method for anti-jamming communication is proposed. Specifically, a novel framework is developed to accelerate the training procedure. Like the Gym toolkit, a virtual environment generator for the production of the spectrum waterfall image is made by CGAN which is trained using the dataset sampled randomly by real transceiver. Due to the variety of the synthesized spectrum waterfall images outputted rapidly through the environment generator, the training efficiency is improved significantly. The experiments show, compared with the online training method, the time cost of the offline method is reduced over 50%.