We present a semiautomatic procedure for the optimization of lithographic process conditions. In the regime of resolution enhancement techniques such as optical proximity correction, off-axis illumination, and phase shifting masks, the design of lithographic photomasks and illumination sources becomes increasingly intricate. Earlier "what you see is what you get" (WYSIWYG) approaches cannot be applied anymore. Instead, the employment of sophisticated design optimization tools is inevitable. In contrast to related efforts using an inverse problem formulation to address this goal, the approach presented here can be considered a direct method: mask and illumination layouts are mutually altered, evaluated, and improved, using an heuristic search algorithm. Thus, not only does this method require very little a priori knowledge of the process, it also allows for a very flexible problem formulation, enabling an easy integration of different model options or even process steps. As an underlying optimization algorithm of the presented procedure, a genetic algorithm has been used, whose flow, data representation, and basic operations are discussed briefly. Different representation types for both mask and illumination setups and the formulation of the optimization objectives are explained in detail. Various well-performing mask and illumination settings demonstrate the feasibility of the proposed approach and point out further potentials: in contrast to single problem specific approaches, this method is applicable to a great variety of different complex optimization tasks in advanced lithography.