Sensitivity to tolerances is a well-known problem in optical design. In many cases, multiple designs having different tolerance sensitivities will solve the optical design problem. Often, the solution with the best “as-designed” performance is not the solution with the best “as-built” performance. In the end, it is not the as-designed quality of the optics that matters; it is only the as-built quality that matters. As we demonstrate in this paper, typical merit functions used in optimization (e.g., RMS spot diameter or RMS wavefront variance of the pre-tolerance system) are often poorly correlated to actual, as-built image quality; in many cases the correlation is extremely poor. One known strategy to avoid this is to add something to the merit function that penalizes design forms that are particularly sensitive. The ultimate success of a merit function is determined by the extent to which it correlates with as-built performance. Such a strategy is of particular importance during a global optimization design phase, in which the optimizer will generate many different design forms, some of which may differ significantly from the starting design, both in appearance and in tolerance sensitivity. In this paper we examine the addition of a “sensitivity” parameter to the merit function. We discuss the selection of the weighting factor for the sensitivity parameter, as well as the correlation of the merit function (both with and without the sensitivity parameter) to as-built performance.