The testing of highly aspheric optics often requires complex test arrangements: these test systems can be multi-element and will have both fabrication and alignment errors present in the test wavefront. It may not be feasible to calibrate such systems with conventional optical shop practice. The use of diffractive imitator optics, with carefully controlled fabrication uncertainties, can be used to characterise these systems. We describe the use of reflective imitator CGH optics as calibration artefacts in the calibration of an optical test system used to test ELT primary mirror segments. The optical test system is designed to have two operational modes: one to measure a spherical reference optic; and one to measure the primary mirror segment. The use of diffractive imitators in this test system is designed to provide traceability between these two operational configurations, to quantify residual alignment aberrations, and to quantify fabrication errors in the test system. We outline the design of the optical test system, the design of three imitator CGH artefacts required to provide traceability between the two optical test modes, and our calibration approach. We demonstrate the calibration performance achieved with this approach. Without the use of these imitator artefacts, the absolute accuracy of the optical test is estimated to be 149 nm RMS wavefront, of which 47 nm RMS is attributed to midspatial wavefront errors and 141 nm RMS is attributed to alignment and prescription errors. The repeatability of this calibration has been established as better than 3 nm wavefront standard deviation, with an absolute accuracy of 19 nm RMS wavefront.