We review our recent progress in the fabrication and understanding of ultra-low mode volume, high Q-factor microcavities for quantum dot based cavity QED experiments. The cavities are realized by the controlled incorporation of defects into 2D photonic crystals that consist of a triangular lattice of air holes within an active Air-GaAs-Air slab waveguide containing InGaAs self-assembled quantum dots. Two specific cavity designs are studied: the L3-cavity consisting of three missing holes along a line and the Y1-cavity consisting of a single missing hole with strongly reduced symmetry. Very good quantitative agreement is obtained between the results of spatially resolved optical spectroscopy and 3D calculations of the photonic bandstructure and cavity mode structure. For both cavity designs, cavity Q-factors up to ~8000 are measured for specific designs with ultra-low mode volumes Vmode< (λ/n)3. The relative contribution of cavity losses due to out of plane coupling to the free space continuum, in-plane losses through the photonic crystal and via scattering due to disorder and fabrication imperfections are probed for both cavity designs. We demonstrate that in-plane loss can be almost completely inhibited by tuning the localized cavity modes deeper into the photonic bandgap and the potential to fine tune the out-of plane losses via subtle modifications of the cavity design parameters. This procedure is shown to result in up to ~3x improvements of the cavity Q-factor. The Y1-design is shown to be particularly suitable for QD based cavity QED experiments, due to its very low mode volume, high Q-factors achievable (~7000) and flexibility for enhancement through careful modification of the cavity design.