Topological phonics has opened new avenues to designing photonic devices along with opening plethora of applications. Recently even though there has been many interesting studies in topological photonics in classical domain, the quantum regime has still remained largely unexplored. Towards this goal, we developed a topological photonic crystal structure for interfacing single quantum dot spin with photon to realize light matter interaction with topological photonic states. Developed on a thin slab of Gallium Arsenide membrane with electron beam lithography, such a device supports two robust counter-propagating edge states at the boundary of two distinct topological photonic crystals at near-IR wavelength. We show chiral coupling of circularly polarized lights emitted from a single Indium Arsenide quantum dot under strong magnetic field into these topological edge modes. Owing to the topological nature of these guided modes, we demonstrated this photon routing to be robust against sharp corners along the waveguide. Our new technology can pave paths for fault-tolerant photonic circuits, secure quantum computation, exploring unconventional quantum states of light and chiral spin networks.