Over half of the light incident on the Earth from the Universe falls within the Far-Infrared (FIR) region of the spectrum. Due to the deleterious effects of the Earth's atmosphere and instrument self-emission, astronomical measurements in the FIR require space-borne instrumentation operating at cryogenic temperatures. These instruments place stringent constraints on the mechanical and thermal properties of the support structures at low temperatures. With high stiffness, tensile strength, strength-to-mass ratio, and extremely low thermal conductivity, carbon fibre reinforced polymers (CFRPs) are an important material for aerospace and FIR astronomical applications, however, little is known about their properties at cryogenic temperatures. We have developed a test facility for exploring CFRP properties down to 4 K. We present results from our ongoing study in which we compare and contrast the performance of CFRP samples using different materials, and multiple layup configurations. Current results include an evaluation of a cryostat dedicated for materials testing and a custom cryogenic metrology system, and preliminary cryogenic thermal expansion measurements. The goal of this research is to explore the feasibility of making CFRP-based, lightweight, cryogenic astronomical instruments.