Osteoarthritis (OA) is the most common form of arthritis, where the protective cartilage on the ends of bones wears down over time, causing pain, tenderness, stiffness, loss of flexibility and bone spurs. Degenerative alterations start before cartilage loss happens, which include surface swelling, cartilage fibrillation, and calcification. Detecting the early degenerative alterations can assist the diagnosis of early-stage OA. In this study, two imaging modalities are applied on human hip-joint specimens in ex vivo imaging, including polarization-sensitive optical coherence tomography (PS-OCT) and multiphoton microscopy (MPM). OCT detects the layered tissue structure of cartilage and bone using backscattered light and PS-OCT is a functional extension of OCT. PS-OCT measures tissue birefringence which is sensitive to the orderly organization of collagen in cartilage. MPM can visualize collagen fibers with sub-cellular resolution. Complementary information about cartilage on the cellular and tissue level can be obtained by the multimodal imaging. Using the multimodal system, the variation of the thickness of the cartilage structural zones, abnormal birefringence caused by collagen alterations and fibrillation, and uneven structure resulted from calcification are imaged and quantified. The imaging results show distinctive features of degenerative alterations in the OA specimen, such as uneven tissue surface, fibrillation, and reduced birefringence. It is shown that PS-OCT has great potential in detecting early stage OA.