We report a highly sensitive anti-cyclic citrullinated peptide (anti-CCP) detection method for early diagnosis of rheumatoid arthritis (RA) using surface-enhanced Raman scattering (SERS)-based immunoassay. Herein, cyclic citrullinated peptide (CCP)-conjugated magnetic beads and anti-human IgG-conjugated hollow gold nanospheres (HGNs) were used as substrates and SERS nano-tags, respectively. First, its detection sensitivity was evaluated using anti-CCP standard solutions. Then quantitative anti-CCP levels, determined by the SERS-based assay, were compared with those obtained from three commercially available anti-CCP assay kits (Immunoscan CCPlus, ImmunnLisa<sup>™</sup> CCP and BioPlex<sup>™</sup> 2200) to assess its potential utility as a clinical tool. Finally, clinical samples from 20 RA patients were investigated using them. In the SERS-based assay, the anti-CCP level in human serum was successfully determined by monitoring the characteristic Raman peak intensity of SERS nano-tags. The diagnostic performance of our SERS-based immunoassay for clinical samples shows a good agreement with those measured by three commercial anti-CCP kits. In addition, our SERS-based assay results are more consistent in the low concentration range (0–25 U/mL) than those achieved by the commercial kits. Accordingly, it is estimated that the SERS-based assay is a potentially useful diagnostic tool for early diagnosis of RA.