Noble metal nano-structured thin films show great electromagnetic enhancement due to the surface plasmon resonance effect. It can be used for surface enhanced Raman scattering (SERS), which means markers of ultra low concentration can be detected, having great potential in biosensing applications. In this work, we introduce glancing angle deposition (GLAD), which is based on the traditional physical vapor deposition, to prepare vertical columnar thin films (CTFs). Anodic aluminum oxide (AAO) is used as template during vacuum deposition, and nanodots of ordered distribution are obtained. Subsequently, these nanodots are used as the pre-constructs in GLAD, and the fabricated columns have a much bigger separation. Rhodamine 6G (R6G) SERS spectra of the nanostructure thin films are measured to study their optical properties. Due to the separation of columns, CTFs on nanodots show stronger intensity than CTFs on blank substrates. What’s more, the uniformity is also improved by the separation, making the substrates more applicable for biosensing. CTFs on nanodots of different heights are prepared. SERS measurements show that as the height increases, SERS effect increases due to much more hot spots of electromagnetic enhancement and coupling. Our method proves to be a feasible, low-cost, large-area preparation method for ultra high sensitivity SERS substrate.