Numerous potential biomedical sensing applications of surface-enhanced Raman scattering (SERS) have been reported, but its practical use has been limited by the lack of a robust sensing platform. Optical fiber SERS probes show great promise, but are limited by the prominent silica Raman background, which requires the use of bulky optics for filtering the signal collection and excitation delivery paths. In the present study, a SERS microprobe has been designed and developed to eliminate the bottlenecks outlined above. For efficient excitation and delivery of the SERS signal, both hollow core photonic crystal fiber and double clad fiber have been investigated. While the hollow core fiber was still found to have excessive silica background, the double clad fiber allows efficient signal collection via the multi-mode inner cladding. A micro filtering mechanism has been designed, which can be integrated into the tip of the optical fiber SERS probe, providing filtering to suppress silica Raman background and thus avoiding the need for bulky optics. The design also assists in the efficient collection of SERS signal from the sample by rejecting Rayleigh scattered light from the sample. Optical fiber cleaving using ultra-short laser pulses was tested for improved control of the fiber tip geometry. With this miniaturized and integrated filtering mechanism, it is expected that the developed probe will promote the use of SERS for minimally invasive biomedical monitoring and sensing applications in future. The probe could potentially be placed inside a small gauge hypodermic needle and would be compatible with handheld portable spectrometers.