Random spin fluctuations in an equilibrium ensemble of paramagnetic spins are shown to contain valuable information about the system itself. We use off-resonant Faraday rotation to passively and sensitively "listen" to the random magnetization fluctuations (spin noise) in atomic alkali vapors. These random fluctuations generate spontaneous spin coherences which precess and decay with the same characteristic energy and time scales as the macroscopic magnetization of an intentionally polarized or driven ensemble. Correlation spectra of the measured spin noise reveals g-factors, nuclear spin, isotope abundance ratios, hyperfine splittings, nuclear moments, and spin coherence lifetimes -- without having to excite, optically pump, or otherwise drive the spin system away from thermal equilibrium. These noise signatures scale inversely with interaction volume, suggesting routes towards non-perturbative, sourceless magnetic resonance of small solid state spin systems.