Our objective is to understand the mechanism that generates catastrophic optical damage in pulsed fiber amplifiers. We measured optical damage thresholds of bulk fused silica at 1064 nm for 8 ns and 14 ps pulses. The 8 ns pulse is single longitudinal mode from a Q-switched laser, and the 14 ps pulse is from a Q-switched mode-lock laser. The beams in both cases are TEM00 mode, and they are focused to a 7.5 μm spot inside a fused silica window. The pulse-to-pulse energy variations are 1% for 8 ns pulses and 5% for 14 ps pulses. Under these conditions optical damage is always accompanied by plasma formation at the focal spot; we found the damage threshold fluences are 3854 ± 85 J/cm2 for the 8 ns pulses and 25.4 ± 1.0 J/cm2 for the 14 ps pulses. These fluences are corrected for self focusing. Both damage thresholds are deterministic, in contrast to the claim often made in the literature that optical damage is statistical in the nanosecond range. The measured damage threshold fluences for 8 ns and 14 ps pulses do not fit a square root of pulse duration scaling rule. We interpret the damage in terms of plasma formation initiated by multiphoton ionization and amplified by an electron avalanche. The damage threshold irradiance can be matched with a simple rate equation model that includes multiphoton ionization, electron avalanche, and electron-hole recombination. The damage morphologies are dramatically different in the nanosecond and picosecond cases because of the large difference in deposited energy. However, both morphologies are reproducible from pulse to pulse. We also measured surface damage thresholds for silica windows polished by different methods. We find that cerium oxide polished surfaces damage at approximately 40% of the bulk threshold, with a large statistical spread. Surfaces prepared using an Al2O3 polish damaged between 50% and 100% of the bulk damage limit, with a substantial fraction at 100%. Surfaces polished using first the Al2O3 polish and then an SiO2 polish exhibit surface damage values equal to the bulk damage value at nearly every point. We also measured damage thresholds for different sized focal spots. Some earlier reports have claimed that damage thresholds depend strongly on the size of the focal spot, but we find the surface threshold is independent of the spot size.