High-energy laser systems are limited by the onset and subsequent growth of damage on constituent optics. This has been extensively studied for optics comprised of fused silica, but less so for other common optical materials. There are very few materials as well characterized as fused silica and, in this work, we explore the growth characteristics of other widely used optical materials with a range of physical parameters, namely sapphire, potassium dihydrogen phosphate, calcium fluoride, and compare them to fused silica. Since current understanding is that material fracture must be present before the fluences used in ns laser systems might cause a surface flaw to grow, we have chosen to study flaws on the exit surfaces created with a Vickers indenter. A range of indenter forces were selected that would produce flaw sizes typical of those that have been seen in laser created damage. Samples with arrays of indents were tested in the in the Optical Science Laser (OSL), a master oscillator power amplifier system, with a front-end pulse shaping capability able to deliver relevant fluences with a large area beam. Samples were tested in vacuum at 351 nm and at atmosphere at 1053 nm with a single shot fired every 45 minutes exposing multiple sites simultaneously. High resolution images of each flaw were taken after every shot to document changes. Additional tests at 1064 nm were conducted of individual sites at a 60 Hz rep rate in the Gigashot Optical Laser Damage (GOLD) system. The probability of growth at 3ω at 5.5 J/cm2 is near 100% for both calcium fluoride and fused silica about 50% for the other materials. The growth rates at 3ω at from 5 to 8 J/cm2 are comparable for all but potassium dihydrogen phosphate which are better than five times lower. At 1ω all the materials had about a factor of five increase in the threshold for growth.