As semiconductor devices shrink in size to accommodate faster processing speeds, the need for higher resolution beam-based metrology equipment and beam-based writing equipment will increase. The electron and ion beams used within these types of equipment are sensitive to very small variations in magnetic force applied to the beam. This phenomenon results from changes in Alternating Current (AC) and Direct Current (DC) magnetic flux density at the beam column which causes deflections of the beam that can impact equipment performance. Currently the most sensitive beam-based microscope manufacturers require an ambient magnetic field environment that does not have variations that exceed 0.2 milli-Gauss (mG). Studies have shown that such low levels of magnetic flux density can be extremely difficult to achieve. As examples, scissor lifts, vehicles, metal chairs, and doors moving in time and space under typical use conditions can create distortions in the Earth's magnetic field that can exceed 0.2 mG at the beam column. In addition it is known that changes in the Earth's magnetic field caused by solar flares, earthquakes, and variations in the Earth's core itself all cause changes in the magnetic field that can exceed 0.2 mG. This paper will provide the reader with the basic understanding of the emerging problem, will discuss the environmental and facility level challenges associated in meeting such stringent magnetic field environments, will discuss some of the mitigation techniques used to address the problem, and will close by discussing needs for further research in this area to assure semiconductor and nanotechnology industries are pre-positioned for even more stringent magnetic field environmental requirements.