The state of the art of quantitative infrared thermography is addressed. We are asking more and more of thermography in the field of energy conservation. An energy and dollar conscious public wants to know how much heat is being lost and how much money can be saved by retrofit. Many mission oriented approaches have been developed to address these questions. However, many fundamental questions about how thermography works and how far its capabilities can be pushed have not been addressed. In addition, the theoretical and practical limits of doing benefit cost studies with thermography have not been seriously addressed. This paper discusses the limitations of the current technology and describes many unanswered but important questions facing the future use of thermography for quantitative heat loss analysis.