Radiologists often recommend further imaging, laboratory or clinical follow-up as part of a study interpretation, but rarely receive feedback as to the results of these additional tests. In most cases, the radiologist has to actively pursue this information by searching through the multiple electronic medical records at our institution. In this work, we seek to determine if it would be possible to automate the feedback process by analyzing how radiologists phrase recommendations for clinical, laboratory or radiologic follow-up. We surveyed a dozen attending radiologists to create a set of phrases conventionally used to indicate the need for follow-up. Next, we mined dictated reports over a 1-year period to quantify the appearance of each of these phrases. We are able to isolate 5 phrases that appear in over 21,000 studies performed during the 1-year period, and classify them by modality. We also validated the query by evaluating one day's worth of reports for follow-up recommendations and assessing the comparative performance of the follow-up query. By automatically mining imaging reports for these key phrases and tracking these patients' electronic medical records for additional imaging or pathology, we can begin to provide radiologists with automated feedback regarding studies they have interpreted. Furthermore, we can analyze how often these recommendations lead to a definitive diagnosis and enable radiologists to adjust their practice and decision-making accordingly and ultimately improve patient care.