1.1.

Visual Hindsight Bias

Visual hindsight bias, the “we saw it all along” effect, is a perceptual subtype of hindsight bias in which prior information about an image enhances our perception of that image.⁵ This perceptual bias is often studied by presenting participants in an experiment with blurry (noisy) images that slowly resolve into clear images and vice versa.^5–7 Participants who start with the clear image have more information than those who start with the blurry image, allowing experimenters to test the effects of this knowledge on perception.

Using a version of this technique, Ref. 5 showed that individuals tend to underestimate the influence that visual hindsight bias has on their own perception. Participants were asked to identify at which point they recognized the face of a celebrity, which started out blurry (non-recognizable) and slowly dissolved into the clear, original image (Fig. 1). When participants were subsequently asked to indicate the level of blurriness at which they themselves first recognized the celebrity’s face, they consistently overestimated the degree of blur at which they previously recognized the celebrity—thinking they originally recognized the image when it was blurrier than they actually did. A similar study looked at how visual hindsight bias progresses from childhood through adulthood.⁷ They found that once children and adults know the identity of a blurry object, they consistently overestimate their peers’ ability to recognize the same blurry object. This expands the findings of the previous study and suggests that visual hindsight bias not only affects our own perception, but also our view of others’ perception as well, in addition to being present across the lifespan.

Fig. 1

Example stimuli used in Ref. 5. This paper provided a clear demonstration of hindsight bias in visual perception. In this example, knowing that the images are of Harrison Ford biases the viewer to recognize Harrison Ford on, say, the second image from the right, even though if you were shown the second image on the right by itself without already knowing its identity, it would be too blurry to recognize.

Visual hindsight bias can be elicited in more controlled conditions that do not allow for the effects to arise solely from hysteresis as well. For example, using priming to bring an object to mind is sufficient to allow people to recognize images they would not otherwise, even when they images are presented in a random order so that participants cannot simply “hold” onto their previous interpretation.⁸ To show this, the authors introduced the random image structure evolution (RISE) method of object distortion, where stimuli are systematically transformed by emerging from noise and then dissolving back into noise.⁸ Right before viewing an object in some level of noise, participants were presented with a word that either matched the following objects’ name, or did not match the objects’ name (i.e., a completely unrelated word). The authors found that primed images that matched the prior word were recognized more easily than images that were primed by an unrelated word at matched noise levels. Furthermore, this enhanced recognition occurred even when stimuli were intermixed, where it could not arise from a decision level bias. A similar study showed participants various celebrity’s faces transforming from blurry to clear or vice versa.⁹ They found that priming participants with the celebrity’s name beforehand increased the effects of visual hindsight bias.⁹ Taken together, these studies provide further evidence that knowledge of the identity of the blurred or distorted image (whether from seeing the clear image, hearing a sound, or simply being told what it is) leads to enhanced perception of the image compared to images that were not preceded by relevant prior information.

1.2.

Visual Hindsight Bias in Experts

Hindsight bias—of the more cognitive variety—has been found in many applied settings and in experts, including in medicine,¹⁰ gambling,¹¹ legal decision making,¹² baseball,¹³ public policy,¹⁴ consumer satisfaction,¹⁵ and terrorist attacks.¹⁶ In professional gambling, for example, expert gamblers often reframe losses in hindsight as an event which in retrospect could have been avoided, or reframe wins as confirmation of skill or ability.¹¹ Looking at how this bias relates to eyewitness testimony, Ref. 12 showed participants a video of a crime and later asked the participants to identify the suspect. After identification, feedback was given to either confirm or disconfirm their choice. The authors found that confirming versus disconfirming the eyewitnesses’ choice had a significant impact on many judgment reports, including the eyewitnesses’ self-assessment of their visual experience of the perpetrator (e.g., view, ability to make out facial features, and ease of making identification).¹² These results suggest that the eyewitness is unable to accurately recall the witnessing experience because of this retrospective information. Ref. 17 and others have argued that this contamination of eyewitness memory has caused the prevailing view of the unreliability of eyewitness testimony and suggest that the original judgment, without feedback, is a reliable source of information—but simply one that can be easily corrupted by hindsight biases, new memory encoding, and more, after the initial identification has occurred.

Medical experts are also not immune to cognitive forms of hindsight bias. In one study, neuropsychologists were asked to estimate the probability of three different diagnoses.¹⁸ Half of the participants, labeled the hindsight group, were told one of the three diagnoses was correct. The other half of participants who did not receive this “correct” diagnosis were called the foresight group. Of the hindsight group, 58% of participants gave a higher probability estimate than the foresight group to the diagnosis they were told was correct.¹⁸

What about the more visual form of hindsight bias, wherein people report being able to visually see information after they have knowledge of this information from an independent source? In one such study, 82% of cases that had initially been deemed normal by 2 to 3 physicians were later discovered to contain tumors “visible in retrospect,” as far back as 53 months prior to diagnosis.¹⁰ Another study looked at visual hindsight bias as it relates to radiologists’ perception of pulmonary nodules.¹⁹ Radiologists were shown a series of abnormal chest images and asked to either manually add blur until they could no longer see the nodule (hindsight bias condition), or reduce the blur until they could see it (foresight condition). Their results suggest that radiologists are influenced by hindsight bias and that the extent of the bias seemed to be exacerbated with more difficult nodules. While participants report their visual perception, making this a form of visual hindsight bias, blur is manipulated continuously by participants. Thus, unlike the technique of Ref. 8, this result could potentially arise from decision-level biases rather than arising purely as an effect of visual recognition.

The consequences of visual hindsight bias in radiology can be acute. One article describes this anecdotally:²⁰ when a radiologist looked at an elderly man’s chest x-ray, they concluded that it was normal. The man, however, later became sick and had an additional scan that showed a noticeable mass that eventually led to his death. The man’s family sued the radiologist for initially missing the mass earlier on, when a diagnosis could have prevented the man’s death. In the lawsuit, the case was sent to a second radiologist whose task was to assess and determine whether the mass could have been seen in the original scan. The second radiologist could indeed see the mass.²⁰ This sequence of events is common in radiology.^21,22 When the case is sent to a second radiologist, this physician has additional information when they look at the image in question compared to what the first radiologist had. Depending on the extent of visual hindsight bias in expert radiologists, this additional information could significantly bias their judgment. It could also bias the jury, thus having important legal implications for the radiologist in question. While mammography is one of the most common areas within radiology to be sued for negligence,²³ there is relatively limited research on hindsight bias in expert radiologists, and even less research on the effects of visual hindsight bias in radiologists who specialize in mammography. Furthermore, many of the studies of visual hindsight bias allow for a more cognitive interpretation—adding blur in a continuous manner to an image could result in biases because of decision-level hysteresis, for example.

The current study investigates to what extent expert radiologists demonstrate visual hindsight bias to mammograms with visible abnormalities. We use the RISE method⁸ to ensure that our results do not arise from decision-level hindsight bias and instead are visual in nature. In order to take into account any response bias that may arise from asking radiologists to distinguish between these two abnormalities, we use receiver operating characteristic analysis to measure performance. To anticipate our results, we find evidence that radiologists are influenced by visual hindsight bias when looking at abnormal mammograms.

2.1.

Participants

Aiming for a minimum of 20 participants, we fortunately were able to collect data from 34 radiologists (12 female, 20 male, 2 preferred not to say; age ranged 28 to 69; mean 40) who read an average of 2100 mammograms per year. All participants gave informed consent and were not compensated. The experiment was conducted at the Radiological Society of North America 2019 Conference in Chicago, Illinois, United States. Informed consent procedures were approved by the Institutional Review Board of the University of California, San Diego.

2.2.

Stimuli and Procedure

On each trial, radiologists viewed a unilateral mammogram for 3 s. The mammograms subtended $\sim 16\times 20\mathrm{deg}$ of visual angle at an estimated viewing distance of $\sim 60\mathrm{cm}$ from the screen. All of the mammograms were abnormal, with half of the mammograms containing a mass and half containing a calcification. All images had verified pathology information and were preclassified by independent radiologists who did not participate in this study.

Each mammogram had five versions of itself with different levels of noise, which were created using a degradation process similar to the RISE procedure developed by Ref. 8. The five copies of each mammogram consisted of: 0% noise, 10% noise, 25% noise, 35% noise, and 45% noise. Figure 2 is a visualization of these levels of noise for five example mammograms.

Fig. 2

Example of the five levels of noise (columns) for five mammogram images (rows).

The experimental structure consisted of 8 blocks (30 trials/block), where in each block participants viewed all 5 versions of 6 different mammograms. Blocks were organized by noise level; for example, the six mammograms might each start out presented at their most degraded level (e.g., all six images with 45% noise would be presented sequentially in a random order), and as the trials progressed the images would cycle through the noise levels until all of the images were shown with no noise. Other blocks would start with each image at the 0% noise level (no noise), and then each of the six images would cycle through each noise level to become increasingly degraded. While the noise levels for each block varied systematically—either becoming increasingly noisy or increasingly clear—the six images within each noise level were presented in a random order, ensuring participants could not anticipate whether a given image had a mass or calcification purely based on the sequence itself, reducing cognitive/decision-level biases.

Whether the block started with 0% or 45% noise levels was counterbalanced across blocks and participants. Participants were told when they moved on to the next block. While the images would repeat within a block (for each noise level), no images would repeat across blocks. There were a total of 280 trials across all 8 blocks. The six mammograms chosen for each block were manually categorized based on the structure of the breast outline to decrease the likelihood of participants recognizing repeating images within a block and using this to infer the mass versus calcification judgment.

Immediately after each image was presented, participants were shown a screen containing a 6 point confidence scale ranging from (1) confident this image is a mass to (6) confident this image is a calcification. Using a standard computer mouse, participants were asked to indicate their diagnostic confidence. We used confidence ratings instead of yes/no answers to allow for ROC analysis and separate decision bias from performance. There was no time constraint imposed on responding. After participants indicated their confidence, they clicked a button to move on to the next trial.

Before the experiment began, participants were told that each mammogram would contain either a mass or a calcification and that the task was to rate their confidence on which abnormality they thought was present using a rating scale. They were informed that the images would start out degraded and become more clear, or start out clear and become more degraded and were shown an example of a mammogram depicting the five levels of noise. Participants were not informed that the same image would repeat with a different level of degradation within a batch.

2.3.

Analysis

Our main measure was the area under the curve (AUC), an atheoretical measure of discrimination ability. This measure collapses each conditions’ ROC to a single measure of performance.

Our main hypothesis concerns visual hindsight bias: that is, whether participants might benefit in seeing the abnormality in the noisiest images if they had previously been exposed to the less noisy versions of those images compared to if they had not been exposed to them. In these two situations, participants see the same images, and these are images that are not possible to recognize the abnormality in without previous experience. The highest noise levels provide the best test of hindsight bias because when perceptual information is strong, your “priors” should not play a strong role; however, when perceptual information is weak, visual priors—about how to organize the parts of the image, what objects are where, etc.—will play more of a role. Thus, the highest noise levels provide the core test of visual hindsight bias.