2.1.

Experimental Setup and Thermograph Measurements

As noted in ISO/TR 13154²⁴ and IEC 80601-2-59,²³ a screening thermograph (ST) system includes an IRT and an external temperature reference source or BB. The reference source should have a known radiance temperature (function of real temperature and emissivity, $\epsilon$) over the range of 33°C to 40°C with an expanded uncertainty (coverage factor of 2 for a $\sim 95\%$ level of confidence) of $\pm 0.3°\mathrm{C}$ or less, and a combined stability and drift of $\pm 0.1°\mathrm{C}$ over the temperature interval for measurement, and its image size should be $\ge 20\times 20\text{pixels}$. A workable target plane (WTP) is a specific region of the target plane that is used for temperature measurement; it should accommodate a subject’s face positioning from 0.75 to 2.2 m above the floor. The WTP image pixel size should be at least $320\times 240$. For ambient conditions, the temperature should be 20°C to 24°C and relative humidity should be 10% to 50%, based on ISO/TR 13154:2017. Forced cooling or heating of the target due to airflow or lighting should be avoided or at least minimized.

Based on ISO/TR 13154 recommendations, a screening setup was established as shown in Fig. 1. The setup consisted of a webcam (C920, Logitech, Lausanne, Switzerland) and two IRTs (IRT-1: $320\times 240\text{pixels}$, A325sc, FLIR Systems Inc., Nashua, NH and IRT-2: $640\times 512\text{pixels}$, 8640 $P$-series, Infrared Cameras Inc., Beaumont, TX) that were mounted in adjacent positions on a tripod. A WTP with dimensions of $320\times 240\text{pixels}$ was identified for each IRT—the entire image for IRT-1 and a subset of the image in the most uniform region for IRT-2. We developed a graphical user interface with MATLAB to simultaneously control the webcam and two IRTs, and to collect images and patient/environmental information. Both IRTs had 30-Hz frame rates, and detailed specifications and fundamental performance testing results—including IRT measurement linearity with respect to a BB—can be found in our prior study.²⁸ The laboratory accuracy of both IRTs with a BB satisfies the standard requirements (i.e., $\le 0.5°\mathrm{C}$). A BB (SR-33, CI Systems Inc.) with a $4\times 4\mathrm{in.}$ emitter (image $\text{size}\ge 20\times 20\text{pixels}$) was set at 35°C then positioned perpendicular to the sightline of the cameras and placed within the frame alongside the subject’s face for temperature drift compensation. We previously verified that BB drift and uncertainty satisfied relevant standards.²⁸ The subject-to-camera distance was 0.6 to 0.8 m to ensure that both the subject’s face and the BB were included in the WTP with a resolution that satisfied recommended specifications (face image $\text{size}\ge 240\times 180\text{pixels}$). Room temperature was maintained between 20°C and 24°C and relative humidity between 10% and 62%, as measured by a weather tracker (Kestrel 4500 NV, Weather Republic LLC, Downingtown, PA). The relative humidity was not exactly controlled within 10% to 50% as suggested by ISO/TR 13154:2017, with 7.5% of the subjects measured with relative humidity between 50% and 62%. Ambient temperature and subject $\epsilon$ (0.98 for human skin³⁶) were entered into the IRT control program as input parameters for calibrating measured temperature. To prevent direct airflow to a subject’s face, an air vent in the room was blocked by a magnetic air deflector. A black, low-reflectivity cloth ($\epsilon =0.97$; Type 822 E0.97, Group 8 technology, Provo, UT) was used as the backdrop.

Fig. 1

Diagram of the screening setup.

To minimize the influence of outside temperature, each subject was instructed to wait for 15 min before measurements started. Once seated, the subject was asked to remove all obstructions from the face (e.g., eyeglasses and hair on forehead) and look at the IRTs. Then, the tripod was adjusted to include the subject’s full face in the WTP. For each subject, four rounds of measurements were performed within $\sim 15\min$ following the procedure shown in Fig. 2. In each round, temperatures were measured with two IRTs (facial images) and several NCITs (forehead temperature). NCIT measurements performed in this study are beyond the scope of the current paper.

Fig. 2

Flowchart of temperature measurement procedure—only data from the steps labeled in green were used in this study.

During each round of imaging, the webcam acquired one standard color image, while each IRT acquired three consecutive frames (acquisition time $\sim 0.1\mathrm{s}$) that were averaged to reduce noise and form a single mean thermal image. In the last round, the subject was instructed to open their mouth to enable thermal imaging of sublingual tissue. Images from the second to fourth rounds of measurements were omitted in the current analysis, to better approximate a realistic screening scenario, but will be evaluated in a future study.

Oral thermometry was used to establish reference temperatures.³² A thermometer (SureTemp Plus 690, Welch Allyn, San Diego, CA) was placed under the subject’s tongue in a sublingual pocket (heat pocket). Then the temperature was read in two different modes, a “fast” mode in several seconds and a “monitor” mode after 3 min. We used the monitor mode data in this study since the monitor mode provides a superior accuracy compared to the fast mode. The monitor mode has accuracy of $\pm 0.1°\mathrm{C}$, which was confirmed against a NIST-traceable contact thermometer (6413, Traceable^® Products, Webster, TX) using a laboratory water bath (89202-926, VWR International, Radnor, PA). The reference temperature ($T_{\mathrm{ref}}$) was calculated as the mean of two oral temperature measurements in monitor mode (during rounds 1 and 3). All subject data were discarded if the difference between two readings was larger than 0.5°C, due to the likelihood of a measurement error.

2.2.

Facial Region Delineation and Temperature Calculations

Temperatures from several facial areas—including the forehead, canthi, mouth, and entire face—were compared to assess impact on fever screening (Fig. 3). Standards documents do not specify a method for delineating facial key points such as canthi in thermal images, so we implemented an image registration approach³⁷ to identify these points by matching facial landmarks on visible-light images to thermal images. However, key points for about half of these images required manual labeling. Based on the identified facial key points, different regions/points on thermal images were defined [Fig. 3(b)].

Fig. 3

Delineated facial regions and critical points on (a) visible-light and (b) thermal images: forehead regions and points (green), canthi region and points (red), mouth region (gray rectangle), and entire face (blue rectangle). Photos are of author Q. Wang.

As shown in Fig. 3, the extended forehead area was determined in the vertical direction by hairline and eyebrows. The maximum single-pixel temperature in the extended forehead area ($T_{\mathrm{FEmax}}$) was first obtained. Then, the center forehead area was defined in the horizontal direction by the canthus points and in the vertical direction by 70% of the height of the forehead area from the bottom edge (eyebrows), and its maximum temperature ($T_{\mathrm{FCmax}}$) was obtained. Five points were defined in the forehead center region rectangle—the center point and the middle points of the top, bottom, left, and right edges. At each of these points, the mean temperature of a $3\times 3\text{pixel}$ region was used to determine results ($T_{\mathrm{FC}}$, $T_{\mathrm{FT}}$, $T_{\mathrm{FB}}$, $T_{\mathrm{FL}}$, and $T_{\mathrm{FR}}$).

Two inner canthus points were identified in each thermal image using the registration method.³⁷ Two small circular regions of interest (ROIs) were defined with a diameter of 13 pixels, using the inner canthus points as their outer edge. The mean and maximum temperatures of the left (${\overline{T}}_{\mathrm{CL}}$ and $T_{\mathrm{CLmax}}$) and right (${\overline{T}}_{\mathrm{CR}}$ and $T_{\mathrm{CRmax}}$) ROIs were obtained. From these values, the mean and maximum of ${\overline{T}}_{\mathrm{CL}}$ and ${\overline{T}}_{\mathrm{CR}}$ (${\overline{T}}_{\mathrm{C}}$ and $T_{\mathrm{Cmax}1}$) and the maximum of $T_{\mathrm{CLmax}}$ and $T_{\mathrm{CRmax}}$ ($T_{\mathrm{Cmax}2}$) were also calculated. An extended canthus region was delineated, which was centered at the two canthus points and having a width of 96 pixels. Its top edge extended upward to the bottom of the eyebrows, and its bottom edge extended downward by the same amount. The maximum temperature ($T_{\mathrm{CEmax}}$) of this area was obtained.

The mouth region was defined by a rectangle that included all the facial key points around the mouth, then the maximum temperature of this region ($T_{\mathrm{Mmax}}$) was identified. The entire face region was defined horizontally by the edges of the face and in the vertical direction by the chin and hairline, and its maximum temperature was determined ($T_{\max }$).

2.3.

Data Analysis

3.1.

Subject Demographics

A total of 596 subjects were recruited; all were at least 18 years old and free of disease affecting the skin in canthi area or forehead one week prior to the screening date. Among these subjects, 33 had two oral temperature readings with difference $>0.5°\mathrm{C}$, and thus were removed from the database. Of the remaining 563 sets of subject data, we excluded 19 from IRT-1 and 23 from IRT-2 due to motion artifacts that degraded image quality. Finally, we had 544 data sets for IRT-1 and 540 data sets for IRT-2. These data sets will be released to the public in the near future. Demographic information for study subjects is summarized in Table 1.

Table 1

Demographics of study subjects.

3.2.

Temperature Measurement Accuracy and Correlation

Temperatures for different facial locations from thermal images were compared with $T_{\mathrm{ref}}$. For each IRT, comparative boxplots of pairwise differences with $T_{\mathrm{ref}}$ were constructed (Fig. 4). For all temperatures from both IRTs, pairwise differences were positive. This is consistent with prior studies, and expected given that sublingual tissue is not typically subjected to the levels of convective heat loss that external tissues experience.⁴² Overall, the forehead region showed the greatest discrepancy, then the canthus regions, and the entire face maximum shows the least difference.

Fig. 4

Boxplots of pairwise differences between $T_{\mathrm{IRT}}$ and $T_{\mathrm{ref}}$. The parameter $T_{\mathrm{IRT}}$ on the $y$ axis represents all the IRT-measured facial temperatures on the $x$ axis.

Pearson correlation coefficients for all measurement locations were highly consistent with the pairwise difference results (Table 2). Among the seven forehead temperatures, $r$ values varied from 0.39 to 0.63 with $T_{\mathrm{FEmax}}$ showing the highest correlation and agreement with $T_{\mathrm{ref}}$, followed by $T_{\mathrm{FCmax}}$. The maximum temperature around the mouth ($T_{\mathrm{Mmax}}$) showed results that were similar to $T_{\mathrm{FEmax}}$. Correlation results for inner canthi temperatures were in general higher than for the forehead, ranging from 0.51 to 0.76. The highest $r$ values ($>0.7$) were found for maximum temperatures across left and right inner canthi regions ($T_{\mathrm{Cmax}2}$), the extended inner canthi region ($T_{\mathrm{CEmax}}$), and the entire face ($T_{\max }$). As $T_{\max }$ had the largest $r$ value, this metric may best estimate $T_{\mathrm{ref}}$.

Table 2

Pearson correlation coefficients (r values) between facial temperatures and Tref.

Facial temperature correlations were further analyzed with scatter plots of $T_{\mathrm{IRT}}$ versus $T_{\mathrm{ref}}$ for selected facial locations (Fig. 5). These included $T_{\mathrm{Cmax}1}$, which aligns with ISO/TR 13154 recommendations (although this approach is not explicitly defined), as well as $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$, which showed the best $r$ values for extended forehead, extended inner canthi, and whole face regions, respectively. As expected, scatter plots show a large group of $T_{\mathrm{ref}}$ values near 37°C, representing the high proportion of afebrile subjects (84% of $T_{\mathrm{ref}}$ readings were between 36.4°C and 37.4°C). $T_{\mathrm{Cmax}1}$ results showed the greatest variability, most notably in the subfebrile range. $T_{\mathrm{FEmax}}$ results showed improved correlation with $T_{\mathrm{ref}}$ in this range, as well as across higher $T_{\mathrm{ref}}$ values. $T_{\mathrm{CEmax}}$ and $T_{\max }$ measurements were less variable than other measurements and well-correlated to $T_{\mathrm{ref}}$, approaching a linear relationship. Data for the two IRT systems showed only minor differences in correlation.

Fig. 5

Scatter plots of $T_{C\max 1}$, $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$ versus $T_{\mathrm{ref}}$, with linear fits and correlation coefficients. (a) IRT-1 and (b) IRT-2.

3.3.

Fever Detection Performance

ROC curve analysis was used to evaluate performance of IRT-based detection of febrile subjects and its dependence on facial measurement location. Figure 6 shows ROC curves for $T_{\mathrm{Cmax}1}$, $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$, while AUC values for all measurements are found in Table 3. In Fig. 6, the $y=x$ line denotes random discrimination; all ROC curves from our study were well above this line. $T_{\max }$ and $T_{\mathrm{CEmax}}$ yielded optimal ROC curves with AUC values $>0.95$, indicating high discrimination effectiveness. IRT-2 showed slightly better performance for $T_{\max }$ than $T_{\mathrm{CEmax}}$. ROC curves for $T_{\mathrm{Cmax}1}$ and $T_{\mathrm{FEmax}}$ indicated lower performance, with the most notable feature being a slow convergence to high Se as Sp decreased. In general, discrimination performance for $T_{\mathrm{IRT}}$ aligned well with correlations to $T_{\mathrm{ref}}$, as shown in Table 2.

Fig. 6

ROC curves for febrile subject ($T_{\mathrm{ref}}\ge 37.5°\mathrm{C}$) detection with (a) IRT-1 and (b) IRT-2 using facial temperatures of $T_{\max }$, $T_{\mathrm{CEmax}}$, $T_{C\max 1}$, and $T_{\mathrm{FEmax}}$.

Table 3

AUC values from the ROC curves of different facial temperatures.

The statistical significance of AUC differences between $T_{\mathrm{Cmax}1}$, $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$ was evaluated using six pairwise tests (Table 4). For both IRTs, there was no statistically significant difference ($p\ge 0.05$) between AUC values of $T_{\max }$ and $T_{\mathrm{CEmax}}$, and these parameters $$ were both significantly higher ($p<0.05$) than $T_{\mathrm{FEmax}}$. When compared to $T_{\mathrm{Cmax}1}$, $T_{\max }$ showed a significantly higher AUC, but $T_{\mathrm{CEmax}}$ showed a significantly greater result for only one of the two IRTs. Overall, this AUC comparison indicated that $T_{\max }$ and $T_{\mathrm{CEmax}}$ should provide the most optimal results when used for fever screening.

Table 4

Results of pairwise comparisons of the AUC values for Tmax, TCEmax, TFEmax, and TCmax1.

Facial temperatures measured by IRTs are typically at least 1°C to 2°C lower than $T_{\mathrm{ref}}$.³⁴ Therefore, it is necessary to either calibrate $T_{\mathrm{IRT}}$ to $T_{\mathrm{ref}}$ or find a suitable cutoff temperature for each IRT during fever screening. For consistency with prior IRT studies,^18,20 we used the latter method. Our results above (Fig. 6 and Table 4) were based on defining fever as $T_{\mathrm{ref}}\ge 37.5°\mathrm{C}$, from which an optimal cutoff temperature can be obtained for each IRT (first row in Tables 5Table 6–7). We also calculated the optimal cutoff temperature under diagnostic thresholds of 37.8°C and 38°C.

Table 5

Optimal IRT cutoff temperatures and related Se/Sp values for Tmax under different Tref thresholds.

Table 6

Optimal IRT cutoff temperatures and related Se/Sp values for TCEmax under different Tref thresholds.

Table 7

Optimal IRT cutoff temperatures and related Se/Sp values for TCmax1 under different Tref thresholds.

According to our results, $T_{\max }$ and $T_{\mathrm{CEmax}}$ provided the best approaches for fever detection and did not exhibit significant differences in performance. However, the approach that most closely adheres to the recommendations in ISO/TR 13154 is $T_{\mathrm{Cmax}1}$. Therefore, we used all three temperatures in evaluating the optimal Se and Sp for different $T_{\mathrm{ref}}$ diagnostic thresholds in Tables 5–7. Cutoff temperatures were calculated to optimize Se and Sp simultaneously.³⁸

These results further illustrate how diagnostic performance decreased from $T_{\max }$ to $T_{\mathrm{CEmax}}$ to $T_{\mathrm{Cmax}1}$. The Se/Sp values for $T_{\max }$ and $T_{\mathrm{CEmax}}$ are similar, which reflects the extent of overlap in their ROC curves (Fig. 6) and minimal AUC differences (Table 4). Performance was good for $T_{\mathrm{Cmax}1}$, yet significantly lower than for $T_{\max }$ and $T_{\mathrm{CEmax}}$. While discrimination of febrile subjects was superior for a cutoff threshold of 38°C, impressive outcomes were also obtained for lower grade fever thresholds at 37.5°C and 37.8°C. The trend of increasing Se/Sp with diagnostic threshold may be due to a larger difference between the temperature of febrile subjects and the normal temperature value where most of the subjects were; a normal temperature with some error is less likely to reach 38°C.

3.4.

Blackbody Compensation

Temperature compensation using a validated BB was recommended by ISO/TR 13154 to improve IRT system stability. To quantify the impact of this compensation, correlation coefficients between the facial temperatures and $T_{\mathrm{ref}}$ (Table 8) and the AUC values based on a diagnostic threshold of 37.5°C (Table 9) were calculated without BB compensation. Comparison of Table 2 with Table 8 shows that the $r$ values with BB compensation increased for all facial temperatures measured by both IRTs. The increases for $T_{\mathrm{Cmax}1}$, $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$ are 7%, 13%, 12%, and 12% for IRT-1, respectively, and 5%, 4%, 6%, and 5% for IRT-2, respectively. Comparison of Table 3 with Table 9 shows that the AUC values with BB compensation also increased for all facial temperatures. The increases for $T_{\mathrm{Cmax}1}$, $T_{\mathrm{FEmax}}$, $T_{\mathrm{CEmax}}$, and $T_{\max }$ are 2%, 3%, 4%, and 4% for IRT-1, respectively, and 2%, 1%, 3%, and 3% for IRT-2, respectively.

Table 8

Pearson correlation coefficients (r values) between facial temperatures and Tref (no BB compensation).

Table 9

AUC values from the ROC curves of different facial temperatures (no BB compensation).

4.1.

Thermographic Screening Accuracy and Standardization

This study was largely based on two international consensus documents described above—IEC 80601-2-59 and ISO/TR 13154.^23,24 The guidance provided by these publications helped ensure that the devices used in this research had a high level of image quality and that the acquisition methods—including instructions to subjects—were optimized to enable accurate measurements. The optimal approaches identified in our study produced results that were equal to or better than most prior relevant works in terms of absolute agreement with, and correlation to, reference measurements, as well as discrimination of febrile subjects.

Our findings showed that the differences between $T_{\mathrm{ref}}$ and temperatures of different facial regions were in the ranges of 1.6°C to 2.8°C for the forehead region, 1.4°C to 2.4°C for the inner canthi regions, 1.7°C to 1.8°C for the mouth region, and 1.2°C to 1.3°C for the maximum face temperature. The magnitude of these results is smaller than results from Nguyen et al.,¹⁷ who showed differences in the range of 2.1°C to 8.7°C between $T_{\mathrm{ref}}$ and facial maximum temperature by three IRTs; similarly Chan et al.¹⁸ showed forehead temperatures differences of 3°C and 3.9°C for febrile and afebrile subjects, respectively. Our results also showed strong correlations between IRT-measured temperatures ($T_{\mathrm{IRT}}$) and $T_{\mathrm{ref}}$, with both IRTs producing $r$ values as high as the 0.75 to 0.80 range. These values are much higher than several prior studies that found $r$ values between IRT and oral temperatures of no greater than $0.45$.^2,17,39 Scatter plots of $T_{\mathrm{IRT}}$ versus $T_{\mathrm{ref}}$ provided in prior studies, such as Chan et al.,¹⁸ also do not show the strong linear trends seen in our $T_{\max }$ and $T_{\mathrm{CEmax}}$ data (Fig. 5). It is likely that this improvement in correlation is due to control methods that help to reduce measurement variability, including stability correction with a BB, reduction of confounding environmental factors, multiframe averaging, and the use of canthi regions and full-face maximums in thermal images.

Strong temperature correlations enabled discrimination between febrile and afebrile subjects to a high degree of accuracy. For a low-grade fever diagnostic threshold of 37.5°C, $T_{\max }$ data produced an AUC value of 0.95 to 0.97 and Se/Sp values in the 0.85 to 0.95 range. For a diagnostic threshold of 37.8°C, Se/Sp values increased to the 0.93 to 0.95 range. These results for relatively low-grade fever detection, as well as findings at higher diagnostic thresholds shown in Table 5, compare favorably with the literature. In a study of airport travelers, Priest et al.²⁰ found an AUC of 0.71 ($\mathrm{Se}/\mathrm{Sp}=0.86/0.71$) for a fever threshold of 37.5°C using full-face maximum temperatures. Nishiura and Kamiya² estimated that the AUC values were 0.79 and 0.75 for threshold temperatures of 37.5°C and 38.0°C, respectively. Nguyen et al.¹⁷ compared IRT performance for fever screening using images of the face and neck, with 37.8°C as the fever threshold. In this study, AUC values of 0.96 and 0.92 were found for two IRTs, yet the corresponding $r$ values of 0.43 and 0.42 do not appear sufficient for high accuracy measurements. Hewlett et al.¹⁹ obtained AUC values of 0.86 and 0.90 for fever thresholds of 37.8°C and 38°C, respectively, but did not report $r$ values or results for 37.5°C. These comparisons provide substantial evidence that an approach based largely on adherence to recently published standards has the potential to advance IRT-based fever-screening capability.

4.2.

Comparison of Facial Temperatures

The 17 facial temperatures extracted from each subject’s thermal image can be categorized by facial region (forehead, canthi, mouth, and entire face) or by measurement location selection method (fixed location versus maximum value of a defined region). Analyzing our extensive clinical testing results provided insight into key trends and potential approaches for optimizing IRT-based fever screening.

IRT system performance was highly dependent on measurement location, with the forehead producing lower accuracy than canthi regions. Temperatures determined from five fixed locations on the forehead ($T_{\mathrm{FC}}$, $T_{\mathrm{FT}}$, $T_{\mathrm{FB}}$, $T_{\mathrm{FL}}$, and $T_{\mathrm{FR}}$) had relatively low correlations ($r<0.50$) with $T_{\mathrm{ref}}$ and larger pairwise differences. Fixed locations in the canthi region showed moderately strong correlations ($r$ values of 0.51 to 0.63) with $T_{\mathrm{ref}}$ and their pairwise differences from $T_{\mathrm{ref}}$ were also relatively small. Similarly, the maximum-value data for canthi regions showed better performance than the forehead or mouth regions. This result aligns with a prior comparison of IRT-based eye and forehead measurements.⁴³ In our study, maximum value of the entire face ($T_{\max }$) provided better performance than the forehead ($T_{\mathrm{FEmax}}$) in terms of correlation and fever detection; this finding is consistent with a prior study that compared maximum temperatures from the full face and forehead ($r$ values of 0.43 and 0.36, respectively).¹⁸ Differences in performance between the forehead and inner canthi are likely due to perfusion of the canthi from the internal carotid (ophthalmic) artery, proximity to large vessels, and relatively thin skin,³² whereas the forehead is less diffusely perfused and more susceptible to convective and evaporative cooling.^43,44 These findings may shed light on the poor sensitivity values found in some NCIT studies.¹⁴

Overall, the maximum value in a region showed better diagnostic performance and correlation with $T_{\mathrm{ref}}$ than the value at a fixed location within this region, with greatest $r$ values and AUC values for $T_{\max }$, followed by $T_{\mathrm{CEmax}}$ and then $T_{\mathrm{FEmax}}$. Prior studies have also found that maximum-values approaches tended to provide greater performance.¹⁸ $T_{\mathrm{CEmax}}$ and $T_{\max }$ yielded similar $r$ values and statistically equivalent AUC values, as well as significantly higher AUC values than $T_{\mathrm{FEmax}}$. Interestingly, Fig. 5 shows that unlike the relatively tight cluster of normal-range data points ($T_{\mathrm{ref}}=36.4°\mathrm{C}$ to 37.4°C), data for $T_{\mathrm{Cmax}1}$ exhibit a tail extending to lower IRT-measured values than other datasets. This feature is also present in the few scatter plots that have been published from clinical IRT data.^18,43 Additionally, we found that individual hairs on the forehead degraded accuracy. The improved performance observed for maximum region temperatures may be due in part to subject-to-subject variations in facial anatomy and physiology that cause unpredictable nonuniformity in spatial temperature distribution. Taking the maximum temperature of a region affords greater robustness to such variations.

As noted above, approaches involving the inner canthi or maximum-temperature locations provided higher levels of performance. Therefore, it is not surprising that $T_{\mathrm{CEmax}}$—which involves both of these features—provided one of the best options of the 17 temperatures tested. The finding that $T_{\max }$ provided slightly better performance than $T_{\mathrm{CEmax}}$ is a more unexpected result, because it was not advocated in ISO/TR 13154 as a “robust measurement site,” as the inner canthi were. However, this approach has been used in a number of prior studies^17,18,20 likely due to its combination of simplicity and effectiveness. These prior studies achieved relatively high-Se/Sp values (0.7 to 0.9) using this approach. In part, this effectiveness stems from the fact that the inner canthi are a key thermal feature in full-face images, as discussed Sec. 4.3. In spite of these benefits, there may be unresolved challenges related to the use of $T_{\max }$, such as confounding physiological factors (e.g., sinusitis) that impact temperature distributions.²⁴

4.3.

Distribution of Thermal Maxima in Full-Face Images

To better understand the results obtained with $T_{\max }$, we evaluated the distribution of locations where maximum temperatures occurred over 3252 thermal images collected by the two IRTs from the first round of measurements. The locations of thermal maxima in full-face images are summarized in Fig. 7 and Table 10. According to Table 10, thermal maxima appeared most commonly (59.5%) in the inner canthi region, followed by oral (21.7%), forehead (8.8%), nasal (4.1%), and temporal (3.6%) regions. The predominance of inner canthi maxima is expected given what is known regarding perfusion in this region. A relatively large fraction of maxima occurred in the oral region, likely due to perfusion from the facial artery which is closer to the external carotid artery than the vessels that perfuse most facial regions. The forehead maximum was typically along the hairline, likely due to the thermal insulation effect of hair. Some thermal maxima appeared in the temporal region, likely due to the superficial temporal arteries. It was unexpected to find maxima in the nasal/nostril region (bottom); whether these are due to some pathologies, such as sinusitis,^24,45 or perhaps exhalation of warm air is not currently known.

Fig. 7

Thermal images illustrating (a) delineation of regions for thermal maxima analysis and (b)–(i) examples of maxima in human subjects. (a) Defined regions, (b) inner canthi, (c) oral (closed), (d) forehead (hairline), (e) temporal, (f) nasal, (g) neck, (h) other (outer canthus), and (i) other (cheek). Photo (a) is of author Q. Wang.

Table 10

Spatial distribution of facial temperature maxima.

4.4.

Quality of a Thermographic Screening System

The IEC 80601-2-59 standard²³ defines an ST as a system composed of an IRT and an external temperature reference source (usually a BB with known temperature and emissivity), and in some cases, a computer and software for data acquisition, processing, and storage. Therefore, most results in this paper, except for the data in Sec. 3.4, are technically not results of two thermal cameras, IRT-1, and IRT-2, rather, two fever-screening systems, (IRT-1 + BB) and (IRT-2 + BB). We have evaluated these two systems in our previous work²⁸ and found that their uniformity, stability, drift, minimum resolvable temperature difference, and laboratory accuracy all satisfied the standard requirements.

The use of a BB for temperature compensation had a moderate impact on IRT screening ability. In our prior study,²⁸ such compensation vastly improved the stability of IRT-2 and enabled the system to meet IEC 80601-2-59 performance specifications. Except to measure and compensate for long-term drift, the use of three-frame averaging in the current study may have improved stability to the point where the BB was no longer critical. If no frame averaging is used, the use of a BB would likely be more critical for fever detection. Additionally, the current study was executed in an environment with relatively stable ambient temperature; it is likely that in a less controlled screening location with larger, more rapid thermal fluctuations, BB compensation would be more important.

While inherent IRT instrumentation quality is critical, the performance also depends on effective implementation. The use of control methods, such as an absorbing background, multiple frame imaging, and thermally stable, forward-facing subjects also likely helps to optimize screening performance. Given that many of these confounding factors have been addressed in our study, the results presented here likely indicate a best-case performance level. As control, if methods we have implemented are removed—which may be necessary in certain real-world screening situations—it is likely that performance will degrade. The degree to which removal of any specific control will impact results is beyond the scope of the current study but may be important for predicting real-world performance.

4.5.

Fever Screening During an Epidemic

The primary purpose of this study was to facilitate the implementation of IRT systems and practices that enable optimal measurement accuracy and highly effective fever screening during epidemics. However, achieving effective screening can be a complex process, as many factors need to be addressed beyond the physics, instrumentation, and acquisition procedures. While our results showed that some facial temperatures had good discrimination abilities with high AUC values, some previous literature claimed that thermography was not highly effective for fever screening during disease outbreaks.^1,2,46,47 This may have been due to device instability,^{7,11,32,34,48} inappropriate temperature reading locations, nonstandard calibration, and environmental controls.²⁸ In a future study, we will address the impact of environmental conditions outside those that the standard recommends—such as higher or lower ambient temperatures that might be encountered near the entry of a building—on IRT accuracy and linearity.

The frequency at which fever presents as a symptom is another impediment to successful screening. In the current COVID-19 outbreak, many of those infected are largely asymptomatic and only 73% have exhibited a fever;³ in 2009, only half of H1N1 outbreak cases had temperatures of $\ge 37.8°\mathrm{C}$;⁸ and a 2011 study indicated that none of the 30 subjects identified as being flu-infected had a temperature of 37.8°C or greater, and only two had a temperature of 37.5°C.²⁰ Therefore, while IRT-based screening can detect individuals with elevated body temperature, it is not a viable stand-alone tool for screening for individuals infected with specific diseases.⁴⁹ It may play an adjunct role along with other screening evaluations. Since fever is only one common symptom of infectious disease, an effective screening process should include evaluation of a range of symptoms.^3,7,8 The future development of an integrative screening system may include thermography along with other optical imaging approaches for evaluation of vital signs, such as pulse rate and respiratory rate, and other physiological parameters.⁸

Given that $T_{\mathrm{CEmax}}$ and $T_{\max }$ provided the best performance, it is worth considering issues that might influence the decision to implement one approach or the other. Acquiring a full-face region for calculation of $T_{\max }$ would likely be easier to accomplish and performed more reliably than determining $T_{\mathrm{CEmax}}$ via auxiliary visible-light imaging and computationally intensive techniques for coregistration of inner canthi regions.³⁷ This may be particularly important in a high-throughput situation where delays due to computer processing or image coregistration errors could become highly inconvenient. However, implementing an approach that blindly determines the maximum temperature from a full-face thermal image may increase the need to identify confounding pathological/physiological conditions such as sinusitis.^24,45,50 To accomplish this task rapidly and effectively may require significant screener training, although automated approaches (e.g., deep-learning algorithms) could also be developed to augment or replace manual assessments.

Another practical challenge involves the identification of an appropriate reference temperature diagnostic threshold, given the diversity of values that have been implemented. The Centers for Disease Control and Prevention (CDC) has recommended the use of 38°C,⁵¹ whereas prior human subject studies have been based on 37.5°C,^7,39,40 37.6°C,⁵² 37.7°C,^32,48 37.8°C,^17,19,20 and 38°C.³⁵ Different thresholds have been used for different outbreaks, such as $\ge 38°\mathrm{C}$ for SARS¹ and 37.7°C for adults and 37.9°C for children in an H1N1 study.⁸ The literature indicates that as the threshold temperature decreases, diagnostic accuracy typically degrades. Uncertainty in normal body temperatures—which can be influenced by circadian rhythm, age, physical exertion, and other factors—can further increase error in screening tasks.^{15,35,39,53–56} For example, studies have shown that the core body temperature in the morning may be significantly lower than in the afternoon.^53–55 Additionally, a recent study indicates that normal body temperature has decreased on average since the establishment of the 37°C threshold 150 years ago.⁵⁷ In spite of these obstacles, our results indicated that IRT systems are capable of detecting low-grade fever (37.5°C) in subjects, which could mean that early-stage infections and those producing only moderate symptoms could be more readily identified. The significance of this ability is demonstrated by the fact that of the subjects with $T_{\mathrm{ref}}$ values over 37.5°C in our study, 60% would not have exceeded the CDC recommended diagnostic threshold of 38°C.

Even if a suitable diagnostic threshold for fever based on body temperature can be defined, determining the IRT cutoff temperature for fever screening requires a variety of considerations. While we calculated optimal cutoff temperature to optimize both Se and Sp, this may not represent an optimal value for real-world use. For a severe disease, lower cutoff values may be needed to minimize false negatives in primary screening. Given the typically low prevalence of diseased individuals in a screening population, the false positive rate in primary screening will be high (and thus the positive predictive value low). On the other hand, it may also be important to balance the burden on the population being screened (e.g., travel delays) and screening personnel (e.g., workload, fatigue, and cost to health agencies).^17,39