1.

Introduction

In recent years, muscle oxygen saturation (${\mathrm{SmO}}_2$) measured via near-infrared spectroscopy (NIRS) has developed into an affordable and readily available technology. The application of this technology in athletic settings is expanding, as fundamental questions are being addressed.¹ One of the major concerns in the application of NIRS in athletic settings is that common limitations of NIRS are not well understood and need to be properly addressed, while still allowing the technology to be utilized for its clear advantages.^2,3

The problem associated with using NIRS to measure quantifiable values has been the unknown path length problem in the modified Beer–Lambert method. Without knowing the photon path length, it is impossible to derive quantifiable values from the returning NIRS signal. This has the effect that NIRS output is in relative values, most often expressed as arbitrary units of hemoglobin (Hb) and myoglobin (Mb). NIRS cannot differentiate between oxyhemoglobin (${\mathrm{O}}_2\mathrm{Hb}$) and oxymyoglobin (${\mathrm{O}}_2\mathrm{Mb}$) or deoxyhemoglobin (HHb) and deoxymyoglobin (HMb)⁴ and therefore for the remainder of this paper the terms ${\mathrm{O}}_2\mathrm{Hb}$ and HHb will be the sum of ${\mathrm{O}}_2\mathrm{Hb}$ and ${\mathrm{O}}_2\mathrm{Mb}$ and HHb and HMb. Because these arbitrary units are relative in nature, direct comparisons are difficult and usually limited to trends in the derived signal. To increase the robustness of the relative values, it is often recommended to use a saturation in a percentage using the following equation:^5,6

Saturation takes into consideration the relative change of total hemoglobin (tHb) and the interaction between ${\mathrm{O}}_2\mathrm{Hb}$ and HHb. The result of this type of saturation equation is ${\mathrm{SmO}}_2$, as mentioned earlier, or an often-applied tissue oxygenation index (TOI). It is important to denote the difference between the two, as they are often used interchangeably. In ${\mathrm{SmO}}_2$, the m indicates that the saturation is intended be isolated to the muscle layer. In TOI, the $T$ indicates the measurement is an average of all tissues under the sensor. Then, term saturation refers to the availability and functionality of a 0% to 100% scale, whereas an index generally refers to a measurement ratio to be compared to a fixed standard. If a parameter includes the term saturation, it should be reliable and valid in terms of 0% to 100% scale. However, large manufacturer differences, including inter-optode distance, number of wavelengths used, spectral width, and algorithmic calculations, all raise questions about the scaling of ${\mathrm{SmO}}_2$ and TOI.^7–9 These differences focus around interindividual and intersite (muscle site selection) variation in results.^10,11 When choosing a new muscle site or changing test participant altogether, the optical properties measured by the NIRS device changes. If the NIRS device cannot adjust for these changes in specific measurement site properties, a large variation is expected and seen. Statistical normalization approaches, such as a physiological calibration, are then often used to address these problems.^7,9,12

The situation becomes even more complicated as no consensus “gold standard” for NIRS-derived values exists to compare against. Certain studies do refer to gold standard comparisons, looking at alternate measurement techniques to validate NIRS. Invasive experiments, such as isolated hindlimb experiments with animal subjects^13,14 or venous blood draw in humans performing exercise protocols,¹⁵ show good results. Other publications comparing NIRS measurements to invasive measures of blood oxygenation levels demonstrate lower levels of success.^16,17 Noninvasive phosphorus magnetic resonance spectroscopy and NIRS comparisons¹⁸ also show good results. Interesting, in particular, is the comparison between functional magnetic resonance imaging (fMRI), which is noted by Cui et al.¹⁹ as a gold standard in human brain imaging and NIRS. FMRI as blood oxygen-level-dependent imaging requires the user to make almost near-identical assumptions about the measurement technique being used as NIRS does.⁶ When debating a specific gold standard to which NIRS-derived ${\mathrm{SmO}}_2$ should be compared, an obvious lack of authority exists as the term is not mentioned in reviews,^6,20,21 with appropriate hesitation. Alternate methods are needed to address the question of validity of ${\mathrm{SmO}}_2$ measurements on a 0% to 100% scale.

A technique used for addressing the difficulty of relative values provided by NIRS, including the interindividual and intersite measurement dilemma, is the normalization process of physiological calibration.^20,21 This technique applies the arterial occlusion method (AOM) to identify the minimum and maximum signals of the NIRS device in question. The AOM functions by applying a suprasystolic cuff to a chosen limb for a 5- to 6-min period, which results in a linear change in the NIRS-derived signal for ${\mathrm{O}}_2\mathrm{Hb}$ to a minimum signal point identified by a plateau. Upon release, the phenomenon of hyperemia²² results in a return of the NIRS signal to a maximum point; the opposite is true for HHb. The minimum signal is a maximally deoxygenated state defined by a disappearance of ${\mathrm{O}}_2\mathrm{Hb}$ in relation to the sum of ${\mathrm{O}}_2\mathrm{Hb}$ and HHb, and maximum signal is a maximally oxygenated state and the disappearance of HHb to the sum ${\mathrm{O}}_2\mathrm{Hb}$ and HHb. Having identified the minimum and maximum points, these can then be set at 0% to 100% as a physiological calibration. This process generates an individual and robust scale for further testing. Considering the debate around a true gold standard, this type of physiological calibration offers a functional test to assess and compare NIRS devices and ${\mathrm{SmO}}_2$ scaling. In athletic settings and other possible applications of NIRS, using a standard AOM to calibrate the NIRS signal for each measurement site is not a feasible approach. Therefore, an NIRS device that provides a reliable and valid 0% to 100% scale with reasonable accuracy would greatly enhance NIRS usability.

To address the question of validation and reliability of an NIRS device, three concepts should be applied:

This paper applied the three identified concepts above in a specific test battery to evaluate the performance of ${\mathrm{SmO}}_2$ on a scale of 0% to 100% provided by a commercially available NIRS device, the Moxy Monitor.

2.

Methods

2.4.

Procedure

A series of tests were selected to address the questions of reliability and validity on a 0% to 100% scale generated by the selected NIRS device. The test battery was designed to examine repeatability, reproducibility, and face validity. To limit the physiological variation and ensure a stable and repeatable environment, all tests used the AOM. For the experimental procedure, participants came into the lab for two session with 1-week separation between each session, as shown in Fig. 1.

Fig. 1

Experimental procedure and aim for each participant. The test battery involved two AOM test sessions including two passive AOM trials and one active AOM trial.

2.4.1.

Arterial occlusion method

The AOM was conducted using a pneumatic tourniquet (Rudolf Riester GmbH, DE) with thigh cuff dimensions of $96\times 13\mathrm{cm}$ inflated to $>300\mathrm{mm}\mathrm{Hg}$. The tourniquet was suited on the right leg of all the participants. Prior to every test, all participants were asked to refrain from strenuous physical activity 24 h prior, to refrain from alcohol consumption and smoking 24 h prior to the experiment, and to maintain individual diet routine. The maximally deoxygenated state plateau identified as ${\mathrm{SmO}}_2$ minimum (${\mathrm{SmO}}_2\min$) was determined by the average of the final 20 s or 10 data points of the AOM, as long as this met the condition of a visual plateau. The maximally oxygenated state identified as ${\mathrm{SmO}}_2$ maximum (${\mathrm{SmO}}_2\max$) was determined as the peak ${\mathrm{SmO}}_2$ output average over 10 s or 5 data points following the end of the AOM as a result of the hyperemic effect.

2.4.2.

Passive trials

Sensors were placed on the VL, VM, RF, and G and the participants assumed a lying supine position. Participants assumed the lying position for 5 min prior to data collection. The data collection started with 60 s of data collection for a baseline measurement, and after 60 s the pneumatic tourniquet was rapidly inflated. The pneumatic tourniquet remained inflated and pressure controlled for the 6 min to find the ${\mathrm{SmO}}_2\min$ plateau. The quality of the arterial occlusion was controlled through pulse oximeter and pulse palpation of the lower leg. After 6 min, the pneumatic tourniquet was released, and an additional 3 min of measurement took place to assess the hyperemic response and to find the ${\mathrm{SmO}}_2\max$ value.

2.4.3.

Active trial

Each participant came in for an initial setup session to determine 1 repetition-maximum (1-RM). Participants executed a series of maximum effort leg extension trials on a leg extension machine (Schnell GmbH, DE). The best trial was taken as their estimated 1-RM. In the active AOM session, each participant was again suited with the pneumatic tourniquet and the sensors placed on the activity-recruited muscles VL, VM, and RF. The participants were then positioned in the knee extension machine and remained in the sitting position for 5 min prior to data collection. The data collection started with 60 s of data collection for a baseline measurement and, after the 60 s, the pneumatic tourniquet was rapidly inflated. The participants then executed continuous leg extension repetitions at 40 rpm at 5% of 1-RM until exhaustion. The pneumatic tourniquet remained inflated and pressure controlled as long as activity took place to identify ${\mathrm{SmO}}_2\min$. Following exhaustion, the pneumatic tourniquet was released, and an additional 3 min of measurement took place to assess the hyperemic response and to find the ${\mathrm{SmO}}_2\max$ value.

3.

Results

All muscle sites display the expected linear decrease in ${\mathrm{SmO}}_2$ with the application of the suprasystolic cuff to a minimum plateau and upon release a hyperemic response in both passive and active conditions (see Fig. 2). The 0% to 100% scale showed a good dynamic range over all muscle sites during passive conditions with $M_{\text{range}}$ of $67.9\%\pm 9.9$ ($M_{\min }=10.1\%\pm 5.7$; $M_{\max }=78.1\%\pm 6.0$), as shown in Fig. 3.

Fig. 2

Predicated linear decrease in ${\mathrm{SmO}}_2$ as a result of the AOM and time during (a) passive and (b) active conditions, with minimum value plateau and hyperemic response on cuff release. Dotted lines indicate start and stop of the AOM. Mean VL (○); mean VM (⋄); mean RF (▵); mean G (x).

Fig. 3

Mean (black squares) and 90% CI (thick darker vertical lines) and 95% CI (thin lighter vertical lines) for ${\mathrm{SmO}}_2\min$ and ${\mathrm{SmO}}_2\max$ for (b) active and (a) passive conditions for all muscle sites: VL, VM, RF, and G. Dashed lines indicate the a priori determined thresholds of ${\mathrm{SvO}}_2$ for assessment of face validity.

3.1.

Repeatability, Interindividual, and Intersite Reproducibility: Passive Trials

Looking at the mean difference of ${\mathrm{SmO}}_2\min$, repeatability during passive trials in all muscle sites showed no statistical difference between passive trials 1 and 2 and all can be considered statistically equivalent using EI (see Fig. 4). For ${\mathrm{SmO}}_2\max$, repeatability during passive trials all muscle sites showed no statistical difference between passive trials 1 and 2, but only the VL and G sites can be considered statistically equivalent (Fig. 4). Bland–Altman plot analysis was used in addition to assess equality on a case-to-case basis. All Bland–Altman results were considered to show suitable agreement between passive trials 1 and 2 (Fig. 5). All muscles show no systemic bias between trials 1 and 2, as the line of equality is clearly within the EI. The data show that the ${\mathrm{SmO}}_2\max$ hyperemia has a greater degree of variation than ${\mathrm{SmO}}_2\min$ as a result of the AOM (Fig. 5). However, none of the plots shows significant relationship between mean and difference in both the max and min responses following a Pearson product–moment correlation. Looking at the mean of ${\mathrm{SmO}}_2\min$ and ${\mathrm{SmO}}_2\max$ for interindividual and intersite reproducibility during passive trials, no statistical difference between the obtained values can be discerned (Fig. 2).

Fig. 4

Mean differences (black squares) and 90% CI (thick darker horizontal lines) and 95% CI (thin lighter horizontal lines). The mean differences for active and passive trials are the difference between passive trials mean and the active trial in question. A priori EI is set at $\pm 5\%$ (dashed lines). Solid line indicates line of equality.

Fig. 5

Bland–Altman plots of passive trials 1 and 2 for ${\mathrm{SmO}}_2\min$ and ${\mathrm{SmO}}_2\max$ for (a) VL, (b) VM, (c) RF, and (d) G. A priori EI is set at $\pm 5\%$ (shaded area). The solid line identifies the mean bias (MB) and dashed lines identify the upper and lower limits of agreement at $\pm 1.96\mathrm{SD}$. For both the MB and the limits of agreement, respectively, dotted lines represent 95% CI.

3.2.

Interindividual, Intersite, and Interactivation Reproducibility: Active Trial

Looking at reproducibility between active trials and passive trials, mean difference comparisons show no difference and statistical equivalency between VL ${\mathrm{SmO}}_2\min$ and VM ${\mathrm{SmO}}_2\min$ using the EI (see Fig. 4). The same is true for VL ${\mathrm{SmO}}_2\max$. RF ${\mathrm{SmO}}_2\max$ and VM ${\mathrm{SmO}}_2\max$ showed no difference but are not statistical equal (see Fig. 4). RF ${\mathrm{SmO}}_2\min$ was not statistically equivalent and different at the 95% CI. The results of ${\mathrm{SmO}}_2\max$ and ${\mathrm{SmO}}_2\min$ on a Bland–Altman plot for the three examined quadriceps muscle sites show acceptable equivalency for VL and VM but not for RF between passive and active trials (see Fig. 6). As with the comparison between passive trials, the active–passive trials show a greater degree of variation in the ${\mathrm{SmO}}_2\max$ hyperemia results. Unlike the passive trial comparison, the active-passive trials show a significant relationship between mean and difference for ${\mathrm{SmO}}_2\min$ in a Pearson product-moment correlation with a tendency toward higher values for ${\mathrm{SmO}}_2\min$ for the active condition for VL ${\mathrm{SmO}}_2\min$ ($r=-0.5251$, $n=15$, $p=0.044$) and RF ${\mathrm{SmO}}_2\min$ [$r=-0.7071$, $n=13$, $p=0.006$ (see Fig. 6)]. Looking at the mean of ${\mathrm{SmO}}_2\min$ and ${\mathrm{SmO}}_2\max$ for interindividual and intersite reproducibility, during active and passive trials only RF ${\mathrm{SmO}}_2\min$ during the active trial shows potential difference in means (see Fig. 2).

Fig. 6

Bland–Altman plots of passive trials mean and active trials for ${\mathrm{SmO}}_2\min$ and ${\mathrm{SmO}}_2\max$ for (a)VL, (b) VM, and (c) RF. A priori EI is set at $\pm 5\%$ (shaded area). The solid line identifies the MB and dashed lines identify the upper and lower limits of agreement at $\pm 1.96\mathrm{SD}$. For both the MB and the limits of agreement, respectively, dotted lines represent 95% CI. For RF ${\mathrm{SmO}}_2\min$ and VL ${\mathrm{SmO}}_2\min$, a significant relationship was found between mean and mean difference (diagonal correlation line) at $R^2=0.2727$ and $R^2=0.4999$, respectively.

3.4.

Effect of Adipose Tissue Thickness

Because of the well-documented effects of ATT on NIRS signal and consequently on ${\mathrm{SmO}}_2$, the effect of ATT on ${\mathrm{SmO}}_2\min$ was plotted for each muscle site using all 22 participant’s data (see Fig. 7). Linear and segmented regressions were calculated for each muscle site. Clearly, a relationship exists between ATT and ${\mathrm{SmO}}_2\min$, as shown by the linear regression for all muscle sites: VL ($r=0.844$, $n=22$, $p<0.001$); VM ($r=0.901$, $n=22$, $p<0.001$; RF ($r=0.872$, $n=22$, $p<0.001$); and G ($r=0.912$, $n=22$, $p<0.001$). When using a segmented regression, all muscle sites identify a single knot at $15.37\pm 1.52\mathrm{mm}$ to optimize the linear regression (see Fig. 7). The results indicate to a certain degree an effectiveness in maximizing the sensitivity of the NIRS-derived ${\mathrm{SmO}}_2$ signal as long as the ATT thickness remains within the recommended penetration depth threshold of 15 mm. ${\mathrm{SmO}}_2$ values obtained over the ATT threshold should be considered suspect.

Fig. 7

Dashed lines indicate the linear model and segmented regression model with one knot point. Dotted lines indicate start and stop of analysis data points and the knot point. The optimal knot minimizes sum of squares residuals calculated at (a) VL 14.595 mm, $R^2=0.798$; (b) VM 17.613 mm, $R^2=0.928$; (c) RF 15.000 mm, $R^2=0.862$; and (d) G 14.270 mm, $R^2=0.907$.

4.

Discussion

The purpose of this experiment was, first, to propose a test battery to investigate reliability and validity of ${\mathrm{SmO}}_2$ on 0% to 100% scale and, second, to apply this test battery to readily available NIRS device. The argument presented makes some assumptions, the first of which is that the process of a physiological calibration through AOM provides reliable and functional information about the range of NIRS-derived oxygenation signals at an individual level. This being true, a reliable and valid 0% to 100% scale can be scrutinized using the AOM. The 0% to 100% scale tested would need to show acceptable results for repeatability, reproducibility, and face validity, and therefore this test battery sets the groundwork to determine device functionality. In this experiment, all participants showed good repeatability and reproducibility during the AOM tests using the Moxy Monitor.

To discuss validity, an inference was made, and therefore as a product of deductive reasoning the term face validity was considered appropriate. The first position was that muscle tissue during activity or occlusion situations would represent the highest metabolic activity in comparison to other peripheral tissue being measured.⁴³ Therefore, when comparing NIRS-derived values for ${\mathrm{SmO}}_2$ against invasive measures of ${\mathrm{SvO}}_2$, the ${\mathrm{SmO}}_2$ values cannot be higher than the measured value for ${\mathrm{SvO}}_2$. ${\mathrm{SmO}}_2$ should be lower, as ${\mathrm{SvO}}_2$ is a combination of venous blood returning from all tissue layers, including adipose and skin tissue; this is the premise of venous blood contamination, which will be discussed later. While this does not establish validity of an NIRS-derived ${\mathrm{SmO}}_2$ value, it does establish thresholds against which measured values can be tested and lends a useful ${\mathrm{SmO}}_2$ range; the same argument is made by McManus et al.⁷ The a priori ${\mathrm{SvO}}_2$ thresholds applied to this study are drawn from ${\mathrm{SvO}}_2$ data collected during AOM experiments and show the extent of oxygenation ranges in metabolic tissue.^41,42 This range has also been established during high-intensity exercise, during small muscle activation, and during full body exercise. Costes et al.¹⁶ showed ${\mathrm{SvO}}_2$ values of $17.8\pm 4.2$ and $7.8\pm 2.3$ in normoxia (N) and hypoxia (H), respectively, following 20 min of steady-state cycling at 80% of ${\mathrm{VO}}_2\max$. Mancini et al.¹⁵ and Macdonald et al.¹⁷ showed ${\mathrm{SvO}}_2$ ranges following small muscle and lower body isolated kicking exercises of $30.4\pm 6.8\%$, and then $40.1\pm 2.2$ (N) and $33.9\pm 1.8$ (H). All three of these investigations included NIRS measurements. While for Mancini et al.¹⁵ the NIRS data collected correlated with the ${\mathrm{SvO}}_2$ data, for both Costes et al.¹⁶ and Macdonald et al.¹⁷ this was not the case under the normoxic condition. This confounds the relationship between NIRS-derived oxygenation signals and ${\mathrm{SvO}}_2$. A reconciliation with these apparent contradictions involves the venous blood contamination problem discussed earlier.⁶ Measured ${\mathrm{SvO}}_2$ is a combination of blood from active skeletal muscles and skin and adipose tissue circulation. The venous blood contamination problem then results in increased or contradictory NIRS values, in comparison to ${\mathrm{SvO}}_2$, because of increased contribution of lower metabolically active tissue to the NIRS signal as a result of, for example, skin blood flow increase for heat dissipation.^44,45 Advancements in the spatially resolved method has attempted to address this problem.⁴⁵ While this advancement has helped in the scaling of NIRS, when looking at device comparison data from this experiment, or alternative studies,^7–9 clearly a functional scale has not been determined. A device measuring ${\mathrm{SmO}}_2$ must distinguish between measured tissue layers to isolate the muscle layer. This is particularly important because certain NIRS devices provide a TOI measurement rather than a ${\mathrm{SmO}}_2$ measurement. This difference should be considered in the analysis and discussion of NIRS data, as the two terms should not be used interchangeably.

To further address and discuss this useful accuracy of ${\mathrm{SmO}}_2$, assumptions about signal contribution need to be made. Under normal conditions, muscles receive near completely oxygenated arterial blood⁸ and therefore it can be assumed changes in oxygenation reported by the NIRS signal can be attributed to changes in Hb and Mb oxygenation at the examined tissue level. The physical demands of the Beer–Lambert law exempt large blood vessels from contributing in a significant way to the NIRS-derived signal, as the concentration of absorbing chromophores is too large to return a signal.⁶ This means that the NIRS-derived signal is mostly from smaller vessel contributions along the lines of arterioles, capillaries, and venules. How much of the signal is derived from which source is a discussion,^46–48 and a commonly used formula is equal parts contribution by all three-vessel systems⁷ or the “one-third, one-third, one-third” model. This assumption led to the discussion by McManus et al.⁷ that the range of ${\mathrm{SmO}}_2$ by the Moxy Monitor is rather large. Interestingly, McManus et al.⁷ apply the same logic using ${\mathrm{SvO}}_2$ thresholds to discuss a potential physiological validity, just applying a different assumption of contribution. The paper does go on to discuss potential difference in NIRS signal as a result of muscle layer specialization—stressing the importance of terminology between for TOI and ${\mathrm{SmO}}_2$. Nonetheless, the one-third, one-third, one-third model stands to be disputed. Boushel et al.⁴⁷ argue for a $10:20:70$ ratio of arterial, capillary, and venous blood in the NIRS signal. An experiment by Poole and Mathieu-Costello⁴⁸ shows that $>90\%$ of total blood volume in the muscles is in the capillaries, which would then again return to the question of tissue layer isolation when talking about signal contribution. Clearly, the one-third, one-third, one-third model for signal contribution is questionable and an adequate model is up for debate. As this paper has no experimental claim to credit or discredit magnitude of signal contribution as was discussed, the assumption that a device advocating a 0% to 100% scale of reasonable accuracy in the form of physiological validity for ${\mathrm{SmO}}_2$ should have results that are smaller than the measured values of ${\mathrm{SvO}}_2$ remains intact as a matter of deduction.

5.

Conclusion

The study illustrates that the retail NIRS device Moxy Monitor is valid and reliable under the conditions of the test battery used. The validity attributed to the device in this paper is a consequence of a series of assumptions, which should be viewed critically. While the repeatability and reproducibility comparison were successful and the functionality of NIRS to measure changes in ${\mathrm{SmO}}_2$-related supply-and-demand parameters does not stand in question, the validity of a 0% to 100% scale is open for discussion. Nonetheless, this type of scaling is of great importance for research and athletic applications in order to compare and contrast data. For this reason, the authors recommend the use of functional scales of 0% to 100% that reflect to an acceptable degree physiological validity, in this case to the reference to ${\mathrm{SvO}}_2$. In the absence of a gold standard, functional scales should be tested in the form of physiological calibration via AOM involving this type of testing battery, including tests of repeatability, reproducibly, and validity.