*in situ*in a handheld device format. The combination of a handheld-type imaging system and a hyperspectral reconstruction algorithm from an RGB camera offers simple instrumentation and operation while avoiding the use of an imaging spectrograph or tunable color filter. This platform could potentially be integrated into a compact, inexpensive, and portable system, while being of great value in high-throughput phenotyping facilities and laboratory settings.

## 1.

## Introduction

Abiotic and biotic stress (e.g., drought, nutrient deficiency, temperature, and pathogen attack) impact plant growth and development, eventually influencing crop yield.^{1} Thus, it is critical to assess stress symptoms during early alteration stages before irreversible damage and yield loss occur.^{2} In addition, as transgenic and conventional plant breeding strategies seek to continue to improve plant input and output traits, researchers in laboratory and industry settings often require quantitative plant phenotyping technologies for specific selection of pathogen-resistant, stress-tolerant, and high-yielding plants.^{3} Indeed, there is a strong need for plant imaging technologies that nondestructively and quantitatively visualize stress traits *in situ*. Importantly, alterations in chlorophyll content are one of the key symptoms that plants display under stresses as a major component of photosynthesis.^{3}

There are a few methods currently available for quantifying chlorophyll content. Since subtle alterations in chlorophyll are not visible until they reach a certain level, destructive biochemical analyses serve as the gold standard. Optical meters, which quantify chlorophyll content as spot measurements, are also used as an advanced method, given the nondestructive nature of the technologies.^{4} However, these methods provide chlorophyll content in a single spot area, lacking the information on spatial distribution. In this respect, fluorescent imaging of chlorophyll has received considerable attention as a nondestructive imaging method.^{5}^{–}^{7} Indeed, chlorophyll fluorescence imaging serves as a valuable tool for quantifying leaf photosynthetic efficiency. On the other hand, quantitative assessment of absolute chlorophyll content using chlorophyll fluorescence is often limited, because fluorescent signals depend on measurement parameters, including the duration (e.g., continuous or pulsed) and the bandwidth (e.g., laser or incoherent light source) of the excitation source as well as quenching of fluorescence. Thus, it would not be straightforward to obtain an absolute value of chlorophyll content when different instruments for detecting chlorophyll fluorescence are utilized. As a result, fluorescence signals in plants often serve as a proxy measure or an indirect indicator of photosynthesis. On the other hand, reflectance measurements are based on elastic light scattering and absorption, in which the wavelength is not changed by the scattering or absorption processes without energy conversion. In this respect, reflectance-based approaches could potentially provide simple and reliable quantification of chlorophyll content while resolving its spatial heterogeneity.

For label-free imaging, conventional hyperspectral imaging systems rely on the use of a mechanical filter wheel, an imaging spectrograph, or a liquid crystal tunable filter, all of which limit the development of simple, compact, and cost-effective systems. To build compact and affordable systems for widespread utilization, it would be desirable to realize a spectrometerless (or spectrometer-free) imaging system. Such a system can be implemented by utilizing an algorithm that can reliably reconstruct full spectral information from RGB image data, which are easily acquired by commonly used three-color CCD (or CMOS) cameras. Fortunately, several methods for spectral reconstruction with RGB data have been studied extensively.^{8}^{–}^{10} Thus, hyperspectral reconstruction approaches using RGB data could potentially lead to simplicity for instrumentation and operation without using bulky and expensive color filters and spectrometers.

In this study, we report that a spectrometerless reflectance imaging method can map out the detailed spatial distributions of chlorophyll content in individual leaves *in situ* in a handheld device format. First, we confirm that reflectance spectral patterns correlate with chlorophyll content in leaves assessed by biochemical analyses. Initially, we utilize a laboratory system coupled with an imaging spectrograph with a high spectral resolution for obtaining a full reflectance spectrum in each $(x,y)$ location. As a testing model, *Arabidopsis thaliana* is used. We also measure the total chlorophyll content in each sample using conventional biochemical assays. Second, we conduct numerical experiments for extracting chlorophyll content without using hyperspectral imaging data. We implement a hyperspectral reconstruction algorithm that reliably reconstructs the full spectral information in plant leaves from RGB data. Third, we test a handheld-type spectrometerless imaging system consisting of a three-color CCD camera and a white-light light-emitting diode (LED) as a detector and a light source, respectively. Finally, we demonstrate detailed chlorophyll imaging in a whole leaf by acquiring RGB data only, without using an imaging spectrograph, in a pilot testing study.

## 2.

## Materials and Methods

## 2.1.

### Plant Models and Biochemical Assays for Chlorophyll Quantitation

A series of 24 leaf samples from *A. thaliana*, a well-known model plant, were utilized for covering a wide range of total chlorophyll content from 0.17 to $1.68\text{\hspace{0.17em}\hspace{0.17em}}\mu \mathrm{g}/\mathrm{mg}$. *A. thaliana* ecotype Col-0 was cultivated at a light intensity of $100\text{\hspace{0.17em}\hspace{0.17em}}\mu \mathrm{E}\text{\hspace{0.17em}}{\mathrm{m}}^{-2}\text{\hspace{0.17em}}{\mathrm{s}}^{-1}$ at 22 °C under a photoperiod of 16-h light/ 8-h dark. Rosette leaves were detached from 4-week-old plants, were incubated in a solution of 3-mM MES (2-[$N$-morpholino]ethanesulfonic acid) at pH 5.7, and were kept under dark for up to 6 days. For conventional biochemical assays, we extracted chlorophyll with 95% ethyl alcohol after incubating the samples at 70 °C for 1 h. Then, absorbance at 665 and 649 nm was measured using a UV/visible spectrophotometer. Finally, total chlorophyll content (i.e., chlorophyll $a$ and chlorophyll $b$) was calculated in the unit of $\mu \mathrm{g}/\mathrm{mg}$ (i.e., chlorophyll/fresh weight) as previously described.^{11}

## 2.2.

### Laboratory Hyperspectral Imaging System

For obtaining experimental spectra from leaf samples, our recently developed system described elsewhere was used.^{12}^{–}^{14} This system allowed us to acquire a matrix of reflectance intensity (also known as a hypercube) $r(x,y,\lambda )$ as a function of the position $(x,y)$ and the wavelength $\lambda $ of light. The specification included a transverse resolution of $\sim 100\text{\hspace{0.17em}\hspace{0.17em}}\mu \mathrm{m}$ with a field of view of $15\times 15\text{\hspace{0.17em}\hspace{0.17em}}{\mathrm{mm}}^{2}$, an imaging depth of $\sim 1\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{mm}$, and a spectral range of 400 to 770 nm with a spectral resolution of 2 nm. A back-directional (angular) filtering scheme in the detection part collected the light reflected from the sample within a narrow solid angle in the exact backward direction. Importantly, this configuration avoided spectral variations originating from different systems, because reflectance spectra, in particular scattering components, are highly sensitive to illumination and detection geometries.^{15} It should be noted that this aspect is crucial to apply a hyperspectral image reconstruction algorithm, trained by hyperspectral data obtained from the laboratory system, to a spectrometerless imaging system.

To remove the stray background light and to compensate for the system responses, the following procedures were implemented:^{13}^{,}^{16} First, after the acquisition of a raw intensity matrix from the sample ${r}_{\text{specimen}}(x,y,\lambda )$, a background intensity matrix ${r}_{\text{background}}(x,y,\lambda )$ without the sample was measured and then subtracted from the raw intensity matrix. Second, by placing a reflectance reference standard (Labsphere, North Sutton, New Hampshire) on the sample stage, a reference intensity matrix ${r}_{\text{reference}}(x,y,\lambda )$ was measured. Because the reflectance reference standard had flat spectral and uniform spatial responses in the entire visible range, ${r}_{\text{reference}}(x,y,\lambda )$ captured the entire system responses. Third, ${r}_{\text{specimen}}(x,y,\lambda )$ was normalized by ${r}_{\text{reference}}(x,y,\lambda )$ such that

## Eq. (1)

$$r(x,y,\lambda )=\frac{{r}_{\text{specimen}}(x,y,\text{\hspace{0.17em}\hspace{0.17em}}\lambda )-{r}_{\text{background}}(x,y,\lambda )}{{r}_{\text{reference}}(x,y,\lambda )-{r}_{\text{background}}(x,y,\lambda )}.$$## 2.3.

### Spectrometerless Imaging System Using a Three-Color CCD

A working prototype of the spectrometerless hyperspectral imaging system was also constructed, as shown in Fig. 1. To directly use a hyperspectral reconstruction algorithm trained using the laboratory system, it was important to minimize any spectral variations resulting from different illumination and detection configurations.^{15} Thus, the illumination and detection configurations of the laboratory system were mimicked in the prototype system. In particular, a telecentric lens with coaxial illumination (magnification of $0.3\times $, Schott Moritex, Japan) allowed us to image the light scattered from the sample in the exact backward direction with respect to the incident light, acting as back-directional angular gating in the reflection mode.^{17} A white-light LED (${0.315}^{\prime \prime}$ LED Spot Light, Edmund Optics, Barrington, New Jersey) was coupled to the telecentric lens via a fiber-optic light guide and was illuminated onto the plant sample. The light reflected from the sample was collected using the same telecentric lens mounted with a three-color CCD (Color Grasshopper3, Point Grey Research, Richmond, British Columbia). The system had a field of view of $70\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{mm}\times 60\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{mm}$ with a pixel size of $56\text{\hspace{0.17em}\hspace{0.17em}}\mu \mathrm{m}$. Similarly to the hyperspectral measurements, an RGB intensity matrix was normalized by a reference signal from the reflectance reference standard to compensate for the spectral and spatial responses of the entire system, including the light source, the fiber-optic light guide, the telecentric lens, and the CCD camera. Thus, it should also be noted that this resultant intensity matrix was relatively independent of the spectral responses of the light source and the camera.

## 2.4.

### Reconstruction of Hyperspectral Image Data from RGB Data

The relationship between a full reflectance spectrum and an RGB reflectance spectrum in each $(x,y)$ position can be modeled such that^{8}^{–}^{10}

^{18}

^{,}

^{19}Then, the problem to reconstruct a full spectrum from an RGB spectrum is to solve the following question: “Would it be possible to compute or estimate $r$ given $S$ and $x$ are known?” In fact, Eq. (2) can be considered as a system of linear equation. However, since the dimension of $S$ is a $3\times N$, an infinite number of solutions of $r$ can exist to satisfy Eq. (2). Fortunately, several methods and techniques for reconstructing full spectra from RGB data have been studied extensively, such as Wiener estimation

^{8}

^{–}

^{10}

^{,}

^{20}

^{–}

^{22}and regression methods.

^{23}

^{–}

^{25}

Suppose that a reflectance spectrum $r$ and a corresponding RGB spectrum $x$ are collected for each sample. Then, ${\widehat{R}}_{m\times N}$ and ${X}_{m\times 3}$ can be formed by stacking the spectra from several independent samples, where $m$ is the number of different samples and $N$ is the number of wavelengths. This reconstruction problem is to learn a conversion matrix $T$ from a training set such that

$T$ can be solved using a method of least squares that minimizes differences between the original spectra and the reconstructed spectra. Once a conversion matrix $T$ is built from a training set, a reflectance spectrum $\widehat{r}$ can be predicted given a new RGB spectrum $x$ as a testing set. In particular, the prediction ability for full spectral construction can be improved with polynomial regression.^{23}

^{,}

^{24}In our case, a possibly nonlinear relationship between full and RGB spectra can be captured by extending $x=({x}_{1},{x}_{2},{x}_{3})$ to $x=(1,{x}_{1},{x}_{2},{x}_{3},{x}_{1}^{2},{x}_{2}^{2},{x}_{3}^{2})$ for polynomials with the second degree.

## 3.

## Results and Discussion

## 3.1.

### Hyperspectral Chlorophyll Imaging

To map out detailed spatial distribution of leaf chlorophyll concentration in the actual unit of $\mu \mathrm{g}/\mathrm{mg}$, we utilized a chlorophyll spectral index and converted the spectral index to an absolute value of chlorophyll concentration. First, for plant chlorophyll content quantification, we exploited the extensively used spectral index (SI) for chlorophyll content^{26}

## Eq. (4)

$$\mathrm{SI}=\frac{r(750\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{nm})-r(445\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{nm})}{r(705\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{nm})-r(445\text{\hspace{0.17em}\hspace{0.17em}}\mathrm{nm})},$$## 3.2.

### Reconstruction of Reflectance Image Data Using RGB Data

We computationally implemented a hyperspectral reconstruction algorithm that can reliably predict detailed spectral information from RGB data. Among several different reconstruction methods, we chose polynomial multivariate regression, described in Sec. 2.4, for reconstructing reflectance spectra for optimal performance. To generate a conversion matrix $T$ in Eq. (3), we imaged 24 *A. thaliana* leaf samples as follows: first, because of the relatively small sample size ($m=24$), we used the entire data set for training. To build a model for reconstructing reflectance spectra using RGB data, an RGB camera response $x$ was expressed in terms of $r$ and $S$ in Eq. (2). We obtained $S$ from the manufacturer (Sony ICX625) that was used in the handheld system [inset in Fig. 4(a)]. Second, we determined a conversion matrix $T$ by applying multivariate polynomial regression in Eq. (3). Finally, we computed a reconstructed reflectance spectrum $\widehat{r}$ for a new RGB spectrum. In this step, to determine the best polynomial degree, $x$ was varied such that $x=(1,{x}_{1},{x}_{2},{x}_{3},{x}_{1}^{2},{x}_{2}^{2},{x}_{3}^{2})$ for second-order polynomial regression, which provided reliable performance in our plant hyperspectral image data.

The overall performance of the reconstruction method was validated utilizing a leave-one-out cross-validation method with all of the plant samples ($m=24$). In this validation, each round of cross-validation used 23 spectra as a training set to reconstruct a full spectrum from the RGB data, and a total of 24 rounds were performed. The accuracy of the reconstructed spectral information was evaluated using goodness-of-fit metrics [i.e., coefficient of determination ${R}^{2}$ and root mean square error (RMSE)]. Given the relatively small sample size, leave-one-out cross-validation was a reasonable validation method for avoiding over-fitting. It should be noted that reconstruction models were built using only the training data set. The representative spectra in Fig. 4(a) show that a full reflectance spectrum can be reliably reconstructed from the RGB data, compared to the original hyperspectral data. To estimate errors of the hyperspectral reconstruction method, we further analyzed differences between the original reflectance spectra and the estimated spectra in the entire wavelength range. Figure 4(b) shows 95% confidence intervals for mean differences between the original reflectance spectra and the estimated spectra. As expected, the wavelength regions around 400 nm and above 700 nm have wider confidence intervals because of the limited spectral range of the RGB channel of the camera, as shown in the inset of Fig. 4(a). After including all of the samples with second-order polynomial multivariate regression, the average RMSE of cross-validation from 24 samples was 0.013 ($\mathrm{min}=0.001$ and $\mathrm{max}=0.030$) and the average adjusted ${R}^{2}$ was 0.99 ($\mathrm{min}=0.966$ and $\mathrm{max}=0.999$), as shown in the inset of Fig. 4(b). This numerical evaluation supports the feasibility that chlorophyll content can be sensitively and accurately predicted using the three-color sensor-based system.

## 3.3.

### Spectrometerless Chlorophyll Imaging

As a pilot test, we acquired hyperspectral image data and RGB image data from *Arabidopsis* samples in a sequential manner. After the samples were imaged using the original laboratory imaging system (with the imaging spectrograph), the identical samples were also imaged using the handheld prototype system with the three-color CCD. Figures 5(a) and 5(b) depict representative chlorophyll images from the original hyperspectral imaging system and the prototype spectrometerless system, respectively. We compared the spatial distribution patterns of chlorophyll content $I{(x,y)}^{\text{hyperspectral}}$ and $I{(x,y)}^{\text{spectrometerless}}$ obtained from the laboratory hyperspectral system and the spectrometerless system. Specifically, we calculated a two-dimensional (2-D) correlation coefficient ${C}_{2\text{-}\mathrm{D}}$, which is defined as

## Eq. (5)

$${C}_{2\text{-}\mathrm{D}}=\frac{\sum _{x}\sum _{y}({I}^{\text{hyperspectral}}-\overline{{I}^{\text{hyperspectral}}})({I}^{\text{spectrometerless}}-\overline{{I}^{\text{spectrometerless}}})}{\sqrt{\left[\sum _{x}\sum _{y}{({I}^{\text{hyperspectral}}-\overline{{I}^{\text{hyperspectral}}})}^{2}\right]\left[\sum _{x}\sum _{y}{({I}^{\mathrm{spectrometerless}}-\overline{{I}^{\mathrm{spectrometerless}}})}^{2}\right]}}.$$## 4.

## Conclusion

This study demonstrates that the proposed spectrometerless imaging method is capable of nondestructive and quantitative *in situ* imaging of chlorophyll content. A reliable spectrometerless (or spectrometer-free) hyperspectral imaging system was developed using a commonly used three-color CCD (or CMOS) camera. Detailed spatial distribution of chlorophyll content in a whole leaf was successfully visualized with a high correlation between the reflectance spectra pattern and the chlorophyll content. Although our hyperspectral reconstruction is currently applicable for *Arabidopsis* samples, due to the *Arabidopsis*-based model training, future systematic studies will allow us to image other type of plants and crops as well as various pigments besides chlorophyll (e.g., anthocyanin and carotenoid). This capability of imaging multiple stress traits would help to deepen our understanding of stress and coping mechanisms in plants. Further, this prototype could potentially serve as a platform for easy integration into an imaging instrument of modest price, potentially leading to widespread uses.

## References

*In situ*measurement of leaf chlorophyll concentration: analysis of the optical/absolute relationship,” Plant Cell Environ., 37 (11), 2508 –2520 (2014). http://dx.doi.org/10.1111/pce.12324 PLCEDV 1365-3040 Google Scholar

## Biography

**Taehoon Kim** is a PhD student in the Weldon School of Biomedical Engineering, Purdue University, West Lafayette, Indiana, USA.

**Jeong-Im Kim** is a research scientist in the Department of Biochemistry, Purdue University, West Lafayette, Indiana, USA.

**Michelle A. Visbal Onufrak** received her BS and MS degrees in electrical engineering from the University of Puerto Rico, Mayaguez, Puerto Rico, USA. She is currently pursuing her PhD in biomedical engineering, Purdue University, West Lafayette, Indiana, USA. Her current research project consists in developing noninvasive optical imaging devices for clinical assessment of precancerous lesions in collaboration with Indiana University School of Medicine.

**Clint Chapple** is a distinguished professor of biochemistry, Purdue University, West Lafayette, Indiana, USA. His areas of expertise include biochemistry and molecular biology of plant secondary metabolism.

**Young L. Kim** received his PhD degree in biomedical engineering and his MS degree in clinical investigation from Northwestern University in 2005 and 2007, respectively. He is currently an associate professor in the Weldon School of Biomedical Engineering and a codirector of the Biomedical Sciences Interdisciplinary Graduate Program at Purdue University, West Lafayette, Indiana, USA. His research interests include light-matter (e.g., natural materials and biological tissue) interactions, biophotonics technology development for sensing, imaging, and treatment, and biostatistical analyses.