Our goal is to use multiple spectroscopy methods in a single system and develop novel multimode spectroscopic data fusion techniques for fish species identification in real-time. We collected spectral signatures of fish fillets from six fish species using four hyperspectral imaging systems: (1) Reflectance spectral imaging in the visible and NIR (VIS-NIR), (2) Reflectance spectral imaging in the short wave infrared (SWIR), (3) Fluorescence visible spectral imaging with UVA and violet excitation, (4) Raman imaging with a 785 nm laser excitation. All fish fillet samples were confirmed by DNA testing. We built multiple classification/ dimension reduction combination methods to calculate the average sensitivity and associated variance for each class and each spectroscopy mode. In our prototype, the derived statistics are used to form policies for Monte Carlo prediction reinforcement learning. We compared the results of our weighted fusion decisions against individual spectroscopy mode decisions to show an overall sensitivity improvement. We believe this is the first reported use of reinforcement learning applied to multimode spectroscopy data classification in food fraud applications.
This study developed multimode hyperspectral imaging techniques to detect substitution and mislabeling of fish fillets. Line-scan hyperspectral images were collected from fish fillets in four modes, including reflectance in visible and nearinfrared (VNIR) region, fluorescence by 365 nm UV excitation, reflectance in short-wave infrared (SWIR) region, and Raman by 785 nm laser excitation. Fish fillets of six species (i.e., red snapper, vermilion snapper, Malabar snapper, summer flounder, white bass, and tilapia) were used for species differentiation and frozen-thawed red snapper fillets were used for freshness evaluation. A total of 24 machine learning classifiers were used for fish species and freshness classifications using four types of spectral data in three different subsets (i.e., full spectra, first ten components of principal component analysis, and bands selected by a sequential feature selection method). The highest accuracies were achieved at 100% using full VNIR reflectance spectra for the species classification and 99.9% using full SWIR reflectance spectra for the freshness classification. The VNIR reflectance mode gave an overall best performance for both species and freshness inspection.
Our goal is to analyze spectral imaging data using multiple optical imaging instruments available in USDA-ARS and SafetySpect laboratories to provide analysis along three axes of classification of fish fillets: 1. farm-raised vs. wild-caught species; 2. fresh vs. frozen fillets; 3. Species A vs. Species B targeting most mislabeled fish types in the US market. We are collecting spectral signatures using four imaging systems: (1) Reflectance spectral imaging in the visible and NIR (400-1000 nm), (2) Reflectance spectral imaging in the short wave infra-red (SWIR) (1000-2500 nm), (3) Fluorescence spectral imaging with UVA and violet illumination, (4) Raman imaging with a 785 nm laser excitation. The fish fillet samples were purchased from online vendors. We image with each of the modalities and then freeze, thaw and reimage each fillet (2 cycles of freeze/thaw) to demonstrate effect of freeze/thaw process in the multimode spectral signatures. All fish fillet samples are DNA tested to ensure the species marketed are not mislabeled. We use feature extraction/selection strategy for different modes of measurements based on the measurement physics and biological/chemical characteristics. We analyze different combinations of feature extraction and selection techniques and operate an exhaustive search, optimization, and fusion to find out the most important features using different imaging modes. This process helps identify which imaging mode (or combination) will have the highest impact and yield 95%+ classification accuracy. This optimization procedure will be based on cost function (sensitivity, specificity, area under the curve) from receiver operating characteristics (ROC) curve.
We present a multimode hyperspectral imaging (HIS) system operating in fluorescence and reflectance modes for food quality and safety applications. The system uses spectral band sequential imaging on the detection side. To ensure constant, high S/N levels, the image acquisition time is optimized for each spectral band. The illumination module uses two independent light sources for fluorescence and reflectance measurements, based on three computer-controlled LED illumination rings. UVA (371 nm, FWHM 16nm, power = 91.7mW/cm²) and blue/violet (418 nm, FWHM = 21nm, power = 38.9mW/cm²) LEDs provide fluorescence excitation. White LEDs (power = 35.8mW/cm²) are used for reflectance. The spectral imaging system incorporated within the detection pathway is able to transition between wavelengths within microseconds over the full bandwidth of the device (450 nm - 800 nm). The system is configured as a tabletop platform with both the illumination and detection located above the food sample. Illumination uniformity is ~90%, spatial resolution is 89μm, and spectral resolution is 8nm.
The system was tested for food safety applications by imaging of pet food spiked with Salmonella enterica, where the ability to identify the bacteria in these samples was compared to existing detection methodologies. As an example of food authentication applications, images captured at 75 wavelengths over the range 450 nm to 810 nm with a 5 nm interval were collected from wild salmon and farmed salmon purchased locally. Wild salmon and farmed salmon were found to have distinctly different reflectance features from 515 nm to 650 nm.