Our goal is to use multiple spectroscopy methods in a single system and develop novel multimode spectroscopic data fusion techniques for fish species identification in real-time. We collected spectral signatures of fish fillets from six fish species using four hyperspectral imaging systems: (1) Reflectance spectral imaging in the visible and NIR (VIS-NIR), (2) Reflectance spectral imaging in the short wave infrared (SWIR), (3) Fluorescence visible spectral imaging with UVA and violet excitation, (4) Raman imaging with a 785 nm laser excitation. All fish fillet samples were confirmed by DNA testing. We built multiple classification/ dimension reduction combination methods to calculate the average sensitivity and associated variance for each class and each spectroscopy mode. In our prototype, the derived statistics are used to form policies for Monte Carlo prediction reinforcement learning. We compared the results of our weighted fusion decisions against individual spectroscopy mode decisions to show an overall sensitivity improvement. We believe this is the first reported use of reinforcement learning applied to multimode spectroscopy data classification in food fraud applications.
This study developed multimode hyperspectral imaging techniques to detect substitution and mislabeling of fish fillets. Line-scan hyperspectral images were collected from fish fillets in four modes, including reflectance in visible and nearinfrared (VNIR) region, fluorescence by 365 nm UV excitation, reflectance in short-wave infrared (SWIR) region, and Raman by 785 nm laser excitation. Fish fillets of six species (i.e., red snapper, vermilion snapper, Malabar snapper, summer flounder, white bass, and tilapia) were used for species differentiation and frozen-thawed red snapper fillets were used for freshness evaluation. A total of 24 machine learning classifiers were used for fish species and freshness classifications using four types of spectral data in three different subsets (i.e., full spectra, first ten components of principal component analysis, and bands selected by a sequential feature selection method). The highest accuracies were achieved at 100% using full VNIR reflectance spectra for the species classification and 99.9% using full SWIR reflectance spectra for the freshness classification. The VNIR reflectance mode gave an overall best performance for both species and freshness inspection.
Our goal is to analyze spectral imaging data using multiple optical imaging instruments available in USDA-ARS and SafetySpect laboratories to provide analysis along three axes of classification of fish fillets: 1. farm-raised vs. wild-caught species; 2. fresh vs. frozen fillets; 3. Species A vs. Species B targeting most mislabeled fish types in the US market. We are collecting spectral signatures using four imaging systems: (1) Reflectance spectral imaging in the visible and NIR (400-1000 nm), (2) Reflectance spectral imaging in the short wave infra-red (SWIR) (1000-2500 nm), (3) Fluorescence spectral imaging with UVA and violet illumination, (4) Raman imaging with a 785 nm laser excitation. The fish fillet samples were purchased from online vendors. We image with each of the modalities and then freeze, thaw and reimage each fillet (2 cycles of freeze/thaw) to demonstrate effect of freeze/thaw process in the multimode spectral signatures. All fish fillet samples are DNA tested to ensure the species marketed are not mislabeled. We use feature extraction/selection strategy for different modes of measurements based on the measurement physics and biological/chemical characteristics. We analyze different combinations of feature extraction and selection techniques and operate an exhaustive search, optimization, and fusion to find out the most important features using different imaging modes. This process helps identify which imaging mode (or combination) will have the highest impact and yield 95%+ classification accuracy. This optimization procedure will be based on cost function (sensitivity, specificity, area under the curve) from receiver operating characteristics (ROC) curve.