1.
Introduction
This paper is focused on the problem of image quality assessment (IQA) for diatom images. Diatoms are a type of unicellular algae that can be found in water areas, such as rivers and ponds. There are currently about 20,000 different species.1 Experts can use these algae as a water quality indicator by counting the number of species found in a sample and its concentration.2 To perform this, manual taxa identification through a microscope is necessary. Digital imaging technology, which is being incorporated into several areas of biological research, can be very useful in the field of microalgae analysis.3 At this point new problems appear, such as automatic detection and classification of these diatoms,4,5 but the first step is related to the perceived image quality. A good IQA plays an important role in this type of problem due to the large number of species and the small differences between them.
Automatic IQA is still a challenge in computer vision, due mainly to the difficulty of developing algorithms that allow simulation of how people perceive the quality in an image. From the image processing point of view, many image features can be used to evaluate the overall image quality, such as color, contrast, contour, luminance, or texture. There are quality assessment metrics to objectively rate image quality. Initially, these methods (also called objective metrics) were based on mathematical differences between two images, taking a known reference image to make a comparison. However, although this type of metrics is still widely used due to their low complexity and high speed, the obtained results do not correlate well with human perception (i.e., human visual system). For this reason, IQA metrics have been evolving to simulate more accurately the human visual system, leading to perceptual metrics. Unlike metrics based only on mathematical parameters, perceptual metrics give a better correlation with the perceived image quality.6
In addition to these metrics, texture features are used to characterize image quality and represent relevant information. It is difficult to provide a clear definition of visual texture, although it can be related to the spatial distribution of intensity values. So, taking into account this assumption, different kinds of texture can be found, such as homogeneous or not homogeneous, smooth or rough, and fine or coarse. Usually, features which define these types of textures are used in computer vision applications related to classification problems.
In this article, the use of different illumination filters at the image acquisition step is proposed to enhance the image and therefore improve its quality. To accomplish this, some diatom images acquired with these filters have been evaluated using IQA metrics and texture features. Moreover, the same set of images have been evaluated by a group of diatom experts, from the point of view of image resolution, focus, and contrast perceived. In Fig. 1, a diatom sample is shown. The first image, Fig. 1(a), was taken without illumination filter, that is, bright field illumination. The same taxon was acquired in Figs. 1(b)–1(d) but using different illumination filters. The comparison between these illumination filters has been done in this study to analyze their effects in terms of objective image quality and subjective human perception of taxa identification.
Fig. 1
Filter effect. Navicula tripunctata (a) No filter, (b) filter F2, (c) filter F3, and (d) filter F6.
This paper is organized as follows: in Sec. 2 an overview of the state of the art on IQA is presented. Section 3 explains the materials used in this research. The techniques and experiments carried out are described in Sec. 4 and the results obtained are summarized in Sec. 5. Finally, conclusions are given in Sec. 6.
2.
State of the Art
2.1.
Image Quality Assessment
Existing IQA methods can be divided in two approaches, subjective assessment and objective assessment. In the first one, a pool of human observers evaluate the quality of a given visual content. Thus, subjective quality methods are based on psychophysical experiments in which the human observers estimate the quality of a group of visual stimuli.7 However, this kind of assessment methods is hard to include in an automatic quality assessment system.8 They are usually used to validate the objective IQA metrics.
The main purpose of objective IQA methods is to give an objective quality value that should be in agreement with subjective human evaluation. Objective image quality metrics can be classified into three main groups, depending on the amount of previous information required:9 full reference (FR-IQA), reduced reference (RR-IQA), and no reference (NR-IQA). In the first case, FR-IQA, a reference image is needed for comparison. FR-IQA metrics offer good performance although a reference image for comparison is not always available, so the application range is limited. Reduced reference methods use information of some features from the reference image, albeit it is not mandatory to have that image. Finally, since the requirement of a reference image (or partial reference information) is a problem in several applications, no-reference metrics (NR-IQA or blind) have been developed, which do not need any information of the reference image,10 that is, these metrics predict the image quality of an image using other information such as the nature of visual human system or the effect of image distortions.11
In this article, NR-IQA metrics together with a psychophysical experiment have been applied to evaluate the improvement of the overall image quality when illumination filters are used. The study was dedicated to diatom images.
2.2.
Textural Features
Image texture information can be useful in tasks such as image segmentation and classification. In image analysis, visual texture is related to the spatial distribution of intensity values (gray tones) and may be described as a pattern that is spatially repeated. To characterize the different types of visual textures, several measures have been studied. First, texture properties were inferred using first-order statistical measures, such as mean, variance, asymmetry, kurtosis or entropy, and among others. These statistical measures are based on the analysis of the image histogram to characterize textural information. However, spatial information is a significant component of textures. For this reason, in 1973 Haralick established a set of 14 textural features based on co-occurrence matrices (“gray-tone spatial-dependence matrices”),12 which added relative spatial information between gray levels in the texture. Co-occurrence matrices are defined by two parameters, the distance (related to the texture size) and orientation among gray levels (0 deg, 45 deg, 90 deg, and 135 deg). Some of the Haralick features, also called second-order statistical measures, are homogeneity, dissimilarity, energy, and correlation. Using these measures, we can characterize the visual texture in an image. For example, the energy or second angular moment has higher values for smoother textures and homogeneity measures are higher when contrast in textures is lower.
In 1978, Tamura proposed a set of computational measures related to six basic textural features.13 For this author, Haralick features “are not obvious visually” and even a random selection of features can give a good accuracy in a classification problem, so he attempted to develop textural features, which are closer to human visual perception. The first of these features is coarseness, which is related to the size or repeating frequency of the texture elements. Bigger elements, or less repeated, are coarser as opposed to fine textures. The second one is contrast, related to gray-level distribution in an image. Sharper images have higher contrast. The third one is directionality, which measures the presence of orientation in the image. The last three are line-likeness, related to the shape of the texture element, regularity of variation of these elements and roughness, as opposed to smooth textures. In Tamura’s work, a comparison with subjective assessments showed that coarseness, contrast, and directionality attained successful results.
Several works in the literature have used visual textural measures of Haralick and Tamura to solve several classification problems based on image texture.14 In this paper, for the complete dataset, some Haralick and Tamura features have been calculated to analyze the effect of oblique illumination filters in terms of texture properties and compare them with the IQA metrics.
2.3.
Oblique Illumination
Some professional image acquisition equipment, such as robotized microscopes and several illumination techniques, are used in addition to the standard bright field illumination. In the study of living cells or organic matter, due to their transparency, they cannot be observed in a clear way; therefore, several important details are lost. Hence, other illumination techniques are used, such as phase contrast. The increased contrast is realized by modulating the attenuation and phase delay of the unscattered light. Through the phase contrast technique, small refraction index variations are obtained, making these structures visible.15,16 Later, the differential interference contrast (DIC) approach was developed to improve the phase contrast method, solving some disadvantages of this technique (such as the halo effect around the structures) and increasing contrast in transparent specimens.17,18
However, phase contrast illumination techniques are expensive and difficult to apply in portable and low-cost microscopes. For this reason, in this article, the use of simple and low-cost illumination filters is proposed. There are other related works aiming to obtain high-resolution images in a cheaper and easier way. For example, Fourier ptychography uses modulated illumination to collect a set of darkfield images and then reconstructed a wild field image with high resolution.19 For this purpose, a programmable LED array20,21 or an LCD liquid display22 is needed to replace the original illumination unit. These approaches achieve good results in terms of image quality, but they have some disadvantages, such as the substitution of the original illumination source, the need to take and store several images, and the complex retrieval algorithm to obtain the high-resolution image. Oblique illumination filters techniques, such as the proposed one, are easier, cheaper, and it is not necessary to substitute the illumination source.23,24
3.
Materials
3.1.
Image Acquisition Equipment
To perform this study, a set of 3360 images were acquired in TIF format with no compression. The microscope used to take them was a portable and low-cost microscope, the model SP30 from Brunel microscopes, with a objective. The digital camera coupled to the Brunel system was the UI-1240LE-C-HQ model from IDS Imaging Development Systems. The main camera specifications can be found in Table 1.
Table 1
Camera specifications.
| Dimensions | |
| Weight | 41 grams |
| Camera type | Color |
| Resolution | 1.3 Mpx |
| Sensor type | CMOS color |
| Sensor model | e2V EV76C560 ACT |
| Sensor size | 1/1.8″ |
| Pixel size | |
| Mount | C or CS |
| Power consumption | 0.3 to 0.7 W |
| Communication interface | USB 2.0 Mini-B |
Furthermore, the microscope was modified to add more functionality, such as automatic image acquisition. To do this, three stepper motors were installed on microscope axis knobs and an Arduino controller is used to manage the microscope stage motion.
3.2.
Filters
The main motivation of this work is to study the improvement of the overall image quality obtained using illumination filters. For that purpose, a wheel of filters, which includes a set of seven filters (F2, …, F7) with different shapes and a region (F1) with no filter, was designed. The wheel allows us to easily change between the filters. The filter system is shown in Fig. 2. In Fig. 2(a), the three-dimensional (3-D) model of the wheel is displayed. The manufacture process was performed using a 3-D printer, which allows to make filters in an easy and cheap way. To obtain the final design, a selection between several wheels was performed. Finally, after some perceptual tests, a unique wheel was selected to carry out the full study because it produced the most significant image enhancement. The printed wheel is shown in Fig. 2(b).
Fig. 2
Filter system. The labels 1, 2, …, 8 are related to filters F1, F2, …, F8. (a) 3-D model and (b) printed wheel.
In the microscope, the filter wheel is located between the light source and the condenser (see Fig. 3). Figure 3 shows a microscope scheme and a general view of the complete system. On the right part of the figure, the most relevant parts of an optical microscope are pointed out. On the left part of the diagram, it is graphically described how this illumination filter works, changing the light direction to obtain different oblique illumination grades depending on the filter shape.
3.3.
Dataset
In this work, 21 taxa were acquired and assessed. The species selection was performed by a diatom expert, looking at a wide array of morphological and structural features such as shape, size, and striae density, which are key factors for taxa identification. In Appendix A, a set of features of these species are summarized. For each evaluated taxon, 20 fields were captured, and each field was acquired using the eight filters shown before in the filter wheel, taking into account that filter 1 is the unfiltered image (bright field). Therefore, the final set of images was composed of diatom images in total.
4.
Methodology
4.1.
Image Acquisition
Once the species were selected by the diatom expert, the fields were captured with all filters. In Fig. 4, a Cyclostephanos dubius diatom field can be observed with the different illumination filters applied. Figure 5 shows some of the images obtained for Cocconeis placentula euglypta taxon. A quality inspection of these images shows differences in terms of resolution, focus, and contrast. Thus, a quantitative evaluation is performed to define the effect of these illumination filters in terms of image quality.
4.2.
Subjective Evaluation
Once all the 3360 images from the studied taxa were acquired using the filter wheel, three taxonomists from the Diatom Laboratory at Enviromental School in Universidad de León (Spain) carried on a psychophysical test to qualitative evaluate their quality. Three image quality properties were analyzed and rated from 1 to 5 (in which 1 is very bad quality and 5 is very good quality). These properties are: (a) resolution, in terms of capability to identify the necessary diatom features in the image, (b) focus, to evaluate the blurriness in the image, and (c) contrast.
After that, for each taxon and property, a histogram of the 160 fields evaluated by the three taxonomists was calculated. To define a quality value for each filter, that allows us to compare them, a linear combination of the histogram values weighted with the score values is done. The weight assignment is presented in Table 2. Figure 6 shows an example of the results obtained for the Nitzschia Umbonata taxon when evaluating three properties. For this example, the best filters are: (a) for resolution is F2, followed by F3; (b) for focus is F2, followed by F5; and (c) for contrast is F3, followed by F6. Table 3 shows the histogram values of this example for the best filters and the final value obtained after the linear combination.
Table 2
Weight assigned to each score.
| Score | Weight |
|---|---|
| 5 | 5 |
| 4 | 3 |
| 3 | 2 |
| 2 | 1 |
| 1 | 0 |
Table 3
Histogram values of the Nitzschia Umbonata scores.
| Resolution | Focus | Contrast | ||||
|---|---|---|---|---|---|---|
| Score | F2 | F3 | F2 | F5 | F3 | F6 |
| 1 | 0 | 0 | 1 | 0 | 0 | 2 |
| 2 | 6 | 4 | 9 | 8 | 4 | 13 |
| 3 | 17 | 12 | 19 | 22 | 14 | 12 |
| 4 | 20 | 36 | 20 | 20 | 37 | 23 |
| 5 | 17 | 8 | 11 | 10 | 5 | 10 |
| Total | 185 | 176 | 162 | 162 | 168 | 156 |
Fig. 6
Nitzschia Umbonata evaluation. F1 represents the unfiltered image and F2 to F8 are the custom filters designed. (a) Resolution histogram, (b) focus histogram, and (c) contrast histogram.
Thus, the best filters for each taxon were calculated. Then, taking into account the results of the 21 taxa analyzed, a summary graph was generated. The histograms for each filter considering resolution, focus, and contrast perception are given in Fig. 7. That is, the total number of times a filter has been selected as the best, or the second best. Observing these results, it can be seen that, in terms of: (a) resolution, the filters F2, F3, and F6 obtain better IQA than the image without filter. In addition, for the second best case, filters F3, F5, and F6 have also been selected more times than F1; (b) focus, the filters F2 and F3 obtain the best results, and (c) contrast, only four filters were selected as the best or the second best filter (F2, F3, F5, and F6), so that in no case did the results of the unfiltered images improve in terms of contrast. Another significant conclusion is that some filters do not improve perceived image quality in any aspect, such as F4, F7, or F8.
4.3.
Objective Image Quality Metrics
The results of the subjective evaluation show that some illumination filters enhance the image quality in terms of resolution, focus, and contrast perceived. To quantitatively and objectively evaluate the image quality, the same set of images has been assessed using seven no-reference IQA metrics.
4.3.1.
Contrast
Local contrast provides an indicator of focus in an image.25 This metric is based on the absolute difference of each pixel with the eight neighboring pixels. The final score is calculated adding the obtained values for each pixel
The contrast value, , for each pixel, is calculated as described as4.3.2.
Entropy of image histogram
Entropy is a statistical measure of the information contained in an image.26 This quality measure is based on the fact that the histogram of a focused image contains more information than an unfocused one. Entropy can be defined as shown in Eq. (3), where is the probability for each gray level.
Sharper images have a higher number of gray levels, so the entropy will be higher.
4.3.3.
Anisotropy
Anisotropy is measured as the variance of the entropy on several directions.27 As it was previously explained, entropy increases with sharpness but, in situations where images are noisy, there is not a fair correlation. Thus, the anisotropy measure approach is based on the fact that degradation in the image damages the directional information and, for this reason, anisotropy decreases as more distortions are added to the image.
This metric is sensitive to noise or blurriness, so it can be used as a good IQA method.
4.3.4.
Sum of modified Laplace transform
This derivative-based metric28 is based on the Laplacian operator [] to rate the sharpness in an image []. The final score of the metric is calculated by adding the resultant absolute values obtained in the convolution process [see Eq. (4)], where and are the images after convolution with the Laplacian operator
4.3.5.
Tenenbaum gradient
This derivative-based method is one of the first metrics proposed for focus analysis in literature.29 It is based on the Sobel operators to convolve the image (both vertical and horizontal). The final score of the metric is calculated as the sum of the square of the gradient vector components [see Eq. (5)], where and are the resultant images after convolution with the Sobel operators. This metric offers an overall measure of the image quality30
4.3.6.
Blind image quality index
This metric consists of a two step-framework for NR-IQA based on natural scene statistics (NSSs).31 NR-IQA methods usually assume that the distortion affecting the image is known, such as white noise, Gaussian blur, or fast fading. In BIQI metric, the first step of the two-step framework is image distortion classification based on how NSS are modified, and the second step is the quality assessment, using a specific algorithm for the selected distortion.
4.3.7.
Natural image quality evaluator
NIQE is also a blind NR-IQA method based on measurable deviations from statistical regularities observed in natural images (constructing an NSS model). No training with human evaluated distorted images is needed, which means that it is a “completely blind” metric.32 The final score for this metric, which gives a quality measure of the analyzed image, is calculated as a distance between the NSS model statistics and those of the image.
5.
Results
5.1.
Objective Quality Metrics
In Fig. 8, the results of the objective image quality metrics are presented. For each metric, the mean and the standard deviation values of all images for the considerated filter are calculated. The resultant values are normalized within the interval [0, 1]. For all metrics, excluding BIQI and NIQE, a higher value means a better image quality. So, in all cases, the values for the unfiltered image (F1) are the smallest one, and the higher values are obtained, in general, by filters F3 and F6. For the last two metrics analyzed, that is, BIQI and NIQE, the results are similar but presented in a different way. In these cases, lower values mean better image quality and the unfiltered images have the higher scores; therefore, images acquired using an illumination filter present a better image quality.
Fig. 8
Objective quality metrics scores. F1 represents the unfiltered image and F2 to F8 are the custom filters designed. Red squares and yellow dots indicate the best and worst results, respectively. (a) Contrast score, (b) entropy score, (c) anisotropy score, (d) SML score, (e) TG score, (f) BIQI score, and (g) NIQE score.
5.2.
Textural Features
To study the effect of using these oblique illumination filters in terms of texture properties, some Haralick statistical measures12 and Tamura features13 were calculated. The results for Haralick contrast, Haralick energy, Haralick homogeneity, and Tamura coarseness are given here (see Fig. 9). Each feature score is normalized within the interval [0, 1] and represents the mean value for all the analyzed species for each filter.
Fig. 9
Textural features scores. F1 represents the unfiltered image and F2 to F8 are the custom filters designed. Red squares and yellow dots indicate the best and worst results, respectively. (a) Haralick contrast score, (b) Haralick homogeneity score, (c) Haralick energy score, and (d) Tamura coarseness score.
In the case of contrast F1 [Fig. 9(a)], the unfiltered image has the lowest value, so the use of illumination filters increases the overall contrast in an image and, in general, higher contrast is related to sharper images. Conversely, Haralick homogeneity results [Fig. 9(b)] reveal that the unfiltered image has the highest homogeneity score. This measure is related to the previous one because the homogeneity in an image is higher when contrast is lower. The next two measures, such as Haralick energy [Fig. 9(c)] and Tamura coarseness [Fig. 9(d)], are related to smooth and coarse textures, respectively. In the first case, F1 has the highest value; therefore, the image is smoother than those which have been acquired using an illumination filter. In the last case, F1 has the lowest score, which means that filtered images are coarser.
We can use these textural measures to relate visual texture properties with image quality. In this way, comparing with the objective image quality metrics, an image with high contrast and coarser texture elements will have a better quality than others with lower contrast and finer texture elements.
5.3.
Examples
Analyzing the results of IQA metrics and textural measures, the best filters are F3 and F6 since in most cases the best scores were obtained by them. Two diatoms fields are presented to visually compare them in Figs. 10 and 11. The differences in terms of contrast and resolution can be observed in both examples, where the F1 (no filter) and the F3 and F6 images are shown.
6.
Conclusions
In the present document, different oblique illumination filters have been tested to analyze if the perceived image quality increases compared to a standard bright field image. To achieve this, a set of seven filters, with different internal shapes, were designed and manufactured using a 3-D printer. Twenty samples of 21 diatom species were acquired and evaluated with the seven illumination filters in addition to the unfiltered image. The complete dataset (3360 images) was evaluated by a group of three diatomist in terms of image resolution, focus, and contrast. The results obtained indicate that oblique illumination filters enhance the perceived resolution in an image and the best filters for these aspects are F2, F3, and F6. However, for focus, the scores obtained for the unfiltered image are very similar to the best ones (F2 and F3). This fact can be explained because these types of illumination filters add a “shadow” effect to the image edges that are perceived, such as an image distortion. In terms of contrast, the best filters are F2, F3, and F6.
Moreover, the same dataset was evaluated using several no-reference objective IQA metrics. For all of them, the images acquired using the proposed illumination filters achieve better scores than the image acquired without filter. From the seven filters analyzed, F3 and F6 obtain the best marks in terms of overall image quality.
Considering the results of texture analysis, and taking into account the objective quality assessment metrics, we can identify a relation between texture features and image quality. Coarser and higher contrast textures in an image are related to a higher quality, whereas finer, smoother, and lower contrast textures are associated with a lower image quality. In the case of taxa identification, the use of illumination filters increases contrast, roughness, and coarseness of the image, and it is perceived as better image quality.
Hence, based on the results obtained, we can assume that the use of these types of low-cost illumination filters improves the global image quality in diatom analysis.
Appendices
Appendix A
In Figs. 12Fig. 13Fig. 14–15, all diatom classes used in this work are illustrated. The unfiltered (F1) original image is shown together with the filtered image with the highest quality, that is F3, F6, etc. Moreover, some features such as taxon name, average size, and outer shape are indicated.
Disclosures
The authors have no relevant financial interests in this article and no potential conflicts of interest to disclose.
Acknowledgments
The authors acknowledge financial support of the Spanish Government under the Aqualitas-retos project (Ref. CTM2014-51907-C2-2-R-MINECO).
References
Biography
Jesus Ruiz-Santaquiteria received his computer engineering degree from the University of Castilla-La Mancha, Spain, in 2016. Currently, he is studying a computer engineering master at the same university and working as a research assistant of the VISILAB group. His current research interests are artificial intelligence, computer vision, image processing, and automation in microscopy.
Jose Luis Espinosa-Aranda received his computer engineering degree and his PhD in computer science from the University of Castilla-La Mancha, Spain, in 2009 and 2014, respectively. Currently, he is a research assistant of the VISILAB group and an associate professor on the University of Castilla-La Mancha, Spain. His current research interests include artificial intelligence and computer vision.
Oscar Deniz is a associate professor at UCLM. His research interests are mainly focused on computer vision and pattern recognition. He is the author of more than 50 refereed papers in journals and conferences. He is a senior member of IEEE. He is the coordinator of European H220 Project “Eyes of Things,” and participates in FP7 AIDPATH and H2020 BONSEYES projects. Reviewer/technical expert for EU programs such as Eurostars and Advisory Board Member of H2020 project TULIPP.
Carlos Sanchez received his electrical engineering degree from the Universidad Autónoma de Madrid, Spain, in 2014. Currently, he is pursuing his PhD degree in industrial engineering at Universidad de Castilla La Mancha. His thesis is being developed at Instituto de Óptica (CSIC) under the supervision of Gabriel Cristobal. His current research interest is microscopy image processing.
Maria Borrego-Ramos is a marine biologist, specialized in taxonomy and systematic of benthic diatoms, and the assessment of water quality using diatom-based indices. She has coauthored eight papers in ISI-indexed international journals. She is interested in management and conservation of water bodies.
Saul Blanco is specialized in taxonomy, ecology, and biogeography of freshwater diatoms, having led several studies about the assessment of water quality by means of diatom-based indices. In the field of systematics and biodiversity, he has discovered or codiscovered more than 20 algal species new to science in different regions of the world. He has authored or coauthored eight books and two book chapters. He has published ca. 100 papers in ISI-indexed international journals (h = 22).
Gabriel Cristobal is currently a research scientist at the Instituto de Optica (CSIC). He was a postdoctoral fellow at the International Computer Science Institute and the Electronic Research Lab (UC Berkeley) from 1989 to 1992. His current research interests are joint representations, vision modeling, and image processing in microscopy. He has been responsible for several national and EU research and development projects. He was a coeditor of the books “Optical and Digital Image Processing” and “Biologically Inspired Computer Vision.”
Gloria Bueno is a lecturer and principal researcher at the School of Engineering in UCLM at Ciudad Real, Spain, since 2002, where she is currently conducted VISILAB group. She holds a PhD in machine vision obtained at Coventry University in 1998. She has carried on her research activities at different research centers, such as Centro de Estudios e Investigaciones Técnicas de Guipuzkoa, Centre National de la Recherche Scientifique, Hôpitaux Civil and Telecommunication School, Univ. Louis Pasteur, and Gilbert Gilkes and Gordon Technology. She is leading different national and European research project in biomedical image processing. Her current interests are in signal and image processing, modeling, and artificial intelligence.
