Lawn plant identification and segmentation based on least squares support vector machine and multifeature fusion

Yinhai She; Xinyan Wang; Tina Gui; Jinkang Cai

doi:10.1117/1.JEI.28.2.023034

23 April 2019 Lawn plant identification and segmentation based on least squares support vector machine and multifeature fusion

Yinhai She, Xinyan Wang, Tina Gui, Jinkang Cai

Author Affiliations +

Journal of Electronic Imaging, Vol. 28, Issue 2, 023034 (April 2019). https://doi.org/10.1117/1.JEI.28.2.023034

Abstract

Different turfs have different growth characteristics, engendering differences in the number of maintenance cycles and amounts of pesticides used; therefore, studying their subtle color and shape differences through image recognition is crucial. Our study proposes an improved least squares support vector machine (LS-SVM) pixel classification method for this purpose. The sensitivity to local color changes in the hue, saturation, and value color space is considered, and the Sobel operator is used to extract the homogeneity as pixel-level color features. The maximum local energy, gradient, and second-order moment matrix of image pixels are obtained as texture features using a Gabor filter. Seven shape features of different plant leaves are calculated, multiple extracted features are used as LS-SVM classifier inputs, and samples are selected and trained with a dynamic threshold. The trained classifier can be used for segmentation. The experiments showed that it could use the local information of the color images and the excellent generalization ability of LS-SVM to segment lawn plants effectively. Under different weather conditions, the penalty coefficient, C, and kernel parameters with optimal generalization were Bayesian optimized to obtain a segmentation rate exceeding 95%. This algorithm yields a higher classification rate for plants with less obvious differences in texture and shape and optimizes space and time complexities.

1. Introduction

Lawn plants are classified based on corresponding standards to enable green units to formulate reasonable plans and select suitable lawn grass species. This can help maintenance staff and promote the construction of modern ornamental gardens and sport lawns. Different turf species have dissimilar growth characteristics, resulting in different stubble heights and thus different maintenance and pruning cycles; therefore, the plant species must be classified. This study focused on tall fescue, ryegrass, bluegrass, carpet grass, and other common lawn grasses and used image recognition technology to classify them. After lawn types are identified, the theoretical basis for setting the height of a lawnmower can be determined according to the respective growth characteristics of the lawns to realize scientific and efficient mowing operations.

In plant identification processes, flowers and fruits may last for only a few weeks,¹^–³ whereas plant leaves remain nearly constant throughout the year.⁴^,⁵ Furthermore, leaf shape is one of the most important visual features for describing many plants; therefore, classification based on leaf characteristics is crucial. Common classification techniques include the k-nearest neighbor (KNN), probabilistic neural network (PNN), and support vector machine (SVM) classifiers; however, these techniques have obvious drawbacks. For example, the KNN technique is expensive for testing every instance,⁶^,⁷ sensitive to noise, and provides irrelevant inputs. Moreover, a PNN has a large network structure and an excessive number of attributes.⁸^,⁹ An SVM classifier cannot easily be trained on large-scale training samples and can only classify them into two categories.¹⁰ This study addresses these shortcomings by effectively classifying leaves using a least squares SVM (LS-SVM) classifier.¹¹

Ingrouille and Laird¹² classified oak species with 27 leaf shape features using the principal composition analysis method. Sixta¹³ used the internal distance of shape context for leaf recognition. Rossatto et al.¹⁴ used the volume fractal dimension and naive Bayesian classification for leaf image recognition. Mallah et al.¹⁵ described a method for improving the recognition rate with a small training set and incomplete feature extraction. They used a K-nearest value classifier combined with a feature vector and density estimation method to improve the recognition rate. The recognition rate reached 96% when three features were combined, whereas it was 91% when four features were used. Du et al.¹⁶ extracted plant leaf shape features and image-invariant moments, and they used a mobile center hypersphere classifier to recognize more than 20 plants. Wu et al.³ orthogonalized the shape and texture features and increased the number of recognition categories to more than 30 using a PNN. Priya et al.¹⁷ used the same feature dataset as Ref. 3 to improve the recognition rate using an SVM. Elhariri et al.¹⁸ combined the color features on the basis of the shape and texture features to recognize more than 30 types.

Most lawn leaves have different shapes, rich colors, and textures. Therefore, these are the main characteristics used to distinguish plant species.¹⁹ Most of the leaves identified in articles, such as Refs. 3 and 12 13.14.15.16.17.–18, are relatively flat and broad. However, the objects of leaf recognition in this paper are all slender and clustered lawn plants, and the background information is relatively more complex. Therefore, the study in this paper is conducive to the development of intelligent lawn mowers.

The main contributions of this study are as follows: (1) based on the LS-SVM segmentation algorithm, an algorithm was developed for effectively identifying and segmenting turf plant images; (2) common turf plants were classified using the improved algorithms; and (3) the accuracy and effectiveness of the turf identification were evaluated qualitatively for different weather conditions.

2. Lawn Recognition Algorithm

Lawn images may contain random noise due to the influence of light, uneven exposure, camera distortion, or other types of interference. To suppress the background interference and random noise and to highlight the object region of the image in the identification process, the image can be segmented using an LS-SVM method with a dynamic threshold. When a dynamic threshold is used for selecting the training sample, a fast, stable, and reasonable training sample can be obtained. When images are identified using the LS-SVM method, the equality constraint in the standard SVM is changed to an inequality constraint, and the quadratic programming problem is transformed into a problem of linear equations. Therefore, the computational complexity is reduced considerably, and the computational speed becomes higher than that of a general SVM. Figure 1 shows a detailed description of the specific process steps for categorizing turf plants using an LS-SVM and a dynamic threshold.

Fig. 1

Flowchart of LS-SVM image identification based on a dynamic threshold.

In the experiment conducted in this study, a Daheng Image Company’s Mercury series mer-231-41u3c camera was used to collect lawn images. The camera resolution was determined to be $1920 \times 1200$ , and it was matched with an m0814-mp2 8-mm-focal-length lens.

During the collection of lawn images, the lawn height was 30 to 180 mm. To ensure that the lawn was in the camera shooting range, the distance between the camera bracket and the lawn level was set at 600 mm, and the camera center was 90 mm from the ground. A total of 240 images of four common lawns, tall fescue, perennial ryegrass, Kentucky bluegrass, and carpet grass, were collected under sunny and cloudy conditions, as shown in Fig. 2.

Fig. 2

Four kinds of lawn images collected: (a) Kentucky bluegrass, (b) perennial ryegrass, (c) tall fescue, and (d) carpet grass.

2.1.

Dynamic Threshold

Due to shadows, different background contrast levels, burst noise, and background gray-level changes, the lawn images cannot be segmented effectively using traditional methods, such as those involving a global fixed threshold²⁰^,²¹ or an Otsu threshold.²² Therefore, a set of dynamic thresholds related to pixel position can be used for local dynamic threshold processing in a local neighborhood. The basic steps of dynamic threshold segmentation are as follows:

1. The whole image is segmented into subimages with a 50% overlap, and the sizes are determined by the average filter. The mean filter determines the size of the subimages, mainly by setting the size of the template window, for instance as a $3 \times 3$ matrix.
2. Determine a histogram for each subimage.
3. Detect whether the histogram of each subimage is bimodal, and if so, interpolate to obtain the threshold, $g_{0}$ , of all the subimages, otherwise no processing is performed.
4. Interpolate the threshold values of the subimages to obtain the threshold value, $g_{t}$ , for the whole image. Here, $f$ is set as the customized pixel value offset, which refers to the size of the mask. Its optimal range is [5, 40].
5. The brighter part of the image is assumed to be the grass sample. In each subimage, pixels satisfying the condition $g_{0} \geq g_{t} + f$ are regarded as the object; the remaining pixels are considered as the background. If $f$ is excessively small, many small areas containing noise will be extracted. By contrast, if $f$ is excessively large, the region cannot be extracted easily. In this scenario, the optimal value of $f$ can be set as 10 based on training with a large number of templates.

2.2.

Color Spatial Feature Extraction

Compared with red, green, and blue (RGB) color space, hue, saturation, and value (HSV) color space can express the tone and brightness of color more directly. Furthermore, it can separate color and brightness information. The equation²³ for conversion from RGB to HSV is as follows:

Eq. (1)

V = \max (R, G, B),

Eq. (2)

S = {\begin{cases} \frac{v - \min (R, G, B)}{v}, & if v \neq 0 \\ 0, & otherwise \end{cases},

Eq. (3)

H = {\begin{cases} 60 (G - B) / [v - \min (R, G, B)], & if v = R \\ 120 + 60 (B - R) / [v - \min (R, G, B)], & if v = G \\ 240 + 60 (R - G) / [v - \min (R, G, B)], & if v = B \end{cases},

where

(R, G, B)

represent the red, green, and blue coordinates, respectively;

S

(saturation) and

V

(value) lie in the range [0, 1], and

H

(hue) lies in the range [0, 360]. To eliminate the influence of light on color, only pixel-level color features are considered from the color space of

S

. Figure 3 shows the

H

,

S

, and

V

components of an image in the HSV color space.

Fig. 3

$H$ , $S$ , and $V$ components of color images: (a) original image, (b) $H$ component, (c) $S$ component, and (d) $V$ component.

In the proposed algorithm, the pixel-level color features that conform to the human visual system are divided into two parts: discontinuity and standard deviation. Here, $P_{x, y} = (P_{x, y}^{H}, P_{x, y}^{S}, P_{x, y}^{V})$ indicates that the pixel is located at $(x, y)$ with three components in an $M \times N$ size image, and ${CF}_{x, y}$ represents the color features of pixel $P_{x, y}$ .

2.2.1.

Compute the noise discontinuity

Discontinuity refers to the inconsistency of pixel gray values at the junction of different regions; it mainly describes the edge amplitude. The discontinuity of color components, $P_{x, y}^{k} (k = H, S)$ , selected in this study is represented by $c_{x, y}^{k}$ . Sobel, Canny, Derish, Laplacian, and other edge operators were used for the edge computation of the same noisy images. The results show that the performance of the second-order differential Laplacian operator is better than that of other traditional edge operators. However, the images must be enhanced and smoothed before the gradient calculation is performed.

Because determining the exact edge position is not necessary, the discontinuity and gradient can be calculated using the simple Sobel operator:

Eq. (4)

c_{x, y}^{k} = \sqrt{G_{x^{'}}^{k^{2}} + G_{y^{'}}^{k^{2}}} (k = H, S),

where

G_{x^{'}}^{k^{2}}

and

G_{y^{'}}^{k^{2}}

are the components of the gradient in the

x^{'}

and

y^{'}

directions, respectively.

2.2.2.

Compute the standard deviation

By assuming that the signal is ergodic, the standard deviation, $υ_{x, y}^{k}$ , describes the intensity variation within a local image window and is calculated for a pixel component, $P_{x, y}^{k} (k = H, S)$ , as follows:

Eq. (5)

υ_{x, y}^{k} = \sqrt{\frac{1}{d^{2}} \sum_{m, n \in Ω_{x, y}} {(P_{m, n}^{k} - μ_{x, y}^{k})}^{2}},

where

x \geq 2

,

m \leq M - 1

,

y \geq 2

,

n \leq N - 1

,

μ_{x, y}^{k}

is the mean value of the color component,

P_{x, y}^{k} (k = H, S)

, in local window,

Ω_{x, y}

and is defined as

Eq. (6)

μ_{x, y}^{k} = \frac{1}{d^{2}} \sum_{m, n \in Ω_{x, y}} P_{m, n}^{k},

where

Ω_{x, y}

is a local window of

d \times d

whose center is pixel

(x, y)

, and

d

is an odd number

> 1

.

2.2.3.

Compute the pixel-level color features

Herein, the local homogeneity of image pixels is defined by pixel-level color features. Homogeneity is closely related to the local information extracted from an image, and it reflects the consistency of the color features in the local region. However, the aim of image segmentation is to divide the image into several homogeneous regions; therefore, the local homogeneity, as a color feature for regional segmentation, plays a major role in the process.

The local homogeneity consists of the standard deviation and discontinuity of the color component, $P_{x, y}^{k} (k = H, S)$ , and it is represented by

Eq. (7)

h (P_{x, y}^{k}) = 1 - E (P_{x, y}^{k}) \times V (P_{x, y}^{k}),

where

Eq. (8)

E (P_{x, y}^{k}) = \frac{υ_{x, y}^{k}}{\max {υ_{x, y}^{k}}} a n d

Eq. (9)

V (P_{x, y}^{k}) = \frac{c_{x, y}^{k}}{\max {c_{x, y}^{k}}} .

Here,

υ_{x, y}^{k}

and

c_{x, y}^{k}

represent the standard deviation and discontinuity, respectively, of the pixel color component,

P_{x, y}^{k} (k = H, S)

, located at

(x, y)

.

Finally, the pixel, $P_{x, y}$ , with ${CF}_{x, y} = [h (P_{x, y}^{H}), h (P_{x, y}^{S})]$ can be obtained at location $(x, y)$ .

Figure 4 shows the pixel-level color characteristics of the $H$ and $S$ components.

Fig. 4

Pixel color feature matrix of the (a) $H$ and (b) $S$ components.

2.3.

Texture Feature Extraction

Texture is usually combined with the color features and applied to image segmentation. In this algorithm, the local energy, local gradient, and local second moment are extracted from the six directional subbands of the Gabor filter as pixel-level texture features. The Gabor filter can not only reduce the influence of illumination and noise but it also preserves the edge information of texture in different scales and directions through its good bandpass and directional selectivity. Figure 5 shows the six directional subbands of the Gabor filter.

Fig. 5

Gabor filter’s six directional subbands: (a) 0 deg, (b) 30 deg, (c) 60 deg, (d) 90 deg, (e) 120 deg, and (f) 150 deg.

2.3.1.

Selection of the color space

The $S$ component of the HSV color space is selected to represent the texture because it closely matches the human perception of lightness, and this color space can control the color and brightness information independently.

2.3.2.

Application of the Gabor filter to the S component

In this study, a Gabor filter with six orientations and two scale bands was used to decompose the $S$ component. Khan et al.’s²⁴ research shows that 4 to 6 orientation subbands could approximate the directional selectivity of the human visual system, and two-level decomposition could be selected for turf images. Figure 6 shows the Gabor filter’s six directional and two scale subbands.

Fig. 6

Gabor filter subband images in six directions and two scales: (a) 2 scale, direction is 0; (b) 2 scale, direction is 30; (c) 2 scale, direction is 60; (d) 2 scale, direction is 90; (e) 2 scale, direction is 120; and (f) 2 scale, the direction is 150.

Fig. 7

Image of each pixel-level texture feature: (a) local energy, (b) local gradient, and (c) local second moment.

2.3.3.

Extraction of the local energy

The local energy of the $S$ component can be calculated²⁴ as follows:

Eq. (10)

S E^{S} (x, y) = {[S (x, y) * G_{m, θ, ϕ, f}^{0} (x, y)]}^{2} + {[S (x, y) * G_{m, θ, ϕ, f}^{e} (x, y)]}^{2},

where * denotes the convolution, and

G_{m, θ, ϕ, f}^{0} (x, y)

and

G_{m, θ, ϕ, f}^{e} (x, y)

denote a pair of parity symmetric filters that, respectively, represent the real and imaginary parts of the Gabor filter.

Here, $S E_{m, n}^{S} (x, y)$ represents the Gabor subband coefficients at location $(x, y)$ that corresponds to the $m (m = 1.0, 2.0)$ scale and $n (n = 0 \deg, 30 \deg, 60 \deg, 90 \deg, 120 \deg, 150 \deg)$ orientation. The local energy, $E_{x, y}$ , is defined as the maximum (in absolute value) of the 12 coefficients at location $(x, y)$ , which is one-pixel texture feature at location $(x, y)$ :

Eq. (11)

E_{x, y} = \max_{m, n} {| S E_{m, n}^{S} (x, y) |} .

2.3.4.

Extraction of the local gradient

The gradient is used to measure the change in pixel value in the $x$ and $y$ directions, which is an important measure of image features. If a region is smooth, its gradient is small. In this algorithm, $G_{x, y}$ is used to represent the maximum value of the 12 gradient magnitudes at location $(x, y)$ , which is another pixel texture feature at location $(x, y)$ :

Eq. (12)

{\begin{cases} G_{x, y}^{m, n} = \sqrt{G_{x^{'}}^{m, n^{2}} + G_{y^{'}}^{m, n^{2}}} \\ G_{x, y} = \max_{m, n} {| G_{x, y}^{m, n} |} \end{cases} .

2.3.5.

Extraction of the local second moment matrix

To describe the texture of lawn leaves, a texture feature based on the local second-order moments can be used, which can be considered as the covariance matrices of two-dimensional random variables. The energy values in the two main directions in the neighborhood are represented by their eigenvalues. When one eigenvalue is greater than the others, the local neighborhood represents the texture and dominates the orientation. The second moment, $M_{x, y}^{m, n}$ , of the Gabor filter subband coefficient, $G_{m, n} (x, y)$ , is expressed as follows:

Eq. (13)

M_{x, y}^{m, n} = G_{σ} (x, y) * (\nabla I) {(\nabla I)}^{T},

Eq. (14)

(\nabla I) {(\nabla I)}^{T} = (\begin{matrix} G_{x}^{m, n^{2}} & G_{x}^{m, n} G_{y}^{m, n} \\ G_{y}^{m, n} G_{x}^{m, n} & G_{y}^{m, n^{2}} \end{matrix}) .

Here, * represents the convolution,

\nabla I

represents the gradient,

G_{x}^{m, n}

and

G_{y}^{m, n}

, respectively, represent the components of the gradient in the

x

and

y

directions, and

G_{σ} (x, y)

is a separable binomial approximation to a Gaussian smoothing kernel with a variance

σ^{2}

:

Eq. (15)

G_{σ} (x, y) = \frac{1}{2 π σ^{2}} e^{- (x^{2} + y^{2}) / 2 σ^{2}} .

Here, $M_{x, y}^{m, n}$ is a symmetric semipositive definite matrix; and $λ_{2} (λ_{1} > λ_{2})$ are defined as the eigenvalues of $M_{x, y}^{m, n}$ ; and $ϕ$ is the main eigenvector of $M_{x, y}^{m, n}$ . When $λ_{1}$ and $λ_{2}$ are negligible, the local neighborhood is approximated by a constant, representing a nontextured area. In contrast, a large value represents a textured area.

In the proposed algorithm, the sum of the eigenvalues $λ_{1}$ and $λ_{2}$ of the second moment matrix, $M_{x, y}^{m, n}$ , is defined as the pixel-level feature, $V_{x, y}^{m, n}$ . The maximum value of the sum of 12 eigenvalues is represented by $V_{x, y}$ , which is the third pixel texture feature:

Eq. (16)

V x, y = \max_{m, n} {| V_{x, y}^{m, n} |} .

Finally, the texture feature, ${TF}_{x, y}$ , of the image pixel, $P_{x, y}$ , is obtained at location $(x, y)$ :

Eq. (17)

{TF}_{x, y} = (E_{x, y}, G_{x, y}, V_{x, y}) .

The image of each pixel-level texture feature is shown in Fig. 7.

2.4.

Shape Feature Extraction

In addition to the color and texture characteristics, shape features constitute another typical characteristic of lawn plants. Therefore, shape features can be added to the classification and recognition processes. In this paper, the method presented in Ref. 25 is used to extract the geometric parameters of four plants by the frequently used Fourier harmonic function. The shapes of different lawn plants have obvious differences, which can be adequately described by their rectangularity, aspect ratio, roundness, and sphericity.

The aspect ratio is defined as the ratio of the length to the width of the rectangular blade.

Rectangularity is defined as the ratio of the blade area to the minimum outer rectangle, which reflects the degree to which the object fills the minimum external rectangle and lies in the range (0, 1).

Circularity is defined as the ratio of $4 π$ times the blade area to the square of the circumference, $l$ ; it reflects the degree of compact correlation between the blade and the circumferential circle.

Sphericity is the ratio between the blade area and the circumference of the smallest circumscribed rectangle.

Eccentricity is defined as the ratio between the long axis and the short axis of the blade.

Lobation is defined as the ratio of the shortest distance between the center of gravity of the blade area and the boundary to the short axis of the blade; it can reflect the amplitude characteristics of the blade boundary.

The circumference-to-diameter ratio is defined as the ratio of the blade circumference to the long axis (Table 1).

Table 1

The lawn shape characteristic formulas.

Shape feature	Formula
Leaf aspect ratio	$K = \frac{a_{R}}{b_{R}}$
Rectangularity	$R = \frac{A_{0}}{a_{R} b_{R}}$
Circularity	$D = \frac{4 π A_{0}}{l^{2}}$
Sphericity	$F = \frac{A_{0}}{2 * (a_{R} + b_{R})}$
Eccentricity	$E = \frac{a}{b}$
Lobation	$L = \frac{l_{\min}}{b}$
Circumference-to-diameter ratio	$C = \frac{l}{a}$

The geometric parameters of the blade region can be calculated according to the extracted blade profile of the lawn, including the blade area, $A_{0}$ , blade area perimeter, $l$ , length, $a_{R}$ , and width, $b_{R}$ , of the minimum external rectangle of the blade region, the long axis $a$ and short axis $b$ of the blade region, and the shortest distance, $l_{\min}$ , from the center of the blade region to the boundary.

Table 2 shows the shape characteristics of lawn plants.

Table 2

Shape characteristics of lawn plants.

	K	R	D	F	E	L	C
Tall fescue	0.22	0.36	0.47	7.54	7.55	0.46	3.67
Kentucky bluegrass	0.14	0.23	0.45	4.03	7.86	0.12	7.57
Carpet grass	0.67	0.51	062	4.11	2.79	0.38	8.08
Perennial ryegrass	0.16	0.71	0.48	5.29	9.44	0.33	7.44

2.5.

Least Squares Support Vector Machine Identification Model

The classification and recognition of lawn plants is essentially a complex multiclass discrimination problem. The classification of lawn plants is a small sample and multiclass classification problem that can be solved using an SVM classifier. Therefore, a classifier model based on an LS-SVM, an improved version of the traditional SVM method, was developed. This method adopts a least squares linear system as the loss function and solves a set of linear equations instead of the complex quadratic programming problem in the SVM. Moreover, this method has a low computational complexity and has the advantages of good generalization and a fast learning speed.

Assume that the given training set is

Eq. (18)

{(x_{1}, y_{1}), \dots (x_{l}, y_{l})}, (x \in R^{n}, y \in {+ 1, - 1}),

where

x

is the input vector,

l

is the number of samples,

n

is the dimension of the input vector, and

y

is the type of input vector.

The basic idea of the LS-SVM method is to find the smallest hyperplane $‖ ω ‖$ in a separable hyperplane, which is the same as the classical SVM. However, the LS-SVM method gives an $e_{i}$ correction for each data point; therefore, the problem of finding the hyperplane optimization is transformed to a convex optimization problem:

Eq. (19)

\min J_{LS} (ω, b, e) = \frac{1}{2} ω^{T} ω + γ \frac{1}{2} \sum_{i = 1}^{l} e_{i}^{2},

Eq. (20)

y_{i} (ω^{T} ϕ (x_{i}) + b) = 1 - e_{i} (i = 1, \dots l) .

The Lagrangian method is used to solve the mentioned optimization problem, which is transformed into solving a linear equation:

Eq. (21)

[\begin{matrix} 0 & - Y^{T} \\ Y & Z Z^{T} + γ^{- 1} I \end{matrix}] [\begin{matrix} b \\ α \end{matrix}] = [\begin{matrix} 0 \\ \vec{1} \end{matrix}],

where

α

is the Lagrange multiplier,

Z = [ϕ {(x_{1})}^{T} y_{1}, \dots ϕ {(x_{l})}^{T} y_{l}]

,

Y = [y_{1}, \dots y_{l}]

,

\vec{1} = [1, \dots 1]

,

α = [α_{1}, \dots α_{l}]

.

In the LS-SVM method, the supporting value is proportional to the error of the data points, $e_{i}$ , and $γ$ is 1. This method nonlinearly maps input parameters to the high-dimensional feature space and constructs the same optimal decision function as the classical SVM according to the principle of structural risk minimization.

Eq. (22)

f (x) = sign (ω_{0} \circ x + b_{0}) = sign [\sum_{i = 1}^{l} α_{i} y_{i} K (x_{i} \circ x) b_{0}] .

Then, each test sample,

x_{i}

, is substituted into

f (x)

, and the type of test sample can be obtained.

3. Results and Analysis

An experiment was performed using a computer with an Intel(R) i5-2520M 2.5 GHz quad-core processor with 4 GB of memory. The algorithm was developed using VS2012 C# along with the HALCON© 12.0.1 image processing library (MVTec Software GmbH, Germany). The proposed algorithm was evaluated using three evaluation criteria. In Sec. 3.1, the segmentation performance of the fuzzy threshold method, maximum between-group variance method (Otsu), and the proposed algorithm are compared with a complex background. In Sec. 3.2, the radial basis function (RBF) kernel function of the LS-SVM and multifeature fusion are selected as the recognition method through several experimental comparisons. In Sec. 3.3, the stability and accuracy of the algorithm under different illumination conditions are verified.

3.1.

Image Segmentation Evaluation Results

Four images with different complex backgrounds were selected from the collected samples to evaluate the proposed segmentation method. Figure 8(a) shows the original images, and Fig. 8(b) shows the segmented images obtained using the proposed algorithm. To further evaluate the segmentation performance of the proposed method, the performance of the proposed method was compared with that of the fuzzy threshold method and Otsu method for various categories, and the results are presented in Figs. 8(c) and 8(d).

Fig. 8

Segmentation effect comparison of several segmentation methods: (a) original images of four species of lawn ( $a_{1}$ Kentucky bluegrass, $a_{2}$ perennial ryegrass, $a_{3}$ tall fescue, and $a_{4}$ carpet grass); (b) image segmentation obtained by the algorithm in this paper; (c) image segmentation obtained by the Otsu method; and (d) image segmentation obtained by the fuzzy threshold method.

As presented in Fig. 8, all three image segmentation algorithms provided good segmentation performance levels for a single plant with a simple background. For the relatively complex background of the last three images ( $a_{2}$ perennial ryegrass; $a_{3}$ tall fescue; $a_{4}$ carpet grass), the proposed algorithm could ignore the remaining small weed areas and identify the region of interest directly; this capability is consistent with the human visual observation characteristics and demonstrates the strong adaptability of the algorithm to changes in lighting conditions.

3.2.

Discrimination Using Support Vector Machines

This study proposes an LS-SVM-based method for segmenting and recognizing common lawn plants. First, a square area of $3 \times 3 pixels$ serves as the convolution unit, and the pixel color features of the $H$ and $S$ components are selected as two color input features. Second, the local energy, gradient, and second moment are selected as the texture input features. The leaf aspect ratio, rectangularity, circularity, sphericity, eccentricity, lobation, and the circumference-to-diameter ratio of the blade region are then selected as shape input features. Finally, a suitable local threshold is selected based on dynamic threshold processing, and 13 feature vectors are constructed as the input for LS-SVM training samples for the recognition and classification of lawn plant images.

In this study, four kinds of common lawn plants were selected as objects to be identified. A total of 240 images of the four selected plants are available, and every kind of these selected plants can be found in the 60 images. A total of 120 lawn images were selected randomly as the SVM training set, and the others were selected as the test set. To observe the recognition accuracy of the SVM RBF, a linear kernel function, polynomial kernel function, RBF, and multilayer perceptron kernel function (sigmoid) were used for comparison. Table 3 shows the characteristics of the identification test data for the different kernel functions and their comparison.

Table 3

Recognition rate using different kernel functions based on multiple features.

Kernel function	Training recognition rate (%)	Test recognition rate (%)
Linear	96.9	87.8
Polynomial	99.8	91.6
Radial basis	100	99.9
Sigmoid	100	92.1

Table 3 shows that when the RBF was used as the kernel function of the SVM, the classification rate for the training and test sets could reach 99.9% or more. This was the best result among all the kernel functions. Bayesian optimization,²⁶ grid search,²⁷ random search, and other methods have been applied by many researchers to determine the values of the penalty coefficient, $C$ , and the kernel parameter $σ$ . After a certain range of sample testing and comparison, the Bayesian optimization was found to be the best choice in this study, and the best values of $C$ and $σ$ were obtained ( $C = 8$ and $σ^{2} = 0.42$ ).

Based on the same characteristics of the input vector, the performance of the traditional SVM method was compared with that of the proposed LS-SVM method. Table 3 shows the classification results. Figure 9 shows that the LS-SVM combined with color, texture, shape, and dynamic threshold parameters provided the best segmentation results, and the overall recognition rate for the four types of turf reached 92.88%. In addition, the recognition results for the four types of turf were compared; the recognition rate for carpet grass was the highest (94.1%), whereas that for Kentucky bluegrass, it was only 85.37%. This is because the difference between the shape and texture characteristics of Kentucky bluegrass and other lawns is not obvious and is affected by illumination and other noise. To further improve the recognition accuracy, the number of shape and texture feature parameters can be increased in a later stage.

Fig. 9

Comparison of classification results using different recognition methods.

3.3.

Lighting Performance Discrimination

To verify the practicability of the proposed algorithm under different weather conditions, a green lawn area was selected as an experimental site. A total of 240 images were collected at two times: once at noon on a sunny day and once at ∼ 3:00 PM on a cloudy day. The experiment showed that the overall recognition rate on a sunny day was 92.3%, whereas that on a cloudy day was higher, mainly because the light on sunny days is direct, and the light on cloudy days is diffused, resulting in a higher contrast between the plants and the background, which makes the recognition rate on cloudy days higher. Furthermore, the average recognition time on a sunny day was 1.42 s higher than that on a cloudy day; nonetheless, this can meet the requirements of actual classification (Table 4).

Table 4

Recognition rate under different weather conditions.

Weather environment	Number of images/frame	Recognition rate (%)	Average time (s)
Sunny day	140	92.3	1.42
Cloudy day	100	95.6	1.36
Total average	—	93.95	1.39

4. Conclusion

This paper proposes an image segmentation method based on dynamic threshold and LS-SVM techniques. This method adopts a $3 \times 3 pixel$ square area as separate units. Then, it selects two pixel color features, local energy, gradient, and the second moment, which are taken as three texture features, and seven blade shape regional characteristics as input characteristics. Next, it applies the feature vector as the LS-SVM input and adds the LS-SVM local dynamic threshold to obtain the training sample used to classify images taken for identification. The experiments showed that the algorithm is highly accurate in different environments. For the recognition of turf plants with similar shapes and texture features, more feature vectors can be added to the LS-SVM recognition model to train the samples. Considering the limitations of the proposed algorithm, it is necessary to further optimize the penalty coefficient, $C$ , and parameters for more complex environments to achieve the classification and recognition requirements. In future work, we plan to optimize and improve the algorithm and to classify more lawn plants to establish a complete database for lawn plant identification and classification. This can provide a theoretical basis for accurately setting the height of cutters for ZTR mowers.

References

1.

R. L. Zimdahl et al., “Weed classification,” Fundamentals of Weed Science, 47 –60 5th ed.Academic Press(2018). Google Scholar

2.

S. Zhang et al., “Label propagation based supervised locality projection analysis for plant leaf classification,” Pattern Recognit., 46 (7), 1891 –1897 (2013). https://doi.org/10.1016/j.patcog.2013.01.015 Google Scholar

3.

S. G. Wu et al., “A leaf recognition algorithm for plant classification using probabilistic neural network,” in IEEE Int. Symp. Signal Process. and Inf. Technol., 11 –16 (2007). https://doi.org/10.1109/ISSPIT.2007.4458016 Google Scholar

4.

S. H. Lee et al., “How deep learning extracts and learns leaf features for plant classification,” Pattern Recognit., 71 1 –13 (2017). https://doi.org/10.1016/j.patcog.2017.05.015 Google Scholar

5.

D. Hall et al., “Evaluation of features for leaf classification in challenging conditions,” in IEEE Winter Conf. Appl. Comput. Vision, 797 –804 (2015). https://doi.org/10.1109/WACV.2015.111 Google Scholar

6.

Q. Liu et al., “A novel locally linear KNN model for visual recognition,” in Proc. IEEE Conf. Comput. Vision and Pattern Recognit., 1329 –1337 (2015). https://doi.org/10.1109/CVPR.2015.7298738 Google Scholar

7.

Q. Liu et al., “A novel locally linear KNN method with applications to visual recognition,” IEEE Trans. Neural Networks Learn. Syst., 28 (9), 2010 –2021 (2017). https://doi.org/10.1109/TNNLS.2016.2572204 Google Scholar

8.

R. Tadeusiewic et al., Exploring Neural Networks with C#, CRC Press, Taylor & Francis Group, Boca Raton, Florida (2014). Google Scholar

9.

S. Y. Ooi et al., “Image-based handwritten signature verification using hybrid methods of discrete radon transform, principal component analysis and probabilistic neural network,” Appl. Soft Comput., 40 274 –282 (2016). https://doi.org/10.1016/j.asoc.2015.11.039 Google Scholar

10.

C. Sukalpa et al., “Text independent writer identification for Oriya script,” in Proc. Tenth IAPR Int. Workshop IEEE Doc. Anal. Syst., (2012). https://doi.org/10.1109/DAS.2012.86 Google Scholar

11.

J. A. K. Suykens and J. Vandewalle, “Chaos control using least-squares support vector machines,” Int. J. Circuit Theory Appl., 27 (6), 605 –615 (1999). https://doi.org/10.1002/(SICI)1097-007X(199911/12)27:6<605::AID-CTA86>3.0.CO;2-Z ICTACV 0098-9886 Google Scholar

12.

M. J. Ingrouille and S. M. Laird, “A quantitative approach to oak variability in some north London woodlands,” London Nat., 65 35 –46 (1986). Google Scholar

13.

T. Sixta, Image and Video-Based Recognition of Natural Objects, Czech Technical University, Prague (2011). Google Scholar

14.

D. R. Rossatto et al., “Fractal analysis of leaf-texture properties as a tool for taxonomic and identification purposes: a case study with species from neotropical Melastomataceae (Miconieae tribe),” Plant Syst. Evol., 291 (1–2), 103 –116 (2011). https://doi.org/10.1007/s00606-010-0366-2 Google Scholar

15.

C. Mallah, J. Cope and J. Orwell, “Plant leaf classification using probabilistic integration of shape, texture and margin features,” 01 –06 (2014) http:www. Actapress.com/Abstract.aspx Google Scholar

16.

J.-X. Du et al., “Leaf shape based plant species recognition,” Appl. Math. Comput., 185 (2), 883 –893 (2007). https://doi.org/10.1016/j.amc.2006.07.072 Google Scholar

17.

C. A. Priya, T. Balasaravanan and A. S. Thanamani, “An efficient leaf recognition algorithm for plant classification using support vector machine,” in Proc. Int. Conf. Pattern Recognit., Inf. and Medical Eng., 428 –432 (2012). https://doi.org/10.1109/ICPRIME.2012.6208384 Google Scholar

18.

E. Elhariri, N. El-Bendary and A. E. Hassanien, “Plant classification system based on leaf features,” in Proc. 9th Int. Conf. Comput. Eng. and Syst., 271 –276 (2014). https://doi.org/10.1109/ICCES.2014.7030971 Google Scholar

19.

Q. P. Wang et al., “Recognition of leaf image based on ring projection wavelet fractal feature,” Lect. Notes Comput. Sci., 6216 240 –246 (2010). https://doi.org/10.1007/978-3-642-14932-0 LNCSD9 0302-9743 Google Scholar

20.

H. K. A. Devi, “Thresholding: a pixel-level image processing methodology preprocessing technique for an OCR system for the Brahmi script,” Ancient Asia, 1 161 –165 (2006). https://doi.org/10.5334/aa.06113 Google Scholar

21.

Y. Solihin and C. Leedham, “The multi-stage approach to grey-scale image thresholding for specific applications,” 1 –27 Nanyang Avenue, Republic of Singapore (2000). Google Scholar

22.

T. Y. Goh et al., “Performance analysis of image thresholding: Otsu technique,” Measurement, 114 298 –307 (2018). https://doi.org/10.1016/j.measurement.2017.09.052 0263-2241 Google Scholar

23.

V. Chernov et al., “Integer-based accurate conversion between RGB and HSV color spaces,” Comput. Electr. Eng., 46 328 –337 (2015). https://doi.org/10.1016/j.compeleceng.2015.08.005 CPEEBQ 0045-7906 Google Scholar

24.

J. F. Khan, R. R. Adhami and S. M. A. Bhuiyan, “A customized Gabor filter for unsupervised color image segmentation,” Image Vision Comput., 27 (4), 489 –501 (2009). https://doi.org/10.1016/j.imavis.2008.07.001 Google Scholar

25.

J. C. Neto et al., “Plant species identification using elliptic Fourier leaf shape analysis,” Comput. Electron. Agric., 50 (2), 121 –134 (2006). https://doi.org/10.1016/j.compag.2005.09.004 CEAGE6 0168-1699 Google Scholar

26.

A. Wilson, A. Fern and P. Tadepalli, “Using trajectory data to improve bayesian optimization for reinforcement learning,” J. Mach. Learn. Res., 15 (1), 253 –282 (2014). Google Scholar

27.

P. C. Bhat et al., “Optimizing event selection with the random grid search,” Comput. Phys. Commun., 228 245 –257 (2018). https://doi.org/10.1016/j.cpc.2018.02.018 CPHCBZ 0010-4655 Google Scholar

Biographies of the authors are not available.

CC BY: © The Authors. Published by SPIE under a Creative Commons Attribution 4.0 Unported License. Distribution or reproduction of this work in whole or in part requires full attribution of the original publication, including its DOI.

Citation Download Citation

Yinhai She, Xinyan Wang, Tina Gui, and Jinkang Cai "Lawn plant identification and segmentation based on least squares support vector machine and multifeature fusion," Journal of Electronic Imaging 28(2), 023034 (23 April 2019). https://doi.org/10.1117/1.JEI.28.2.023034

Received: 28 November 2018; Accepted: 27 March 2019; Published: 23 April 2019

Access the abstract

JOURNAL ARTICLE
9 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 7 scholarly publications.

Explore citations on Lens.org

KEYWORDS

Image segmentation

Detection and tracking algorithms

Feature extraction

Image processing algorithms and systems

Image filtering

Image processing

Algorithm development

1.

Introduction

2.

Lawn Recognition Algorithm

Fig. 1

Fig. 2

2.1.

Dynamic Threshold

2.2.

Color Spatial Feature Extraction

Eq. (1)

Eq. (2)

Eq. (3)

Fig. 3

2.2.1.

Compute the noise discontinuity

Eq. (4)

2.2.2.

Compute the standard deviation

Eq. (5)

Eq. (6)

2.2.3.

Compute the pixel-level color features

Eq. (7)

Eq. (8)

Eq. (9)

Fig. 4

2.3.

Texture Feature Extraction

Fig. 5

2.3.1.

Selection of the color space

2.3.2.

Application of the Gabor filter to the S component

Fig. 6

Fig. 7

2.3.3.

Extraction of the local energy

Eq. (10)

Eq. (11)

2.3.4.

Extraction of the local gradient

Eq. (12)

2.3.5.

Extraction of the local second moment matrix

Eq. (13)

Eq. (14)

Eq. (15)

Eq. (16)

Eq. (17)

2.4.

Shape Feature Extraction

Table 1

Table 2

2.5.

Least Squares Support Vector Machine Identification Model

Eq. (18)

Eq. (19)

Eq. (20)

Eq. (21)

Eq. (22)

3.

Results and Analysis

3.1.

Image Segmentation Evaluation Results

Fig. 8

3.2.

Discrimination Using Support Vector Machines

Table 3

Fig. 9

3.3.

Lighting Performance Discrimination

Table 4

4.

Conclusion

References

Show All Keywords

Keywords/Phrases

Search In:

Publication Years