Image-based neural architecture automatic search method for hyperspectral image classification

Zhonggang Hu; Wenxing Bao; Kewen Qu; Hongbo Liang

doi:10.1117/1.JRS.16.016501

6 January 2022 Image-based neural architecture automatic search method for hyperspectral image classification

Zhonggang Hu, Wenxing Bao, Kewen Qu, Hongbo Liang

Author Affiliations +

Journal of Applied Remote Sensing, Vol. 16, Issue 1, 016501 (January 2022). https://doi.org/10.1117/1.JRS.16.016501

Fig. 1

View Large | Download | View In Article Context

Feature extraction of HSI: (a) a 3×3 neighboring block, (b) a 5×5 neighboring block, and (c) a masked image.

Fig. 2

View Large | Download | View In Article Context

Abstract description of NAS method. The search strategy selects an architecture from a defined search space. The architecture is passed to the performance evaluation strategy, which returns the estimated performance of the architecture to the search strategy. Architecture D is an optional architecture in the search space.

Fig. 3

View Large | Download | View In Article Context

HSI data preprocessing process, where X1, X2, and X3 are the training, validation, and test datasets, respectively. I1, I2, and I3 are the training, validation, and test index sets, respectively.

Fig. 4

View Large | Download | View In Article Context

I-NAS for the HSIC instances. The model can take an input of any size and produce an output with a corresponding size. In training, the predicted label is selected in the output according to the position of the training pixel. The cross-entropy represents the gradient of the cross-entropy objective function [Eq. (1)]. Architecture D is an optional architecture in the search space.

Fig. 5

View Large | Download | View In Article Context

CNN framework of cells stacking for HSI classification.

Fig. 6

View Large | Download | View In Article Context

Cell structure search process diagram.

Algorithm 1

I-NAS for HSI classification.

Require: Initialize number of training group $X^{1}$ , validation group $X^{2}$ , and operation set $O$ .

1: for each pixel do

2: Create the index according to each class of label.

3: end for

4: split the sample set carrying position information into training, validation and test dataset according to

X^{1}

and

X^{2}

.

5: Mask the training, validation and test dataset respectively.

6: Architecture search phase:

7: initialize search epochs, learning rate

ξ

and

ε

, the architecture variable

B

, and the CNN weight

w

.

8: for every search epoch do

9:

w \leftarrow w - ξ \nabla_{w} L_{train} (w, B)

;

10:

B \leftarrow B - ε \nabla_{B} L_{val} (w - ξ \nabla_{w} L_{train} (w, B), B)

.

11: end for

12: choose the best

B^{*}

according to the performance on validation dataset.

13: according to

o^{(n, m)} = \arg \max_{o \in O} B_{o}^{* (n, m)}

, acquire the best I-NAS architecture

A r

.

14: Train and test the optimal I-NAS:

15: initialize training epochs, the weight

w^{*}

of

A r

and learning rate

ξ

.

16: for every training epoch do

17:

w^{*} \leftarrow w^{*} - ξ \nabla_{w}^{*} L_{train} (w^{*})

.

18: end for

19: for every test epoch do

20: according to test dataset carrying position information, acquire the predict by I-NAS.

21: end for

22: obtain OA, AA, Kappa by evaluating the predict and test labels.

Fig. 7

View Large | Download | View In Article Context

IN dataset. (a) False-color image. (b) Ground truth map. (c) Class name and color code and the corresponding number of classes.

Fig. 8

View Large | Download | View In Article Context

UP dataset. (a) False-color image. (b) Ground truth map. (c) Class name and color code and the corresponding number of classes.

Fig. 9

View Large | Download | View In Article Context

The optimal structure of a cell based on the I-NAS method on IN.

Fig. 10

View Large | Download | View In Article Context

Optimized structure of another cell based on the I-NAS method on IN.

Fig. 11

View Large | Download | View In Article Context

The optimal structure of a cell based on I-NAS method on UP.

Fig. 12

View Large | Download | View In Article Context

Optimized structure of another cell based on I-NAS method on UP.

Table 1

Classification results of different methods for IN dataset.

Class	SVM	3DCNN	SSRN	ADGAN	CapsNet	P-NAS	I-NAS
1	67.10 ± 1.67	89.80 ± 1.23	71.08 ± 2.48	98.04 ± 2.99	68.50 ± 9.25	100 ± 0.00	100 ± 0.00
2	70.11 ± 2.58	90.00 ± 2.22	94.67 ± 4.81	97.72 ± 2.18	93.94 ± 2.78	86.52 ± 3.59	89.76 ± 4.42
3	63.09 ± 5.81	86.82 ± 1.71	88.17 ± 6.18	95.55 ± 3.57	88.77 ± 5.11	96.97 ± 1.58	96.85 ± 1.81
4	44.02 ± 4.27	94.18 ± 1.93	99.13 ± 9.49	96.02 ± 2.67	71.03 ± 9.33	99.87 ± 0.25	98.82 ± 2.07
5	85.97 ± 2.37	92.20 ± 1.06	92.61 ± 7.25	96.43 ± 3.13	87.05 ± 3.29	96.41 ± 2.35	95.78 ± 1.72
6	92.25 ± 1.35	99.50 ± 1.32	97.03 ± 5.56	97.76 ± 2.89	98.83 ± 1.26	98.47 ± 1.01	99.32 ± 0.55
7	81.81 ± 1.66	82.75 ± 2.30	50.68 ± 4.39	69.23 ± 9.65	72.64 ± 9.61	100 ± 0.00	100 ± 0.00
8	98.47 ± 1.03	99.61 ± 0.97	98.55 ± 4.10	99.54 ± 0.78	99.85 ± 0.42	100 ± 0.00	99.95 ± 0.15
9	50.11 ± 5.74	99.20 ± 0.97	68.49 ± 3.70	71.78 ± 9.71	56.66 ± 9.62	100 ± 0.00	100 ± 0.00
10	68.38 ± 1.97	87.71 ± 5.12	94.58 ± 1.05	95.73 ± 1.64	94.71 ± 3.01	92.57 ± 1.11	96.19 ± 1.62
11	82.22 ± 2.59	91.19 ± 0.92	89.55 ± 5.26	96.90 ± 1.84	97.25 ± 1.34	91.83 ± 3.20	90.49 ± 4.09
12	68.26 ± 2.29	93.10 ± 1.15	74.24 ± 9.72	97.14 ± 2.33	87.01 ± 5.15	94.24 ± 1.55	95.32 ± 1.97
13	94.03 ± 2.92	99.58 ± 1.32	99.69 ± 9.71	96.93 ± 5.99	96.94 ± 4.58	100 ± 0.00	99.84 ± 0.30
14	94.95 ± 0.81	96.22 ± 0.27	98.02 ± 7.36	99.76 ± 0.31	98.25 ± 1.58	99.42 ± 0.34	98.24 ± 2.32
15	57.62 ± 5.29	75.21 ± 2.11	91.75 ± 9.40	97.66 ± 1.95	88.12 ± 8.86	99.74 ± 0.52	99.12 ± 1.07
16	83.68 ± 6.22	99.00 ± 2.67	94.36 ± 9.39	84.02 ± 8.05	91.13 ± 7.57	97.77 ± 2.72	100 ± 0.00
OA (%)	76.83 ± 1.19	91.68 ± 1.44	90.74 ± 3.49	87.97 ± 6.56	94.39 ± 0.63	94.25 ± 0.39	94.64 ± 1.23
AA (%)	75.08 ± 1.49	92.31 ± 0.57	87.66 ± 5.68	93.19 ± 2.72	85.91 ± 2.97	97.11 ± 0.32	97.49 ± 0.47
$K \times 100$	73.69 ± 1.31	89.40 ± 0.95	89.39 ± 4.02	96.76 ± 0.92	93.47 ± 0.74	93.39 ± 0.45	93.83 ± 1.41

Table 2

Classification results of different methods for UP dataset.

Class	SVM	3DCNN	SSRN	ADGAN	CapsNet	P-NAS	I-NAS
1	95.38 ± 1.37	86.20 ± 3.23	88.88 ± 7.36	89.59 ± 8.60	95.80 ± 0.81	95.96 ± 3.10	96.66 ± 1.88
2	94.60 ± 0.88	93.11 ± 2.11	92.21 ± 1.73	91.44 ± 9.65	99.83 ± 0.24	95.91 ± 1.68	96.86 ± 1.46
3	65.34 ± 4.27	63.09 ± 6.54	92.79 ± 5.99	95.95 ± 3.47	68.74 ± 7.59	99.22 ± 0.73	98.28 ± 1.54
4	77.73 ± 7.36	95.81 ± 1.77	96.19 ± 0.82	88.65 ± 6.25	96.75 ± 1.49	94.41 ± 2.06	97.44 ± 0.54
5	94.57 ± 2.83	94.14 ± 4.78	97.76 ± 3.05	97.73 ± 2.29	99.90 ± 0.18	99.49 ± 0.43	100 ± 0.00
6	67.55 ± 4.42	93.06 ± 1.99	97.03 ± 0.36	91.07 ± 9.49	99.49 ± 0.61	99.32 ± 0.55	97.34 ± 1.09
7	61.03 ± 6.42	57.81 ± 5.44	68.23 ± 9.55	93.56 ± 4.87	77.27 ± 4.73	99.97 ± 0.07	99.91 ± 0.13
8	78.50 ± 5.96	76.10 ± 2.86	90.16 ± 3.77	72.77 ± 9.66	87.03 ± 3.23	97.45 ± 1.10	99.25 ± 0.40
9	99.89 ± 0.06	83.20 ± 4.92	100 ± 0.00	73.05 ± 9.67	95.88 ± 4.11	95.02 ± 3.69	99.92 ± 0.05
OA (%)	84.45 ± 1.88	87.32 ± 1.89	91.20 ± 1.58	81.25 ± 9.33	95.44 ± 0.81	96.71 ± 0.71	97.45 ± 0.75
AA (%)	81.62 ± 1.98	82.51 ± 2.17	91.58 ± 1.60	88.20 ± 6.06	91.31 ± 1.68	97.42 ± 0.83	98.41 ± 0.36
$K \times 100$	79.87 ± 2.25	83.56 ± 2.71	88.17 ± 2.17	87.53 ± 9.14	94.16 ± 1.04	95.65 ± 0.93	96.62 ± 0.98

Fig. 13

View Large | Download | View In Article Context

Classification maps obtained by all considered algorithms on IN dataset. (a) Ground truth, (b) SVM, (c) 3DCNN, (d) SSRN, (e) ADGAN, (f) CapsNet, (g) P-NAS, and (h) I-NAS.

Fig. 14

View Large | Download | View In Article Context

Classification maps obtained by all considered algorithms on UP dataset. (a) Ground truth, (b) SVM, (c) 3DCNN, (d) SSRN, (e) ADGAN, (f) CapsNet, (g) P-NAS, and (h) I-NAS.

Fig. 15

View Large | Download | View In Article Context

Impact of different number of training samples on OA results for training. OA results were obtained by all algorithms on (a) IN dataset and (b) UP dataset.

Fig. 16

View Large | Download | View In Article Context

CA results for each class with different number of total labeled samples for training over the I-NAS on (a) IN dataset and (b) UP dataset.

Table 3

Time consumption of different algorithms on the IN dataset.

Method	Training time (s)	Testing time (s)
SVM	0.15 ± 0.01	3.18 ± 0.08
DCNN	1502.29 ± 6.03	57.68 ± 3.98
SSRN	3368.62 ± 51.05	47.58 ± 5.97
ADGAN	752.49 ± 4.92	3.24 ± 0.10
CapsNet	1293.09 ± 43.26	14.77 ± 0.96
P-NAS	1670.81 ± 7.94	102.53 ± 2.34
I-NAS	153.74 ± 4.38	0.33 ± 0.04

Table 4

Time consumption of different algorithms on the UP dataset.

Method	Training time (s)	Testing time (s)
SVM	0.09 ± 0.01	5.96 ± 0.48
DCNN	458.29 ± 3.11	246.78 ± 57.58
SSRN	1358.58 ± 70.69	82.25 ± 1.39
ADGAN	1227.62 ± 4.77	14.61 ± 0.05
CapsNet	871.47 ± 38.74	78.99 ± 1.57
P-NAS	1145.18 ± 15.68	319.97 ± 2.79
I-NAS	427.60 ± 6.68	1.20 ± 0.18

Table 5

Total number of trainable parameters in different algorithms (MB).

Method	Datasets
Method	Indian Pines	Pavia University
SSRN	0.330108	0.189316
CapsNet	0.262329	0.174164
P-NAS	0.067012	0.073683
I-NAS	0.082724	0.089795

Table 6

Time consumption and number of parameters for P-NAS and I-NAS on the IN dataset within the architecture search phase.

Method	Architecture search time (s)	Number of parameters (MB)
P-NAS	3090.35	0.288820
I-NAS	453.85	0.188084

Table 7

Time consumption and number of parameters for P-NAS and I-NAS on the UP dataset within the architecture search phase.

Method	Architecture search time (s)	Number of parameters (MB)
P-NAS	2453.18	0.286947
I-NAS	1262.30	0.185987

Access the abstract

JOURNAL ARTICLE
21 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

CITATIONS

Cited by 3 scholarly publications and 1 patent.

Explore citations on Lens.org

KEYWORDS

Image classification

Hyperspectral imaging

Convolution

Image processing

Neural networks

Feature extraction

Statistical modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years