2.1.

Aging Process of Asphalt Pavement

Based on previous studies and our field investigation, asphalt pavement shows a dramatic decline of bitumen proportion in the preliminarily aged condition due to the effects of volatilization, absorption, oxidation, and photochemical reaction on the road surface, resulting in a brittle and vulnerable pavement [Fig. 1(b)].^5,24 Along with the further erosion of bitumen, moderately aged road surfaces expose their stone content [Fig. 1(c)]. The asphalt pavement then enters the heavily aged stage in which the rock pebbles are exposed at the road surface as relatively small granules due to continuous tearing as well as weathering effect [Fig. 1(d)]. During the aging processes, the deformability and structural strength of the road pavement are degraded. Eventually, a wide range of road pavement distresses occur in forms of linear cracks, alligator cracks, potholes, and rutting [Fig. 1(e)]. In order to quantify the degree of asphalt pavement aging, the Munsell Neutral Value Scale Card (MNVSC) was used to describe the color change of the asphalt pavement quantitatively during our field inspection.^18,25 The MNVSC is a standard scale of neutral grays placed in sequence between black (0% reflectance) and white (100% reflectance) in 37 levels. Colors within each level are nondifferentiable to human eyes under standard viewing conditions. The standard color names and their corresponding values and spectral reflectance established by ISCC-NBS are illustrated in Table 1. According to the field investigation and visual interpretation, the aging conditions of asphalt can be categorized into three classes, preliminarily aged [N0.5/ to N4.25/], moderately aged (N4.25 to N6.75/], and heavily aged (N6.75/, N8.75/]. The range of (N8.75/, N9.5/] was not included in the classification to match the color of any kind of asphalt pavement. In addition, the pavement distresses were also excluded because the spectral features of distresses could not be appropriately or accurately characterized by MNVSC.

Fig. 1

The aging processes of asphalt pavement over time. (a) Fresh road pavement, (b) preliminarily aged pavement, (c) moderately aged pavement, (d) heavily aged pavement, and (e) pavement distresses (e.g. linear crack, alligator crack, pothole, etc.)

Table 1

ISCC-NBS color names, values, and spectral reflectance of MNVSC and the corresponding asphalt pavement conditions.

2.2.

Study Area and Data Acquisition

In order to investigate the spectral features of the aged asphalt road pavement, and the possibility of mapping pavement conditions from remotely sensed data, several asphalt roads located in the southwest of Beijing, China, were selected (Fig. 2). The East Baiyang Road (marked in red) in the south of the image has three lanes in both directions. The Yangguang South Street (marked in yellow) and the East Ring Road (marked in green) are located in the west and the east of the image, respectively. The number of lanes of the Yangguang South Street is the same as that of the Baiyang East Road. The East Ring Road has four lanes in each direction. In the middle of the image are the two significantly narrower roads, the Liangxiang East Area No. 14 and No. 16 Roads, linking the Baiyang East Road and the Yangguang South Street. Both possess two or more lanes in their main roads.

Fig. 2

The study area and the road segments draped on the Worldview-2 image.

Although few distresses were found on these road pavements, it is obvious that each road was in a different aging condition. For instance, the Liangxiang East Area No. 14 Road was just repaved in early 2013 and is now preliminarily aged, whereas the Liangxiang East Area No. 16 Road and the Yangguang South Street appear to be in a worse condition because they were built in 2006 and have not been repaved since then. In contrast, the majority of the pavement in both the Baiyang East Road and the East Ring Road is moderately aged, showing the loss of bitumen and some outcrops of stone component.

In this study, Worldview-2 satellite images were chosen for mapping the asphalt pavement conditions (Fig. 2). The Worldview-2 sensor has eight bands spanning from visible to near-infrared wavelengths (from 0.45 to $1.04\mu \mathrm{m}$) that are useful to identifying the information about the aging of asphalt road pavement. In principle, the WorldView-2 multispectral imagery has a resolution of 1.84 m. In this study, however, the WorldView-2 dataset provided by Kosmos Company of Remote Sensing was resampled by the data provider to 2-m spatial resolution and was captured on September 21, 2013. Subsequently, preprocessing was performed including radiometric calibration, and atmospheric and geometric correction to obtain the original land surface reflectance for further spectral analysis. First, based on the gain and offset data stored in the image metadata, WorldView-2 imagery was calibrated using ENVI 5.3 software package. The atmospheric correction was then conducted for the Worldview-2 image to retrieve land surface reflectance using the FLAASH module integrated with ENVI 5.3. In the module, the atmospheric model mid-latitude summer and aerosol model urban were applied. Additionally, given that accuracy assessment required an accurately geolocation match between the Worldview-2 image and field GPS points, image registration based on 11 GPS points was also conducted using a first-order polynomial and nearest neighbor resampling, resulting in an RMS error of 0.83. Finally, in order to avoid the impact of off-road areas, we delineated the road areas into five polygons based on information from field investigation and visual interpretation of the Worldview-2 image (Fig. 2).

2.3.

Spectral Measurements with Analytical Spectral Devices Spectrometer

Field measurement of pavement spectra was conducted three times in the study area between April and July in 2015. An analytical spectral devices full range (ASD Field Spec-FR, ASD Corporation) was used to record spectral data. The instrument has three detectors covering the visible- and near-infrared (VNIR) and a short-wave infrared (SWIR1 and SWIR2) band, with a spectral sampling interval of 1.4 nm for the VNIR detector and 2.0 nm for the SWIR detectors. Field spectral measurement was conducted between Beijing local time 11:00 am and 13:00 pm under a clear sky, and the dark current was removed at every beginning of measurement. The optical fiber bundle collected the reflected radiation with a 25-deg conical field of view at 50 cm above the pavement, corresponding to a $22\mathrm{cm}\times 22\mathrm{cm}$ region. In addition to the ground targets, a white reference was also measured with the Spectralon Panel (Labsphere Inc., North Sutton, New Hampshire) to standardize and calculate the spectral reflectance of all the ground targets. The spectrum of every target including the white reference was recorded five times per measurement, and then the mean of five spectra was used in further processing. In addition to the asphalt pavement, the spectral information of other related objects including vegetation, cement sidewalks, white traffic lines, and bare soil was also collected, and ancillary information such as digital photos, pavement distress types, pavement health condition, and the asphalt color evaluated using the Munsell Neutral Value Scale (glossy collection) were collected and recorded through the field work. In total, a spectral library consisting 500 spectra was developed and used in the following spectral analysis.

The spectra were preprocessed including spectral smoothing and removal of the water absorption bands using the software package of ViewSpecPro 6.0.15, and a MATLAB Toolbox for spectral automatic postprocessing developed by NERC Field Spectroscopy Facility.²⁶

2.4.

Asphalt Pavement Condition Mapping Using MESMA

Some asphalt pavement pixels are covered by some other objects (e.g., vegetation, cement sidewalks, soil) even though the Worldview-2 image has a relatively fine spatial resolution of 2 m (Fig. 3). Spectral mixture analysis (SMA) was, therefore, performed to calculate the fraction of pavement from the mixed pixels. MESMA is based on the principle of linear spectral mixture analysis shown by the following equation:

Fig. 3

A portion of the Worldview-2 false color image in the Baiyang East Road area.

The accuracy of the model can be evaluated using root mean square error (RMSE)

Fig. 4

The flowchart of the MESMA with the Worldview-2 image.

Based on the GPS and other road attribute data taken in situ, a spectral library was first developed using a VIPER tool from several regions of interest (ROIs) on the Worldview-2 image. The library contains about 40 pixels for each representative surface type, covering the asphalt pavements in preliminarily aged, moderately aged, and heavily aged conditions, as well as other related objects including vegetation, cement sidewalk, white traffic line, bare soil, and shade. The library was optimized by analyzing the endmember average root mean square error (EAR), a metric proposed by Dennison and Roberts,³² to select the most representative endmembers with the lowest EAR value for each land cover class. EAR was calculated for each endmember by averaging the RMSE of the set of models that use that endmember to unmix the spectra belonging to the same land cover class

Meanwhile, because too many or too few endmembers will increase model error,²⁷ and the aging process is progressive in each pavement condition, five endmembers for each type of aged pavements and three nonpavement endmembers for every other class were extracted in this study. In addition, one shade endmember that we selected from the mean of the top three shade endmembers with the lowest EAR was used as well. As illustrated in Fig. 5, the spectra of the selected endmembers were used to build the spectral unmixing model. A coupled strategy is proposed in our study by combining the two-endmember model (EM2) with the three-endmember model (EM3). Namely, if the RSME by EM2 and that by EM3 is greater than $\sigma$, EM3 is adopted; otherwise, EM2 should be adopted. This strategy is simple but quite effective per our experiment.^31,33

Fig. 5

The endmember spectra resample to Worldview bands and used in the spectral unmixing model.

Viper tools can provide a maximum of four spectral libraries including one for shade endmember. Each of these libraries refers to one type of endmember in a mixed pixel. Specifically, Viper tools define two shade options, photometric shade and nonphotometric shade. The former defaults to zero reflectance, whereas the latter enables a user to provide a shade spectrum from a spectral library. In this study, we chose the filtered shade endmember by EAR as the input. The EM2 models (Table 2) included 27 endmembers in the first library (namely, 15 of three kinds of aging pavement and 12 of four nonpavement objects) and the shade endmember in the second library. They were processed in a constrained mode with a set of recommended parameters by Roberts et al.³⁴ (e.g., minimum and maximum fraction constraints of $-0.05$ and 1.05, shade constraint of 0.8, RMSE of 0.025, and residual constraint of 0.025). In addition, 180 EM3 models (Table 2) in the same constraint mode as above were applied to the image as well, in which the models were comprised of 15 asphalt pavement endmembers in the first library, 12 endmembers of other objects in the second library and one shade endmember in the third library. After generating the fraction image of each class of land cover (i.e., PA pavement, MA pavement, HA pavement, vegetation, sidewalks, traffic line, and soil), shade normalization following 31 and 33 was performed, that is, each nonshade fraction was divided by the sum of the nonshade fractions for that pixel. For those pixels modeled by EM2 models, the resulting shade normalized fraction would be 100%. Finally, for more effective visual interpretation for pavement in different aging degrees, a classification image for aging pavement distribution was generated based on the criteria: the value of a pixel would be set to 1 if the fraction of a pavement endmember in it is $>0.5$.

Table 2

The statistics of the EM2 and EM3 models in the experiment.

2.5.

Accuracy Assessment Procedure

In order to evaluate the accuracy of the fraction image of every target, the Worldview-2 multispectral image was pan-sharpened with its corresponding 0.5-m panchromatic image, to generate a multispectral image of 0.5-m resolution for the study area. Then, the classification of pan-sharpened image was performed by visual interpretation, which was based on the priori knowledge from our three exhaustive field investigations in the study area. A classification image consisting of five classes (PA pavement, MA pavement, HA pavement, shade, and nonpavement land covers) was produced and visually inspected for accuracy, and errors were manually corrected. The visually classified image was then used to calculate the fraction of each class within each $4\times 4\text{pixel}$ neighborhood, from which a 2-m fraction image was generated and used as reference in the following accuracy assessment. Afterward, a number of samples for each type of endmember was selected from the 2-m reference image and used to conduct linear regression analysis with the modeled fraction at the sample pixel location in the fraction image generated from the MESMA analysis. To further assess the accuracy of the classification result, a number of random sample points of the asphalt pavement in the study area was selected from the field measurements, regardless of the endmembers used in the unmixing analysis.

3.1.

Spectral Analysis of the Aged Asphalt Pavement

The asphalt pavement shows different spectral patterns in different conditions of aging due to the change of ingredients and structure of the road pavement. Some representative spectral curves of the asphalt pavement for the first three conditions are illustrated in Fig. 6. The spectral curves labeled as P1, P2, and P3 were taken from the road pavement in the preliminarily aged condition. Specifically, P1 was from fresh asphalt, whereas P2 and P3 had been just lightly aged. The average reflectance of these three pavement samples was very low and the peak reflectance was $<10\%$ at the wavelength of 0.35 to $2.5\mu \mathrm{m}$. Two hydrocarbon absorption zones were found around the wavelengths of 1.72 and $2.3\mu \mathrm{m}$, which is consistent with the findings of Herold.⁵ The photos M1, M2, and M3 in Fig. 6 were taken of the pavements in a moderately aged condition, in which the lithologic content was exposed to the road surface. The overall spectral reflectance in this condition increased as expected, and the absorption of hydrocarbon observed for samples in the lightly aged condition could no longer be found. Instead, a strong absorption at the wavelength of $2.33\mu \mathrm{m}$ by the lithologic component of the pavement was found. Moreover, the spectral slope from visible- to near-infrared appears to be sharper. In the heavily aged condition (e.g., H1, H2, and H3 in Fig. 6), the milled rock pebbles and less asphalt in the pavement resulted in a visibly flatter pavement. Hence, the overall reflectance increased considerably (e.g., a peak of $\sim 24\%$ observed for H3). The reflectance in the visible band continuously increases further through the aging process, and three weak absorption features related with iron oxide emerged at 0.52, 0.67, and $0.87\mu \mathrm{m}$.^16,35 After a turning point at $0.78\mu \mathrm{m}$, the reflectance showed a steady decline with a strong absorption by mineral at $\sim 2.33\mu \mathrm{m}$.

Fig. 6

The spectra and corresponding digital photos and color values of the asphalt pavement in preliminarily aging stage (P1, P2, and P3), moderately aging stage (I1, I2, and I3), and heavily aging stage (L1, L2, and L3).

In sum, it is clear that the asphalt pavement in different aging conditions presents different spectral patterns from 0.35 to $2.5\mu \mathrm{m}$, especially in visible- and near-infrared bands. This makes it possible to map the asphalt pavement conditions based on Wordview-2 imagery.

3.2.

Pavement Condition Mapping

A synthetic unmixing analysis based on the coupled strategy proposed in Sec. 2.4 was performed with a threshold value ($\sigma$) of 0.01. This means that if the difference of RMSE by EM2 and EM3 is $>0.01$, the EM3 model is used to unmix the pixel; otherwise, the EM2 model is used to retrieve the fraction of the endmembers. Table 2 shows the number and percentage of the pixels processed by the different models. As can be seen, 81.5% of the pixels were unmixed by the EM2 model, whereas 18.11% of the pixels were unmixed by the EM3 model.

The fraction maps of the pavement, shade, and nonpavement endmembers are shown in Fig. 7. It can be seen that the Liang Xiang East Area No. 14 Road has a much bigger fraction of PA endmember, suggesting this road pavement was mostly in the PA condition [Fig. 7(a)]. The East Ring Road and the Baiyang Road had a larger fraction of MA endmember [Fig. 7(b)], and the Yangguang South Street had a larger fraction of HA endmember [Fig. 7(c)]. The fraction of the other nonpavement land covers was mapped in Fig. 7(d), and that of shade was illustrated in Fig. 7(e).

Fig. 7

The fraction maps for the (a) preliminarily aged, (b) moderately aged, (c) heavily aged pavements, (d) nonpavement land covers (i.e., vegetation, sidewalks, traffic line, and soil), and (e) shades based on the subpixel unmixing approach.

For more effective visual interpretation, the value of a pixel is set to 1 if the fraction of a pavement endmember in it is $>0.5$. The three thresholding fraction images of the heavily, moderately, and preliminarily aged pavements were used to generate an RGB false composite image (Fig. 8). The image classification showed a fairly good agreement between the modeled road surface condition and the actual measurement (Fig. 8).

Fig. 8

The fraction image for the heavily aging pavement (HA), the preliminarily aging pavement (PA), and the moderately aging pavement (MA) in RGB and the photos taken in A, B, C, and D sites.

Specifically, it can be seen that the pavement health conditions in the sites of A, B, C, and D are visually consistent with what is shown in the photos taken at these locations. For example, the surface of the pavement that was evaluated as heavily aged (in red) shows to be in a bright gray color (meaning less asphalt) on the corresponding photo. The pavement of the Liangxiang East Area No. 14 Road was classified as being in the preliminary aging stage.

The photos taken of site A (Fig. 8) suggest that the road had been repaved recently and therefore was in relatively the best condition. The Yangguang South Street had been in service for about 9 years without any maintenance and was mostly recognized as heavily aged (bright red pixels in Fig. 8). The intersection of the East Ring Road and the Baiyang East Road was moderately aged. In site D, the different lanes of the Baiyang East Road appear to be in different aging stages.

3.3.

Accuracy Assessment

According to the accuracy assessment procedure described in Sec. 2.5, the pan-sharpened Worldview-2 multispectral image was classified into five classes (PA pavement, MA pavement, HA pavement, shade, and nonpavement land covers) by visual interpretation. Afterward, 20 samples for each type of endmember were selected from the 2-m reference image and used to conduct linear regression analysis with the modeled fraction at the sample pixel location in the fraction image generated from the MESMA analysis. The results are presented in Fig. 9 and show that the MESMA-derived fraction was well correlated with the reference fraction ($R^2>0.7$). For the classification result, 246 random sample points of the asphalt pavement in the study area were selected from the field measurements (Fig. 10). The relationship between the Munsell color values and the aging stages collected in field investigation was used to match the classes of the 165 pavement samples with their color values (Fig. 10). Statistical analysis showed that 81.71% of the points could be correctly matched with a Kappa coefficient of 0.77.

Fig. 9

Correlation analysis between the modeled and reference fraction: (a) preliminary aged pavement, (b) moderately aged pavement, (c) heavily aged pavement, (d) nonpavement objects, and (e) shade.

Fig. 10

The matching result between the color values and the classification results. (The size of the circles indicates the number of the matched or nonmatched points. The color values from 0.5 to 8.75 in $x$-axis correspond to the Munsell color values in Table 1. $Y$-axis presents five types of land cover classification results from the experiment above, i.e., PA: preliminarily aging pavement; MA, moderately aging pavement; HA, heavily aging pavement; NP, nonpavement objects; SD, shade.)

3.4.

Statistical Analysis of Pavement Conditions

To quantitatively evaluate the aging condition of the asphalt road pavement in the study area, the area and percentage of the pavement in the three aging conditions were calculated and listed in Table 3. The results suggest that the Liangxiang East Area No. 14 Road was in the best condition, with 59.40% and 25.45% of its pavement being preliminarily aged and moderately aged, respectively (Fig. 11). In contrast, the Yangguang South Street and the Liangxiang East Area No. 16 Road show a significantly different pattern. The percentage of heavily aged pavement on these two roads reached up to 88.17% and 80.02%, respectively, presenting an urgent need for maintenance. For the East Ring Road, about 53.03% was classified as being moderately aged while 44.53% as heavily aged. It is obvious that the information derived from remotely sensed data is valuable for the management department to make informed plans for effective and timely road maintenance.

Table 3

The area of the road pavements in the three aging conditions.

Fig. 11

The percentage of the area of the aged pavements in the roads (%).

3.5.

Discussion

The application of the proposed method in mapping road pavement condition in the study area has been shown preliminarily to be effective. The coupling strategy of two- and three-endmember models allows for more accurate mapping of asphalt pavement conditions, especially near the road edges. For example, the EM2 models built for the study area are shown in Fig. 12. Different colors are used to indicate different two-endmember models used in the unmixing analysis. Specifically, 82% of the image pixels (i.e., the ratio of the number of modeled pixels to the number of entire image pixels without background) were unmixed successfully using the EM2 model except those near the border of roads and greenbelts (e.g., black pixels in the areas A and B in Fig. 12) because of the lack of appropriate endmembers in the models. In contrast, the EM3 model was more effective for those mixed pixels that are located between roads and greenbelts except those covered by vehicles (e.g., black pixels in the site A in Fig. 13) and 98.5% of valid pixels (i.e., the ratio of the number of modeled pixels to the number of entire image pixels without a background) with a low average RMSE were modeled successfully. It was found that the EM2 model was more appropriate for analyzing those pixels within the central road area, which was much less impacted by surrounding objects. The EM3 model was more suitable for analyzing those pixels near the road edges, where more nonpavement objects such as soil and vegetation tended to be captured in a pixel as well.

Fig. 12

The EM2 model image in the study area. (Color pixels indicate different EM2 models, and black pixels indicate unsuccessful unmixing.)

Fig. 13

The EM3 model image in the study area. (Color pixels indicate different EM3 models, and black pixels indicate unsuccessful unmixing.)

The analysis result is clearly valuable for the highway maintenance department to make timely and suitable maintenance plans. However, our approach also has some weakness. First of all, the shade endmember is simply treated as a low reflective object, despite that shading is more complicated in reality. The proposed method can be further improved by using multiseasonal images to remove or reduce the shades by the objects near roads prior to the subpixel unmixing analysis. Second, based on the in situ measured pavement spectral data, it can be seen in Fig. 3 that more spectral signatures can be observed in the wavelengths $>1.0\mu \mathrm{m}$, especially in those $>2.0\mu \mathrm{m}$. Unfortunately, the Worldview satellite does not acquire data to cover this spectral range. On the other hand, the distress condition of asphalt road pavement cannot be accurately detected well with our proposed approach due to the fact that its spectral signature is not very distinct. LiDAR might be a good data source for providing the 3-D geometric information of the distress pavement. In addition, bridging the generated pavement condition maps with the operational regulations of the maintenance department is yet to be studied for more practical applications.