The rapid development of remote sensing technologies provides interesting possibilities for the further development of nationwide mapping procedures that are currently based mainly on passive aerial images. In particular, we assume that there is a large undiscovered potential in multitemporal airborne laser scanning (ALS) for topographic mapping. In this study, automated change detection from multitemporal multispectral ALS data was tested for the first time. The results showed that direct comparisons between height and intensity data from different dates reveal even small changes related to the development of a suburban area. A major challenge in future work is to link the changes with objects that are interesting in map production. In order to effectively utilize multisource remotely sensed data in mapping in the future, we also investigated the potential of satellite images and ground-based data to complement multispectral ALS. A method for continuous change monitoring from a time series of Sentinel-2 satellite images was developed and tested. Finally, a high-density point cloud was acquired with terrestrial mobile laser scanning and automatically classified into four classes. The results were compared with the ALS data, and the possible roles of the different data sources in a future map updating process were discussed.
Nationwide topographic databases presenting, for example, buildings, roads, water areas, ground elevation, and some land use information provide the basic material for map production and numerous applications exploiting geospatial data. Currently, there are increasing demands for up-to-date and detailed data. Keeping the databases up to date and further developing their contents and maintenance processes thus becomes more and more important (see, e.g., Refs. 1 and 2). In Finland, as in many other countries, aerial images covering the entire country are acquired regularly at a few years’ intervals and used as the main remotely sensed data source in map updating. Data from single-channel airborne laser scanning (ALS) are also used, primarily to create high-quality digital terrain models (DTMs). The first scanning of the entire country with a minimum density of is planned to be finished in 2019. Multitemporal ALS datasets are not yet commonly available, and the intensity information of the ALS is not generally in active use. Some objects in the National Land Survey of Finland (NLS) Topographic Database3 are currently updated continuously (e.g., roads) or yearly (e.g., buildings) using information obtained, for example, from municipalities. More extensive updating of the database is done at a few years’ intervals following aerial image acquisitions. This updating process is mainly based on visual interpretation of the images.
In the present article, we investigate and discuss the possibilities that the newest remote sensing techniques could bring to map updating, especially considering the increasing demands for updating frequency, automation, and detailed contents of databases. Our work is not directly linked to the operational updating process of the NLS Topographic Database or its development in the near future. Instead, the objective is to consider the topic from a wider perspective and provide ideas for further research and development. Previous research has suggested that the updating of map databases could be made more automatic, for example, by developing automated change detection approaches (see, e.g., Refs. 45.–6). This previous research has mainly concentrated on datasets currently in operational use, such as aerial images and height information derived from images or single-channel ALS. The results have been encouraging, but challenges have also been reported. These challenges include shadows, and the diversity of objects and aerial image values over large areas or at different times of the year (see, e.g., Ref. 7).
Due to their favorable spatial resolution and viewpoint, aerial data acquisitions by cameras and laser scanners are likely to remain the main approach for nationwide topographic mapping in the future. In particular, we assume that there is a large undiscovered potential in multitemporal ALS. The newest developments in this field include multispectral ALS, which provides accurate 3-D information and intensity in three bands simultaneously.8 Unlike traditional passive aerial imaging, the technique is independent of external illumination conditions and the data do not include shadows. This is a very interesting novelty in the field of mapping. For example, the feasibility of using the technique for land cover classification has been demonstrated (see, e.g., Refs. 910.–11). Matikainen et al.10 also tested change detection between buildings and roads on a map and in multispectral ALS data from one date. According to our knowledge, change detection between multitemporal multispectral ALS datasets has not yet been studied. When such data are available, change detection approaches utilizing both height and multispectral intensity comparisons between different dates can be developed.
In addition to ALS, remote sensing technologies, in general, have developed rapidly in recent years,12 and the use of multisource data has been considered promising in different fields. In forestry, for example, data assimilation using multisource remotely sensed data has been suggested as a potential approach for providing updated forest inventory data.13 Holland et al.14 discussed the numerous ground-based and airborne survey techniques currently available and how the rapid development of systems challenges a mapping agency in collecting and maintaining data. This development means that standard aerial data sources could be complemented with other remotely sensed datasets more easily than before. In order to effectively utilize multisource remotely sensed data in nationwide mapping in the future, the advantages and challenges of different data sources and their possible roles in an integrated map updating process should be understood. Therefore, we also investigated the potential of satellite images and ground-based data to complement multispectral ALS.
One limitation of aerial data is that data acquisitions are relatively expensive and time consuming. On a country-wide level, they can normally only be repeated once in a few years, which can be a challenge when considering the increasing demands for up-to-date databases. Different from aerial techniques, satellite systems provide frequent coverage of large areas. The Sentinel satellites of the European Space Agency (ESA) are currently particularly interesting.15 They provide large amounts of data that are freely available. In our study, we investigated the use of Sentinel-2 optical satellite images for change detection. The spatial resolution of Sentinel-2 images is not high enough for mapping small details, such as individual buildings. They could, however, be used to monitor larger changes in the landscape.
In addition to demands on updating frequency, there are needs for very detailed geospatial information and 3-D city models, especially in urban areas. To meet these needs, very detailed remotely sensed data from the ground level are also increasingly available due to advanced terrestrial mobile mapping systems with laser scanners and cameras.16,17 Data from mobile laser scanning (MLS) are more accurate and denser than before and can be rapidly and flexibly collected. Applications of such data vary from their application in smart cities, urban planning, and street mapping to their application in autonomous driving systems. Interestingly, for mapping applications, the viewing perspective of the data is different from aerial systems. In our study, we also investigated the use of MLS data for detailed urban mapping.
The specific objectives of our study were as follows:
1. To investigate the potential use of multitemporal multispectral ALS data for automated change detection for the first time: a method was developed and tested to detect the most important height and intensity changes by direct comparison of two multispectral ALS datasets.
2. To investigate the potential of satellite images and MLS data to support map updating and complement ALS data:
3. To discuss the possible roles and integration of multispectral ALS data, satellite images, and MLS data in a future multisource map updating process.
The multitemporal multispectral ALS data were considered the elementary data and reference for other datasets. The level of detail in the multispectral ALS data corresponds approximately to that of aerial images currently used in map updating work, and in the future, multispectral ALS could thus have a central role in nationwide mapping. After discussing related work in Sec. 2 and presenting our study area and data in Sec. 3, the methods and experimental results are presented in Sec. 4. Section 5 concentrates on analyzing the quality of the results from the viewpoint of map updating. Finally, the possible roles of the different data sources in a future map updating process are discussed in Sec. 6. The conclusions of the study are presented in Sec. 7. Considering the individual parts of the study, our aim is not to find the most optimal or best-performing methods in this study. Rather, we aim to present practical methods that allow the discussions mentioned above, give satisfactory results, and can be later used as a starting point for further developments.
All remotely sensed datasets used in the study have been acquired with the most novel and advanced systems currently available. The combination of multispectral ALS data, Sentinel-2 satellite images, and MLS data from the same area is unique and has not been discussed in previous literature. This is also the first study investigating change detection from multitemporal multispectral ALS data.
Multisource Data Analysis Approaches in Mapping
The integration and fusion of different remotely sensed datasets have received much attention during recent decades (see, e.g., Refs. 18 and 19) although wider discussions on the possible roles of airborne, spaceborne, and ground-based remote sensing techniques for a given application are not common. In the field of mapping, typical examples of data integration in research literature include the use of ALS and aerial image data or satellite image data for mapping buildings (see, e.g., Refs. 20 and 21) and land cover (see, e.g., Ref. 22). As mentioned above, Holland et al.14 discussed the numerous survey techniques from the viewpoint of a mapping agency. Data analysis methods were not presented in their article. The integration of ALS and MLS data has been discussed, for example, by Rutzinger et al.23 regarding the extraction of vertical walls. Zhou and Vosselman24 compared curbstone mapping results from ALS and MLS data. Zhu and Hyyppä25 presented an approach for the 3-D modeling of railway environments by utilizing both ALS and MLS data. Bamber et al.26 presented a digital elevation model of Greenland based on various spaceborne and airborne datasets and digitized map data.
Mapping with Multispectral ALS Data
The feasibility of conventional single-channel ALS data for creating DTMs and mapping elevated objects, such as buildings and trees, is well known (see, e.g., Ref. 27). Several studies related to change detection and map updating have also been carried out. Change detection between multitemporal single-channel ALS datasets has been discussed, for example, by Murakami et al.,28 Richter et al.,29 and recently by Tran et al.30 Since the emergence of the first operational multispectral ALS system, Optech Titan, in late 2014, the potential of this single-sensor technology for several applications has also been demonstrated. These include land cover classification (see, e.g., Refs. 89.10.–11, 31, and 32), tree species classification,33,34 road mapping,35 and map updating.10 In particular, the ability of the technique to map ground-level classes—such as low vegetation, roads, paved and unpaved surfaces—is a valuable addition to the conventional ALS technique.
The Optech Titan sensor has three channels: infrared 1550 nm (channel 1), near-infrared 1064 nm (channel 2), and green 532 nm (channel 3). Each channel produces a separate point cloud. The most common approach to utilizing the multispectral intensity information has been to produce multispectral intensity images in raster format. A point-based approach that exploits the nearest neighbor approach in order to merge intensity information from different channels has also been used.9 Radiometric correction of Optech Titan data was discussed by Yan and Shaker.36 To our knowledge, change detection based on multitemporal multispectral ALS data has not yet been investigated.
Change Detection from Optical Satellite Images
Monitoring Earth surface dynamics and tracking changes from satellite images has been an important research topic in the past, and it still is. For example, multispectral optical data from Landsat satellite series have been applied for land use mapping for decades. The opening of the Landsat archive for free use has further increased the use of Landsat data.37 A review of change detection using Landsat has been presented by Zhu.38 Generally, a number of change detection techniques have been developed, and it continues to increase.39–41 When selecting a change detection method, the characteristics of the remotely sensed images used and the phenomenon under study should be considered.42 In a built-up area, this could include, for example, the type and density of urban structures and vegetation, and seasonal changes in the landscape.
The Copernicus Sentinel-2 mission43 is a constellation of twin polar-orbiting satellites with global coverage and a revisit time of five days at the equator with the two satellites. The satellites Sentinel-2A and Sentinel-2B were launched in June 2015 and March 2017, respectively. Each of the satellites carries a multispectral optical instrument with a swath width of 290 km. They acquire information in 13 spectral bands (from 443 to 2190 nm) at different resolutions (10, 20, and 60 m). The service is mainly focused on land monitoring and security. All the data are available for scientific and commercial use, free of charge. One of the first insights into the use of Sentinel-2 data for urban areas was provided by Lefebvre et al.44 In their study, change detection approaches were divided into image-to-image comparison and postclassification comparison. They used the fusion of the independent image classifications of Sentinel-2 and Landsat 8 to update the Copernicus High Resolution Imperviousness Layer. Pesaresi et al.45 tested the applicability of a classification method called symbolic machine learning on Sentinel-2 imagery and also demonstrated the added value of Sentinel-2 products over Landsat-derived products for mapping built-up areas. Being a recently launched system, the real usability of Sentinel-2 data has not yet been widely and fully analyzed. There are also many commercial satellite systems providing data with higher spatial resolution, but they exceed the scope of this study.
Mapping of Urban Objects from MLS Data
Much research has been carried out on the mapping of urban objects from MLS data including pole-like objects (e.g., Refs. 4647.48.–49) and more general object classes (see, e.g., Refs. 50 and 51). More recently, Li et al.52 segmented pole-like road furniture, i.e., street lights and traffic signs, into different components based on their spatial relations and compared a proposed knowledge-driven method and different machine learning classifiers to label these components. Also, the extraction of building facades has been studied23,51 as well as the retrieval of road markings and curbstones.53 The MLS classification studies reviewed above follow the object-based approach, that is, a point cloud is first segmented into objects and then the objects are classified. Also, two other approaches exist, namely the point-based approach54 and the voxel- or supervoxel-based approach (see, e.g., Ref. 55) where either single points or (super) voxels are classified. In this study, we adopted the object-based approach because it can preserve the spatial relations between different objects.
The most common object recognition workflow starts with the removal of ground and building facade points.23,48,50,51 Then, a connected component analysis48,51 or some more advanced segmentation method, such as graph-cut,50 is carried out on the above-ground points. Finally, the above-ground objects are classified into different classes, such as cars, lamp posts, and traffic signs. The classification algorithms may apply machine-learning methods50,51 or knowledge-based rules.48 Some approaches only concentrate on finding objects of a pole-like shape.46,47,49
Study Area and Data
Study Area, Reference Data, and Map Data
Our study area is located in Espoonlahti, in the city of Espoo, about 15 to 20 km from the center of Helsinki. The area is a growing suburban area with high-rise and low-rise residential areas and an industrial area. Areas that are not built-up are mainly covered by coniferous and mixed forest. In method development and testing, an “intensive study area” covered by both ALS and Sentinel-2 datasets was used. This area is illustrated in Fig. 1 (the center point is approx. 60° 9′ 18” N, 24° 38’ 24” E). The Sentinel-2 satellite images used in the study covered a larger area. On the outskirts of this larger area, there are also some agricultural fields and a golf course. These areas, however, were not used in method development or evaluation. The study area is located on the coast, and the multispectral ALS data and Sentinel-2 images covered some water areas, but these were also excluded from the analyses as our study concentrated on the changing suburban area.
A permanent land cover classification test field consisting of reference points with known land cover is located in the study area (see Refs. 10 and 56). In the present study, 427 “unchanged” reference points were used as training points in Sentinel-2 satellite image analyses to define training segments for developing a change detection method (with the classes “changed” and “unchanged”). The points used in the study were selected so that they cover part of the intensive study area, and the density of the points is feasible for segment-based analysis of Sentinel-2 satellite images (; see Fig. 1). Part of the intensive study area remained outside the training points for quality evaluation purposes (quality evaluation in this study was based on ALS classifications rather than the reference points). In addition, only those points were used that remained unchanged during the time period of interest, that is, between August 2015 and June 2016. To select unchanged points, two versions of the reference points were used. The first one is the same as that used in Ref. 10 and corresponds to the situation in August 2015. The second is a newer one corresponding to the situation in June 2016.57 The main data sources used in the manual checking and updating of the land cover information of these point sets were multispectral ALS data acquisitions from the corresponding points of time. To select unchanged points for the satellite image analysis, the land cover was generalized into three classes: “built-up area,” “tree,” and “low vegetation.” Unchanged points considering these classes were selected.
For evaluating the quality of change detection results from multitemporal multispectral ALS data, a specific set of test points was collected. These were collected from changes that are important for topographic mapping and included 34 points on new buildings, 56 points on new roads, and 57 points on new clear-cut forest areas. The selection of these points was based on visual comparison of the two multispectral ALS datasets used in the study. Map data were used as an aid. Some of the new buildings and roads were still under construction, but they were already clearly visible in the ALS data from the second date. The clear-cut points were selected from areas where trees had been cut, but the construction of new roads and buildings had not yet begun.
Map data used in the study included information extracted from the Topographic Database maintained by the NLS.3 Buildings and roads from early 2015 were used in change detection tests with multispectral ALS data. Water areas and agricultural polygons were used to mask these areas out of analyses. Water areas were excluded from multispectral ALS and Sentinel-2 analyses. Agricultural areas were excluded from Sentinel-2 analyses. Road vectors in the database represent the center lines of the roads, and the width of the roads is not available. In change detection, we used buffered road vectors. The widths of the buffers were based on approximate information available on the widths of different types of roads. Different versions of the topographic database data, and openly available map and aerial image data from the city of Espoo (map service58) were also used as an aid at various stages of the study. It should be noted that due to the active construction work in the area, we did not have map data that exactly corresponded to the situation at the time of data acquisitions.
Multispectral ALS Data
Multispectral ALS data from two dates, August 21, 2015, and June 14, 2016, were used in the study. The trees were in leaf in both of these datasets. The point density in preprocessed data was about 8 to for each channel in the August 2015 data and about 11 to in the June 2016 data. The main steps in preprocessing of the August 2015 data included basic preprocessing, the range correction of intensity values, cutting out overlap points, removing some error points, and the creation of digital surface models (DSMs), DTMs, and three-channel intensity images in raster format (for more details, see Ref. 10). The same procedures, except the procedure for DTM generation, were also applied to the June 2016 data. The pixel size of the DSMs and DTM used in the study was 1 m on the ground. The pixel values in the DSMs represent the maximum height of points from all channels. The pixel size of the three-channel intensity images was 0.2 m, and the intensity values represent the average intensity of first-pulse and only-pulse laser points divided by 100.
An additional preprocessing step compared to those described in Ref. 10 was a calibration to adjust the intensity information from June 2016 so that it would be comparable with the August 2015 data. Originally, the intensity levels from the two data acquisitions were not comparable, which means that direct change detection between the intensity values is not possible without adjustment. The adjustment used 18 natural and manmade targets found in the study area. The targets included building roofs, parking places, sports fields, and beaches. The ALS points falling to each of the areas were inspected to only include target hits, not, for example, cars or reflective lines on an asphalt-covered car park. One of the targets was discarded because its behavior in the time series data was anomalous. For each channel, linear fit with scalar adjustment was found between the mean intensities from the two dates. Intensities in the later dataset were then adjusted with this linear model. To avoid extrapolation and negative intensity values, values below the minimum mean intensity value of the reference targets were not adjusted. According to visual evaluation, the intensity levels in the two datasets matched each other relatively well after the adjustment.57
In addition to the preprocessed height and intensity data, land cover classifications carried out for the data10,57 were used in the study. The land cover classifications were carried out using object-based random forest59 classification. The classification results from the first date, together with the DSMs and intensity images from both dates, were used as input data in change detection tests based on multispectral ALS data. The classification results from both dates were used as reference data in evaluating change detection results based on Sentinel-2 satellite images.
Sentinel-2 Satellite Image Data
All Sentinel-2 images from our study area were downloaded from the Copernicus Open Access Hub (previously known as the Sentinels Scientific Data Hub60). Starting from July 15, 2015, and concluding on June 30, 2016, a total of 44 Sentinel-2 images were downloaded. Many of these images had a high percentage of cloud cover, and only nine images were finally used in the study (Table 1). The selection criterion for images was that they should be completely cloud-free over our study area.
The acquisition dates of the remotely sensed datasets used in the study.
|Optech Titan multispectral ALS data||August 21, 2015, June 14, 2016|
|Sentinel-2 satellite images||August 17, 2015, August 24, 2015, October 3, 2015, October 6, 2015, April 10, 2016, May 10, 2016, May 13, 2016, May 30, 2016, June 29, 2016|
|MLS data||May 9, 2017|
Sentinel-2 data at the time of image acquisition were only available in Level-1C format,61 so it was necessary to manually produce a level-2A product, which represents bottom-of-atmosphere information (i.e., atmospherically corrected data). For this purpose, the sen2cor tool provided by the ESA62 was used. Nowadays, users can also download level-2A data directly from the hub.
Having all the images processed to the level-2A product level, the calculation of a normalized difference vegetation index (NDVI) and normalized difference built-up index (NDBI) was done for each image. NDVI was selected because it is the most frequently used vegetation index and it easily distinguishes healthy vegetated areas, whereas NDBI is suitable for highlighting urban areas. Appropriate Sentinel-2 bands and the official software SNAP (Sentinel-2 Toolbox)63 were used to calculate the indices:
The MLS data for the study were collected using a Finnish Geospatial Research Institute (FGI) proprietary MLS system called Roamer. Roamer used a Riegl VUX-1HA laser scanner to collect 3-D data of the scenery. The frequency of the scanning mirror was 250 Hz, resulting in cross-track profiles; the scanner emitted 1 017 000 pulses/s. In the online signal processing, the scanner could resolve for up to seven echoes per pulse, depending on the object characteristics. Each 360 deg profile corresponded to 4068 transmitted pulses, resulting in an angular resolution of 1.5 mrad. The point spacing was right below the scanner due to the sensor elevation of about 2.8 m from the ground. The point spacing on a plane perpendicular to the laser rays was 77 mm at a 50-m distance to the scanner. The positioning of the MLS system was carried out using the NovAtel SPAN GNSS-IMU (Global Navigation Satellite System—Inertial Measurement Unit; Flexpak6 receiver and ISA-100C IMU) positioning system for raw observables. The trajectory and orientation data were postprocessed with base station data from Trimnet virtual data service using Waypoint Inertial Explorer (NovAtel Inc., Calgary, Canada). The data were then georeferenced into a 3-D point cloud using Riegl RiWorld software (Riegl, Horn, Austria) and using previously acquired system offsets and calibration. The point cloud data were stored in LAS 1.2 format for delivery and processing.
The data used in the study were acquired on May 9, 2017, when trees were not yet in full leaf. Roamer was driven mainly on streets and, in some cases, on pedestrian routes in order to cover the area, buildings, and construction sites of interest. An excerpt of the data is presented in Fig. 2, showing the 3-D point cloud colored by the pulse return amplitude in tones of gray.
Methods and Experimental Results
In this section, we present the specific methods developed and tested in the present study. These include:
1. a new change detection method that uses multitemporal multispectral ALS data from two dates
2. a new continuous change monitoring method that uses a time series of Sentinel-2 satellite images
3. a method for urban scene classification that uses MLS data from one date (following Ref. 64).
The experimental change detection results (methods 1 and 2) and classification results (method 3) obtained by using these methods are also presented in this section. Quality analysis of the results, inter-relationships between the different datasets, and their possible use in a future multisource map updating process will be discussed in further sections.
Change Detection from Multitemporal Multispectral ALS data
Change detection method
The objective of the change detection test was to study the feasibility of direct height and intensity comparisons between multitemporal multispectral ALS datasets for map updating. A change detection process relying on two datasets was developed and mainly implemented with the eCognition software (Trimble Germany GmbH, Munich). The basic steps of the method are illustrated in Fig. 3. The main input data for the method include multispectral intensity images and DSMs from the two dates, and the results show the changes that have occurred in height or intensity. The land cover classification results from the first date are used as a starting point for the analysis. This ties the detected changes to land cover classes and also allows the use of different change detection rules for different classes. The land cover classes were generalized for the purpose of change detection (see Table 2).
The relation between land cover classes in the suburban study area10 and change detection from multitemporal multispectral ALS data.
|Class from land cover classification||Input class in change detection based on intensity||Input class in change detection based on DSM||Output classes from change detection|
|Building||—||Building||Building, height increase|
|Building, height decrease|
|Tree||—||Tree||Tree, height increase|
|Tree, height decrease|
|Asphalt||Artificial ground||Ground||Artificial ground, NDVI increase|
|Ground, height increase|
|Ground, height decrease|
|Gravel||Artificial ground||Ground||See above|
|Rocky area||Natural ground||Ground||Natural ground, NDVI decrease|
|Ground, height increase|
|Ground, height decrease|
|Low vegetation||Natural ground||Ground||See above|
The most typical changes in our study area include the felling of trees due to the beginning of construction work and various other changes related to the construction of new buildings and roads. The felling of trees and the construction of new buildings are basically clearly visible in the height data. The same applies to changes in existing buildings, such as their expansion, demolition, and changes in height. The change detection of objects previously classified as “buildings” and “trees” was thus studied on the basis of the two DSMs. Height changes in ground objects were also analyzed because these can reveal new buildings and the extensions of previous ones.
Changes in ground-level objects, such as the construction of new roads, may also be visible in the height data, but it can be expected that intensity changes are the most important cue in this case. Changes in ground-level objects were thus also studied on the basis of the two intensity images, in practice, pseudo-NDVI values calculated from the images. A pseudo-NDVI can be calculated from Optech Titan channels 2 and 3.9 The aim was to detect changes from natural to artificial ground, indicating new roads, parking places, and construction sites. The opposite case (i.e., change from artificial to natural ground) was also included. This is a less typical change in practice, but it can occur, for example, when construction work is finished and newly sowed grass begins to grow on roadsides or in courtyards.
The most typical uninteresting changes detected with our approach include large vehicles, such as buses on roads, and temporary buildings or other constructions not presented on the map. Special rules exploiting the existing map data were developed to recognize these cases. The parameter values used in change detection and the reasoning for the selection of these values are shown in Table 3. Generally, the parameter values were based on general knowledge or analyses and experiments carried out in our training area. For extending the method to larger areas and different seasons, more sophisticated learning approaches could be utilized.
The parameter values used in change detection from multitemporal multispectral ALS data.
|Threshold values for height changes||Less than m or greater than 2.5 m||The expected minimum height for a building or one floor in a building; trees are also normally higher than this|
|Threshold values for pseudo-NDVI changes||Less than or greater than 0.2||According to histogram analysis in the training area, intensity changes in ground classes were normally within the range of to 0.2|
|Threshold value for eliminating height changes with small object size||Buildings are normally larger than this; the threshold value helps to avoid a large number of very small changes in the results|
|Threshold value for eliminating intensity changes with a small object size||Experiments were carried out in the training area to find a suitable value; ground-level objects normally presented in the topographic database are also clearly larger than this|
|Threshold values for separating vehicles from the classes “ground, height increase” and “building, height decrease”: Maximum overlap with buffered road objects from the map, Object size||Experiments and object size measurements were carried out in the training area; it should be noted that moving vehicles can appear as elongated objects in ALS data|
|The criterion for separating temporary buildings from the class “building, height decrease”: The number of overlapping building objects on the map||0||These buildings were not presented on the map; it is thus likely that they were building-like objects or temporary buildings that are not of interest for mapping|
The land cover classification method and the quality of the results have been described by Matikainen et al.10 Segmentation was carried out with the multiresolution segmentation method,65,66 and the parameter values were the same as those that were used in land cover classification. The multiresolution segmentation method is a widely used segmentation method based on bottom-up region merging. It uses an optimization procedure to minimize the average heterogeneity of segments.66 By varying parameters and input data, users can create segments with different sizes and shapes. These segmentation results can form a hierarchical structure, where segments on an upper level consist of one or several segments on a lower level. In our ALS change detection test, we created only one segment level based on the DSM data and one based on the intensity data and these were separated from each other. The segment hierarchy, however, was utilized to create segments inside certain land cover classes and to merge neighboring segments with similar changes before analyzing the sizes of changed areas.
Change detection results
The method development, including the setting of parameter values, was carried out in a subarea (about excluding water). Another subarea (about excluding water and a small land area outside the intensive study area) was used for testing and evaluating the usefulness of the detected changes. The results for this test area are shown in Fig. 4. Figure 5 shows some details of the results, together with the DSMs, intensity images, and land cover classification results from the first date.10 The figures include changes that are potentially of interest for map updating (i.e., changes classified as “vehicles” or “temporary buildings” were excluded).
Continuous Change Monitoring with Sentinel-2 Satellite Images
Change monitoring method
The method developed for continuous change monitoring from Sentinel-2 satellite images consists of preprocessing, segmentation, and change detection. An overview of the method is presented in Fig. 6.
Preprocessing included atmospheric correction and the calculation of an NDVI and NDBI, as described in Sec. 3.3. Segmentation was performed in a hierarchical manner using the multiresolution segmentation algorithm65,66 and appropriate bands and parameter values (see Table 4) in the eCognition software. The basic idea of this hierarchical segmentation method was already described in Sec. 4.1.1. In the Sentinel-2 change detection test, the first and the last image of the time series were used as input data. The first image was first segmented to obtain initial segments on a higher hierarchical level, and a lower hierarchical level of segments was then created inside the initial ones using the last image. In this way, boundaries already detected from the first image remained and new ones only detected from the last image were added. The final segments on the lower level were analyzed in change detection. The boundaries of changed areas should be visible in these segmentation results. Water and agricultural polygons from the topographic database were used as thematic layers in segmentation and excluded from further change analysis. Agricultural areas have their specific behavior considering spectral changes, and they require separate rules for change detection. In our intensive study area covered with reference points, there were only a couple of agricultural areas, and therefore, this topic was not studied in our case. Generally, most of the nonbuilt-up land in Finland is covered by forest, and the study could be later extended to agricultural areas. The parameters for segmentation were selected based on visual analysis and interpretation of segmentation results while varying their values. A total of 1686 segments (polygons) were created. The segmentation results for a subarea can be seen in Fig. 7.
The parameters used in the segmentation of Sentinel-2 images for change detection.
|Parameter||Segmentation level I||Segmentation level II|
|Input level||Pixels||Segmentation level I|
|Source image||August 17, 2015||June 29, 2016|
|Image bands in use||Red, green, blue, near-infrared (each with equal weight)||Red, green, blue, near-infrared (each with equal weight)|
|Thematic information||Water and agricultural areas||Water and agricultural areas|
To develop rules for the change detection, training segments with no changes during the time frame were analyzed. There are natural changes in reflectance values during the year, such as the normal, expected changes in vegetation, and the purpose of analyzing the training segments was to obtain an understanding of these changes. The basic assumption was that changes in interest when mapping are larger than these normal seasonal changes. Training segments were selected on the basis of reference points that remained unchanged (see Sec. 3.1). Every segment that had at least one reference point within its boundaries was selected as a training segment. The total number of training segments was 100, and all of them were considered unchanged.
For each of the training segments , the median value was calculated from NDVI and NDBI bands for each date to represent the segment reflectance. Median values were used instead of mean values in order to avoid the influence of outliers and averaging in the case of mixed segments. The differences of median values between every two consecutive dates were then calculated for the segments (). For each time fragment (two consecutive dates), the minimum and maximum segment differences were found, and these were the threshold values for change detection rules:
Time series statistics, including the average, minimum, and maximum values of the median differences for each time fragment, are shown in Fig. 8. Change detection for a given segment was based on the threshold values:
An additional rule applicable to an NDVI was based on the assumption that the NDVI value should decrease if a change from a vegetation area to a built-up area occurs. This was the change we were mostly interested in. In our case, this additional condition was met by only using the first inequality of the criterion above. This was possible because all the minimum threshold values () were negative, that is to say, they were related to an NDVI decrease, and all the maximum threshold values () were positive, that is to say, they were related to an NDVI increase.
Change monitoring results
After having set the rules using the training segments, the median value of the NDVI and NDBI was calculated and change detection was performed for all 1686 segments. The change detection results based on the NDVI decrease rule are shown in Fig. 9. The results shown in the figure are a combination of all the changes detected within the selected time span (August 2015 to June 2016). A total of 432 polygons were detected to have changed.
Detailed 3-D Mapping with MLS
Methodology for urban scene classification
Our method for classifying MLS data consists of three steps: data preprocessing, ground removal, and classification.64 In the first phase, we remove noise and partition data into blocks. Then, ground points are removed. Lastly, above-ground points are clustered using connected component analysis and these clustered components are classified into different categories. The workflow of the method is shown in Fig. 10.
On occasion, the scanner produces noise, in other words, points in the air, which are mainly due to multiple reflections. Long-distance echoes, that is to say, objects at a longer range than the given measurement rate permits, are also not solved for the correct range. Presumably, some misinterpretations are caused by scattering from aerosols (dust) and other scanner online echo detection faults (e.g., direct or reflected sunlight entering the scanner optics). The noise causes mistakes in the following stages if not treated properly. Because of the low intensity of these noisy points, a low-intensity threshold was used to remove them. In this paper, this threshold was empirically defined to be 1200.
MLS datasets can be very large and the processing can be computationally expensive if we process the whole dataset in one time. Additionally, there are many urban objects that are scanned multiple times, which will cause shifts of objects. In order to solve these problems, the data are partitioned into blocks based on trajectory points and their sequenced time. Then, these generated, sequenced blocks are cropped into a regular size along the trajectory direction, 30 m along the trajectory, and 60 m across the trajectory.
Ground point removal
Compared with the points of other classes, ground points have smaller height differences locally. Therefore, ground points are extracted by filtering out points with large height differences in their local neighborhood. Here, the neighborhood is defined to include the 100 nearest neighboring points of a certain point. The threshold for a large height difference was set to be 0.15 m. In addition, we filtered out points that were too high compared with the height of a trajectory point at the same time instant. Ground points are removed and the rest of the points remain for the next phases of processing.
Clustering and classification
After ground points are removed, a connected component analysis67 is carried out to obtain above-ground components. The distance threshold in the connected component analysis was set to be 0.5 m. Rules were defined and applied to classify the components into three classes: “buildings,” “trees,” and “pole-like road furniture.” Building and nonbuilding components are first detected in the following way. From every above-ground component, planar patches are extracted using surface growing. If there is at least one detected planar patch satisfying the following conditions, the above-ground component is classified as a building component: (1) the area of the patch should be large enough (the threshold was set to be ) (2) the normal of the detected planar patch should be parallel to a horizontal direction, that is, the angle between the normal of the detected plane and the horizontal plane should be less than 5 deg; and (3) the patch should mostly contain points with a prominent planarity, that is, points whose local neighborhood are situated approximately on a plane. The planarity for a point is calculated based on the eigenvalues of the covariance matrix of the points in the neighborhood of using the equation . The point is classified as prominently planar if planarity is larger than linearity and scatterness . In a building component, the ratio of points with a prominent planarity had to be larger than 0.65. Apart from the conditions 1 to 3 above, the horizontal distance between at least one pair of points in a detected vertical planar patch should be larger than 2.5 m in order to remove narrow nonbuilding segments.
Next, trees are extracted from the nonbuilding components. One sent laser pulse can reflect multiple times from vegetation because of gaps between small branches or leaves. In contrast, bare ground or a building facade can only give a single return for one transmitted pulse. In addition, trees are typically higher than other types of vegetation. Therefore, a nonbuilding component is classified as a tree if the following two conditions are met: (1) the percentage of multiple returns in the component is high enough () and (2) the highest point of the component is 2 m or higher from ground level.
Last, pole-like road furniture are recognized from the remaining nonbuilding and nontree components. This is performed by first applying a slice analyzing method and then a cylinder mask method for each component. We first cut the component into slices in a vertical direction. If the horizontal diameter of any slice is larger than 0.5 m, the component is discarded; otherwise, it is considered as a pole candidate. Next, for each pole candidate, we calculate the shift of center points and the difference of the diameters between every two adjacent slices. If the shift of center points or difference in the diameter between any two adjacent slices is too large (), the slices are removed from the pole candidate. If the number of remaining slices is lower than 3, the component is removed from the pole candidates. For the remaining pole candidates, a cylinder mask47 is used to perform a final check regarding if the candidate is pole-like road furniture. A detailed description of a cylinder mask can be found in Ref. 47; here, we use the center points of slices to define the cylinder axis. The main difference to the method of Ref. 47 is that we use unorganized point clouds, whereas in Ref. 47, the scan line information, also called profile or row and column information, is used. With the scan line information, neighborhood searches are faster, but such information is not always available, and thus, the method used in this paper is more generic.
Figure 11 shows objects of different classes detected from the MLS data. According to visual evaluation, most objects were correctly classified. Most misdetections come from connected objects such as trees connected with buildings. This is the main drawback of object-based classification. Some tree trunks with only a few branches were misclassified as pole-like objects. It is difficult to distinguish them from pole-like road furniture. Except for the classified objects, the rest of the objects remained unclassified with the method and were removed. For example, there were some trees that were visible in the original data but not in the final results because they were not recognized as trees with the set of classification rules in use.
Analysis of the Experimental Results
Potential of Multitemporal Multispectral ALS Data for Change Detection and Map Updating
When changes detected from the multitemporal multispectral ALS data are combined with map data, an illustrative presentation of the changes is obtained (Fig. 4). It can be seen that many changes related to construction work in the study area were detected. The most prominent of these was the felling of several forest areas before the beginning of construction work. These changes were classified as “tree, height decrease.” Several construction sites previously classified as “ground” are also visible in the results because the height of these areas has increased or their pseudo-NDVI value has decreased. All these changes are obviously interesting for map updating, although their direct linking to map objects is not straightforward. Buildings are normally presented on the map in their finalized form, not as construction sites. Forest areas are not presented as polygon objects on the present Finnish topographic maps because they are considered to belong to the background. Larger clear-cut areas, however, are presented. Many small changes are also visible in the results. The most typical of these is the felling of individual trees. Such changes are not needed for nationwide topographic mapping, but they are interesting for detailed city models that also represent trees. The short time interval between data acquisitions also affected the nature of the changes detected in this study, for example, there were many construction sites instead of finalized buildings.
To better understand the usefulness and nature of the changes detected from multitemporal multispectral ALS data, two evaluations were carried out. The first one was a numerical one and its objective was to test how well new objects interesting to topographic mapping were detected. It was based on the reference points collected on new buildings, new roads, and new clear-cut forest areas. The results are summarized in Table 5. They show that 95% of the changes were detected. Two buildings points and five road points were not detected as changes. All the clear-cut points were detected. The classification of the detected changes varied depending on the land cover class of the segments on the first date.
A comparison of the changes detected from multitemporal multispectral ALS data to reference points.
|The class of the reference point||Detected as changed||Not detected as changed||Sum||Detected change class (number of points)|
|New building||32 (94.1%)||2 (5.9%)||34||Tree, height increase (1); tree, height decrease (7); ground, height increase (24)|
|New road||51 (91.1%)||5 (8.9%)||56||Tree, height decrease (44); natural ground, NDVI decrease (7)|
|New clear-cut forest area||57 (100%)||0||57||Tree, height decrease (57)|
|All||140 (95.2%)||7 (4.8%)||147|
The second evaluation was a visual one, and the objective was to understand the nature of changes that were detected by direct automatic comparison of the multitemporal datasets. The evaluation was based on visual inspection of the two multispectral ALS datasets with some help from existing map data. The information given by this evaluation could be used in further research and development. Basically, the detected differences can include changes that are interesting to mapping but also many other changes, which, from the viewpoint of map updating, could be considered as false detections. Categorical classification of the detected changes as desired and undesired ones was not included in this study. This topic is closely related to the definition of mapping rules for different objects and requires special attention if the method is further developed for operational use. A total of 10 changes were picked from each change class in the results, including vehicles and temporary buildings. If the number of changes in a class was smaller than 10, each change was evaluated. The changes were picked evenly from attribute tables listing the detected DSM and intensity changes. The results are reported in Table 6.
A visual evaluation of changes detected from multitemporal multispectral ALS data.
|Change class (number of changes evaluated / total number of changes in the class)||Visual interpretation of detected changes|
|Building, height increase (9/9)||Six changes in construction sites, typically in buildings under construction|
|Three changes in the industrial area; two of these were related to construction or structural changes visible in the data and one was an unclear change|
|Building, height decrease (8/8)||One demolished building|
|Four changes in the industrial area; these were visible changes in building-like objects|
|Three changes related to minor geometric differences in data from besides high buildingsa|
|Tree, height increase (10/12)||Seven changes with trees in the first dataset and a building under construction or a crane in the second one|
|Two changes inside a building that was under construction in the first dataset and had been partly misclassified as a tree in land cover classification|
|One change with trees in both datasets; the change was probably due to minor geometric differencesa|
|Tree, height decrease (10/349)||Eight changes most probably related to the felling of trees, the cutting of tree branches or trees falling (it is sometimes difficult to define exactly)|
|Two changes, probably related to minor geometric differencesa|
|Ground, height increase (10/133)||Six changes related to new buildings or building parts|
|Two changes in the industrial area; one of these was probably a large vehicle or a container and, in the other case, there are some unknown objects in the second dataset|
|One temporary building-like object in a construction site|
|One new tree|
|Ground, height decrease (9/9)||Eight changes in construction sites or other similar areas|
|One unknown change in a courtyard|
|Vehicle (10/38)||Nine vehicles are present in one dataset but not in the other|
|One change in a new built-up area under construction that is not a vehicle|
|Temporary building (10/50)||Seven temporary buildings or building-like objects (such as large vehicles) are in the first dataset but not in the second one|
|There is one change with a power line that is in the first dataset but not in the second one|
|One change due to minor geometric differencesa|
|One change due to the cutting of trees on a forest edge that had been misclassified in the land cover classification|
|Natural ground, NDVI decrease (10/28)||Nine changes from vegetated ground to nonvegetated ground or other objects|
|One unknown change in vegetation cover|
|Artificial ground, NDVI increase (10/10)||Seven changes, probably new vegetation (it is sometimes difficult to determine exactly)|
|One new building and one construction site with changes in materials and reflectance|
|One unknown change|
A visual evaluation of the detected changes (Table 6) showed that most of them were real changes that had occurred in the area. However, the changes were not always important for mapping or the importance was difficult to determine. For example, many small changes had occurred in the industrial area. In an industrial area, there are containers, large vehicles, and other large manmade objects, and changes in such objects are visible in the change detection results. They can be difficult to separate from small buildings in aerial data, even visually. Seven out of the 96 evaluated changes were interpreted as false detections due to minor geometric differences in the data from two dates. For example, in some cases, the boundary of a large building was in slightly different positions in the two DSMs. This was probably related to the lack of laser points besides an obscuring building. In some cases, there was a real change in the area but the classification of the change was incorrect due to a misclassification of the object from the first ALS dataset. For example, parts of a building that was under construction in both datasets had been originally misclassified as tree, and these parts were now assigned to the class “tree, height increase.”
Some new buildings were present in both ALS datasets although they were still missing from the map data used in the study. These buildings were not detected as changes and they are missing from the final results, that is to say, they are presented neither as map buildings nor as changes. Some examples of these can be seen in Fig. 5(a). In a real mapping situation, it would be important that the contents of the map correspond exactly to the situation in the first dataset.
Potential of Sentinel-2 Satellite Images for Continuous Change Monitoring
From the viewpoint of data integration in map updating, an important question related to the Sentinel-2 monitoring process is its ability to map changes that are meaningful for mapping and appear in the same areas as those detected from higher-resolution datasets. In our study, this was evaluated by comparing the results with the multispectral ALS classifications from the beginning and ending of the monitoring period. Reference changes were derived from the land cover classifications of the two multispectral ALS datasets from August 2015 and June 2016. The ALS results originally had six land cover classes, which were combined into three classes: “built-up area” (including the original classes “asphalt,” “gravel,” and “building”), “low vegetation” (originally “low vegetation” and “rocky area”), and “tree” (“tree”). The three classes were the same as those that were used to select unchanged reference points for the Sentinel analysis. Cross-classification of the August 2015 and June 2016 results was carried out, and a reference change map with changed and unchanged areas was obtained. Water areas and areas outside the ALS data coverage were excluded. Three different Sentinel-2 change detection results created with the three different change rules (NDVI change, NDBI change, and NDVI decrease) were first evaluated by using the entire ALS data coverage (i.e., also including the areas of training segments). The best result was then also evaluated with the training segments excluded.
In practice, the Sentinel-2 segment vectors were first converted into raster format with the same pixel size as the ALS classifications (0.2 m). Segments with more than 90% of their area inside the ALS data coverage were selected for further analyses. This included a total of 223 segments. For each segment, the percentage of changed pixels in the ALS change map was calculated. The segments were then divided into four change categories using 25% intervals based on this reference change percentage. From the Sentinel-2 change detection results, the number of changed segments in each of these categories was calculated and divided by the total number of segments in the reference change category. The results are presented in Table 7. Among the three different change detection rules, the one with an NDVI decrease as the change criterion gave the best correspondence with the reference results. The criterion based on NDBI change clearly gave the lowest correspondence. The results based on the NDVI decrease criterion were thus also evaluated with the training segments excluded. There were more changes in the western part of the study area outside the training points (see Fig. 9), which is reflected in these results. Overall, the correspondence between the changes detected with the NDVI decrease rule and the ALS land cover classifications was good. In the highest change category (75% to 100% of the segment changed according to the reference data), 100% of the segments were detected to have changed. In the lowest category (0% to 25%), the percentage was only 9%.
A comparison of Sentinel-2 change detection results with reference change categories derived from two multispectral ALS classifications. The best results were obtained with the NDVI decrease rule, and they are highlighted bold in the table.
|Reference||Sentinel-2 change detection results Rule NDVI change/NDBI change/NDVI decrease/NDVI decrease, training segments excluded||Total number of segments All/training segments excluded||Percentage of segments detected as “changed” (%)|
|The change percentage of a segment from multispectral ALS classifications||Detected as “changed” (number of segments)||Detected as “not changed” (number of segments)|
A more detailed analysis of the reference classifications showed that the most common changes inside the changed Sentinel-2 segments obtained with the NDVI decrease rule were a change from tree to built-up area (39.3% of changed pixels), a change from tree to low vegetation (34.9%), and a change from low vegetation to built-up area (18.7%). It can thus be seen that most of the changes detected from the Sentinel data were clearly related to construction. Visual analysis of the detected changes and reference changes was also carried out. The overall accuracy of the original ALS classifications was as high as 96%.57 However, there were some false positive changes in the reference change map due to classification errors (e.g., noise in forested areas, sports fields with artificial cover, nonhomogeneous gravel areas). In Sentinel-2 images, small image shifts (1 to 2 pixels) and some special vegetation types (e.g., waterfront vegetation) caused false-positive changes. Such shifts occurred in the early stage of Sentinel-2A image acquisition, and it is expected that this effect will decrease in the future. A suburban area is a mixture of different built-up and vegetated surfaces, and for small segments, even small shifts can cause remarkable changes in reflectance. Also, the small temporal difference between the Sentinel-2 and multispectral ALS acquisitions caused errors in active construction sites.
Despite the minor issues discussed above, we can conclude that the presented Sentinel-2 change monitoring method is feasible for detecting changes in a growing built-up area. Being fully automated, this method allows an easy and affordable way of performing continuous change detection over large areas. In the future, further development of the method can be carried out, for example, to better analyze special cases, such as waterfront vegetation and to extend the analyses to agricultural areas.
Potential of MLS Classification Results to Complement ALS Data in Mapping Urban Areas
The ability of the classified MLS point cloud to complement ALS data and provide finer details for mapping an urban area was evaluated by visual interpretation. The MLS classification test did not involve change detection. The comparison was thus based on the ALS data from the second date and concentrated on the appearance of buildings and poles in ALS and MLS data. These are important objects for detailed city models of urban areas.
Figure 12 shows a comparison of ALS and MLS data in a small area of active construction. The upper row shows a top view of the multispectral ALS point cloud and the MLS point cloud, and the lower row shows the results from building extraction. The buildings extracted from the ALS data are based on the classification of the June 2016 multispectral ALS data in raster format.57 Figure 13 presents a side view of one building in the ALS data, classified MLS data, and combined ALS and MLS data. A comparison for poles is presented in Fig. 14.
The comparisons show the complementary nature of the MLS and ALS point clouds. Building roofs are well presented in the ALS data, and the number of reflections from building walls is small. In MLS data, the situation is quite the opposite, and wall structures are very well presented. A combination of these two datasets would allow accurate modeling of both building roofs and facades. ALS and MLS data also provide two alternative ways for defining the outlines of 2-D buildings, ALS according to the roof outlines, and MLS according to the position of the walls. This might be useful, especially if map data present the bases of the buildings instead of roof edges, as for example the NLS Topographic Database.68 In such cases, building outlines derived from MLS data could correspond to maps better than those derived from ALS data. However, MLS data acquired from roads also have deficiencies when considering building mapping, such as the lack of courtyards.
Poles are another interesting example, where MLS data could be used to complement ALS data. The tops of poles such as street lamps are often visible in dense ALS data, but the sides of the poles are not visible. In the MLS data, the poles are clearly visible and they can be extracted. Poles are not needed for nationwide topographic mapping, but they are interesting for detailed city mapping and modeling, street lighting design, and optimization,69 and also for power line monitoring.70
Discussion on Multisource Data Integration in Future Map Updating
As discussed in Sec. 1, the most important remotely sensed datasets currently used in nationwide mapping are multispectral aerial images and single-channel ALS data. In the ALS data, the geometric information is clearly more important and more widely used than the intensity information. The advantages, challenges, and main applications of aerial images and single-channel ALS data are summarized in Table 8. The information is based on the authors’ general knowledge of the field. Table 9 lists corresponding information for multispectral ALS data, Sentinel-2 satellite images, and MLS data. In the case of these new datasets, the advantages and challenges are evaluated in comparison to the conventional techniques.
The advantages, challenges, and main applications of the remote sensing techniques currently used in nationwide mapping.
|Data source||Advantages||Challenges||Main applications|
|Aerial images||A high-flying altitude and the possibility to cover large areas effectively||Cannot be acquired through clouds or in darkness||The mapping of the terrain surface, land cover and 3-D objects, often based on visual interpretation|
|Multispectral data with very high spatial resolution||Reflectance is dependent on illumination and viewing geometries that are more complex than in ALS|
|Stereo images allow height measurements||Shadows complicate interpretation, especially with automated methods|
|One cannot “see” objects under trees and a DTM is difficult to derive|
|Single-channel ALS data||3-D information and intensity in one channel, directly available||The intensity information is greyscale||The creation of DTMs|
|Can be acquired in darkness||National data can be composed of data from several different scanners with different approaches of intensity measurement and calibration||The mapping of 3-D objects in cities and in forests|
|Can see terrain and objects under trees, and a high-quality DTM can be obtained||High point density requires a lower flying altitude, reducing efficiency; new ALS techniques, however, can alleviate this problem||Corridor mapping applications|
|Well-suited for automated analyses|
The advantages, challenges, and potential applications of multispectral ALS data, Sentinel-2 satellite images, and MLS data for nationwide mapping. In this case, the advantages and challenges are evaluated in comparison to the conventional techniques (Table 8), unless otherwise mentioned.
|Data source||Advantages||Challenges||Potential applications|
|Multispectral ALS data||3-D information and intensity in three channels||New technology – not yet widely available||A substitute or an addition to conventional techniques in the mapping of terrain surface, land cover and 3-D objects|
|High potential for automated classification and change detection||A relatively low flying altitude is required, especially for the green channel, which reduces efficiency and increases costs||Detailed characterization and classification of 3-D objects and ground-level classes|
|Change detection can be performed on the basis of spectral data, in addition to geometry||Each channel produces a separate point cloud||The creation of low-resolution virtual reality applications for large areas by visualizing the data directly as a point cloud|
|Current sensor technology does not provide homogeneous point spacing|
|The artefacts of intensity data|
|Sentinel-2 satellite images||Freely available data||Relatively low spatial resolution compared to aerial images||Continuous monitoring and change detection over large areas, which could be used to focus more detailed data acquisitions and map updating work; rapidly changing areas could be mapped more frequently|
|Regular data acquisitions||Relatively low spatial resolution compared to the contents of topographic databases|
|13 spectral channels||Cloudiness may hamper image acquisition|
|More stable illumination and viewing geometry than in aerial images|
|High potential for automated classification and change detection|
|Higher spatial resolution than in previous, freely available satellite images, allowing smaller changes to be mapped and new change-based applications|
|MLS data||Very detailed 3-D information and intensity in one or two channels||Typically, MLS does not provide full coverage of areas||Highly detailed 3-D mapping of objects|
|High flexibility in data acquisition||Data acquisition over large areas is laborious||Complementing ALS data|
|High-quality data can be acquired based on needs, e.g., more complex scenes can be collected with double density||The point density of the data is not homogeneous||Replacing aerial data in small areas, for example, updating changed areas detected by satellite image change detection – this is especially suited for urban areas|
|Matching data of various strips can be challenging when working in conditions of poor GNSS coverage||The collection of reference data to support ALS and satellite image analyses and evaluate their quality (for land cover or other applications)|
|Multispectral MLS is not yet available|
Multispectral ALS data have a similar spatial resolution compared to current operational techniques and, in the future, they could be used to complement, or possibly even replace, these in the mapping of terrain surface, land cover, and 3-D objects. The main benefits compared to aerial images are the lack of shadows and the independence of external illumination conditions, which make automated analyses easier to develop. Multitemporal multispectral ALS data allow direct change detection between height and intensity data from two dates. The change detection results obtained in the present study would probably be a useful aid in manual map updating work by showing where changes have occurred in the data. For automated mapping of the changed objects and for filtering out irrelevant changes, further analysis methods need to be developed. Multispectral ALS data could also be used in various low-resolution virtual reality applications, such as digital twin applications, by visualizing the data directly as a point cloud. The challenges of current sensor technology include nonhomogeneous point spacing and artifacts in intensity data. The behavior and calibration of the intensity data are important topics for further investigation in order to fully understand and utilize multitemporal multispectral data. A practical limitation of using multispectral ALS for nationwide mapping is also the low flying altitude required by the current systems. This makes the technique less effective than single-channel ALS and much less effective than aerial imaging. Generally, however, ALS technology is likely to develop in the future, and techniques such as single-photon lidar (see, e.g., Ref. 71) are already available for the effective scanning of large areas with one channel.
Sentinel-2 satellite images enable an easy and affordable way of performing continuous change detection over large areas. Due to the constant imaging geometry, sun-synchronous orbit and processing operations, the data are well suited for repetitive automated analyses. The spatial resolution is also higher than in previous, freely available satellite images, which makes new change detection applications possible. It is still not sufficient for the mapping of topographic objects, such as buildings or roads, but changes can be detected approximately at city block level. The method could provide useful information when deciding when and where more detailed data acquisitions should be carried out. This could possibly save the costs of aerial data acquisitions and allow for more frequent mapping of rapidly changing areas. Changes automatically detected from Sentinel-2 satellite images could also give hints on the areas that operators should inspect more carefully when doing practical updating work from higher resolution datasets. However, to find the smallest changes, such as individual new buildings inside an old built-up area, visual inspection, or automated analysis of higher resolution data is also needed.
Considering satellite images, a clear problem in our study area was the cloudiness. Within about 1 year, only nine cloud-free images were obtained. This would probably be sufficient for the mapping application discussed in this article, but more real-time change detection applications are hard to achieve in Finland or other areas with similar weather conditions. Having both Sentinel-2 satellites operational in orbit increases the possibility of getting useful images. If more frequent cloud-free images were available, the robustness of the results would also increase. The time gap between consecutive images would be smaller and the index value differences in unchanged areas would also be smaller, which would make real changes more obvious. In addition, it should be noted that in the northern parts of the world, low sun angles in wintertime can also have a significant influence on image data and this should be considered when using interseasonal data. However, Sentinel-2 data are not even available for this region in the midwinter period from December to mid-January.
MLS data can be used for detailed mapping and 3-D modeling of objects. In particular, they are feasible for the mapping of objects along roads in urban areas. As shown in the results, the technique is also well suited for complementing ALS data in regard to details such as building facades and poles. Occasionally, MLS might even be used to replace aerial data in map updating and other applications due to its higher flexibility. Satellite image monitoring could trigger the use of MLS. When local changes are detected in satellite images, an MLS system could be used to rapidly acquire detailed data from those areas. The data could be used to check and update buildings and other objects in the street environment. More extensive mapping could be done at a few years’ intervals following aerial data acquisitions. The limitation of MLS data, however, is the visibility of objects further away from roads (see also Ref. 23). Even the complete outlining of buildings may be challenging if the back of the building is not visible to roads. Occlusion by other objects such as parked cars can also be a significant problem in MLS data.24 It can be expected that regular acquisitions of aerial data are thus also important. In some applications, such as pole mapping in a city, the combination of satellite image monitoring to detect areas of interest (e.g., new city blocks) and MLS-based mapping of the objects could be a major approach. However, more information is needed on the potential of the combined use of satellite images and MLS data for various applications. The results should be compared to those obtained by using current airborne datasets or multispectral ALS data. MLS data could also be used as reference data for ALS and satellite image classifications both to help in the analysis and method development and to evaluate the quality of the classification results.
To summarize our study, in Fig. 15, we depict a possible future map updating process. The process is an adaptive one, and it could exploit different remote sensing techniques in a flexible way. Ideally, this could lead to an increasing level of automation and updating frequency. The central ideas of the process are the following:
1. Openly available satellite images, such as Sentinel-2, could be used for continuous monitoring and the rough detection of changes. This information could then be used to define areas for more detailed data acquisitions with ALS and MLS techniques.
2. Multispectral ALS data could be used instead of, or in addition to, conventional aerial images in order to achieve a higher level of automation. When multitemporal multispectral ALS data are available, direct comparison between height and intensity information becomes possible and helps to map changes in the area.
3. In selected areas, more detailed mapping with ground-level systems could also be used.
Generally, the diversity of remote sensing techniques is increasing (see, e.g., Ref. 14), the interest in noncentralized mapping solutions is also increasing, and it is likely that multisource approaches will become more popular in the future. In Finland, a new National Topographic Database is under preparation.1 In maintaining the new database, topographic data obtained from municipalities and cooperation with other organizations will have important roles. There are also techniques not discussed in this article that can become important in practical work. One of these is unmanned aerial vehicles (UAVs), which could be used instead of aerial and ground-based techniques to some extent. Both aerial imaging and laser scanning are possible from UAVs. The advantages of UAVs include higher flexibility compared to conventional aerial systems and easier coverage of areas compared to ground-based techniques. For example, Nevalainen et al.72 discussed a test in which UAV imagery was acquired and processed in order to produce 3-D topographic data of an urban area. Taking all these developments into account, it is important to find optimal approaches for utilizing multisource data with different levels of detail and to develop practical methods for integrated mapping processes. As an example from forestry, data assimilation has been suggested as an approach to combine multisource, remotely sensed data and information obtained from growth models in order to obtain better forest inventory data.13 In the field of mapping, changes of interest have a different and more abrupt nature. However, the use of statistical methods to combine new types of multisource data and to take their various information contents into account should also be developed for map updating.
Three remote sensing techniques with different levels of detail were analyzed considering their potential for nationwide mapping, and their possible integration in a future map updating process was discussed. First, automated change detection from multitemporal multispectral ALS was developed and tested for the first time. The results show that direct comparisons between height and intensity data from different dates reveal even small changes related to the development of a suburban area. A major challenge in future work is to link the changes with objects that are of interest for mapping and to separate essential changes from unessential ones, especially with automated methods. In the future, multispectral ALS data could have a central role in nationwide mapping. Second, a method for continuous change monitoring from Sentinel-2 satellite images was developed and tested. The results were evaluated on the basis of multispectral ALS data classifications from two dates, and a good correspondence was found. The major application potential of satellite image change detection could be the continuous monitoring of large areas in order to show where important changes have occurred and where more detailed data acquisitions are needed. The third technique tested in the study was MLS. A high-density point cloud was acquired and classified automatically into four classes. Comparisons with the multispectral ALS point cloud showed the complementary nature of these two datasets. The MLS can be used in detailed city modeling and when adding important features, such as building facades and poles, to an ALS point cloud. Occasionally and for some specific applications, continuous monitoring from satellite images and flexible data acquisition by MLS might also replace aerial data acquisitions. This, however, requires further studies. Generally, the diversity of remote sensing techniques is increasing, which provides flexibility in the selection of data sources. Ideally, this could increase the level of automation in nationwide mapping and the frequency of its updates, and it could help to improve the contents of topographic databases. To fully benefit from this potential, it is important to find optimal approaches for utilizing multisource data with different levels of detail and developing practical methods for integrated mapping processes.
The authors wish to thank Harri Kaartinen for participating in MLS data acquisition, TerraTec Oy for co-operation in Optech Titan data acquisition, and the NLS Department of Production and the City of Espoo for map data. The financial support from the Academy of Finland (through the projects “Integration of Large Multisource Point Cloud and Image Datasets for Adaptive Map Updating” (project decision number 295047), “Centre of Excellence in Laser Scanning Research” (307362), “Urban Morphology and Atmospheric Boundary Layer Modeling” (277734), and Business Finland (through the Co-creation project “EODIE—Earth Observation Data Information Extractor” (5332/31/2018) is gratefully acknowledged.
Leena Matikainen is working as a specialist research scientist at the Finnish Geospatial Research Institute FGI. She received her MSc (Tech.), LicSc (Tech.), and DSc (Tech.) degrees in 1995, 2005, and 2012, respectively, from the Helsinki University of Technology/Aalto University in the field of surveying/remote sensing. Her research interests include automated interpretation of laser scanner and image data for mapping and map updating applications. She is the author of more than 40 scientific publications.
Miloš Pandžić works as a junior researcher at the BioSense Institute, University of Novi Sad. He is a PhD student at the Faculty of Civil Engineering, University of Belgrade, where he also received his BSc and MSc degrees in 2013 and 2016, respectively. His research interests include Earth observation, remote sensing, image processing, and data analytics.
Fashuai Li works as a joint postdoc of Wuhan Geomatic Institute and Wuhan University. He received his BSc and MSc degrees in geodesy at the China University of Mining and Technology (Beijing), respectively, in 2011 and 2014. He then earned his PhD degree in photogrammetry and computer vision at the University of Twente in 2019. In 2017, he visited FGI as a research scientist. His research focuses on the interpretation of mobile laser scanning data.
Kirsi Karila is a specialist research scientist at the FGI. She received her MSc (Tech.) and LicSc (Tech.) degrees from the Helsinki University of Technology in 2004 and 2008, respectively. Her current research interests include mapping and monitoring applications using radar, lidar, and optical remote sensing data sets.
Juha Hyyppä is a professor of remote sensing and photogrammetry and director of FGI remote sensing. He is also PI of the Academy of Finland Centre of Excellence in Laser Scanning Research. He is interested in new laser scanning technologies, drone and mobile mapping, autonomous driving, and using them in various applications. He has more than 200 Web of Science articles with an H-index of 45.
Paula Litkey (LicSc Tech.) is a senior research scientist with the Department of Remote Sensing and Photogrammetry, FGI. Her research interests include computational methods and laser scanning.
Antero Kukko is a research professor of mobile mapping at the FGI. He received his PhD degree in photogrammetry from Aalto University in 2013. He is currently leading the Mobile Laser Scanning and Autonomous Driving Group. He is also an adjunct professor at the Department of Built Environment, Aalto University, and Co-PI of the Centre of Excellence in Laser Scanning Research. He has authored or coauthored over 100 scientific journal publications.
Matti Lehtomäki (MSc) is a research scientist with the Department of Remote Sensing and Photogrammetry, FGI. He is also studying toward a PhD degree at Aalto University. His research interests include computational methods, algorithm development, laser scanning, and computer vision.
Mika Karjalainen received his MSc, LicSc, and DSc (Tech.) degrees from Aalto University (formerly Helsinki University of Technology), School of Science and Technology, Espoo, Finland, in 1997, 2008, and 2010, respectively. He is currently working as a research manager in the Department of Remote Sensing and Photogrammetry of the FGI. His research interests include applications of satellite images, SAR interferometry, radargrammetry, and digital photogrammetry.
Eetu Puttonen is a research manager and a research group leader at the FGI, National Land Survey of Finland. He received his MSc and PhD degrees in physics from the University of Helsinki in 2007 and 2012, respectively. His current research interests include novel laser scanning techniques, automated map updating applications, and related algorithm development. He is an Academy of Finland Research Fellow and a member of the Centre of Excellence in Laser Scanning Research.