KR20230140832A - UAV and Artificial intelligence-based urban spatial information mapping Method and Apparatus - Google Patents

Abstract

A method and device for mapping urban spatial information using drones and artificial intelligence are presented. The urban spatial information mapping method using drones and artificial intelligence proposed in the present invention includes the steps of producing a multispectral image, a vegetation index image, and a terrain image through a sensor of an unmanned aerial vehicle, the produced multispectral image, a vegetation index image, and It includes the step of confirming images and variables for random forest-based land cover classification using terrain images, and the step of verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the identified images and variables.

Description

Urban spatial information mapping method and apparatus using drones and artificial intelligence {UAV and Artificial intelligence-based urban spatial information mapping Method and Apparatus}

The present invention relates to a method and device for mapping urban spatial information using drones and artificial intelligence.

A land cover map is a map of the physical state of the land surface according to a certain classification system and is basic data for understanding the current status of a region. At the city level, it is used as a scientific basis for urban planning such as land suitability, environmental evaluation, and urban regeneration. At the academic level, it is used as basic research data for various purposes such as thermal environment analysis, air flow simulation, and ecosystem investigation. In Korea, the government is updating the land cover map across the country every 1 to 10 years depending on budget security. However, in the process of analyzing data acquired using satellite images and digital aerial images, there is a problem of inconsistency in the update cycle and cover data, and a minimum classification standard is set to include areas under 2500 m ² in the surrounding classification items. There are limitations in terms of low precision for small areas, so a land cover construction plan with high productivity and accuracy is needed.

Recently, with the development of UAV and on-board sensor technology, spectroscopic accuracy and positioning accuracy have increased, and it is being used in fields that require accuracy such as remote sensing and aerial surveying. In addition, in addition to the general RGB camera, multispectral and thermal infrared sensors can be attached to one sensor, such as combining a VHR (Very High Resolution) RGB sensor with a 4-band multispectral sensor or a 4-band multispectral sensor with a thermal infrared sensor. With the possible emergence of UAVs, various images can be acquired through a single flight. In addition, UAV images have fewer temporal and spatial constraints compared to satellite and aerial images and have the advantage of being easy to acquire high-resolution images.

In recent prior art, high spatial resolution UAV images are used to classify and analyze land cover in a specific area. However, high-resolution images have the characteristic of high variability in spectral values for the same object. This is because the higher the spatial resolution of an image, the more pixels are needed to express one object, and the pixel value contains the image structure and background information associated with the object. Therefore, when pixel-based image analysis methods are used for high-resolution UAV images, the accuracy of the acquired images may be significantly reduced due to the variability of spectral values because the number of pixels in the acquired images is often tens of millions or more.

Recently, object-based image analysis has emerged as a new paradigm that can control the variability of spectral values, replacing pixel-based approaches that only use spectral information in images. Object-based image analysis is different from pixel-based approaches that classify each pixel separately because it groups homogeneous and consecutive pixels and creates and classifies objects to perform image classification. Therefore, this object-based approach can reduce the variability of spectral values caused by empty space, shadows, textures, etc. In addition, it is possible to comprehensively consider information such as accumulation, shape, and texture, and has the advantage of improving classification accuracy by integrating and utilizing various geographic information such as vector data other than image data.

Using object-based image analysis, many researchers are using data collected from UAVs to map land cover in small and medium-sized areas, along with various remote sensing and machine learning methods.

In the prior art, a fixed-wing UAV was used to acquire 4-band multispectral images and achieve 95% accuracy for 5 items (banana land, bare land, building, other vegetation, and water). did.

In another prior art, RGB and NIR cameras were mounted on a rotary-wing UAV to photograph a river area in the area of 200 m x 100 m to map land cover, and four items (grass, water, trees, road) An accuracy of 78% was achieved.

In another conventional technology, UAV-based RGB images are used to set 7 items (building, grass, road, trees, water, soil, shadow) for residential areas and compare image filters that can be used for object-based image analysis. did. In this way, previous research uses object-based image analysis to classify or detect specific objects for various purposes.

In another prior art, a UAV platform was constructed to predict corn yield using a UAV-based RGB sensor. Research on the use of UAVs in the environmental field is being conducted in various regions, but the use of land cover classification required for urban environment analysis is insufficient.

[1] USGS, https://www.usgs.gov/landsat-missions/landsat-soil-adjusted-vegetation-index (archived on 22th December 2021) [2] Kindu, M.; Schneider, T.; Teketay, D.; Knoke, T. Land Use/Land Cover Change Analysis Using Object-Based Classification Approach in Munessa-Shashemene Landscape of the Ethiopian Highlands. Remote Sens. 2013, 5, 2411-2435. https://doi.org/10.3390/rs5052411 [3] Dronova, I.; Gong, P.; Wang, L. Object-based analysis and change detection of major wetland cover types and their classification uncertainty during the low water period at Poyang Lake, China. Remote Sens. Environ 2011, 115, 3220-3236. https://doi.org/10.1016/j.rse.2011.07.006 [4] Zhou, W.; Troy, A. An object-oriented approach for analyzing and characterizing urban landscape at the parcel level. Int. J.Remote Sens. 2008, 29, 3119-3135. https://doi.org/10.1080/01431160701469065 [5] 42. Qian, Y.; Zhou, W.; Yan, J.; Li, W.; Han, L. Comparing Machine Learning Classifiers for Object-Based Land Cover Classification Using Very High Resolution Imagery. Remote Sens. 2015, 7, 153-168. https://doi.org/10.3390/rs70100153 [6] Breiman, L. Random forests. Mach. Learn. 2001, 45, 5-32. https://doi.org/10.1023/A:1010933404324

The technical task to be achieved by the present invention is to identify images that are effective in producing a land cover map including driveways, sidewalks, street trees, and street green space based on high-resolution images acquired by UAV in urban areas, and to utilize the data by producing a land cover map. The goal is to provide urban spatial information mapping methods and devices using drones and artificial intelligence to improve performance.

In one aspect, the urban spatial information mapping method using a drone and artificial intelligence proposed in the present invention includes producing a multispectral image, a vegetation index image, and a terrain image through a sensor of an unmanned aerial vehicle, and the produced multispectral image. , Confirming images and variables for random forest-based land cover classification using vegetation index images and terrain images, and verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the identified images and variables. Includes steps.

The step of producing a multispectral image, vegetation index image, and terrain image through the sensor of the unmanned aerial vehicle is red, green, blue, red edge, and NIR band (Near Infrared band). It produces orthoimages including DSM (Digital Surface Model), DEM (Digital Elevation model), and LST (Land Surface Temperature).

Using multispectral images according to an embodiment of the present invention, red and near infrared (NIR) wavelength bands are used, and NDVI (Normalized Difference Vegetation Index) and plant cover, which are indicators of vegetation, are higher than predetermined standards. Calculates the Soil Adjusted Vegetation Index (SAVI), an indicator used to correct NDVI for the effect of soil brightness in low-lying areas, where SAVI defines a soil brightness correction factor to accommodate land cover types, and Red ) and NIR values, and through difference calculation with DEM (Digital Elevation Model), which represents the height value of the ground excluding features in DSM (Digital Surface Model), it represents only the elevation value of features excluding the ground. Produce a normalized digital surface model (nDSM).

All images produced according to an embodiment of the present invention are resampled based on the multispectral image, and the layers of all produced images are equally rescaled into linear bands,

In the step of identifying images and variables for land cover classification, the impact of potential outliers is reduced, the relative size of numerical data is removed to reflect the same importance, normalized to a common value range, and image segmentation is performed. A layer stack process is performed to merge all produced images to generate input data for a single raster image.

The step of confirming images and variables for random forest-based land cover classification using the produced multispectral images, vegetation index images, and topographic images is performed by using raster data with specific shape characteristics according to the similarity of spectral characteristics. Through image segmentation that creates each object by grouping individual pixels, the object shape and size are changed according to object-based weight settings, and the average pixel value is calculated according to the pixel group of adjacent objects to determine the pixels that should be included in each object. Determine the creation of each object by setting object-based weights by adjusting spectral detail, spatial detail, and minimum segment size.

Random forest-based land cover classification that generates a plurality of variables using a plurality of statistical values assigned to each object created through the image segmentation and optimizes the plurality of variables according to the importance of the plurality of variables. By performing, building, car road, sidewalk, forest, grass, street tree, street grass, bare soil, water ( Land cover is classified into categories that include water.

The step of verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the confirmed images and variables is the producer accuracy (confusion matrix), which calculates accuracy by organizing the classification results of object items and actual items into a matrix. Producer's accuracy, user's accuracy, and overall acuuracy are calculated respectively.

According to embodiments of the present invention, the importance of UAV acquired images is evaluated to produce land cover maps needed in the field of urban environmental research, and the usability of using them for land cover classification in medium and small areas is considered. To this end, we produce multispectral images, vegetation index images, and topographic images that can be produced using drone sensors, and use object-based image analysis (OBIA) and random forest to analyze land cover in urban areas. Images and variables effective for classification were identified. In addition, land cover is classified through image and variable reduction, and the usability of UAV images for land cover classification is examined through accuracy comparison and verification.

Figure 1 is a flowchart illustrating a method for mapping urban spatial information using drones and artificial intelligence according to an embodiment of the present invention.
Figure 2 is a diagram showing the configuration of an urban spatial information mapping device using a drone and artificial intelligence according to an embodiment of the present invention.
FIG. 3 is a diagram illustrating the process of data collection, preprocessing, segmentation, classification, and verification steps implemented to derive an object-based urban spatial information map according to an embodiment of the present invention.
Figure 4 is a diagram showing a target site according to an embodiment of the present invention.
Figure 5 is a diagram showing arbitrary point locations for verification of classification accuracy according to an embodiment of the present invention.
Figure 6 shows the results of UAV image production according to an embodiment of the present invention.
Figure 7 shows the image segmentation results according to an embodiment of the present invention.
Figure 8 is a diagram showing training data characteristics according to land cover according to an embodiment of the present invention.
Figure 9 is a diagram showing the importance of variables ranked 40th among 126 variables according to an embodiment of the present invention.
Figure 10 is a diagram showing the results of applying an RF model to the entire target area according to an embodiment of the present invention according to an embodiment of the present invention.
Figure 11 is a diagram showing examples of types of sidewalk blocks on the sidewalk within the target site according to an embodiment of the present invention.

In urban areas, land cover maps are important spatial information used as basic data in the field of environmental assessment and research, but there is a lack of usability for small and medium-sized areas due to limitations such as the data update cycle and production costs. In the present invention, we identify images that are effective in producing a land cover map including driveways, sidewalks, street trees, and street green space based on high-resolution images acquired by UAV in urban areas, and propose a land cover production method and usability of the data.

According to an embodiment of the present invention, the importance of 126 variables used in learning was calculated through random forest classifier based on UAV and object-based image analysis. Through this, the importance of UAV images was evaluated, the model was optimized through variable optimization, and compared and analyzed with the initial model using 126 variables. When calculating variable importance, nDSM, NDVI, LST, SAVI, Blue, Green, Red, and Rededge images were distributed in the order of the top 40. It was shown that nDSM and LST images, in addition to multispectral and vegetation indices, can be effectively used in land cover classification in urban areas. As a result of the accuracy verification, the kappa coefficient of the initial model was 0.728 and the kappa coefficient of the optimized model was 0.721, so there was no significant change in the classification results despite reducing the variables. It was found that the roadway and sidewalk were distinguishable, and when the road was classified into the roadway and sidewalk, the roadway was classified better than the sidewalk, and through the spectral variation characteristics of surrounding objects, street trees and street green space were distinguished from forests and grasslands. Confirmed. It is believed that the application of the object-based land cover classification technique using UAV can contribute to the field of urban environmental research due to its advantages such as data timeliness and economic feasibility. Hereinafter, embodiments of the present invention will be described in detail with reference to the attached drawings.

Figure 1 is a flowchart illustrating a method for mapping urban spatial information using drones and artificial intelligence according to an embodiment of the present invention.

The proposed urban spatial information mapping method using drones and artificial intelligence includes the step of producing a multispectral image, a vegetation index image, and a terrain image through a sensor of an unmanned aerial vehicle (110), the produced multispectral image, a vegetation index image, and A step of confirming images and variables for random forest-based land cover classification using topographic images (120) and a step of verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the confirmed images and variables ( 130).

In step 110, a multispectral image, a vegetation index image, and a terrain image are produced through the sensor of the unmanned aerial vehicle. According to an embodiment of the present invention, red, green, blue, red edge, NIR band (Near Infrared band) and DSM (Digital Surface Model) and DEM (Digital Elevation model) and LST (Land Surface Temperature) can be produced.

According to an embodiment of the present invention, red and near infrared (NIR) wavelength bands are used using multispectral images, and NDVI (Normalized Difference Vegetation Index), which is an indicator of vegetation, and plant cover are determined by predetermined standards. Calculates the Soil Adjusted Vegetation Index (SAVI), an index used to correct NDVI for the effect of soil brightness in lower regions. SAVI defines a soil brightness correction factor to accommodate land cover types, and Red ( It can be calculated as the ratio between Red) and NIR values. Through difference calculation with DEM (Digital Elevation Model), which represents the height value of the ground excluding features in the DSM (Digital Surface Model), the elevation of features excluding the ground It is possible to create a normalized digital surface model (nDSM) that represents only values.

All images produced according to an embodiment of the present invention are resampled based on the multispectral image, and the layers of all produced images are equally rescaled into linear bands. In the step of identifying images and variables for land cover classification, the impact of potential outliers is reduced, the relative size of numerical data is removed to reflect the same importance, normalized to a common value range, and image segmentation is performed. A layer stack process is performed to merge all produced images to generate input data for a single raster image.

In step 120, images and variables for land cover classification based on random forest are confirmed using the produced multispectral image, vegetation index image, and topographic image.

According to an embodiment of the present invention, the object shape and size can be changed according to object-based weight settings through image segmentation that generates each object by grouping individual pixels in raster data with specific shape characteristics according to the similarity of spectral characteristics. .

At this time, the average pixel value is calculated according to the pixel group of adjacent objects, the pixels that should be included in each object are determined, and spectral detail, spatial detail, and minimum segment size are determined. You can adjust the creation of each object by adjusting and setting object-based weights.

A random forest (Random Forest) that generates a plurality of variables using a plurality of statistical values assigned to each object created through image segmentation according to an embodiment of the present invention, and optimizes the plurality of variables according to the importance of the plurality of variables. By performing land cover classification based on forest, buildings, car road, sidewalk, forest, grass, street tree, street grass, and bare ground are classified. Land cover is classified into categories including bare soil and water.

In step 130, the accuracy of land cover classification based on the unmanned aerial vehicle image is verified using the identified image and variables. At this time, the producer's accuracy, user's accuracy, and overall accuracy are calculated through a confusion matrix that calculates accuracy by organizing the object item classification result and the actual item into a matrix. do.

Figure 2 is a diagram showing the configuration of an urban spatial information mapping device using a drone and artificial intelligence according to an embodiment of the present invention.

The proposed urban spatial information mapping device 200 using drones and artificial intelligence includes an image production unit 210, a land cover classification unit 220, and an accuracy verification unit 230.

The image production unit 210 according to an embodiment of the present invention produces multispectral images, vegetation index images, and topographic images through sensors of an unmanned aerial vehicle. According to an embodiment of the present invention, red, green, blue, red edge, NIR band (Near Infrared band) and DSM (Digital Surface Model) and DEM (Digital Elevation model) and LST (Land Surface Temperature) can be produced.

According to an embodiment of the present invention, red and near infrared (NIR) wavelength bands are used using multispectral images, and NDVI (Normalized Difference Vegetation Index), which is an indicator of vegetation, and plant cover are determined by predetermined standards. Calculates the Soil Adjusted Vegetation Index (SAVI), an index used to correct NDVI for the effect of soil brightness in lower regions. SAVI defines a soil brightness correction factor to accommodate land cover types, and Red ( It can be calculated as the ratio between Red) and NIR values. Through difference calculation with DEM (Digital Elevation Model), which represents the height value of the ground excluding features in the DSM (Digital Surface Model), the elevation of features excluding the ground It is possible to create a normalized digital surface model (nDSM) that represents only values.

All images produced according to an embodiment of the present invention are resampled based on the multispectral image, and the layers of all produced images are equally rescaled into linear bands. In the step of identifying images and variables for land cover classification, the impact of potential outliers is reduced, the relative size of numerical data is removed to reflect the same importance, normalized to a common value range, and image segmentation is performed. A layer stack process is performed to merge all produced images to generate input data for a single raster image.

The land cover classification unit 220 according to an embodiment of the present invention uses the produced multispectral image, vegetation index image, and topographic image to identify images and variables for random forest-based land cover classification.

According to an embodiment of the present invention, the object shape and size can be changed according to object-based weight settings through image segmentation that generates each object by grouping individual pixels in raster data with specific shape characteristics according to the similarity of spectral characteristics. .

At this time, the average pixel value is calculated according to the pixel group of adjacent objects, the pixels that should be included in each object are determined, and spectral detail, spatial detail, and minimum segment size are determined. You can adjust the creation of each object by adjusting and setting object-based weights.

A random forest (Random Forest) that generates a plurality of variables using a plurality of statistical values assigned to each object created through image segmentation according to an embodiment of the present invention, and optimizes the plurality of variables according to the importance of the plurality of variables. By performing land cover classification based on forest, buildings, car road, sidewalk, forest, grass, street tree, street grass, and bare ground are classified. Land cover is classified into categories including bare soil and water.

The accuracy verification unit 230 according to an embodiment of the present invention verifies the accuracy of land cover classification based on the unmanned aerial vehicle image using the confirmed image and variables. At this time, the producer's accuracy, user's accuracy, and overall accuracy are calculated through a confusion matrix that calculates accuracy by organizing the object item classification result and the actual item into a matrix. do.

FIG. 3 is a diagram illustrating the process of data collection, preprocessing, segmentation, classification, and verification steps implemented to derive an object-based urban spatial information map according to an embodiment of the present invention.

The process for deriving an object-based urban spatial information map according to an embodiment of the present invention includes the data acquisition (311) and image preprocessing (312) steps, as shown in FIG. 3, using the sensor of the unmanned aerial vehicle. A step of producing a multispectral image, a vegetation index image, and a terrain image (310), a multispectral image, a vegetation index image, and a terrain image including image segmentation (321) and classification (322). It can be simplified into five main steps: checking images and variables for random forest-based land cover classification using 320 and accuracy assessments 330.

First, through basic data acquisition (311), orthoimages were produced by acquiring UAV flights and GCPs so that multiple images could be overlapped in the same space. Afterwards, in an image preprocessing step (312) to identify images and variables that are effective in classifying land cover in urban areas, the same weight was assigned to all images by removing differences in the relative sizes of the numerical data. Next, an object was created by grouping pixels with similar characteristics through image segmentation and object creation (321). Based on this, training data was created, an RF classifier (Random Forest Classifier) was trained by applying all variables that could be extracted from UAV images, and variable importance was calculated (322). Through this, effective images and variables for land cover classification in urban areas are identified, the model is retrained by reducing images and variables, and accuracy is compared and verified (330) to provide information on land cover classification in urban areas using UAV images and OBIA. Usability was considered. Below, the process of data collection, preprocessing, segmentation, classification, and verification steps implemented to derive an object-based urban spatial information map according to an embodiment of the present invention will be described in more detail.

Figure 4 is a diagram showing a target site according to an embodiment of the present invention.

The target site for urban spatial information mapping according to an embodiment of the present invention is to select a target site that can produce land cover maps of urban areas using UAV images and verify usability. As shown in Figure 4, various land cover types are selected in small areas. The area around Changwon National University, Changwon-si, Gyeongsangnam-do, South Korea, was selected. The UAV shooting range was set to a rectangle of 623 m x 1138 m, including the outer area of the university. The terrain is surrounded by mountains with an altitude of approximately 600m. Buildings with a height of 5 to 8 stories are mainly distributed within the school, and it consists of an area with a wide pedestrian path made of sidewalk blocks and an area with both a roadway and a pedestrian path. Additionally, landscaping facilities such as lawns and trees are distributed throughout the area.

The UAV used in the step of producing multispectral images, vegetation index images, and terrain images through the sensors of the unmanned aerial vehicle, including the data collection and preprocessing steps according to an embodiment of the present invention, is the fixed-wing eBeeX developed by sensefly, Switzerland. It weighs about 1.4 kg and has a wingspan of 116 cm. It can fly at a speed of 40~110km/h for up to 90 minutes, and the aircraft can be equipped with various sensors such as RGB, thermal infrared, and 5 band multispectral, and has built-in RTK/PPK functions.

In the present invention, Duet-T (RGB+Thermal) and RedEdge-MX (Red, Green, Blue, Rededge, NIR) were used. The shooting date according to the embodiment of the present invention was June 21, 2021, and was conducted on a clear day without clouds. The flight was conducted with the GSD (ground sample distance) set to 13cm/pixel when the RedEge-MX sensor was installed, and the GSD of the thermal sensor was set to 20cm/pixel when the Duet-T sensor was installed. Both species and lateral redundancy were set to 75%. And as shown in Figure 4, 25 GCPs (Ground Control Points) were acquired to correct the location accuracy of the drone image.

UAV images collected according to an embodiment of the present invention were each produced as an orthoimage using Pix4D Mapper software, and were divided into red, green, blue, red edge, and NIR bands. ) and DSM (Digital Surface Model), DEM (Digital Elevation model), and LST (Land Surface Temperature) orthoimages were produced. And using multispectral images, NDVI (Normalized Difference Vegetation Index) and SAVI (Soil Adjusted Vegetation Index) were calculated using equations (1) and (2). NDVI is an indicator that uses red and NIR wavelength bands and is closely related to vegetation. The higher the NDVI value, the higher the NIR reflectivity, which corresponds to high density of vegetation and healthy plants. SAVI is an index used to correct NDVI for the effect of soil brightness in areas of low vegetation cover. SAVI, derived from Landsat satellite imagery, defines a soil brightness correction factor (L) of 0.5 to accommodate most land cover types, and is calculated as the ratio between Red and NIR values [1]. And through difference calculation with DEM, which represents the height value of the ground excluding features in DSM, nDSM (normalized digital surface model), which represents only the elevation value of features excluding pure ground, was created. This is because it is believed that if nDSM is used in the classification process, buildings and trees existing in the terrain can be more effectively classified.

(One)

(2)

All produced images were resampled with GSD to 15cm/pixel based on multispectral images in order to recognize the horizontal green space between roads planted for landscaping. And since the produced image layers are also used in the next step, classification, all input layers were equally rescaled into a linear band with an 8-bit range from 0 to 255. This step reduces the impact of potential outliers in the segmentation and classification steps and normalizes them to a common value range by removing the relative size of the numerical data to reflect the same degree of importance. Lastly, since one raster image is required as input data when performing image segmentation, Red, Green, Blue, Red Edge, NIR, NDVI, SAVI, LST, and nDSM are merged. A layer stack process was carried out.

In the step of identifying images and variables for random forest-based land cover classification using multispectral images, vegetation index images, and topographic images including image segmentation and classification according to an embodiment of the present invention, image segmentation is a spectral characteristic. This is a process of grouping individual pixels in raster data with similar and specific shape characteristics into one object, and the shape and size of the object vary depending on the object-based weight setting [2]. This technique calculates the average pixel value by considering groups of neighboring pixels and determines which pixels each object should contain. Image segmentation generalizes the area within the raster to maintain all data features into a larger continuous area rather than at a fine pixel level [3]. Therefore, the reliability of object-based image analysis is determined by appropriate image segmentation, and classification accuracy may vary depending on the segmentation results.

ArcGIS pro 2.8 software was used to segment images according to an embodiment of the present invention. In ArcGIS, the weights used to create objects must be considered when segmenting an image. Object creation can be adjusted by modifying the spectral detail and spatial detail to values between 1 and 20. Image segmentation is designed to improve classification processing speed by eliminating complexity and file size due to fine spatial resolution, but since the relationship between input weight values and image segmentation results is difficult to clearly explain, analysts must use trial and error to determine appropriate standards. However, a method must be found [4, 5]. Reducing the detail of either weight reduces the complexity and file size of the raster data. In general, image segmentation with high spectral detail and low spatial detail is preferred because of its short processing time. And by adjusting the minimum segment size, objects smaller than this size can be merged with the most similar adjacent segments.

In an embodiment of the present invention, spectral detail was sequentially read in 5 units from 10 to 20, and spatial detail was changed in 5 units to read object creation. Next, the spectral detail and spatial detail were adjusted in increments of 1, and the minimum segment size was sequentially applied in increments of 10 from 20 to 60 to merge very small object sizes with adjacent objects to select the final weight.

Since the land cover map serves as basic data for various studies at the city level, the present invention sets up classification items to contribute to the analysis of the urban environment. As shown in Figure 3, the classification items include building, car road, sidewalk, forest, grass, street tree, street grass, and bare ground. It was divided into 9 items: bare soil and water. Here, the roads were subdivided into roads for cars and roads for people, and further subdivided into forests, street trees, grasslands, and street green spaces. The reason is that air pollutants such as fine dust and NOx are emitted in spaces where cars drive, but since these do not occur in India, the classification was subdivided to better specify the space where pollutants are generated. In the case of street trees and street green space, if the area is below a certain standard, they are classified as road land cover in the Ministry of Environment's land cover map. Street trees and street green space were added because they are land covers that can be used in various aspects such as reducing fine dust in the city, wind paths, and reducing the thermal environment.

The statistical values assigned to objects from UAV-based images to classify land cover are count, area, mean, max, range, STD (standard deviation), sum, variety, majority, minority, median, PCT (percentile), NMS ( (mean nearby STD). Through this, a total of 126 variables were constructed by calculating 14 statistical values for each image for one object. Although this study seeks to classify land cover and street green space, the spectral characteristics of forests, street trees, grasslands, and street green space may show similar characteristics. However, while forests and grasslands are forests or grasslands in which the surrounding objects have similar spectral characteristics, street trees and street green spaces contain land cover rather than vegetation such as roads, sidewalks, and buildings, resulting in large deviations from the surrounding spectral characteristics. Therefore, an NMS variable was added to classify street trees and street green space by calculating the standard deviation of the average value of objects within 3 m from each object.

For supervised classification, training data consisted of polygons in the range of 1000-5000 for 8 items excluding water. In the case of water, only 165 polygons were selected because vegetation within the stream covered most of the area near the stream. These training polygons were selected manually through screen interpretation, taking into account the distribution among land cover classes and the different shades of color that characterize the study area.

RF (Random Forest) [6] consists of several sets of decision trees and bootstrap aggregating (in other words, bagging). This is a method of combining basic classifiers trained on slightly different training data. One decision tree is trained using the same samples, but other samples are not selected at all and are trained on different decision trees. A single decision tree is very sensitive to noise in training data, but the results derived through voting on multiple trees by lowering the correlation between trees have a strong advantage against noise.

Although the RF classifier has many strengths, it is difficult to secure the explanatory power of the independent and dependent variables because it is a black box model in which it is unknown on what basis the results were obtained. Therefore, the increase in variables makes OBIA classification a highly subjective and time-intensive task. To solve this. Advantages can be gained by optimizing variables by estimating which variables play an important role in prediction performance through a measure called variable importance.

Collecting training data samples to train an RF classifier is very important. An adequate number of samples must be collected to provide a statistically valid representation of the map's accuracy, and because the area ratio of land cover is not uniform, accuracy assessments may become meaningless even if sampled to be statistically unbiased. Therefore, in the embodiment of the present invention, an appropriate number of samples were collected for each land cover item among the total 254,456 image objects, and 22,353 (8.8%) were used as training data.

In RF learning according to an embodiment of the present invention, the ratio of verification data among training data was set to 30%. Additionally, the number of trees ranges from 100 to 1000, the maximum depth of tree ranges from 5 to 10, and the minimum leaf size ranges from 1 to 3. The RF model was explored. Ultimately, the number of trees was set to 500, the maximum tree depth was 10, and the minimum leaf size was 2. Then, variable importance was calculated, variables were optimized, and RF models were trained and compared under the same conditions.

Figure 5 is a diagram showing arbitrary point locations for verification of classification accuracy according to an embodiment of the present invention.

Verification of the accuracy of the land cover map classified according to the embodiment of the present invention is the producer's accuracy through a confusion matrix that calculates accuracy by organizing the classification results of object items and actual items into a matrix, as shown in Figure 5. and user's accuracy and overall accuracy were calculated, respectively. As reference data for accuracy verification, 500 randomly selected verification points were visually read based on the original image captured by a UAV.

Figure 6 shows the results of UAV image production according to an embodiment of the present invention.

As a result of UAV image collection, Red, Green, Blue, Red Edge, and NIR (Near Infrared) images were collected from the Rededge-MX sensor, and a total of 5,925 images were acquired. The Duet-T sensor collected RGB and LST images, and acquired 1,132 images each. The image results produced by combining the collected images were used to create 9 images, as shown in Figure 6. As a result of image geometry correction based on GCP, the root mean square deviation (RMSE) of the Duet-T sensor-based image was 0.033m in the x-direction and 0.048m in the y-direction, and the root mean square deviation (RMSE) of the image based on the Rededge-MX sensor was 0.048m in the y-direction. was found to be 0.056m in the x-direction and 0.054m in the y-direction.

Figure 7 shows the image segmentation results according to an embodiment of the present invention.

As shown in FIG. 7, the selection of image segmentation weights according to an embodiment of the present invention showed a tendency for the boundaries between roads and grasslands to be merged rather than created at spectral detail 10, and at 15, roads and grasslands tended to be merged in some areas. There was a section where there was no distinction between and India. Spectral detail 20 showed a tendency for objects to have very fine boundaries even within the same land cover. In this case, the boundary between land covers is clear, but the size of the object is very small, so it may be sensitive to noise in the image. And from spectral detail 15 and spatial detail 15, the roadway and sidewalk began to be distinguished, but the boundary between the asphalt-paved roadway and the bare gravel made of gravel was not created. Based on this, the spectral detail was increased to explore conditions where the boundary between the driveway and the bare dirt is clear. Through detailed adjustment, it was confirmed that under spectral detail 18 and spatial detail 10 conditions, the boundaries for each land cover were clear and objects were not formed in excessive detail. When the minimum segment size was set to 40, it was judged that the size of the object was not subdivided into very small pieces but divided into appropriate sizes, and the boundary of the street green space that exists instead of the dividing line in the middle of the roadway became clear, so the weight was selected at 40. did. Therefore, for object-based land cover classification, spectral detail of 18, spatial detail of 10, and minimum segment size of 40 were finally selected (see Figure 6).

Figure 8 is a diagram showing training data characteristics according to land cover according to an embodiment of the present invention.

The characteristics of the training data for each class used for training are shown in Figure 8. Looking at nDSM, buildings, forests, and street trees were clearly distinguished compared to other land covers based on their characteristics. NDVI and SAVI can be seen to have a clear tendency to increase in classes representing vegetation, and in the case of street grass in NDVI, the range of the box plot appears to be wide due to areas where hay or soil is exposed. . In addition, in the case of non-vegetation, the range almost overlaps, and although vegetation and non-vegetation can be distinguished at a single wavelength, detailed distinctions such as car road, sidewalk, and bare soil are difficult. LST was able to confirm that water was clearly classified as low, and then vegetation such as forests and grasses existed in a low range. Artificial covering materials such as buildings and driveways were characterized by high LST values, but in the case of bare soil, it was found to be widely distributed from the vegetation range to the artificial cover range. In the red, green, and blue images, the box plots of the driveway, sidewalk, and backyard, whose ranges continuously overlap, were found to be distinct. In particular, the ranges of the driveway, sidewalk, and backyard were distinct in the blue image. It turned out to be possible. NIR and Rededge showed that the ranges of the box plots for most classes overlapped, except for water. The range of the roadway and sidewalk appeared to overlap in all images except the blue image, but the box plot range for the roadway, which was mostly made of black asphalt, was narrower than the sidewalk, which was made up of blocks of various colors.

Figure 9 is a diagram showing the importance of variables ranked 40th among 126 variables according to an embodiment of the present invention.

As a result of training the RF model using all 126 variables, the variable importance is as shown in Figure 9, and among the 126 variables, the 40th ranked variable with an importance exceeding 0.6% was expressed. Looking at variable importance, nDSM (8), NDVI (7), LST (7), SAVI (6), Blue (6), Green (4), Red (1), Rededge (1) ) The number was distributed in video order. nDSM and NDVI occupy positions from 1st to 12th, and from 13th onwards, various images and statistical values exist together. nDSM is considered effective in distinguishing between buildings and trees with height values, and NDVI is effective in distinguishing between vegetation and non-vegetation. In the case of the vegetation index, NDVI showed higher importance than SAVI. In the case of NDVI, there is less overlap between the range of forests, grasslands, and street trees, and buildings, roads, and sidewalks, but SAVI overlaps more ranges than NDVI. It is judged as LSTs are distributed between 16th and 33rd places, and as shown in the training data characteristics, it is believed that they were important in distinguishing between the building, roadway, and sidewalk groups and the forest, grassland, street trees, street green space, and water groups. . Referring to prior technologies, most of them mainly used multispectral images and vegetation indices such as NDVI, but the results of the present invention suggest that LST images can also be used in urban area classification. And among single spectral wavelengths, the blue image ranked highest in variable importance, which is believed to be due to the high variable importance for distinguishing between driveways, sidewalks, and bare areas. Looking at the statistical values of variable importance, median (6), PCT75 (6), mean (6), NMS (5), min (5), majority (5), max (4), minority (3) The number is distributed in order. For nDSM, median, PCT75, and mean were ranked high, and for NDVI, mean, PCT75, and medain were ranked high. In NMS, which was used to distinguish street trees and street green space, nDSM and Green images were ranked high.

Figure 10 is a diagram showing the results of applying the RF model to the entire target area according to an embodiment of the present invention.

Figure 10(a) is a diagram showing an initial model applying 126 variables, and Figure 10(b) is a diagram showing an optimized model.

Optimized by reducing the model to 6 images of nDSM, NDVI, SAVI, LST, Blue, and Green and 7 variables of PCT75, mean, NSM, median, majority, max, and min through variable importance according to an embodiment of the present invention. And the results of applying the model to the entire target area are shown in Figure 10. As a result of predicting land cover through a model using all 126 variables, the area ratio by land cover was forest (19.2%), roadway (16.8%), street tree (16.6%), street green space (14.8%), and India (13.5%). %), buildings (11.5%), grassland (5.3%), bare ground (2.2%), and water (0.1%). In the optimization model through variable reduction, forest (19.1%), roadway (17.6%), and street green space were used. (16.4%), street trees (14.7%), sidewalks (12.2%), buildings (11.6%), grassland (5.8%), bare ground (2.5%), and water (0.1%), with the overall area ratio difference excluding India. It was similar at less than 1%, and despite reducing the variables, there was no significant change in the classification results.

When interpreting the land cover classification results qualitatively, it was found that forests and street trees were well distinguished, and in the case of water, it was found to be classified very clearly even though it occupied a very small area within the target area and had the least amount of training data. It was found that buildings and driveways were well classified from the artificial covering material, and it was confirmed that driveways and sidewalks were classified separately, but in areas where shadows existed, it was found that driveways tended to be classified as sidewalks. Shadows are one of the major problems that impede the accuracy of information extraction in remote sensing, and additional research on shadow detection and shadow correction is deemed necessary. In addition, even small trails in the forest were found to be classified as sidewalks, and the surrounding grassland tended to be misclassified as street green space. Additionally, there was a tendency for the Naji area to be classified as India.

As a result of accuracy verification according to an embodiment of the present invention, the Kappa coefficient of the initial model to which all variables were applied was calculated to be 0.728, and the overall accuracy was calculated to be 76.0%. The Kappa coefficient of the optimization model was 0.721, and the overall accuracy was 75.4%, showing that the accuracy did not decrease significantly even though the variables were reduced (Tables 3 and 4). User accuracy according to variable optimization was found to be within ±3% for buildings, driveways, sidewalks, forests, grasslands, street trees, and street green spaces, showing no significant change compared to the initial model, and increased to 4.9% for water. And, as there was a tendency to predict India as Naji more than the initial model, the accuracy was found to be lowered by -13.8%. In the case of producer accuracy, buildings, forests, street trees, open fields, and water were within ±3%, showing no significant changes compared to the existing model. And while the maker accuracy of sidewalks and street trees decreased to -7.5% and -6.0%, respectively, grassland and driveways increased by 7.0% and 5.6%, respectively. Therefore, when examining the change in accuracy, it was found that buildings, forests, and street trees were not significantly affected by variable reduction. In the prior art, nDSM and NDVI are used to ensure classification accuracy for buildings and trees, and when nDSM was used as input data, improvements were reported for several items, including buildings, in quantitative and qualitative analysis of the classification results. . Therefore, in the present invention, it is judged that there is little change in variable optimization for buildings, forests, and street trees, largely influenced by nDSM and NDVI, which rank high in variable importance.

Figure 11 is a diagram showing examples of types of sidewalk blocks on the sidewalk within the target site according to an embodiment of the present invention.

In the optimization model, the items with user and producer accuracy of more than 90% were buildings and water, and as mentioned earlier, buildings are judged to have a large impact on nDSM. As shown in Figure 8, among the optimization variables, water had the greatest difference in LST from other land covers, and for vegetation, which is the section where the box plots overlap, the classification accuracy is judged to be high due to NDVI. In the case of driveways and sidewalks, when classified as a road that combines driveways and sidewalks as in other studies, the manufacturer's accuracy was 89.6% and the user's accuracy was 80.0%, which is similar to the classification accuracy of the prior art in the range of 75 to 95%. It appeared. However, as India was overpredicted, the user accuracy was low at 65.7%. This is believed to be due to the tendency to misclassify bare areas with similar spectral characteristics as sidewalks, as the blocks constituting the sidewalks in the target site contain a variety of colors such as beige, red, green, and gray, as shown in Figure 11. Accordingly, 46% of Naji's reference data was classified as India, and the Naji creator's accuracy was evaluated as very low at 31.5%. When evaluating accuracy by combining forests and street trees into trees, the user accuracy was high at 97.2% and the producer accuracy was 96.3%, but when subdivided into forests and street trees, the accuracy was found to decrease. This is because misclassification occurs between the two items, and it is judged to be influenced by the tree's shadow when read on the screen. It is believed that these errors can be improved by using clinical maps as auxiliary data in the future.

The device described above may be implemented with hardware components, software components, and/or a combination of hardware components and software components. For example, devices and components described in embodiments may include, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA), It may be implemented using one or more general-purpose or special-purpose computers, such as a programmable logic unit (PLU), microprocessor, or any other device capable of executing and responding to instructions. A processing device may execute an operating system (OS) and one or more software applications that run on the operating system. Additionally, a processing device may access, store, manipulate, process, and generate data in response to the execution of software. For ease of understanding, a single processing device may be described as being used; however, those skilled in the art will understand that a processing device includes multiple processing elements and/or multiple types of processing elements. It can be seen that it may include. For example, a processing device may include a plurality of processors or one processor and one controller. Additionally, other processing configurations, such as parallel processors, are possible.

Software may include a computer program, code, instructions, or a combination of one or more of these, which may configure a processing unit to operate as desired, or may be processed independently or collectively. You can command the device. Software and/or data may be used on any type of machine, component, physical device, virtual equipment, computer storage medium or device to be interpreted by or to provide instructions or data to a processing device. It can be embodied in . Software may be distributed over networked computer systems and stored or executed in a distributed manner. Software and data may be stored on one or more computer-readable recording media.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., singly or in combination. Program instructions recorded on the medium may be specially designed and configured for the embodiment or may be known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. -Includes optical media (magneto-optical media) and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, etc. Examples of program instructions include machine language code, such as that produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

As described above, although the embodiments have been described with limited examples and drawings, various modifications and variations can be made by those skilled in the art from the above description. For example, the described techniques are performed in a different order than the described method, and/or components of the described system, structure, device, circuit, etc. are combined or combined in a different form than the described method, or other components are used. Alternatively, appropriate results may be achieved even if substituted or substituted by an equivalent.

Therefore, other implementations, other embodiments, and equivalents of the claims also fall within the scope of the claims described below.

Claims (14)

Producing a multispectral image, a vegetation index image, and a terrain image through the sensor of an unmanned aerial vehicle;
Confirming images and variables for random forest-based land cover classification using the produced multispectral image, vegetation index image, and topographic image; and
Verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the confirmed images and variables
Urban spatial information mapping method including. According to paragraph 1,
The step of producing multispectral images, vegetation index images, and terrain images through the sensors of the unmanned aerial vehicle,
Red, Green, Blue, Rededge, NIR band (Near Infrared band), DSM (Digital Surface Model), DEM (Digital Elevation model), and LST (Land Surface Temperature) Producing orthovideos containing
Urban spatial information mapping method. According to paragraph 2,
Using the multispectral image, Red and Near Infrared (NIR) wavelength bands are used, and NDVI (Normalized Difference Vegetation Index), which is an indicator of vegetation, and soil brightness in areas where plant cover is lower than a predetermined standard are measured. Calculates the Soil Adjusted Vegetation Index (SAVI), an indicator used to correct NDVI for impacts;
The SAVI defines a soil brightness correction factor to accommodate land cover types and is calculated as the ratio between Red and NIR values,
Through difference calculation with DEM (Digital Elevation Model), which represents the height value of the ground excluding features in DSM (Digital Surface Model), nDSM (normalized digital surface model) that represents only the elevation value of features excluding the ground is produced.
Urban spatial information mapping method. According to paragraph 3,
All the produced images are resampled based on the multispectral image, and the layers of all the produced images are equally rescaled into linear bands,
In the step of identifying images and variables for land cover classification, the impact of potential outliers is reduced, the relative size of numerical data is removed to reflect the same importance, normalized to a common value range, and image segmentation is performed. A layer stack process is performed to merge all produced images to generate input data for a single raster image.
Urban spatial information mapping method. According to paragraph 1,
The step of confirming images and variables for random forest-based land cover classification using the produced multispectral image, vegetation index image, and topographic image is,
Through image segmentation, which creates each object by grouping individual pixels in raster data with specific shape characteristics according to the similarity of spectral characteristics, the object shape and size are changed according to object-based weight settings,
Calculate the average pixel value according to the pixel group of adjacent objects to determine which pixels should be included in each object,
Adjusts the creation of each object by setting object-based weights by adjusting spectral detail, spatial detail, and minimum segment size.
Urban spatial information mapping method. According to clause 5,
Random forest-based land cover classification that generates a plurality of variables using a plurality of statistical values assigned to each object created through the image segmentation and optimizes the plurality of variables according to the importance of the plurality of variables. By performing, building, car road, sidewalk, forest, grass, street tree, street grass, bare soil, water ( classification of land cover into categories that include water.
Urban spatial information mapping method. According to paragraph 1,
The step of verifying the accuracy of land cover classification based on unmanned aerial vehicle images using the confirmed images and variables is,
Producer's accuracy, user's accuracy, and overall accuracy are calculated through a confusion matrix that calculates accuracy by organizing the classification results of object items and actual items into a matrix.
Urban spatial information mapping method. An image production department that produces multispectral images, vegetation index images, and terrain images using the sensors of unmanned aerial vehicles;
A land cover classification unit that identifies images and variables for random forest-based land cover classification using the produced multispectral images, vegetation index images, and topographic images; and
Accuracy verification unit that verifies the accuracy of land cover classification based on unmanned aerial vehicle images using the confirmed images and variables
An urban spatial information mapping device including a. According to clause 8,
The video production department,
Red, Green, Blue, Rededge, NIR band (Near Infrared band), DSM (Digital Surface Model), DEM (Digital Elevation model), and LST (Land Surface Temperature) Producing orthovideos containing
Urban spatial information mapping device. According to clause 9,
The video production department,
Using the multispectral image, Red and Near Infrared (NIR) wavelength bands are used, and NDVI (Normalized Difference Vegetation Index), which is an indicator of vegetation, and soil brightness in areas where plant cover is lower than a predetermined standard are measured. Calculates the Soil Adjusted Vegetation Index (SAVI), an indicator used to correct NDVI for impacts;
The SAVI defines a soil brightness correction factor to accommodate land cover types and is calculated as the ratio between Red and NIR values,
Through difference calculation with DEM (Digital Elevation Model), which represents the height value of the ground excluding features in DSM (Digital Surface Model), nDSM (normalized digital surface model) that represents only the elevation value of features excluding the ground is produced.
Urban spatial information mapping device. According to clause 10,
The video production department,
All the produced images are resampled based on the multispectral image, and the layers of all the produced images are equally rescaled into linear bands,
In the step of identifying images and variables for land cover classification, the impact of potential outliers is reduced, the relative size of numerical data is removed to reflect the same importance, normalized to a common value range, and image segmentation is performed. A layer stack process is performed to merge all produced images to generate input data for a single raster image.
Urban spatial information mapping device. According to clause 8,
The land cover classification department,
Through image segmentation, which creates each object by grouping individual pixels in raster data with specific shape characteristics according to the similarity of spectral characteristics, the object shape and size are changed according to object-based weight settings,
Calculate the average pixel value according to the pixel group of adjacent objects to determine which pixels should be included in each object,
Adjusts the creation of each object by setting object-based weights by adjusting spectral detail, spatial detail, and minimum segment size.
Urban spatial information mapping device. According to clause 12,
The land cover classification department,
Random forest-based land cover classification that generates a plurality of variables using a plurality of statistical values assigned to each object created through the image segmentation and optimizes the plurality of variables according to the importance of the plurality of variables. By performing, building, car road, sidewalk, forest, grass, street tree, street grass, bare soil, water ( classification of land cover into categories that include water.
Urban spatial information mapping device. According to clause 8,
The accuracy verification unit,
Producer's accuracy, user's accuracy, and overall accuracy are calculated through a confusion matrix that calculates accuracy by organizing the classification results of object items and actual items into a matrix.
Urban spatial information mapping device.