KR102557775B1 - Drone used 3d mapping method - Google Patents

Abstract

The present invention relates to a method for producing a 3D map using a drone, comprising: collecting drone image data of a map production area captured using a drone; collecting satellite image data of the mapping area; Obtaining corrected drone image data by correcting perspective distortion of the drone image data; acquiring corrected satellite image data by correcting shadow distortion of the satellite image data; Performing image matching on the mapping area using the corrected drone image data and the corrected satellite image data based on the ground control point of the mapping area; and constructing an orthoimage through modeling of the matched image obtained through the image matching to produce a 3D map; By including the image distortion, it is possible to accurately represent the actual situation of a desired region and to effectively acquire 3D terrain information.

Description

3D map production method using drone {DRONE USED 3D MAPPING METHOD}

The present invention relates to a method for producing a 3D map using a drone, and specifically, collects drone image data and satellite image data for a map production area, and corrects perspective distortion of the drone image data and shadow distortion of the satellite image data. After performing image matching for the map production area using corrected drone image data and corrected satellite image data based on the ground reference point, by constructing an ortho image through modeling of the matched image to create a 3D map, It relates to a 3D map production method using a drone capable of accurately representing the actual situation of a desired area by correcting distortion and effectively acquiring 3D terrain information.

As is well known, an unmanned aerial vehicle (UAV) is a single-use or reusable unmanned aerial vehicle (UAV) that flies autonomously or remotely controlled by aerodynamic force without a pilot, and can carry weapons or general cargo. It can be defined as a powered flight vehicle that can be used, and such an unmanned aerial vehicle system is also referred to as a drone.

With the development of drone technology as described above, various technologies using drones are being researched and developed in various forms. Drones were initially developed mainly for military purposes, but the fields of application have gradually expanded, and recently, facility management, coastal surveillance, and environmental monitoring It is used for various purposes such as surveillance of large buildings, surveillance of forest fires, surveillance of forests, unmanned night patrol, unmanned delivery service, pesticide sprayer, crime detection, criminal tracking, operation, extreme sports filming, terrain modeling, structure modeling, etc. , It is also used for photographing tourist attractions.

On the other hand, high-resolution satellite images are being photographed and provided, and due to the rapid development of GIS (geographic information system) technology, conditions are being created to acquire various topographical information and systematically manage them.

In particular, digital topographic map production and image map production, which are geographic information-based data through high-medium-resolution optical satellites such as IKONOS, QuickBird, SPOT, and KOMSAT among artificial satellites, and urban topographic information and disaster information obtained using RADARSAT equipped with an active sensor Management, various construction management, resource management, environmental management, etc. utilization is rapidly increasing.

In addition, it has become necessary to establish an information system for various purposes by linking geospatial information data with GIS, and integration of various information obtained using medium and low resolution satellite image data with high resolution image data and integration of multi-sensor data In order to diversify the utilization of image information, various researches and inventions are being conducted.

As described above, various techniques are being researched and developed for the production of digital orthographic images and 3D maps that accurately express the actual situation of the desired area by using inverted images taken by drones and satellite image data, etc. am.

1. Korean Patent Publication No. 10-2011-0082903 (published on July 20, 2011)

The present invention collects drone image data and satellite image data for the map production area, corrects perspective distortion of drone image data and shadow distortion of satellite image data, and corrects drone image data and corrected satellite image based on ground control points. After performing image matching for the map production area using data, by constructing an ortho image through modeling of the matching image to produce a 3D map, it is possible to correct image distortion to accurately represent the actual situation of the desired region. The purpose of this study is to provide a method for producing a 3D map using a drone that can effectively acquire 3D terrain information.

Objects of the embodiments of the present invention are not limited to the above-mentioned purposes, and other objects not mentioned above will be clearly understood by those skilled in the art from the description below. .

According to an embodiment of the present invention, collecting drone image data in which a map production area is photographed using a drone; collecting satellite image data of the mapping area; Obtaining corrected drone image data by correcting perspective distortion of the drone image data; acquiring corrected satellite image data by correcting shadow distortion of the satellite image data; Performing image matching on the mapping area using the corrected drone image data and the corrected satellite image data based on the ground control point of the mapping area; and constructing an orthoimage through modeling of the matched image obtained through the image matching to produce a 3D map; a method for producing a 3D map using a drone may be provided.

In addition, according to an embodiment of the present invention, in the step of obtaining the corrected drone image data, a 3D map production method using a drone for correcting the perspective distortion using depth information of the drone image data may be provided. .

In addition, according to an embodiment of the present invention, in the step of obtaining the correction satellite image data, after generating a shadow sample for correcting the shadow distortion and selecting a correction target range using the generated shadow sample, the A method of producing a 3D map using a drone performed in a manner of generating a shadow distortion correction image according to pixel information on a target range of correction may be provided.

In addition, according to an embodiment of the present invention, the performing of the image matching may include detecting a first ground reference point corresponding to the ground reference point from the corrected drone image data and corresponding to the ground reference point from the corrected satellite image data. After detecting the second ground control point, a 3D map production method using a drone that performs the image registration using the first ground control point and the second ground control point may be provided.

In addition, according to an embodiment of the present invention, in the step of producing the 3D map, a 3D map production method using a drone in which the matched image is modeled in a point cloud form or a mesh form may be provided.

In addition, according to an embodiment of the present invention, in the step of producing the 3D map, when the point cloud is modeled, a digital surface model (DSM), a digital elevation model (DEM), and A method of producing a 3D map using a drone modeled using at least one selected from digital terrain models (DTMs) may be provided.

In addition, according to an embodiment of the present invention, in the step of producing the 3D map, a 3D map production method using a drone for constructing the orthoimage by rearranging the modeled matched image according to image coordinates is provided. can

The present invention collects drone image data and satellite image data for the map production area, corrects perspective distortion of drone image data and shadow distortion of satellite image data, and corrects drone image data and corrected satellite image data based on ground control points. After performing image registration for the map production area using , by constructing an ortho image through modeling of the registration image to produce a 3D map, it is possible to correct image distortion to accurately represent the actual situation of the desired region, as well as 3D terrain information can be obtained effectively.

1 is a flowchart showing a process of producing a 3D map using a drone according to an embodiment of the present invention;
2 is a diagram illustrating a 3D map production system using a drone according to an embodiment of the present invention;
3 to 5 are views for explaining the correction of perspective distortion of drone image data according to an embodiment of the present invention;
6 is a diagram for explaining correction of shadow distortion of satellite image data according to an embodiment of the present invention;
7 to 9 are diagrams for explaining image matching according to an embodiment of the present invention;
10 is a diagram illustrating modeling using a numerical surface model according to an embodiment of the present invention;
11 is a diagram illustrating an orthoimage constructed according to an embodiment of the present invention.

Advantages and characteristics of the embodiments of the present invention, and methods for achieving them will become clear with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and only these embodiments make the disclosure of the present invention complete, and common knowledge in the art to which the present invention belongs. It is provided to completely inform the person who has the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numbers designate like elements throughout the specification.

In describing the embodiments of the present invention, if it is determined that a detailed description of a known function or configuration may unnecessarily obscure the subject matter of the present invention, the detailed description will be omitted. In addition, terms to be described later are terms defined in consideration of functions in the embodiment of the present invention, which may vary according to the intention or custom of a user or operator. Therefore, the definition should be made based on the contents throughout this specification.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a flowchart illustrating a process of producing a 3D map using a drone according to an embodiment of the present invention, and FIG. 2 is a diagram illustrating a 3D map production system using a drone according to an embodiment of the present invention. 3 to 5 are diagrams for explaining correction of perspective distortion of drone image data according to an embodiment of the present invention, and FIG. 6 illustrates correction of shadow distortion of satellite image data according to an embodiment of the present invention. 7 to 9 are diagrams for explaining image matching according to an embodiment of the present invention, and FIG. 10 illustrates modeling using a numerical surface model according to an embodiment of the present invention. FIG. 11 is a diagram illustrating an orthoimage constructed according to an embodiment of the present invention.

Referring to FIGS. 1 to 11 , it is possible to collect drone image data of a map production area captured using a drone 10 (step 110).

In the step 110 of collecting such drone image data, the drone 10 may be flown and collected according to the coordinates of the ground reference points surveyed in the map production area.

For example, the drone 10 includes a flying means, a photographing means, a wireless communication module, a GPS receiver, a pressure sensor, a gyro sensor, etc. in the main body of the drone, and flies unmanned to a desired area according to wireless control, and photographs the area. Thus, the drone image data can be wirelessly transmitted to the map production device 30 for map production. Such a mapping device 30 may be provided in a control center that performs wireless control of the drone 10 or may be included in the control center.

Here, the drone main body may be provided with various components for operation and control of the drone 10. Using a flight means including a propulsion motor, a propeller, etc. It can fly according to the route, and it can acquire drone image data for 3D map production by using a photographing means.

In addition, the drone 10 may receive a coordinate signal transmitted from the control center using a wireless communication module, and use a GPS receiver to receive a GPS signal from a global navigation system (GPS) provided in the satellite image collection device 20. may be received, the air pressure around the drone may be measured in real time to detect the altitude of the drone body using a pressure sensor, and it may be provided to detect vertical inversion of the drone body using a gyro sensor.

Here, the drone 10 may obtain various satellite information related to location information through a global positioning system (GPS) and an inertial navigation system (INS) of the satellite image collection device 20. there is.

The drone 10 having the configuration as described above obtains the coordinates of the ground reference point by surveying the ground reference point in order to fly unmanned in the mapping area, determines the flight path according to the coordinates of the ground reference point surveyed in the mapping area, After flying the drone 10 with reference to the coordinates of the ground control point surveyed according to the determined flight path and photographing the area, the photographed drone image data may be collected.

Next, the satellite image collection device 20 may collect satellite image data of the mapping area captured (step 120).

In the step 120 of collecting such satellite image data, as the satellite image collection device 20, it can be collected using a satellite navigation system (GPS) and an inertial navigation system (INS).

For example, the satellite image collection device 20 may include a satellite navigation system (GPS) and an inertial navigation system (INS). The satellite navigation system (GPS) is composed of at least 24 or more GPS satellites, and the GPS satellites The precise location of each GPS satellite can be calculated using each unique signal and trajectory parameter, and each location information of the GPS satellites calculated using the GPS receiver provided in the drone 10 or the map production device 30 The exact location of the drone 10 or the mapping device 30 may be calculated through trilateration and trilateration.

In addition, the inertial navigation system (INS) is a system that helps the drone 10 fly to a desired location by calculating and controlling the rotation and positional movement of the drone 10, including a gyro and an accelerometer that serve as an inertial sensor. And it can receive and collect high-resolution satellite image data captured through the inertial navigation system (INS).

The satellite navigation system (GPS) and the inertial navigation system (INS) as described above may be built into the satellite image collection device 20 as an integrated system.

As described above, various satellite information provided by the satellite image collection device 20 including the satellite navigation system (GPS) and the inertial navigation system (INS) can be provided to the drone 10 and the map production device 30. .

Next, the map production device 30 may acquire corrected drone image data by correcting the perspective distortion of the drone image data (step 130).

In step 130 of acquiring such corrected drone image data, perspective distortion can be corrected using the depth information of the drone image data. The coordinate position of the pixel is obtained, and after calculating the local normal vector through the position information of the neighboring pixels in each pixel, the normal vector of the plane is calculated based on this, and the rotation is converted through the normal vector of the calculated plane. Perspective distortion can be corrected by calculating a rotation axis and rotation angle to be used for , and performing rotation conversion on each pixel to obtain a mapping position.

For example, the drone image data may include stereo image data, and after obtaining disparity information using the stereo image as a parallel vision stereo model, depth information of the image may be generated using the disparity information.

Here, a disparity image can be generated using the disparity information. After preprocessing including image correction to compensate for the camera's visual error and stereo matching to find matching points for each pixel of the left and right images, The distance of the corresponding point for each pixel by the horizontal displacement of the left image and the right image can be expressed as an image. It can be created by detecting pixels and matching points.

The coordinate position of each pixel in the camera coordinate system can be obtained from the depth information generated as described above. For distortion correction, the camera coordinate system can be converted into a camera coordinate system in which the front optical axis of the camera is the Z-axis, with the camera focus as the origin, using the depth information and the camera factor.

Here, the depth information becomes the distance of the Z axis in the camera coordinate system, and the coordinates in the image coordinate system can be converted into positional information on the coordinate system that becomes the center of the image. It shows that the coordinates of the image are converted to the coordinate system (b).

In addition, the distance f to the viewport where the image is projected can be calculated using the following Equation 1 or Equation 2 using the viewing angle and resolution of the camera.

At this time, Equation 1 calculates f through the vertical viewing angle fov _v of the camera and the vertical resolution h of the camera, and Equation 2 calculates f through the horizontal viewing angle fov _h of the camera and the vertical resolution w of the camera. , the calculation results of the two equations are the same.

The position in the camera coordinate system (P _c (x, y) = ( x _c ,y _c ,z _c )) can be converted.

Through this process, positional information in the camera coordinate system as shown in (c) of FIG. 3 can be obtained.

Next, in each pixel constituting a plane image, a local normal vector for a plane belonging to a pixel can be obtained through pixels surrounding one pixel. As shown in FIG. 5, in P(x,y) A spatial relationship between neighboring pixels may be expressed using a camera coordinate system.

For example, if the position in the camera coordinate system corresponding to the position P(x,y) in the image coordinate system is P _c (x, y), the position information P _c of the points located above and below the pixel in the camera coordinate system (x,y+1), P _c (x,y-1) and the positional information P _c (x+1,y), P _c (x-1,y) in the camera coordinate system of the points on the left and right Using Equation 4, two bets can be generated.

Then, using the fact that the vector generated as the cross product of the two vectors is perpendicular to the two vectors, the cross product of the two vectors is obtained through the following Equation 5 to obtain the local normal vector N _xy at the pixel P(x,y). can be saved

When the local normal vectors of each pixel obtained as described above are summed up, the normal vector N in the plane area can be obtained, and if the normal vector in the plane area is parallel to the Z axis through the rotation transformation of the stereo image, the plane image can be corrected with an image parallel to the xy plane to remove perspective distortion. At this time, the unit vector of the plane normal vector subjected to the rotation conversion becomes N'(0.0,1).

In addition, the rotation conversion matrix R, which rotates from the plane with the normal vector N to the plane with the normal vector N', can be obtained through the following Equation 6 by obtaining the unit vector u serving as the rotation axis and the rotation angle θ.

At this time, the unit vector u, which is the axis of rotation change, can calculate a regular vector perpendicular to the plane formed by the two normal vectors through the cross product of the two normal vectors as shown in Equation 7 below, and the rotation angle θ is two The angle between the normal vectors may be calculated as in Equation 8 below.

When the position P _c (i, j) of each pixel in the camera coordinate system is rotated through the conversion matrix as described above through the following Equation 9, the position after rotation conversion P' _c (i, j) = (x' _c , y' _c , z' _c ) can be calculated.

At this time, since P' is a coordinate on _the camera coordinate system, it must be converted back to the image coordinate system to use it for image processing. After converting to ' _v , y' _v ), the origin can be set back to the original point to obtain the pixel P'(x',y') corresponding to the pixel P(x,y) during correction conversion.

As described above, the depth information is generated using the drone image data, the coordinate position of each pixel is obtained using the generated depth information, and the local normal vector is calculated from the position information of the neighboring pixels in each pixel. After that, calculate the normal vector of the plane based on this, calculate the rotation axis and rotation angle to be used for rotation conversion through the calculated normal vector of the plane, and obtain the mapping position by performing rotation conversion for each pixel (eg For example, perspective distortion can be corrected by way of acquisition of the position of the camera coordinate system through rotation transformation, conversion to the image coordinate system, circular correction, etc.).

Next, the map production apparatus 30 may acquire corrected satellite image data by correcting shadow distortion of the satellite image data (step 140).

In the step 140 of acquiring such correction satellite image data, a shadow sample for shadow distortion correction is created, a correction target range is selected using the generated shadow sample, and shadow distortion is performed according to pixel information for the correction target range. It may be performed in a manner of generating a correction image.

First, the generation of shadow samples was generated by extracting pixels belonging to the road area (S1 area) with distorted information through shadow and the road area (S2 area) not affected by shadow, respectively. ) method, the spectral distance was selected (for example, by comparing the results of each shadow sample generated from an arbitrary spectral distance) to create a shadow sample, and the determined optimal spectral distance was used. By doing so, each shadow sample including shadow sample 1 as a correction target and shadow sample 2 as a reference target can be created.

For example, a shadow sample 1 to be corrected may be generated for a shadow area using a region expansion technique, and a shadow sample 2 to be referenced may be generated for an area unaffected by shadows. Here, each sample may be generated by extracting a cluster of pixels representing pixels to be corrected.

The area expansion technique as described above can generate a shadow sample based on a preset seed pixel and a spectral distance. It is possible to select pixels (ie, pixels in the S2 region) that are clearly undeveloped roads as seed pixels necessary for generating samples of the S1 region and the S2 region through image reading.

In addition, in the case of the spectral distance, since the sample must be created with clustered pixels under the influence of the pixel value of the seed pixel, the final spectral distance value can be selected in consideration of the characteristics of the sample according to the spectral distance. After comparing the generated samples using the numerical value as the spectral distance value, a final sample can be selected from among the samples generated under the influence of the seed pixel. Here, the random number is a value of 11 steps from 20 to 120 at intervals of 10, which can be equally applied to both the case of generating shadow sample 1 in the S1 area and the case of creating the shadow sample 2 in the S2 area. there is.

Second, in selecting the range to be corrected, all pixels belonging to the pixel value range of shadow sample 1 are considered as correction targets, but they can be corrected to different levels depending on the degree of distortion. Shadow sample 1 in S1 area and S2 area By analyzing the shadow sample 2 of , the pixel value range of the shadow sample 1 can be divided into a first correction level, a second correction level, and a third correction level to select a correction target range.

Here, the first correction level is the pixel value range in which the pure S1 area is relatively greatly distorted, and includes pixels belonging to the range from the minimum value of the S1 area to the minimum value of the S2 area, and the second correction level is the first correction level. Corresponding to the region receiving relatively lower distortion (ie, S1>S2), it includes pixels belonging to the range from the minimum value of the region S2 to the pixel value at the point where the sample histograms of the region S1 and S2 intersect, and the third The correction level may include pixels belonging to the range from the pixel value at the point where the sample histograms of the S1 and S2 regions intersect to the maximum value of the S1 region corresponding to the region that does not require correction (ie, S1<S2). there is.

For example, it is possible to establish an operation for correcting the values of pixels to be corrected existing in the previous image to be similar to the values of pixels to be referenced using shadow sample 1 in the S1 area and shadow sample 2 in the S2 area. Since the shadow sample 1 of the area includes pixels with distorted information due to the shadow, it can be converted to be included in the pixel range of the undistorted shadow sample 2 of the area S2. That is, the pixel value of shadow sample 1 can be converted to be as similar as possible to the calcination value of shadow sample 2.

This pixel value conversion process can be performed according to the first correction level, the second correction level, and the third correction level, because the degree of each received shadow effect can be different even in the case of pixels in the shadow area. Here, pixel value conversion may be performed based on the average pixel value of shadow sample 2.

In addition, the pixels extracted as correction targets can be divided into three areas and corrected as shown in FIG. 6. ① is the range of pixel values that do not overlap with the S2 area at all and are determined to belong to the S1 area. Since it is an area that has undergone a large distortion based on the average pixel value of , it is an area that needs to be corrected with a relatively large numerical value, and ② is the range where the pixel values of the S1 and S2 areas overlap. It is an area relatively larger than the ratio, and since it is located closer to the average pixel value of the S2 area than ①, correction toward the average pixel value of the S2 area is possible with only a small width value, and ③ is a pixel that may belong to the S1 area. However, since the probability of being a pixel belonging to the S2 region is greater, even if the pixel value is not moved to the center of the average pixel value of the S2 region, the overall correction result is not greatly affected, so the original value can be maintained without numerical conversion. .

On the other hand, shadow sample 1 of the S1 area and shadow sample 2 of the S2 area are basic variables for building a correction model for correcting shadow distortion (ie, the value of a point that is an index for dividing each area in FIG. 6, S1 (including the minimum value of the area, the minimum value of the S2 area, the intersection value of the S1 area and the S2 area, the maximum value of the S1 area, etc.) .

Third, generating a shadow distortion correction image can perform a numerical conversion to move the pixel values belonging to the S1 region to the maximum within the range centered on the average pixel value of the S2 region, and the minimum value of the S1 region extracted from the shadow sample. It is possible to provide an image in which shadow distortion is corrected (ie, corrected satellite image data) by inputting the minimum value of the region S2 and the pixel values of the point where the two groups intersect.

For example, pixels to be corrected may perform correction processes corresponding to the first correction level, the second correction level, and the third correction level, respectively. It can be determined by referring to the value extracted from shadow sample 2 of the area.

This correction image can be converted numerically through input of basic variables including the minimum value of the S1 region, the minimum value of the S2 region, the intersection value of the S1 region and the S2 region, and the maximum value of the S1 region, and their calculation. For example, the correction operation may be performed using a spatial model language (SML) or the like.

Here, the correction operation can be performed individually and sequentially for the three regions (ie, ①, ②, ③) as shown in FIG. 6, where the ratio of the S1 region is relatively greater than that of the S2 region. can be performed first and applied to the area ① composed of pure S1 areas. The reason is that since the pixel values of the correction area ① are relatively smaller than the pixel values of the ② area, if the pixel values of the ① area are converted first, they are included again in the correction area ② and redundant conversion occurs for the same pixel. At this time, the area ③ can maintain its original value.

After creating a shadow sample for shadow distortion correction through the above-described process and selecting a correction target range using the generated shadow sample, a shadow distortion correction image (i.e., correction) according to pixel information for the correction target range satellite image data) can be created.

Next, the map production device 30 may perform image registration for the map production area using the corrected drone image data and the corrected satellite image data based on the ground control point of the map production area (step 150).

In step 150 of performing such image matching, a first ground reference point corresponding to the ground control point is detected from the corrected drone image data, and a second ground control point corresponding to the ground control point is detected from the corrected satellite image data. Image matching can be performed using the ground control point and the second ground control point, and the first ground control point and the second ground control point can be detected using the image size, respectively.

For example, SIFT (scale invariant feature transform) can be used to image the corrected drone image data and the corrected satellite image data by detecting the first ground control point and the second ground control point, respectively. After detecting the first ground control point according to , and detecting the second ground control point according to the image size in the corrected satellite image data, the first description vector and the second description vector are used to identify the first ground control point and the second ground control point. are generated, respectively, and image matching may be performed by measuring the similarity between the generated first narration vector and the second narration vector.

Here, the first ground control point and the second ground control point may be designated and detected in consideration of the ground control point surveyed in the mapping area.

Specifically, a plurality of Gaussian images corresponding to a drone-captured image (i.e., corrected drone image data) and a difference operation image of the plurality of Gaussian images are used, but the first ground reference point can be detected in consideration of the image size. , As shown in FIG. 7, a plurality of Gaussian images to which a Gaussian filter of a constant multiple is applied to the drone video image is obtained, and a plurality of difference operation images are obtained by performing difference operation of each adjacent image in the obtained plurality of Gaussian images. can be obtained.

And, as shown in FIG. 8, in order to find the first ground reference point in the plurality of difference operation images obtained, a point of 8 pixels around a specific point (X) in the current image and a point of 9 pixels in adjacent images on both sides (that is, , 18 points), and according to the result of comparing the points of a total of 26 pixels based on the specific point, the point with the smallest or largest value is selected, and it is possible to detect the maximum and minimum points by repeating this. can

Next, the points selected according to the detection result may be post-processed according to the size of the image, and the first ground control point may be detected in such a way that the most stable points are selected.

Meanwhile, in the corrected satellite image data, the second ground control point can also be detected similarly to the detection method of the first ground control point as described above.

Next, the first ground control point detected from the corrected drone image data is described as a first description vector, and the second ground control point detected from the corrected satellite image data is described as a second description vector. Image matching can be performed by matching 2 ground control points using a first description vector and a second description vector, and image gradients and directions can be obtained for pixels around a specific point.

In addition, for the rotate-invariant property, the descriptor (keypoint descriptor) and the gradient direction (image gradients) can be rotated relative to the point direction, and gradients of all levels are calculated and shown in FIG. It can be represented by small arrows as shown.

Here, the descriptor is formed in the form of a vector including values of histogram entries in all directions, and the lengths of each right arrow as shown in FIG. 9 may correspond to the values of histogram entries.

As described above, after the detection of the first ground control point and the second ground control point according to FIGS. 7 and 8 and the description of the first ground control point and the second ground control point according to FIG. 9 are completed, corrected drone image data and corrected satellite image Image registration of data can be performed.

For example, when image matching is performed using similarity comparison between the first ground control point and the second ground control point, the number of first ground control points in the corrected drone video image is N, and the second ground control point extracted from the corrected satellite image data If the number of is M, a total of N*M keypoint matching can occur, which has the advantage of requiring fewer operations than the existing template matching method based on pixel-to-pixel comparison.

Here, in the case of image matching determination, as shown in FIG. 9, weights are given with a Gaussian model for the light blue line part, which is the matching point, and the total score for each point can be summed in the final matching determination. there is. That is, image matching can be performed by obtaining a high score at a point where the sky blue line portion is concentrated.

Next, in the mapping apparatus 30, a 3D map may be produced by constructing an orthoimage through modeling of the matched image obtained through image matching (step 160).

In the step 160 of producing such a 3D map, the matched image may be modeled in the form of a point cloud or mesh. In the case of modeling in the form of a point cloud, a digital surface model (DSM) or a digital elevation model (DEM) : It can be modeled using at least one selected from a digital elevation model) and a digital terrain model (DTM).

Here, the digital surface model (DSM) is ground surface information including all artificial features such as trees and buildings, and is a point cloud filled with changing elevation values using an aviation LiDAR (light detection and ranging) system. ) can be derived. Here, you may also include building roofs, treetops, power lines, and other feature elevations.

In addition, the digital elevation model (DEM) is ground surface data that does not include natural features such as trees and artificial features such as buildings. There is a raster method in which the elevation is divided into grids and a triangular irregular network (TIN) method in which the ground surface is expressed by dividing into irregular triangles. A smooth digital elevation model is obtained by removing non-surface points such as bridges and roads. can do.

On the other hand, a digital terrain model (DTM) is numerical information on the location and elevation of points distributed at a reasonable density, including linear features of the surface topography covered with nothing, and can be obtained through stereo photogrammetry, A digital elevation model can be derived from the point distribution and contour lines at regular intervals through interpolation.

In addition, in step 160 of producing a 3D map, an orthoimage may be constructed by rearranging the modeled registered image according to image coordinates.

Meanwhile, when the matched image is modeled in a mesh form, depth information may be calculated from the image of the matched image, and a mesh image of the mapping area may be formed based on the calculated depth information. An ortho image can be constructed by rearranging the generated texture according to the image coordinates and mapping it to the mesh image of the mapping area.

Therefore, the present invention collects drone image data and satellite image data for the map production area, corrects perspective distortion of drone image data and shadow distortion of satellite image data, and corrects drone image data and correction satellite based on ground control points. After performing image registration for the map production area using image data, by constructing an ortho image through modeling of the matching image and producing a 3D map, image distortion can be corrected to accurately represent the actual situation of the desired region. In addition, 3D terrain information can be effectively obtained.

In the above description, various embodiments of the present invention have been presented and described, but the present invention is not necessarily limited thereto. It will be readily apparent that branch substitutions, modifications and alterations are possible.

10 : Drone
20: satellite image collection device
30: map production device

Claims (7)

Collecting drone image data in which the map production area is photographed using a drone;
collecting satellite image data of the mapping area;
Obtaining corrected drone image data by correcting perspective distortion of the drone image data;
acquiring corrected satellite image data by correcting shadow distortion of the satellite image data;
Performing image matching on the mapping area using the corrected drone image data and the corrected satellite image data based on the ground control point of the mapping area; and
Including; constructing an orthoimage through modeling of the matched image obtained through the image matching to produce a 3D map;
In the obtaining of the corrected drone image data, the perspective distortion is corrected using depth information of the drone image data,
After generating the depth information using the drone image data, acquiring the coordinate position of each pixel using the generated depth information, and calculating a local normal vector through the position information of neighboring pixels in each pixel. , Calculate the normal vector of the plane based on the local normal vector, calculate the rotation axis and rotation angle to be used for rotation conversion through the calculated normal vector of the plane, and calculate each pixel according to the calculated rotation axis and rotation angle. The perspective distortion is corrected by performing the rotation conversion on , and obtaining a mapping position.
The step of acquiring the corrected satellite image data may include generating a shadow sample for correcting the shadow distortion, selecting a correction target range using the generated shadow sample, and then shadowing according to pixel information for the correction target range. It is performed in a way to generate a distortion correction image,
The shadow sample is generated by extracting pixels belonging to S1 region, which is a road region having information distorted through shadow, and S2 region, which is a road region that is not affected by shadow, respectively;
In the correction target range, all pixels belonging to the pixel value range of the shadow sample are considered as correction targets, but selected to be corrected to different levels according to the degree of distortion,
The shadow distortion correction image is generated by performing a numerical conversion of moving pixel values belonging to the S1 region within a range centered on the average pixel value of the S2 region.
A method for creating 3D maps using drones. delete delete The method of claim 1,
In the performing of the image matching, after detecting a first ground control point corresponding to the ground control point from the corrected drone image data and detecting a second ground control point corresponding to the ground control point from the corrected satellite image data, A method for producing a 3D map using a drone to perform the image matching using the first ground control point and the second ground control point. According to claim 1 or claim 4,
In the step of producing the 3D map, the 3D map production method using a drone in which the matching image is modeled in a point cloud form or mesh form. The method of claim 5,
The step of producing the 3D map is a digital surface model (DSM), a digital elevation model (DEM), and a digital terrain model (DTM) when the map is modeled in the form of a point cloud. A method for producing a 3D map using a drone modeled using at least one selected from the following. The method of claim 6,
In the step of producing the 3D map, the 3D map production method using a drone builds the orthoimage by rearranging the modeled matching image according to image coordinates.