KR102003187B1 - Method and apparatus for modeling resultant image to which each of ground control points including location information of survey site is matched using images captured by unmanned air vehicle - Google Patents

Abstract

The present invention provides a method for modeling a result image to which each of a plurality of ground control points (GCPs) including location information by measurement in a survey site is matched by using a plurality of images photographed by an unmanned air vehicle flying a route including a plurality of way points of the survey site. The way point which is connected to each point of points corresponding to the plurality of GCPs is determined to identify and analyze first images including the determined way point so as to determine a center point of each point, thereby matching the determined center point of each point with the GCP which is an estimated GCP. Moreover, two-dimensional or three-dimensional result image is modeled by using a result of matching and the plurality of images.

Description

TECHNICAL FIELD [0001] The present invention relates to a method and an apparatus for modeling a resultant image in which each of GCPs (Ground Control Points) including position information of a surveying object is matched using images photographed by an unmanned aerial vehicle CONTROL POINTS INCLUDING LOCATION INFORMATION OF SURVEY SITE IS MATCHED USING IMAGES CAPTURED BY UNMANNED AIR VEHICLE}

Embodiments relate to a method and apparatus for performing a survey on a survey site using images photographed by a unmanned aerial vehicle such as a drone, And a method and apparatus for modeling a resultant image, each of the ground control points being matched.

Surveying is an indispensable task in building a building or facility.

In surveying a survey site (for example, a commercial site), there are cases where a person can not directly measure at least part of the survey site due to environmental or geographical factors of the survey site. In this case, measurements can be made on the survey site using a unmanned aerial vehicle such as a drone. By using images (photographs) of survey sites photographed using an unmanned aerial vehicle for surveying, surveying can be possible for areas that can not be directly measured by people at the survey site.

In the case of performing the survey using the unmanned aerial vehicle, the inner work of matching the ground control point (GCP) actually measured on the survey site to the images taken by the unmanned aerial vehicle should be performed. If the size of the photographed images is large and the number of them is large, such internal work requires a great deal of time and resources. In addition, when performing such matching manually, it is difficult to guarantee the accuracy of the matching between the GCP and the photographed image.

Accordingly, the GCP actually measured for the survey site can be accurately and quickly matched to the image (s) photographed using the unmanned aerial vehicle, and thus, a calibration result such as a two-dimensional high-precision registration image or a three- There is a need for a method and apparatus that can conveniently generate images.

Korean Registered Patent No. 10-1532582 (June 24, 2015) discloses a method for processing cadastral survey data. In order to register the land in the land study or restore the boundaries registered in the cadastral study on the land surface, In order to efficiently and systematically maintain and manage the obtained cadastral survey data when setting a reference point to measure the boundary or area of the cadastral survey data.

The information described above is for illustrative purposes only and may include content that does not form part of the prior art and may not include what the prior art has to offer to the ordinary artisan.

One embodiment uses each of a plurality of GCPs including location information by surveying on a survey site using a plurality of images taken by an unmanned aerial vehicle flying a path that includes a plurality of waypoints on a survey site And provide a method of modeling the matched resultant image.

According to an aspect, there is provided an image processing method comprising the steps of: using a plurality of images taken by an unmanned aerial vehicle flying through a path including a plurality of waypoints on a survey site, performed by a computer system, A method of modeling a resultant image in which a plurality of GCPs (Ground Control Points) including a plurality of GCPs are matched, the method comprising: photographing the survey site including points corresponding to the plurality of GCPs; The method comprising: determining a waypoint associated with each point in the plurality of waypoints corresponding to the plurality of GCPs; identifying first ones of the plurality of images including the determined waypoint; Determining a center point of each point by analyzing first images, Matching an estimated GCP associated with each of the points with a GCP corresponding to each of the plurality of GCPs; and determining each of the estimated GCPs associated with the points as an estimated GCP associated with each of the points, There is provided a method of modeling a resultant image, the method comprising modeling a two- or three-dimensional result image using the plurality of images and a result of matching the plurality of GCPs.

Wherein the plurality of images are obtained by photographing the survey site so as to have a predetermined rate of overlapping of photographs, and each of the plurality of images may be obtained by photographing a part of the survey site.

A GCP recognition plate may be disposed at each of the points corresponding to the plurality of GCPs of the survey target site, and the plurality of images may be obtained by photographing the survey target site on which the GCP recognition plate is disposed.

The step of determining the center point of each of the points may determine a center point of the GCP recognition plate disposed at each of the points and may determine the center point of the GCP recognition plate as the estimated GCP.

The step of determining the center point of each point may include analyzing the first images using a machine learning model, a CNN (Convolution Neural Network) model, or a DNN (Deep Neural Network) model in which the shape of the GCP recognition plate is trained, The center point of each point can be determined.

The matching step includes recognizing an outline of a graphic object corresponding to the GCP recognition plate disposed at each of the points, extracting a pixel value corresponding to a center point of the graphic object based on the recognized outline, Of the GCP with the coordinate value of the GCP corresponding to each of the plurality of GCPs.

The matching step may include recognizing an outline of the figure using a machine learning model, a CNN (Convolution Neural Network) model, or a DNN (Deep Neural Network) model in which the shape of the GCP recognition plate is trained, The corresponding pixel value can be extracted.

Wherein the step of recognizing the outline includes recognizing a plurality of outlines of a graphic form corresponding to the GCP recognition plate disposed at each of the points and determining an average of the plurality of outlines as an outline of the graphic, An outline beyond a predetermined standard deviation among a plurality of outlines may be excluded from the calculation of the average.

The step of determining a waypoint associated with each of the plurality of points may determine a closest waypoint for each of the plurality of waypoints as a waypoint associated with each of the plurality of waypoints.

In determining the waypoint associated with each of the points, one waypoint of the plurality of waypoints may be associated with one of the points.

The resultant image may be a single image for the survey site generated by combining the plurality of images and may be a calibrated image according to a result of matching each of the estimated GCPs of the points to each of the plurality of GCPs .

According to another aspect, there is provided an image processing method comprising the steps of: using a plurality of images taken by an unmanned aerial vehicle flying through a path including a plurality of waypoints on a survey site, performed by a computer system, Wherein the plurality of images are images of the survey site including points corresponding to the plurality of GCPs, the method comprising the steps of: Determining a waypoint associated with a first one of the plurality of waypoints corresponding to the plurality of GCPs, identifying first ones of the plurality of images including the determined waypoint, ; Determining a center point of the first point by analyzing the first images; Matching an estimated GCP associated with the first point to a GCP corresponding to the first one of the plurality of GCPs, the estimated GCP associated with the first point being a center point of the determined first point; Determining the waypoint, identifying the step, determining the center point, and performing the matching on each of the points. And modeling the resulting image, wherein each of the plurality of GCPs is associated with each of the plurality of GCPs, and modeling a two- or three-dimensional result image using the plurality of images. Method is provided.

According to another aspect, there is provided a method for measuring a position of a measurement target, comprising: a plurality of GCPs including position information by measurement on the measurement target site, using a plurality of images taken by an unmanned aerial vehicle flying a route including a plurality of waypoints Wherein each of the plurality of images includes a point corresponding to the plurality of GCPs, and wherein the plurality of images include a plurality of GCPs, An image acquiring unit

And a processor for analyzing and processing the acquired plurality of images, wherein the processor determines a waypoint associated with each point of points corresponding to the plurality of GCPs of the plurality of waypoints, Determining a center point of each point by analyzing the first images, determining a center point of each determined point as an estimated GCP associated with each of the points, Matching an estimated GCP associated with each of the points with a GCP corresponding to each of the plurality of GCPs; matching each of the estimated GCPs associated with the points to each of the plurality of GCPs; Lt; RTI ID = 0.0 > 2D < / RTI >

A GCP recognition plate may be disposed at each of the points corresponding to the plurality of GCPs of the survey target site, and the plurality of images may be obtained by photographing the survey target site on which the GCP recognition plate is disposed.

The processor may determine a center point of the GCP recognition plate disposed at each of the points and determine the center point of the GCP recognition plate as the estimated GCP.

The processor recognizes an outline of a graphic object corresponding to a GCP recognition plate disposed at each of the points, extracts a pixel value corresponding to a center point of the graphic object based on the recognized outline, With the coordinate values of the GCPs corresponding to the respective points among the plurality of GCPs.

Determining a waypoint associated with each point in a plurality of points corresponding to a plurality of GCPs based on the actual surveying and comparing the first images containing the determined waypoints to a machine learning model trained in the shape of a GCP awareness plate, Neural Network) model or a DNN (Deep Neural Network) model, it is possible to accurately determine a center point (estimated GCP, AIGCP) of a point corresponding to each of a plurality of GCPs. In addition, matching of the estimated GCP (i.e., AIGCP) with the actual GCP can be improved by using such a machine learning model, CNN model, or DNN model.

According to the embodiment, there is no need to manually perform an inner work task of matching the GCP with images photographed by the unmanned aerial vehicle, and the matching between the GCP and the photographed images can be automatically and accurately performed.

In addition, a two-dimensional high-precision matching image or a three-dimensional modeling image as a result image can be easily generated, and resource and time consumption can be efficiently consumed due to internal work required for a survey using an unmanned aerial vehicle.

FIG. 1 illustrates a method of modeling a matching result image using a GCP including positional information of a measurement target using images taken by an unmanned aerial vehicle according to an exemplary embodiment of the present invention.
2 is a block diagram illustrating the structure of a computer system for modeling a resultant image and an unmanned aerial vehicle for photographing a survey site, according to an embodiment.
3 is a flowchart illustrating a method of modeling a GCP matching result image including positional information of a measurement target using images taken by an unmanned aerial vehicle according to an embodiment.
4 is a flow diagram illustrating a method for matching an estimated GCP (i.e., AIGCP) determined based on image analysis and a GCP surveyed at a survey site, according to an example.
5 is a flowchart illustrating a method of recognizing an outline of a figure corresponding to a GCP recognition plate disposed on a survey site according to an example.
FIG. 6 illustrates a method of photographing a survey site by flying a path including a plurality of waypoints on a measurement target, according to an example.
FIG. 7 illustrates a method for determining a plurality of images taken by an unmanned aerial vehicle, a waypoint associated with a GCP, and images including the waypoint, according to an example.
FIG. 8 shows a method of modeling an image photographed by an unmanned aerial vehicle as a calibrated resultant image, according to an example.

In the following, embodiments will be described in detail with reference to the accompanying drawings. Like reference symbols in the drawings denote like elements.

FIG. 1 illustrates a method of modeling a matching result image using a GCP including positional information of a measurement target using images taken by an unmanned aerial vehicle according to an exemplary embodiment of the present invention.

1, a calibration result image 130 is modeled using the images of the survey site 120 photographed by the unmanned air vehicle 110 in the case of performing the survey using the unmanned air vehicle 110 .

The survey site 120 may represent a zone (a zone of the land) to be surveyed, including a commercial site, a construction site, and the like.

The illustrated unmanned aerial vehicle 110 may be, for example, a drone. The unmanned aerial vehicle 110 can photograph the survey site 120 while flying a predetermined path on the survey site 120. The plurality of images of the photographed survey site 120 may be communicated to the computer system 100 and the computer system 100 may analyze and process the plurality of delivered images to model the calibrated resultant image 130 have.

A predetermined number of GCPs may be included on the survey site 120, and the unmanned air vehicle 110 may photograph the survey site 120 including the GCP.

The GCP may be a reference point (ground reference point) on the measurement target site 120 and a point at which the position information is known by the actual measurement. The GCP recognition plate may be disposed on the survey site 120 at a location corresponding to the GCP. The surveyor can obtain the location information of the GCP by measuring the location where the GCP recognition plate is placed. For example, in the illustrated survey site 120, there are four GCPs, and a GCP recognition board may be disposed at a point corresponding to each GCP. The measurement for acquiring the position information of the GCP can be performed before the photographing of the measurement target site 120 of the unmanned air vehicle 110. [ Alternatively, the measurement for acquiring the location information of the GCP may be performed after the photographing of the measurement target site 120 of the unmanned air vehicle 110. [ That is, after the GCP recognition plate is placed on the measurement target site 120, the measurement object 120 of the unmanned air vehicle 110 is photographed and the GCP location information (that is, the location information of the point where the GCP recognition plate is located) The measurement can be performed at any time before and after the photographing of the survey target site 120 of the unmanned aerial vehicle 110. [

The unmanned aerial vehicle 110 can photograph the survey site 120 in which the GCP recognition plate is disposed and the images generated according to the photographing can be used for modeling the resultant image 130. [

The computer system 100 analyzes images photographed by the unmanned air vehicle 110 using, for example, a machine learning model, a CNN (Convolution Neural Network) model or a DNN (Deep Neural Network) model, The point corresponding to the GCP can be identified as the estimated GCP (i.e., AIGCP), and the identified estimated GCP can be matched to the actual measured GCP.

The computer system 100 may model the calibrated resultant image 130 using the images obtained by the unmanned aerial vehicle 110 and the matching results between the estimated GCP and the actually measured GCP.

The resulting image 130 may be, for example, a two-dimensional high-precision registration image or a three-dimensional modeling image. The modeled resultant image 130 may be used for construction and precision measurement of the survey site 120, and the like. For example, the area of a partial area of the measurement target site 120 may be calculated using a two-dimensional high-precision matching image, or the amount of building material to be used may be calculated based on the calculated area.

The positional information (for example, the positional coordinates) of the actually measured GCP can be precisely matched (mapped) to the images photographed by the unmanned aerial vehicle 110 through the embodiment. Thus, compared to manually performing internal work related to this matching operation, time and resource consumption can be minimized, and the accuracy of matching can also be dramatically increased.

 The detailed structure and function of the computer system 100 and the unmanned air vehicle 110 and the GCP including the location information of the survey site 120 using the images taken by the unmanned air vehicle 110 130 will be described in more detail with reference to Figs. 2 to 8 to be described later.

2 is a block diagram illustrating the structure of a computer system for modeling a resultant image and an unmanned aerial vehicle for photographing a survey site, according to an embodiment.

Referring to Fig. 2, specific configurations of the unmanned aerial vehicle 110 and the computer system 100 are described.

As described above with reference to FIG. 1, the unmanned aerial vehicle 110 is an apparatus for photographing the survey site 120 by flying the survey site 120, for example, a drone, an unmanned aerial vehicle or other automatic or radio- . For example, the unmanned aerial vehicle 110 may be a plug-in DGPS drones or a plug-in RTK drones.

The unmanned aerial vehicle 110 may fly a predetermined path including a plurality of waypoints on the survey site 120. The predetermined path may be set by the user of the unmanned air vehicle 110. [ For example, a user of the unmanned air vehicle 110 can set a predetermined path through a user terminal associated with the unmanned air vehicle 110 (for example, a terminal equipped with an application related to control of a smart phone or a controller or the unmanned air vehicle 110) have. A user of the unmanned aerial vehicle 110 can designate a plurality of waypoints on a map representing the survey site 120 through a user terminal associated with the unmanned air vehicle 110, It can be set as a path. An exemplary flight path including a plurality of waypoints to the survey site 120 of the unmanned aerial vehicle 110 will be described in more detail with reference to FIG.

The unmanned aerial vehicle 110 can photograph the survey site 120 at each way point on the route. At this time, the center of the photographed image may be a waypoint. Alternatively, the location where the unmanned object 110 photographs the measurement target site 120 may be a position separate from the waypoint.

The waypoint is a position specified on the map, and its positional information (e.g., coordinate value) may be a known value. The position where the unmanned object 110 photographs the measurement target site 120 may be the center point of the photographed image. The center point of the photographed image may be on a path that the unmanned air vehicle 110 travels, and the positional information (e.g., coordinate value) may be a known value.

The unmanned air vehicle 110 may include a communication unit 240, a camera 250, a processor 260, and a storage unit 270.

The communication unit 240 may be a configuration for allowing the unmanned air vehicle 110 to communicate with the computer system 100 and other devices such as a user terminal. In other words, the communication unit 240 is a configuration for the unmanned air vehicle 110 to transmit / receive data and / or information wirelessly or wiredly to the computer system 100 and other devices such as a user terminal, A network interface card, a network interface chip, and a networking interface port, or a software module such as a network device driver or a networking program.

The unmanned aerial vehicle 110 may communicate with the computer system 100 or the user terminal via the communication unit 240 or may transmit the captured images to the computer system 100. [

The processor 260 may manage the components of the unmanned aerial vehicle 110 and may be configured to control a flight of the unmanned air vehicle 110 to a predetermined path. For example, the processor 260 may perform processing and calculation of data necessary to control the flight of the unmanned aerial vehicle 110. [ The processor 260 may be at least one processor of the unmanned aerial vehicle 110 or at least one core in the processor.

The camera 250 may be a device for photographing the survey site 120 during flight. The camera 250 can generate an image (image file) by photographing the survey target site 120. [

The storage unit 270 may include storage for storing images generated by the camera 250. The storage unit 270 may be any internal memory of the unmanned aerial vehicle 110 or an external memory device such as a flash memory, an SD card, or the like mounted on the unmanned air vehicle 110. In addition, the storage unit 270 may store information related to a predetermined path for flight of the unmanned aerial vehicle 110 (for example, information on maps and waypoints).

The computer system 100 models the calibration result image 130 for the survey site 120 by acquiring images of the survey site 120 from the unmanned air vehicle 110 and analyzing the acquired images, (E.g.

The computer system 100 may be a server or other computing device for analyzing and processing images obtained from the unmanned aerial vehicle 110. The computer system 100 uses the plurality of images taken by the unmanned aerial vehicle 110 so that each of the plurality of GCPs including the positional information by the survey on the survey site 120 is matched with the resultant image 130, Can be modeled.

The computer system 100 may be, for example, a smart phone, a personal computer, a server computer, a laptop computer, a laptop computer, a tablet, an Internet Of Things device, And may be a terminal used by a user such as a wearable computer. The computer system 100 may be a terminal that communicates with the unmanned air vehicle 110 and controls the unmanned air vehicle 110. Alternatively, the computer system 100 may be embedded in the unmanned aerial vehicle 110, unlike the one shown in FIG.

The computer system 100 may include an image acquisition unit 210, a processor 220, and a display unit 230.

The image acquisition unit 210 may be a configuration for acquiring images photographed by the unmanned aerial vehicle 110. For example, the image acquisition unit 210 may acquire images via the wired or wireless network from the unmanned air vehicle 110, or may acquire images via the external memory device of the unmanned air vehicle 110.

The image acquisition unit 210 may correspond to the communication unit of the computer system 100. [ That is, the image acquisition unit 210 may be a configuration in which the computer system 100 communicates with the unmanned air vehicle 110 and other devices such as a server. In other words, the image acquiring unit 210 may be a computer system (not shown) configured to transmit / receive data and / or information wirelessly or wired to the unmanned air vehicle 110 and other devices such as a server, A network interface card, a network interface chip, and a networking interface port, or a software module such as a network device driver or networking program.

The processor 220 may be a component for managing components of the computer system 100 and for executing programs or applications used by the computer system 100. [ For example, the processor 220 may analyze and process images photographed by the unmanned aerial vehicle 110, and may perform data and calculations necessary to model the resulting image 130. [ The processor 220 may be at least one processor of the computer system 100 or at least one core within the processor. The detailed functions and operations of the processor 220 will be described in more detail with reference to FIGS. 3 to 8 to be described later.

The display unit 230 may be a display device for outputting data input by a user of the computer system 100 or for outputting images photographed by the unmanned air vehicle 110 or a modeled resultant image 130. [

In an embodiment, the processor 220 of the computer system 100 may determine the estimated GCP by analyzing and processing images photographed by the unmanned aerial vehicle 110, and may compare the determined GCP with the GCP based on the actual measurement have. The processor 220 may model the calibrated resultant image 130 using the results of this matching and the images taken by the unmanned aerial vehicle 110.

In this regard, a method of determining the estimated GCP of the processor 220 and a detailed method of GCP matching based on the determined estimated GCP and the actual measurement will be described in more detail with reference to FIGS. 3 to 8 to be described later.

The technical features described above with reference to FIG. 1 can be applied to FIG. 2 as it is, so that redundant description is omitted.

3 is a flowchart illustrating a method of modeling a GCP matching result image including positional information of a measurement target using images taken by an unmanned aerial vehicle according to an embodiment.

3, using the plurality of images taken by the unmanned air vehicle 110 flying by the path including the plurality of waypoints on the survey site 120, performed by the computer system 100, A method for modeling a matching result image 130 of each of a plurality of GCPs including positional information by measurement on the destination site 120 will be described.

The processor 220 may acquire a plurality of images of the survey site 120 photographed by the unmanned aerial vehicle 110 through the image acquisition unit 210. In step 310, For example, the images may be acquired from the unmanned aerial vehicle 110 via wired or wireless communication, or may be acquired via the external memory device of the unmanned aerial vehicle 110. [

The plurality of images may be taken of the survey site 120 including points corresponding to a plurality of GCPs based on the actual survey for the survey site 120. [ For example, the GCP recognition plate may be disposed at each of the points corresponding to the plurality of GCPs of the measurement target site 120, and the plurality of images may be obtained by photographing the measurement target site 120 in which the GCP recognition plate is disposed.

Each of the plurality of photographed images may have photographed a part of the survey site 120. The plurality of images may be photographed so that the measurement target site 120 has a predetermined rate of photographic duplication, and thus the plurality of images may include the entire area of the measurement target site 120 without fail. The predetermined photographic overlap ratio may be, for example, 60% to 80%.

The unmanned air vehicle 110 can photograph the measurement target site 120 at each way point on the route and the center of the captured image can be the corresponding waypoint. Alternatively, the location where the unmanned object 110 photographs the measurement target site 120 may be a position separate from the waypoint.

The waypoint is a position designated on a map for setting the path of the unmanned air vehicle 110, and its location information (e.g., coordinate value) may be a known value.

In step 320, the processor 220 determines whether or not the points corresponding to the plurality of GCPs based on the actual measurement of the survey site 120 among the plurality of waypoints included in the (flight) path of the unmanned air vehicle 110 The waypoint associated with each point can be determined.

In one example, the processor 220 may determine the closest waypoint for each point of the points corresponding to the GCPs among the plurality of waypoints that the path of the unmanned air vehicle 110 includes as waypoints associated with each point have.

Determining a waypoint associated with each point of the points corresponding to the GCPs of step 320, wherein one of the plurality of waypoints included in the path of the unmanned air vehicle 110 corresponds to the GCPs It can be determined to be associated only with a point of one of the plurality of points. If more than one waypoints are determined to be associated with one of the points corresponding to the GCPs, the processor 220 can ignore the remainder except for one waypoint.

However, one of the points corresponding to the GCPs may be associated with a plurality of waypoints. In other words, there may be a one-to-many association between a point corresponding to a GCP and a waypoint, but the inverse may not hold.

Or determining a waypoint associated with each point of the points corresponding to the GCPs of step 320, wherein one waypoint / one point corresponds to one waypoint / one point closest to that one waypoint / It may be determined to be associated only with a point / one waypoint.

At step 330, the processor 220 may identify the first images containing the waypoints determined at step 320, among the plurality of images photographed. That is to say, the processor 220 can identify, as the first images, images containing a determined waypoint that is associated with each point of the points corresponding to the GCPs from the plurality of captured images.

At step 340, the processor 220 may determine the center point of each point of the points corresponding to the GCPs by analyzing the identified first images. That is, the first images identified in step 330 may be images to be analyzed to determine the center point of each point of the points corresponding to the GCPs in step 340.

In general, since the number of images taken by the unmanned aerial vehicle 110 is very large and the capacity of each image is also very large, it is inefficient to analyze all of the images taken by the unmanned aerial vehicle 110 in step 340 have.

In order to solve this problem, in the embodiment, the first images including the waypoint determined in step 320 are first identified by the step 330, the image analysis is performed on the identified first images, The center point of the point associated with the waypoint can be determined. Therefore, through the embodiments, a faster image analysis process and a center point determination process of each point can be performed.

Determining the center point of each point of the points corresponding to the GCPs may be to determine the center point of the GCP recognition plate disposed at each of the points.

In determining the center point of the GCP recognition plate, a machine learning model, a CNN (Convolution Neural Network) model, or a DNN (Deep Neural Network) model in which the shape of the GCP recognition plate is trained can be used. The shape of the GCP recognition plate may be, for example, a circle or a quadrangle, and the GCP recognition plate may be indicated by a symbol such as X or H in a circle or a rectangle.

The processor 220 may determine the center point of each point of the points corresponding to the GCPs by analyzing the first images using the trained machine learning model, the CNN model, or the DNN model.

The processor 220 may determine the center point of each point determined in step 340 as the estimated GCP. The estimated GCP may also be referred to as AIGCP in that it is determined according to the results of analyzing the first images using a machine learning model, a CNN model, or a DNN model.

Training using the machine learning model, the CNN model, or the DNN model may be performed by the computer system 100 or may be performed by a separate server external to the computer system 100. For example, the processor 220 may determine the center point of each point of the points corresponding to the GCPs by analyzing the first images using a machine learning model, CNN model, or DNN model built on a separate external server.

The processor 220 may determine the center point of the GCP awareness plate located at each point of the points corresponding to the GCPs as the estimated GCP (associated with each respective point) by analyzing the first images.

At step 350, the processor 220 determines the center point of each point determined at step 340 as the estimated GCP associated with each respective point, the estimated GCP at each of the plurality of GCPs (based on the actual measurement) It can be matched with the corresponding GCP. The processor 220 may use a machine learning model, a CNN model, or a DNN model to perform an operation of matching an estimated GCP with a GCP based on an actual survey.

Processor 220 maps the location information (e.g., coordinate values) of the GCP based on the actual measurement corresponding to each point to the location information of the estimated GCP associated with each point to thereby match the estimated GCP to the GCP based on the actual survey .

A more specific method of matching the estimated GCP to the GCP based on the actual measurement will be described in more detail with reference to FIGS. 4 and 5, which will be described later.

According to step 350, each of all the estimated GCPs determined from the photographed images may be matched to each of the GCPs based on the actual measurement.

At step 360, the processor 220 determines each of the estimated GCPs associated with the points corresponding to the plurality of GCPs (i.e., the estimated GCPs estimated through step 350) A two-dimensional or three-dimensional result image 130 may be modeled using a result of matching each of the plurality of GCPs and a plurality of photographed images.

The resulting image 130 may be a single image for the survey site 120 generated by compositing the plurality of images and may be a calibrated image according to the result of matching each of the determined estimated GCPs to each of the plurality of GCPs. The resulting image 130 will be described in more detail with reference to FIG.

With the performance of the matching operation according to step 350, the error contained in the photographed images can be calibrated. Thus, the resulting image 130 modeled and generated by step 360 may contain accurate metrology data (location information).

The technical features described above with reference to FIGS. 1 and 2 can be applied to FIG. 3 as it is, so that redundant description is omitted.

4 is a flow diagram illustrating a method for matching an estimated GCP (i.e., AIGCP) determined based on image analysis and a GCP surveyed at a survey site, according to an example.

Referring to Fig. 4, the above-described step 350 will be described in more detail.

At step 410, the processor 220 may recognize an outline of a graphic corresponding to a GCP recognition plate disposed at each point corresponding to each of a plurality of GCPs (based on an actual measurement).

At step 420, the processor 220 may extract pixels (e.g., pixel values) corresponding to the center point of the graphic based on the recognized outline.

In step 430, the processor 220 may match the position corresponding to the extracted pixel (pixel value) with the coordinate value of the GCP corresponding to each of the plurality of GCPs based on the actual measurement. In other words, the processor 220 can map the coordinate values obtained by actually measuring the position information of the GCP recognition plate at the position of the pixel corresponding to the center point of the extracted GCP recognition plate.

By the above-described steps 410 to 430, the matching between the determined estimated GCP and the GCP based on the actual measurement can be completed.

On the other hand, in the matching of the step 350, a machine learning model, CNN model or DNN model in which the shape of the GCP recognition plate is trained can be used. The processor 220 can recognize the outline of a graphic object corresponding to the GCP recognition plate using a machine learning model, a CNN model, or a DNN model, and extract a pixel value corresponding to the center point of the graphic object.

As described above, training using a machine learning model, a CNN model, or a DNN model may be performed by the computer system 100 or may be performed by a separate server external to the computer system 100. [ For example, the processor 220 recognizes an outline of a graphic object corresponding to a GCP recognition plate using a machine learning model, a CNN model, or a DNN model built in an external server, and extracts a pixel value corresponding to the center point have.

The technical features described above with reference to FIGS. 1 and 2 can be applied to FIG. 3 as it is, so that redundant description is omitted.

5 is a flowchart illustrating a method of recognizing an outline of a figure corresponding to a GCP recognition plate disposed on a survey site according to an example.

With reference to Fig. 5, step 410 described above will be described in more detail.

At step 510, the processor 220 may recognize a plurality of outlines for a graphic corresponding to a GCP awareness plate disposed at each point corresponding to each of the plurality of GCPs (based on the actual measurement).

In step 520, the processor 220 may determine an average of the recognized plurality of outlines as an outline of the figure. In step 520, an interpolation may be used to determine the outline of the figure.

For example, the processor 220 may determine that the outline exceeding a predetermined standard deviation of the plurality of outlines recognized in step 510 is excluded from the calculation of the average, and calculates the average using the remaining outlines, The outline can be determined.

3 to 5, the center point of the GCP awareness plate corresponding to each GCP can be determined more accurately, and therefore, a high accuracy of matching of the GCP to the center point can be achieved.

Also, according to the embodiment, there is no need to manually match the GCP based on the actual measurement to the photographed image (s), and such matching can be performed automatically, resulting in modeling of the resulting image 130 Thereby maximizing the efficiency of the system.

The steps described above with reference to Fig. 3 may be performed in the following order.

For example, the processor 220 may determine a first point of the points corresponding to the plurality of GCPs based on the actual measurement of the measurement target site 120 among the plurality of waypoints included in the (flight) path of the unmanned air vehicle 110, (Step < RTI ID = 0.0 > 1). ≪ / RTI > The processor 220 may identify the first images containing the determined waypoints of the plurality of images photographed (step 2). The processor 220 may determine the center point of the first point by analyzing the identified first images (step 3). The processor 220 may determine the determined center point of the first point as the estimated GCP associated with the first point and determine the estimated GCP associated with the determined first point with the GCP corresponding to the first one of the plurality of GCPs based on the actual measurement (Step 4). The processor 220 may perform the first to fourth steps for each of the points corresponding to the plurality of GCPs based on the actual measurement (step 5).

Next, the processor 220 determines whether each of the determined estimated GCPs is a result of matching each of the plurality of GCPs based on the actual measurement and the resultant image 130 of two-dimensional or three-dimensional Can be modeled.

The technical features described above with reference to FIGS. 1 to 4 can be applied to FIG. 5 as it is, so that redundant description will be omitted.

FIG. 6 illustrates a method of photographing a survey site by flying a path including a plurality of waypoints on a measurement target, according to an example.

In FIG. 6, a route including a plurality of waypoints 610 through which the unmanned air vehicle 110 travels, and GCPs 620 displayed on the survey site 600 are shown. Each of the illustrated GCPs 620 may represent a GCP awareness plate disposed on the survey site 600. The survey site 600 may correspond to the survey site 120 described above.

The unmanned aerial vehicle 110 can fly to pass through the waypoints 610 and can photograph the survey site 600 at each waypoint. The images of the measurement target site 600 may be photographed so as to have a predetermined degree of overlapping. Thus, the unmanned aerial vehicle 110 can capture the survey target site 600 without fail.

The processor 220 may determine from the images taken by the unmanned aerial vehicle 110 a waypoint associated with a point corresponding to each GCP. For example, as shown, the processor 220 may determine a waypoint A associated with a point corresponding to GCP (a) and a waypoint B associated with a point corresponding to GCP (b) , Determine a waypoint (C) associated with a point corresponding to GCP (c), and determine a waypoint (D) associated with a point corresponding to GCP (d).

The processor 220 can determine the center point of the point corresponding to GCP (a) by analyzing the images including the waypoint A and determine the determined center point as the estimated GCP for GCP (a). Processor 220 may match the determined estimated GCP with GCP (a). Processor 220 may repeat the same process for waypoints B through D and GCP b through d.

Processor 220 may model the resulting image 130 using the matched results and the photographed images. As a result, in accordance with the matching operation between the estimated GCP and the GCP based on the actual measurement, a corrected result image 130 can be generated in which the errors included in the photographed images are corrected.

The technical features described above with reference to FIGS. 1 to 5 may be applied to FIG. 6 as it is, so that redundant description is omitted.

FIG. 7 illustrates a method for determining a plurality of images taken by an unmanned aerial vehicle, a waypoint associated with a GCP, and images including the waypoint, according to an example.

Each of the images 710-1 through 740-4 may represent an image photographed by the unmanned aerial vehicle 110. As shown, the images 710-1 through 740-4 may have a predetermined degree of redundancy.

Each of the images 710-1 through 740-4 may have been photographed at each of the waypoints 710 and each of the waypoints 710 may be photographed at each of the images 710-1 through 740-4 It can be a center point of each.

As shown in the illustrated example, the waypoint A closest to the point corresponding to GCP (a) may be determined as a waypoint associated with a point corresponding to GCP (a). At this time, the images 710-2 to 740-4 including the waypoint A may be identified as the first images described above with reference to step 330, and the object of image analysis in step 340 ≪ / RTI >

The technical features described above with reference to FIGS. 1 to 6 may be applied to FIG. 7 as it is, so that redundant description is omitted.

FIG. 8 shows a method of modeling an image photographed by an unmanned aerial vehicle as a calibrated resultant image, according to an example.

The illustrated image 810 schematically shows the resultant image when the images photographed by the unmanned air vehicle 110 are directly modeled. The images photographed by the unmanned air vehicle 110 may include errors due to the specification of the unmanned air vehicle 110, the shooting environment, and the like, so that the modeled image 810 may be distorted.

Therefore, an appropriate calibration is required for images photographed by the unmanned aerial vehicle 110, and according to this calibration, the calibrated resultant image 820 can be modeled. The resulting image 820 may correspond to the result image 130 described above. The resulting image 820 may be, for example, a two-dimensional high-precision matched image or a three-dimensional modeling image. The resulting image 820 may be used for construction and precision measurements for the survey site 120, and so on. For example, the area of a partial area of the measurement target site 120 may be calculated using a two-dimensional high-precision matching image, or the amount of building material to be used may be calculated based on the calculated area.

Through the above-described embodiments, since each of the GCPs based on the actual measurement can be accurately matched to the images photographed by the unmanned aerial vehicle 110, calibration for the photographed images can be finely performed. In addition, this matching operation can be performed automatically.

Thus, as shown, a calibrated resultant image 820 with high accuracy can be easily obtained.

The technical features described above with reference to FIGS. 1 to 7 can be applied to FIG. 8 as it is, so that redundant description will be omitted.

The apparatus described above may be implemented as a hardware component, a software component, and / or a combination of hardware components and software components. For example, the apparatus and components described in the embodiments may be implemented within a computer system, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable array (FPA) A programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to execution of the software. For ease of understanding, the processing apparatus may be described as being used singly, but those skilled in the art will recognize that the processing apparatus may have a plurality of processing elements and / As shown in FIG. For example, the processing unit may comprise a plurality of processors or one processor and one controller. Other processing configurations are also possible, such as a parallel processor.

The software may include a computer program, code, instructions, or a combination of one or more of the foregoing, and may be configured to configure the processing device to operate as desired or to process it collectively or collectively Device can be commanded. The software and / or data may be in the form of any type of machine, component, physical device, virtual equipment, computer storage media, or device , Or may be permanently or temporarily embodied in a transmitted signal wave. The software may be distributed over a networked computer system and stored or executed in a distributed manner. The software and data may be stored on one or more computer readable recording media.

The method according to an embodiment may be implemented in the form of a program command that can be executed through various computer means and recorded in a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, and the like, alone or in combination. The program instructions to be recorded on the medium may be those specially designed and configured for the embodiments or may be available to those skilled in the art of computer software. Examples of computer-readable media include magnetic media such as hard disks, floppy disks and magnetic tape; optical media such as CD-ROMs and DVDs; magnetic media such as floppy disks; Magneto-optical media, and hardware devices specifically configured to store and execute program instructions such as ROM, RAM, flash memory, and the like. Examples of program instructions include machine language code such as those produced by a compiler, as well as high-level language code that can be executed by a computer using an interpreter or the like. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments. For example, it is to be understood that the techniques described may be performed in a different order than the described methods, and / or that components of the described systems, structures, devices, circuits, Lt; / RTI > or equivalents, even if it is replaced or replaced.

Claims (15)

A plurality of GCPs, including position information by measurement on the survey site, using a plurality of images taken by an unmanned aerial vehicle flying a route including a plurality of waypoints on a survey site, A method of modeling a resultant image in which each of a plurality of ground control points is matched,
Wherein the plurality of images are obtained by photographing the survey site including points corresponding to the plurality of GCPs,
Determining a waypoint associated with each point of the plurality of waypoints corresponding to the plurality of GCPs;
Identifying first ones of the plurality of images including the determined waypoints;
Determining a center point of each point by analyzing the first images;
Matching an estimated GCP associated with each of the plurality of GCPs with a GCP corresponding to each of the plurality of GCPs; And
Modeling a two- or three-dimensional result image using the plurality of images and a result of matching each of the plurality of GCPs with each of the plurality of GCPs associated with the points,
/ RTI > of the resulting image. The method according to claim 1,
Wherein the plurality of images are obtained by photographing the survey site so as to have a predetermined rate of photograph duplication,
Wherein each of the plurality of images has taken a portion of the survey target. The method according to claim 1,
A GCP recognition plate is disposed at each of the points corresponding to the plurality of GCPs of the measurement target object, the plurality of images are obtained by photographing the survey target place on which the GCP recognition plate is disposed,
Wherein determining the center point of each point comprises:
Determining a center point of the GCP awareness plate disposed at each point and determining the center point of the GCP awareness plate as the estimated GCP. The method of claim 3,
Wherein determining the center point of each point comprises:
Determining a center point of each point by analyzing the first images using a machine learning model, a CNN (Convolution Neural Network) model or a DNN (Deep Neural Network) model in which the shape of the GCP recognition plate is trained; How to model. The method of claim 3,
The matching step comprises:
Recognizing an outline of a figure corresponding to a GCP recognition plate disposed at each of the points;
Extracting a pixel value corresponding to a center point of the graphic object based on the recognized outline; And
Matching a position corresponding to the pixel value with a coordinate value of a GCP corresponding to each of the plurality of GCPs
/ RTI > of the resulting image. 6. The method of claim 5,
An outline of the figure is recognized by using a machine learning model, a CNN (Convolution Neural Network) model or a DNN (Deep Neural Network) model in which the shape of the GCP recognition plate is trained, and a pixel value corresponding to the center point of the figure is extracted How to model the resulting image. 6. The method of claim 5,
The step of recognizing the outline may include:
Recognizing a plurality of outlines of a graphic form corresponding to a GCP recognition plate disposed at each of the points; And
Determining an average of the plurality of outlines as an outline of the figure,
Lt; / RTI >
Wherein an outline beyond a predetermined standard deviation of the plurality of outlines is excluded in the calculation of the mean. The method according to claim 1,
Wherein determining the waypoint associated with each of the points comprises:
Determining a closest waypoint for each of the plurality of waypoints as a waypoint associated with each of the plurality of waypoints. The method according to claim 1,
Wherein in determining the waypoint associated with each of the points, one waypoint of the plurality of waypoints is associated with one of the points. The method according to claim 1,
Wherein the resultant image is a single image for the survey site generated by combining the plurality of images, the image being a calibrated image according to a result of matching each of the estimated GCPs of the points to each of the plurality of GCPs, How to model an image. 11. A computer program stored in a computer-readable medium for performing the method of any one of claims 1 to 10. A plurality of GCPs, including position information by measurement on the survey site, using a plurality of images taken by an unmanned aerial vehicle flying a route including a plurality of waypoints on a survey site, A method of modeling a resultant image in which each of a plurality of ground control points is matched,
Wherein the plurality of images are obtained by photographing the survey site including points corresponding to the plurality of GCPs,
Determining a waypoint associated with a first one of the plurality of waypoints corresponding to the plurality of GCPs;
Identifying first ones of the plurality of images including the determined waypoints;
Determining a center point of the first point by analyzing the first images;
Matching an estimated GCP associated with the first point to a GCP corresponding to the first one of the plurality of GCPs, the estimated GCP associated with the first point being a center point of the determined first point;
Determining the waypoint, identifying the step, determining the center point, and performing the matching on each of the points. And
Modeling the resulting image, comprising: matching each of the estimated GCPs associated with the points to each of the plurality of GCPs, and modeling the two- or three-dimensional resultant image using the plurality of images . A plurality of images of a plurality of GCPs (Ground Control Points) including position information by measurement on the survey site, using a plurality of images photographed by an unmanned aerial vehicle flying a path including a plurality of waypoints on a surveying object A computer system for modeling the matched result image,
Wherein the plurality of images are obtained by photographing the survey site including points corresponding to the plurality of GCPs,
An image obtaining unit obtaining the plurality of images; And
The processor for analyzing and processing the acquired plurality of images
Lt; / RTI >
The processor comprising:
Determining a waypoint associated with each point in the plurality of waypoints corresponding to the plurality of GCPs,
Identify first ones of the plurality of images that include the determined waypoints,
Determining a center point of each point by analyzing the first images,
Determining a center point of each of the determined points as an estimated GCP associated with each of the points, matching an estimated GCP associated with each of the points with a GCP corresponding to each of the plurality of GCPs,
Model each of the estimated GCPs associated with the points to a respective one of the plurality of GCPs and a two-dimensional or three-dimensional resultant image using the plurality of images. 14. The method of claim 13,
A GCP recognition plate is disposed at each of the points corresponding to the plurality of GCPs of the measurement target object, the plurality of images are obtained by photographing the survey target place on which the GCP recognition plate is disposed,
Wherein the processor determines a center point of a GCP awareness plate disposed at each of the points and determines a center point of the GCP awareness plate as the estimated GCP. 15. The method of claim 14,
The processor comprising:
Recognizes an outline of a figure corresponding to the GCP recognition plate disposed at each of the points,
Extracting a pixel value corresponding to a center point of the figure based on the recognized outline,
And matches a position corresponding to the pixel value with a coordinate value of a GCP corresponding to each of the plurality of GCPs.

Images