KR102239562B1 - Fusion system between airborne and terrestrial observation data - Google Patents

Abstract

Provided is a fusion system between aerial observation data and terrestrial observation data. The system includes: a geometric transformation part processing a first geometric transformation with respect to at least one of aerial observation data and terrestrial observation data; a similarity analysis part setting reference data estimated to be the same mutually from the aerial observation data and terrestrial observation data by conducting similarity analysis between the aerial observation data and the terrestrial observation data, in which the first geometric transformation with respect to the at least one is processed; a matching information extraction part conducting a second geometric transformation with respect to at least one of aerial geometric information of the aerial observation data and terrestrial geometric information of the terrestrial observation data, in which the first geometric transformation with respect to at least one of the observation data is processed, based on the reference data, thereby extracting matching information of a geometric transformation model between the aerial observation data and the terrestrial observation data; and a data fusion part registering the aerial observation data and the terrestrial observation data, in which the second geometric transformation is conducted, as fusion data of the same coordinate system based on the matching information. Therefore, the present invention is capable of securing a great deal of space information with high precision for creating a map.

Description

Fusion system between aerial observation data and ground observation data {FUSION SYSTEM BETWEEN AIRBORNE AND TERRESTRIAL OBSERVATION DATA}

The present invention relates to a fusion system between aerial observation data and ground observation data, and more particularly, aerial observation data and ground observation data in a complementary correction method of observation data acquired from heterogeneous platforms such as an aerial observation system and a ground observation system. The present invention relates to a fusion system between aerial observation data and ground observation data that can be fused by matching observation data.

According to the increasing demand for 3D spatial information such as smart city, digital twin, autonomous driving, etc., various observation sensor platforms such as aerial photography, aerial lidar, mobile mapping system, and ground lidar The utilization has also increased.

For example, it is possible to create spatial information data by observing a relatively wide area through aerial observation sensors such as unmanned aerial vehicles and medium-sized aircraft, and to construct a relatively local area with precision through ground observation in the form of vehicles, robots, and backpacks.

However, in constructing spatial information, the positional accuracy acquired from the observation sensor acts as a very important factor. In general, location registration is performed by measuring the position and attitude value of an aerial or ground multi-sensor platform through GNSS/INS (Global Navigation Satellite System / Inertial Navigation System) and applying it to observation data. Position estimation through positioning satellite observation and position estimation technology through inertial and geomagnetic observations cause errors in position accuracy depending on the satellite reception state, the performance of the observation sensor, and the surrounding environment.

In particular, in urban areas and complex construction sites where the strength of the network and satellite signals may be weak, errors can occur up to several meters in the vertical direction.

At this time, as a technology to correct the positional accuracy, the correction method through direct observation through the surrounding environment and sensor elements such as convective layer conditions, temperature, and air pressure, the correction method through indirect observation based on the ground reference point, and the correction method through continuous observation data have.

In the case of the correction method using direct observation or continuous observation data, errors due to measurement errors of the environment and sensor elements are largely involved, and in the case of the correction method using continuous observation data, the cumulative error is accompanied, which improves the quality of the result data. It's hard to guarantee.

For this reason, in the generation of spatial information, observation data correction is generally mainly based on a ground reference point-based correction method.

In surveys such as triangulation, leveling, trilateral surveying, etc. in the past, there is an advantage that it is possible to estimate the positional precision of the generated spatial information data as well as the correction of the observation data using the ground reference point.

However, the ground reference point-based correction method has a disadvantage in that it takes a lot of time and cost to obtain the ground reference point.

Compared to past point-based surveying, the recent aviation or vehicle-based observation method has the advantage of being able to observe a wide area in a short time, but has a disadvantage in that it requires a large number of ground reference points.

The technical problem to be achieved by the present invention is an aerial observation that can secure a lot of spatial information with high precision by fusion of ground observation data that accurately implements the environment around the ground and aerial observation data that observes a wide area over a certain altitude from the ground. It is to provide a fusion system between data and ground observation data.

The object of the present invention is not limited to the objects mentioned above, and other objects not mentioned will be clearly understood by those skilled in the art from the following description.

According to an aspect of the present invention for achieving the above technical problem, a fusion system between aerial observation data and ground observation data includes a geometric conversion unit that processes a first geometric transformation on at least one of aerial observation data and ground observation data, A similarity analysis unit configured to perform a similarity analysis between the aerial observation data processed with the first geometric transformation on the at least one of the ground observation data and the ground observation data to set reference data that are mutually estimated to be identical to each other from the aerial observation data and the ground observation data; , Based on the reference data, a second geometry for at least one of the aerial geometric information of the aerial observation data and the ground geometric information of the ground observation data for which a first geometric transformation has been processed for at least one of the observation data A matching information extraction unit for extracting matching information of a geometric transformation model between the aerial observation data and the ground observation data by performing the transformation, and the aerial observation data and the ground observation data subjected to a second geometric transformation based on the matching information are identical. It includes a data fusion unit that registers as fusion data of the coordinate system.

In another embodiment, when the first geometric transformation is processed on any one of the aerial observation data and the ground observation data, the observation data subjected to the first geometric transformation is the position data of the unprocessed observation data and the It is processed with reference to attitude data defining the observation direction of the observation data, and the second geometric transformation refers to the position data and the attitude data of the observation data for which the first geometric transformation is not processed, and the first geometric transformation is When performed on the processed observation data, and the first geometric transformation is processed on all of the aerial observation data and the ground observation data, the first geometric transformation defines the observation direction of the aerial observation data. The aerial observation data and the aerial observation data in which all the first geometric transformation is performed are processed by referring to attitude data different from the attitude data for ground, which defines the attitude data and the observation direction of the ground observation data, and with reference to the different attitude data. Can be performed on ground observation data.

In another embodiment, when at least one of the aerial observation data and the ground observation data is obtained from an image sensor, the similarity analysis unit performs a color change based on pixel coordinates among geocoding image data information belonging to the observation data. Analyzes and performs similarity analysis to set the reference data by extracting geometric information estimated as feature geometry, and at least one of the aerial observation data and the ground observation data acquires the observation data from a 3D survey sensor In this case, the similarity analysis unit may set the reference data by analyzing at least one of a geometric shape gradient and a laser intensity gradient of point group data belonging to the observation data to extract the geometric shape information.

In another embodiment, the similarity analysis unit, when the observation detection precision at at least one of the aerial observation data and some locations of the ground observation data is lower than the detection threshold, the aerial observation data lower than the observation detection threshold Filtering the aerial observation data and the ground observation data corresponding to a position lower than the detection threshold so as not to set the reference data, by referring to the dragon location data and the ground location data of the ground observation data lower than the detection threshold. can do.

In addition, the matching information extraction unit, the matching information extraction unit, at a location where any one of the aerial observation data and the ground observation data is filtered, performs a second geometric transformation on the geometric information of the observation data higher than the observation detection threshold. As a result, it is possible to extract matching information of the geometric transformation model.

In another embodiment, a matching information verification unit may further include a matching information verification unit to re-extract the matching information by verifying the matching information based on the reference data.

Here, the matching information verification unit randomly samples feature geometries for a sample from among the reference data estimated to be the same from the aerial observation data and the ground observation data converted from the second geometric shape, and the mutual separation of the sample feature geometries is minimum. Calculates matching information related to the sample feature geometry to be, and re-extracts the matching information of the geometric transformation model with a second geometric transformation consisting of matching information related to the sample feature geometry, and the data fusion unit re-extracts the re-extracted The aerial observation data and the ground observation data, which are second geometrically transformed based on the matching information, may be registered as fusion data of the same coordinate system.

As another example, the matching information verification unit calculates a most probable value for minimizing the amount of separation between the second geometrically transformed aerial observation data and the reference data estimated to be the same from the ground observation data, and the Analyzing the predicted error probability between the reference data estimated to be the same based on the minimum separation amount used in calculating the maximum value, and when the predicted error according to the predicted error probability exceeds the allowable error, the aerial observation data And a process of sampling and extracting geometric shape information estimated as different feature geometry from the ground observation data as new reference data, and removing reference data determined as abnormal data from the geometric transformation model generated by the matching information extraction unit. At least one of processing, processing of changing at least some parameters applied to the geometric transformation model, and processing of changing at least some parameters used in the analysis model equation applied at the time of calculating the maximum value, or changing the analysis model equation The similarity analysis unit and the matching information extraction unit are controlled to perform one, and according to the constraint condition after the processing is performed, the matching information extraction unit re-extracts the matching information of the geometric transformation model by the second geometric transformation, and the data The fusion unit may register the second geometrically transformed aerial observation data and the ground observation data as fusion data of the same coordinate system based on the re-extracted matching information.

In another embodiment, when the aerial observation data is obtained from any one of an image sensor and a three-dimensional survey sensor, and the ground observation data is obtained from a sensor different from the sensor of the aerial observation data, the three-dimensional The observation data obtained from the survey sensor uses virtual image data converted from point cloud data, and the location data of the observation data related to the 3D survey sensor is defined as the location data of the virtual image data, and the observation data The posture data of may be defined as a viewing direction of the virtual image data.

In another embodiment, the geometric transformation model uses external geometric data consisting of position data and attitude data defining an observation direction, and internal geometric data calculated based on a geometric parameter defined in a sensor set to generate the observation data. In the case of a physical sensor model, the aerial geometric information includes at least position data for aviation and attitude data for aviation, and the ground geometric information includes at least position data for ground and posture data for ground, When the geometric transformation model is a replacement sensor model using a replacement model parameter other than the external geometric data, the aerial geometry information and the ground geometry information are respectively an aviation replacement model parameter and a ground replacement model parameter It may include.

Details of other embodiments are included in the detailed description and drawings.

According to the present invention, ground observation data that accurately implements the environment around the ground and aerial observation data that observes a wide area over a certain altitude from the ground are fused. Information can be obtained.

In addition, for fusion, aerial observation data and ground observation data are converted into similar geometric shapes, and the accuracy of data fusion is improved by establishing a geometric conversion model of both observation data based on the reference data commonly estimated through similarity analysis. Can be improved.

In addition, when establishing a geometric transformation model, optimal matching information can be calculated by utilizing both reference data that minimizes the amount of separation between the reference data of the aerial observation data and the ground observation data.

1 is a conceptual block diagram of a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention.
2A to 2C are block diagrams showing various configuration examples of a geometric conversion unit.
3 is a flowchart of a fusion method using a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention.
4 is a diagram illustrating aerial observation data as an orthogonal image acquired in a vertical direction with respect to the ground and aerial observation data acquired in a non-vertical direction with respect to the ground.
5 is a diagram illustrating ground observation data acquired from an image sensor, a 3D survey sensor, and a multi-sensor platform composed of these sensors.
6 is a diagram illustrating that observation data acquired from a 3D survey sensor is geometrically transformed into a top view.
7 is a diagram illustrating a process of setting reference data from observation data obtained from a 3D survey sensor.
8 is a diagram illustrating a process of setting reference data from aerial observation data and ground observation data acquired from image sensors.
9 is a diagram illustrating a process of setting reference data from ground observation data acquired from a 3D lidar sensor.
10 is a flowchart illustrating an embodiment of re-extracting matching information by verifying matching information based on reference data.
11 is a flowchart of another embodiment of re-extracting matching information by verifying matching information based on reference data.
12 is a diagram illustrating the fusion of aerial observation data and ground observation data.

Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings and the following description. However, the present invention is not limited to the embodiments described herein and may be embodied in other forms. Rather, the embodiments introduced herein are provided so that the disclosed contents may be thorough and complete, and the spirit of the present invention may be sufficiently conveyed to those skilled in the art. The same reference numbers throughout the specification indicate the same elements. Meanwhile, terms used in the present specification are for explaining embodiments and are not intended to limit the present invention. In this specification, the singular form also includes the plural form unless specifically stated in the phrase. As used herein, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, actions and/or elements in which the recited component, step, operation and/or element is Or does not preclude additions.

In addition, "unit" to "module" generally refer to components such as software (computer programs) and hardware that can be logically separated. Therefore, the module in this embodiment refers not only to the module in the computer program, but also to the module in the hardware configuration. Therefore, the present embodiment is a computer program for making them function as modules (a program for executing each step in a computer, a program for making a computer function as each means, a program for realizing each function in a computer) ), also describes the system and method. However, for convenience of explanation, phrases equivalent to "save" and "save" are used, but these words are stored in a memory device or stored in a memory device when the embodiment is a computer program. It means to control like letting you do. In addition, the "sub" to the module may correspond to the function one-to-one, but in mounting, one module may be configured as one program, multiple modules may be configured as one program, and conversely, one module may be configured as multiple programs. do. Further, a plurality of modules may be executed by one computer, or one module may be executed by a plurality of computers by a computer in a distributed or parallel environment. In addition, another module may be included in one module. In addition, hereinafter, "connection" is adopted in the case of logical connection (data exchange, instruction, reference relationship between data, etc.) in addition to physical connection. The term "predetermined" means that it is determined before the target processing, as well as before the processing according to the present embodiment is started, even after the processing according to the present embodiment is started, and before the target processing, the It is used by including the meaning of what is determined according to the situation or state of the time, or the situation or state up to that time.

In addition, a system or device is configured by connecting a plurality of computers, hardware, devices, etc. by means of communication such as a network (including a one-to-one communication connection), and a single computer, hardware, device, etc. This includes cases where it is realized. The terms "device" and "system" are used as terms of mutual agreement. Of course, the "system" does not include anything but an artificial decision, a social "mechanism" (social system).

In addition, when processing by each unit or module or a plurality of processing within each unit or module, the target information is read and input from the storage device for each processing, and after the processing is performed, the processing result is displayed. It is to write to the storage device. Therefore, the description of the read input from the storage device before processing and the writing to the storage device after processing may be omitted in some cases. Further, the storage device herein may include a hard disk, a random access memory (RAM), an external storage medium, a storage device through a communication line, a register in a CPU (Central Processing Unit), and the like.

Hereinafter, a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention will be described in detail with reference to FIGS. 1 and 2C.

1 is a conceptual block diagram of a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention, and FIGS. 2A to 2C are block diagrams showing various configuration examples of a geometric conversion unit.

The fusion system 100 between aerial observation data and ground observation data may be applied to a mobile mapping system or a system for autonomous driving according to a purpose.

The system 100 includes an input unit 102 for inputting aerial observation data and ground observation data, a geometric conversion unit 104, a similarity analysis unit 106, a matching information extraction unit 108, and a matching information verification unit 110. And a data fusion unit 112.

Aerial observation data is acquired by sensors mounted on an aerial mobile platform, and is observation data of objects around the ground photographed at a certain altitude or higher from the ground. The aerial mobile platform may be an airplane, a glider, a helicopter, an airship (including a non-rigid airship), a hot air balloon, a manned or unmanned vehicle such as a drone.

Ground observation data is observation data of objects around the ground obtained by sensors mounted on a ground mobile platform. The ground mobile platform may be a device platform such as a vehicle, a robot, a backpack, a hand-held, and a tripod.

Sensors that acquire aerial and ground observation data may be at least one of an image sensor and a 3D survey sensor, and a navigation sensor mounted on an aerial and ground mobile platform includes positioning information, navigation information such as position, attitude, and speed of the platform. Can be detected to obtain observation data for navigation.

The image sensor is a sensor that is mounted on a ground or aerial mobile platform and captures an image of surrounding objects, such as terrain and features, and acquires observation data for images, and may be a surveying or non-surveying camera or a stereo camera. Not limited.

The 3D survey sensor is a sensor that is mounted on a platform and acquires 3D geographic data related to objects around it, for example, terrain and feature related data to acquire observation data for 3D surveying, and is an active remote sensing sensor. For example, the 3D survey sensor 104 may be a laser or ultrasonic sensor, and in the case of a laser sensor, it may be a Light Detection and Ranging (LiDAR) sensor. Such a lidar sensor scans a laser on an object to obtain data and detects the parallax and energy change of the electromagnetic wave reflected and returned from the object, and calculates the distance and reflection intensity to the object.

The navigation sensor acquires the attitude of the vehicle through a positioning device that acquires the moving position of the platform through a satellite navigation system (GPS), an Inertial Measurement Unit (IMU), and an Inertial Navigation System (INS). It may be configured with a posture acquisition device or the like.

Aerial and ground observation data may be generated to include various data by performing predetermined processing on sensor data acquired from the above-described sensors.

The aerial observation data may include aerial geometric information. The aerial geometric information may be different according to the type of the geometric transformation model to be used in the matching information extraction unit 108.

For example, a physical sensor model using internal geometric data calculated based on a geometric parameter defined in a sensor set to generate the observation data and external geometric data consisting of position data and attitude data defining the observation direction in which the geometric transformation model is used. model), the aviation geometric information may include aviation external geometric data including aviation position data and aviation attitude data and aviation internal geometric data.

When the geometric transformation model is not external geometric data, but a replacement sensor model using a replacement model parameter applied to Equation 1, the aviation geometric information may include a replacement model parameter for aviation.

[Equation 1]

Like aerial observation data, ground observation data may have different ground geometric information depending on the type of the geometric transformation model.

For example, when the geometric transformation model is a physical sensor model, the ground geometric information may include external geometric data for the ground and internal geometric data for the ground having position data for the ground and attitude data for the ground.

When the geometric transformation model is a replacement sensor model using a replacement model parameter applied to Equation 1, the ground geometry information may include a replacement model parameter for the ground.

Here, the use of the aviation and ground replacement model parameters for the aviation geometry information and the ground geometry information is a case where there is no, insufficient, or low-precision data for aviation and ground external geometry data.

In addition, the aerial and ground observation data is related to the above-described geometric information, time information on the acquisition time of each sensor, unique data related to device specific properties for each sensor, and point cloud data when observation data is acquired by a 3D survey sensor. The object information for which the attribute of the extracted object is estimated based on at least one of information, information related to geocoding image data, information related to point cloud data, and information related to geocoding image data when observation data is obtained from an image sensor. Can include.

The internal geometric data is an inherent value of the sensor itself, and is an error in observed data for each sensor due to a geometric parameter maintained regardless of whether the platform or the like is moved.

Here, the geometric parameter for the image sensor may be at least one of a focal length, a main point position, a lens distortion parameter, and a sensor format size. The geometric parameter for the 3D survey sensor may be at least one of an incidence angle of each laser, a distance scale, a distance direction offset, and an axial offset, for example, a lidar sensor. In addition, the geometric parameter for the navigation sensor may be at least one of a scale in an axial direction and an offset in an axial direction.

If, in the case of a multi-sensor platform in which an image sensor and a lidar sensor are combined, aerial and ground observation data define the internal geometry of each sensor and the geometric relationship between each sensor according to the position and attitude of the platform on which the sensors are mounted. Geometric modeling-related information may be included based on geometry information composed of external geometry. In this case, the observation data may be fused by combining image data and point cloud data obtained from an image sensor and a 3D survey sensor, and the point cloud data may have color information, or the image information may have three-dimensional position coordinates for each pixel. I can have it.

Aerial and ground observation data may include sensor-specific data. For example, the unique data of the image sensor includes at least one of the sensor's illuminance, ISO, shutter speed, shooting date/time, time synchronization information, sensor model, lens model, sensor serial number, image file name, and file storage location. can do.

The unique data of the 3D survey sensor include LiDAR information, for example, the sensor's photographing date and time, sensor model, sensor serial number, laser wavelength, laser intensity, laser transmission/reception time, laser observation angle/distance, laser pulse, and electronic time. It may include at least one of a delay, a standby delay, and a sensor temperature.

The unique data of the navigation sensor includes sensor model information of GNSS/IMU/INS, GNSS reception information, GNSS satellite information, GNSS signal information, GNSS navigation information, ionospheric/convective layer signal delay information of GNSS signal, DOP information, earth behavior information, Dual/multi GNSS equipment information, GNSS base station information, wheel sensor information, gyro sensor scale/bias information, accelerometer scale/bias information, position/position/speed/acceleration/angular velocity/angular acceleration information and predicted error amount, navigation information filtering model And at least one of erasing error information, a photographing date/time, and all or part of time synchronization information.

The time information may be a time when observation data of each sensor is generated.

The aerial and ground observation data may further include information related to 3D point cloud data based on the 3D survey sensor and information related to geocoding image data based on the image sensor, based on sensor data and geometric information acquired from the sensor.

Information related to 3D point group data includes 3D coordinate data for point group, color information for point group, class information for point group estimated from the object extracted from 3D coordinate data for point group, and laser when the 3D survey sensor is a laser sensor. It may include at least one of strength information.

The geocoding image data related information may include at least one of geocoding coordinate data for each pixel, geocoding color information, and geocoding class information whose type is estimated from the object extracted from the geocoding coordinate data. .

The aerial and ground observation data may include object information extracted in relation to an attribute of an object extracted based on at least one of information related to 3D point cloud data and information related to geocoding image data. The object information may include a type of an object, an overall shape, a specific geometric shape such as a line, plane, circle, or sphere constituting at least a part of the object, a color, and a texture-related attribute.

When aerial observation data is obtained from either an image sensor or a 3D survey sensor, and ground observation data is acquired from a sensor different from the sensor of the aerial observation data, the input unit 102 is obtained from a 3D survey sensor. The obtained point cloud data may be converted into virtual image data, the position data of the observation data may be defined as the position data of the virtual image data, and the attitude data of the observation data may be defined as the viewing direction of the virtual image data.

The geometric conversion unit 104 converts the aerial observation data and the ground observation data to have a similar geometric shape by processing a first geometric transformation on at least one of the aerial observation data and the ground observation data.

Various processing of the geometric conversion units 104a and 104c is performed by referring to the position data and attitude data constituting the geometric information of the unprocessed observation data, as shown in FIGS. 2A and 2B. 1 Geometric transformation can be performed.

For example, as shown in FIG. 2A, the ground observation data is geometrically converted by the ground geometric conversion module 104b, but the aerial observation data is not geometrically converted. In this case, the ground observation data may be converted similarly to the geometry of the aerial observation data based on the aerial geometric information such as the aerial position data, the aerial attitude data, and the aerial internal geometric data of the aerial observation data.

As another example as shown in FIG. 2B, the aerial observation data is geometrically converted by the aerial geometric conversion module 104d, but the ground observation data is not geometrically converted. In this case, the aerial observation data may be converted similarly to the geometry of the ground observation data based on ground geometric information such as ground position data, ground attitude data, and ground internal geometric data of the ground observation data.

As another example, as shown in FIG. 2C, when the first geometric conversion is processed on all of the aerial observation data and the ground observation data using the aerial geometric conversion module 104f and the ground geometric conversion module 104g, the geometric conversion The unit 104e may process the first geometric transformation by referring to attitude data different from the aviation attitude data and the ground attitude data.

As shown in FIGS. 2A to 2C, the similarity analysis unit 106 performs a similarity analysis between the aerial observation data and the ground observation data on which the first geometric transformation has been processed for at least one of them, so that the aerial observation data and the ground observation data Reference data estimated to be identical to each other are set.

When at least one of aerial observation data and ground observation data is acquired from an image sensor, the similarity analysis unit 106 analyzes a color change based on pixel coordinates among geocoding image data information belonging to the observation data, and Similarity analysis can be performed to set the reference data by extracting the geometric information estimated to be.

When at least one of the aerial observation data and the ground observation data acquires observation data from a 3D survey sensor, the similarity analysis unit 106 is a geometric shape change degree and a laser intensity change degree of point cloud data belonging to the observation data. Reference data may be set by analyzing at least one of them to extract the geometric shape information.

In addition, when the observation detection accuracy is lower than the detection threshold at at least one of the aerial observation data and some of the ground observation data, the similarity analysis unit 106 The aerial observation data corresponding to a position lower than the detection threshold and the ground observation data may be filtered so as not to set the reference data with reference to the dragon location data.

The reason that the observation detection accuracy is low in the observation data is that the observation data of the sensor in the occluded area that is not observed is not obtained or is defective, depending on the type of the sensor, or environmental factors during sensor observation such as rainy weather, snowfall, and dark conditions. Is caused by. Regarding the observation of the occluded area, it is difficult to secure data other than the road surface and the building surface to which the laser is irradiated due to the linearity of the laser of the 3D survey sensor. In addition, in the case of aerial observation data, it is difficult to secure building side data due to photographing geometry, and the observation detection accuracy is low.

In order to overcome this, the similarity analysis unit 106 may filter and use an area capable of securing high precision from all observation data of heterogeneous platforms based on location information.

The similarity analysis unit 106 may perform a predetermined pre-process before performing the similarity analysis.

The above-described pre-processing includes at least one of image luminance adjustment pre-processing, image resolution adjustment pre-processing, image sharpness adjustment pre-processing, image geometry transformation, image coordinate system transformation, and pre-processing of extracting boundary information of objects from observation and ortho-image related data I can. This pre-processing is performed to improve the accuracy of the similarity analysis by minimizing the amount of deformation such as geometric information, coordinate system, and resolution due to differences in observed geometric characteristics such as sensor's unique data, observation environment, scale/rotation/displacement value, etc. For example, when an orthogonal image of aerial observation data is acquired in clear weather at noon, and when image data of ground observation data is acquired in rainy weather at sunset, the resolution of both data is different, and an error in similarity analysis may be caused. In order to solve this, a pre-process of adjusting the resolution of both data to a value close to each other may be performed.

According to the operation and function of the similarity analysis unit 106, the accuracy of matching is performed by analyzing the similarity between these data by using object information of reference data and observation-related data, for example, the type of object or geometric shape information such as a specific geometric shape. Can improve.

Based on the reference data, the matching information extracting unit 108 includes at least one of the aerial geometric information of the aerial observation data and the ground geometric information of the ground observation data for which the first geometric transformation has been processed for at least one of the observation data. A second geometric transformation is performed to extract matching information of a geometric transformation model between the aerial observation data and the ground observation data.

The matching information extraction unit 108 may extract matching information using a geometric transformation model. In this case, the geometric transformation model is a physical sensor model using external geometric data and internal geometric data of a sensor that has acquired aerial observation data and ground observation data, or when it is difficult to use external geometric data, It may be a replacement sensor model such as a linear equation, a polynomial equation, or a proportional equation expressed as a replacement model parameter.

Hereinafter, an embodiment in which the matching information extracting unit 108 extracts matching information using a physical sensor model will be mainly described.

2A and 2B, when the first geometric transformation is processed on either of aerial observation data and ground observation data, the second geometric transformation is the position data and attitude data of the observation data for which the first geometric transformation is unprocessed. With reference to, the first geometric transformation may be performed on the processed observation data.

In another embodiment, the second geometric transformation may be performed on both observation data by referring to geometric information such as the observed data processed with the first geometric transformation and the attitude data completely different from the unprocessed observation data.

As shown in FIG. 2C, when the first geometric transformation is processed on all of the aerial observation data and the ground observation data, the aerial observation in which the first geometric transformation is all performed with reference to different attitude data utilized in the first geometric transformation. It can be performed on both data and ground observation data.

The matching information extraction unit 108 performs a second geometric transformation on the geometric information of the observation data higher than the observation detection threshold at a location where any one of the aerial observation data and the ground observation data is filtered, so that the geometric transformation model is matched. Information can be extracted.

The matching information extraction unit 108 generates a geometric transformation model between reference data and aerial and ground observation data based on a coordinate system set by geometric transformation modeling.

Specifically, the matching information extraction unit 108 may generate a geometric conversion model between reference data and observation data based on aviation and ground geometric information consisting of aviation, ground external geometric data, and aviation, ground internal geometric data. I can. In addition, the set coordinate system may be set to one of a coordinate system of unprocessed observation data, a coordinate system based on different attitude data, and an absolute coordinate system. As a result, observation-related data can be converted into a standardized coordinate system and corrected.

In another embodiment, the matching information extracting unit 108 may generate a geometric transformation model between reference data based on aerial and ground external geometric data among geometric information. In consideration of the fact that the positions and attitudes of the sensors acquiring aerial observation data and ground observation data are different from each other, the external geometry may be preferentially utilized when generating the geometric transformation model.

The matching information extraction unit 108 may calculate an objective function that minimizes an amount of separation from observation-related data through a geometric model to which a plurality of parameters are applied.

In the similarity analysis unit 106, even if the observation detection precision of both aerial observation data and ground observation data is lower than the detection threshold and all of them are filtered, the matching information extraction unit 108 is lower than the detection threshold, but the observation detection precision is high. By performing the second geometric transformation, a geometric transformation model may be established, and matching information for residual observation data may be extracted.

The matching information verification unit 110 may re-extract the matching information by verifying the matching information based on the reference data.

The matching information verification unit 110 may be performed in two ways, and may be a random sampling method and an expected error analysis method for reference data.

According to the random sampling method, the matching information verification unit 110 randomly samples the sample feature geometries among the reference data estimated to be the same from the second geometrically-transformed aerial observation data and the ground observation data. Matching information related to the feature geometry for the sample with the minimum amount of separation is calculated. The matching information verification unit 110 re-extracts the matching information of the geometric transformation model as a second geometric transformation composed of matching information related to the sample feature geometry, and transmits the re-extracted matching information to the data fusion unit 112.

Specifically, the matching information verification unit 110 samples a plurality of aerial and image observation data sharing the same estimated feature geometry for each sample. These sampling sets are acquired in plurality for different standard feature geometries. At the same time, the reference data having the feature geometry estimated to be the same as the sample feature geometry are interlocked to be selected. Subsequently, when the aerial and ground observation data and reference data that share the same estimated sample feature geometries are arranged according to the second geometric-transformed geometric information, the sample feature geometries and reference data of the aerial and ground observation data We select the sample feature geometry of the observed data whose mutual separation is determined to be the minimum in the feature geometry of. This sample feature geometry is re-selected as reference data, and matching information related thereto is re-extracted.

According to the predicted error analysis method, the matching information verification unit 110 calculates a maximum value for minimizing the amount of separation between the reference data estimated to be the same from the second geometrically transformed aerial observation data and the ground observation data. Based on the minimum separation amount used at the time of calculation, the predicted error probability between the same estimated reference data is analyzed.

The most probable value is calculated by a predetermined analysis model equation, and some parameters among a plurality of parameters may be applied to this model equation.

By calculating the calculated maximum value as described above, geometric correction of aerial and ground observation data using the reference data is performed, and the time limit correction is the result of applying the reference data to the geometric transformation model and the geometric error of the observation data corresponding to the reference data. This is a process of analyzing a vehicle and calculating a parameter whose geometric error amount is minimal or optimized as a maximum value.

The maximum value may be at least one of error information of an external geometry and an internal geometry. In addition, the maximum probable value may be calculated as at least one of error information of geometric shape information generated by at least one of the above-described information and point cloud data related information, geocoding image data related information, object information, and map related information.

When the predicted error according to the predicted error probability exceeds the allowable error, the matching information verification unit 110 samples the aerial observation data and the geometric information estimated by different feature geometries from the ground observation data as new reference data. Extracting processing, processing of removing reference data determined as abnormal data from the geometric transformation model generated by the matching information extraction unit 108, processing of changing at least some parameters applied to the geometric transformation model, and calculating the maximum value At least one of the processing of changing at least some parameters used in the analysis model equation applied to or changing the analysis model equation may be performed. The matching information verification unit 110 may control the similarity analysis unit 106 and the matching information extraction unit 108 to perform the above-described processing, and according to the constraint condition after performing the processing, the matching information extraction unit 108 May re-extract the matching information of the geometric transformation model through the second geometric transformation.

The data fusion unit 112 registers the second geometrically-transformed aerial observation data and the ground observation data as fusion data of the same coordinate system defined by the geometric transformation model based on the re-extracted matching information.

Various information belonging to aerial observation data and ground observation data are registered for each same location in the same coordinate system, so that specific location information can be accurately identified as high-precision location data among aerial observation data and ground observation data, as well as various port information. It is useful for realizing structured spatial information.

In the present embodiment, it has been described focusing on fusing both observation data based on the matching information re-extracted by the matching information verification unit 110, but when the matching information verification unit 110 is omitted, the matching information extracting unit ( 108), both observation data may be fused.

According to the present embodiment, ground observation data that accurately implements the environment around the ground and aerial observation data that observes a wide area of a certain altitude or higher from the ground are fused. Space information can be secured.

In addition, for fusion, aerial observation data and ground observation data are converted into similar geometric shapes, and the accuracy of data fusion is improved by establishing a geometric conversion model of both observation data based on the reference data commonly estimated through similarity analysis. Can be improved.

In addition, when establishing a geometric transformation model, optimal matching information can be calculated by utilizing both reference data that minimizes the amount of separation between the reference data of the aerial observation data and the ground observation data.

Hereinafter, a fusion method using a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention will be described with reference to FIGS. 1 to 12.

3 is a flowchart of a fusion method using a fusion system between aerial observation data and ground observation data according to an embodiment of the present invention.

4 is a diagram illustrating aerial observation data as orthogonal images acquired in a vertical direction with respect to the ground and aerial observation data acquired in a non-vertical direction with respect to the ground, and FIG. 5 is an image sensor, a three-dimensional survey sensor, and these sensors. It is a diagram illustrating ground observation data acquired from a multi-sensor platform consisting of.

6 is a diagram illustrating a process of setting reference data from observation data obtained from a 3D survey sensor, and FIG. 7 shows a process of setting reference data from aerial observation data and ground observation data obtained from image sensors. It is a drawing. 8 is a diagram illustrating a process of setting reference data from ground observation data acquired from a 3D lidar sensor. 9 is a diagram illustrating that observation data acquired from a 3D survey sensor is geometrically transformed into a top view.

First, a first geometric transformation is processed for at least one of aerial observation data and ground observation data received from the input unit 102 (S305).

Aerial and ground observation data may be generated to include various data by performing predetermined processing on at least one of an image sensor and a 3D survey sensor and sensor data acquired from navigation sensors.

The aerial observation data by the image sensor may be an orthogonal image photographed in a vertical direction on the ground as shown in FIG. 4(a) and/or an image photographed in a non-vertical direction with respect to the ground as in FIG. 4(b).

Ground observation data is image data in a direction upward from the ground as shown in FIG. 5(a), point cloud data by a 3D survey sensor as shown in FIG. 5(b), or a multi-sensor platform of an image sensor and a 3D survey sensor It may be data obtained by combining the acquired image data and point cloud data.

The aerial observation data may include aerial geometric information. The aerial geometric information may be different according to the type of the geometric transformation model to be used in the matching information extraction unit 108.

For example, when the geometric transformation model is a physical sensor model, the aerial geometric information may include external geometric data for aviation and internal geometric data for aviation having position data for aviation and attitude data for aviation. When the geometric transformation model is an alternative sensor model using a substitute model parameter applied to Equation 1, the aircraft geometric information may include a substitute model parameter for aviation.

Like aerial observation data, ground observation data may have different ground geometric information depending on the type of the geometric transformation model. For example, when the geometric transformation model is a physical sensor model, the ground geometric information may include external geometric data for ground and internal geometric data for ground having position data for the ground and attitude data for the ground. When the geometric transformation model is a replacement sensor model using the replacement model parameter applied to Equation 1, the ground geometry information may include a replacement model parameter for the ground.

In addition, the aerial and ground observation data is related to the above-described geometric information, time information on the acquisition time of each sensor, unique data related to device specific properties for each sensor, and point cloud data when observation data is acquired by a 3D survey sensor. The object information for which the attribute of the extracted object is estimated based on at least one of information, information related to geocoding image data, information related to point cloud data, and information related to geocoding image data when observation data is obtained from an image sensor. Can include.

The internal geometric data is an inherent value of the sensor itself, and is an error in observed data for each sensor due to a geometric parameter maintained regardless of whether the platform or the like is moved.

The aerial and ground observation data may further include information related to 3D point cloud data based on the 3D survey sensor and information related to geocoding image data based on the image sensor, based on sensor data and geometric information acquired from the sensor.

Information related to 3D point group data includes 3D coordinate data for point group, color information for point group, class information for point group estimated from the object extracted from 3D coordinate data for point group, and laser when the 3D survey sensor is a laser sensor. It may include at least one of strength information.

The geocoding image data related information may include at least one of geocoding coordinate data for each pixel, geocoding color information, and geocoding class information whose type is estimated from the object extracted from the geocoding coordinate data. .

The aerial and ground observation data may include object information extracted in relation to an attribute of an object extracted based on at least one of information related to 3D point cloud data and information related to geocoding image data.

When aerial observation data is obtained from either an image sensor or a 3D survey sensor, and ground observation data is acquired from a sensor different from the sensor of the aerial observation data, the input unit 102 is obtained from a 3D survey sensor. The resultant point cloud data is converted into virtual image data based on the reflection intensity value or altitude value of the laser, the position data of the observation data is defined as the position data of the virtual image data, and the attitude data of the observation data is virtual image data. It can be defined as the viewing direction of.

The geometric conversion unit 104 converts the aerial observation data and the ground observation data to have a similar geometric shape by processing a first geometric transformation on at least one of the aerial observation data and the ground observation data.

Various processing of the geometric conversion units 104a and 104c is performed by referring to the position data and attitude data constituting the geometric information of the unprocessed observation data, as shown in FIGS. 2A and 2B. 1 Geometric transformation can be performed.

For example, as shown in FIG. 2A, the ground observation data is geometrically converted by the ground geometric conversion module 104b, but the aerial observation data is not geometrically converted. In this case, the ground observation data is based on aviation geometric information such as aviation position data, aviation attitude data, and aviation internal geometric data of aviation observation data, as shown in FIG. Can be converted similarly to 6 is a diagram illustrating that observation data acquired from a 3D survey sensor is geometrically transformed into a top view.

As another example of FIG. 2B, the aerial observation data is geometrically converted by the aerial geometric conversion module 104d, but the ground observation data is not geometrically converted. In this case, the aerial observation data may be converted similarly to the geometry of the ground observation data based on ground geometric information such as ground position data, ground attitude data, and ground internal geometric data of the ground observation data.

As another example, as shown in FIG. 2C, when the first geometric conversion is processed on all of the aerial observation data and the ground observation data using the aerial geometric conversion module 104f and the ground geometric conversion module 104g, the geometric conversion The unit 104e may process the first geometric transformation by referring to attitude data different from the aviation attitude data and the ground attitude data.

For example, the first geometric transformation of the aerial observation data in the non-vertical direction and the ground observation data of FIG. 5(a) as shown in FIG. 4(b) is different attitude data as shown in FIG. It can be implemented by referring to the posture data of the image.

Next, as shown in FIGS. 2A to 2C, the similarity analysis unit 106 performs a similarity analysis between the aerial observation data and the ground observation data on which the first geometric transformation has been processed for at least one of the aerial observation data and the ground observation data. Reference data that are estimated to be identical to each other are set (S310).

When at least one of the aerial observation data and the ground observation data is obtained from the image sensor, the similarity analysis unit 106 performs color change based on pixel coordinates among geocoding image data information belonging to the observation data, as shown in FIG. 8. After analyzing, it is possible to perform a similarity analysis to set reference data by extracting geometric information estimated as a feature geometry.

When at least one of the aerial observation data and the ground observation data acquires observation data from a 3D survey sensor, the similarity analysis unit 106 performs a geometrical geometry of point cloud data belonging to the observation data, as shown in FIGS. 7 and 9. Reference data may be set by extracting the geometric shape information by analyzing at least one of a change in shape and a change in laser intensity.

The above-described geometric shape information may be a simple shape such as a line, a polyline, a surface, a column, a sphere, etc. that is easy to match, or an object such as a lane of a road or a road surface mark.

In addition, when the observation detection accuracy is lower than the detection threshold at at least one of the aerial observation data and some of the ground observation data, the similarity analysis unit 106 The aerial observation data corresponding to a position lower than the detection threshold and the ground observation data may be filtered so as not to set the reference data with reference to the dragon location data.

Before performing the similarity analysis, the similarity analysis unit 106 may perform the preprocessing already mentioned through FIG. 1.

Next, the matching information extraction unit 108 is based on the reference data, the first geometric transformation of at least one of the observation data is processed for aviation geometric information of the aerial observation data and the ground geometric information of the ground observation data A second geometric transformation for at least one of is performed to extract matching information of a geometric transformation model between aerial observation data and ground observation data (S315).

The matching information extraction unit 108 may extract matching information using a geometric transformation model. In this case, the geometric transformation model may be a physical sensor model, or when it is difficult to use external geometric data, as shown in Equation 1, an alternative sensor model such as a linear equation, polynomial equation, or proportional equation expressed as an alternative model parameter. have.

Hereinafter, an embodiment in which the matching information extracting unit 108 extracts matching information using a physical sensor model will be mainly described.

2A and 2B, when the first geometric transformation is processed on either of aerial observation data and ground observation data, the second geometric transformation is the position data and attitude data of the observation data for which the first geometric transformation is unprocessed. With reference to, the first geometric transformation may be performed on the processed observation data.

In another embodiment, the second geometric transformation may be performed on both observation data by referring to geometric information such as the observed data processed with the first geometric transformation and the attitude data completely different from the unprocessed observation data.

As shown in FIG. 2C, when the first geometric transformation is processed on all of the aerial observation data and the ground observation data, the aerial observation in which the first geometric transformation is all performed with reference to different attitude data utilized in the first geometric transformation. It can be performed on data and ground observation data.

The matching information extraction unit 108 performs a second geometric transformation on the geometric information of the observation data higher than the observation detection threshold at a location where any one of the aerial observation data and the ground observation data is filtered, so that the geometric transformation model is matched. Information can be extracted.

When using the geometric transformation model according to the physical sensor model, the matching information extraction unit 108 may utilize external geometric data and internal geometric data of a sensor that has obtained aerial observation data and ground observation data.

In the similarity analysis unit 106, even if the observation detection precision of both aerial observation data and ground observation data is lower than the detection threshold and all of them are filtered, the matching information extraction unit 108 is lower than the detection threshold, but the observation detection precision is high. By performing the second geometric transformation, a geometric transformation model may be established, and matching information for residual observation data may be extracted.

The matching information verification unit 110 verifies the matching information based on the reference data and re-extracts the matching information (S320).

The matching information verification unit 110 may be performed in two ways, and may be a random sampling method and an expected error analysis method for reference data.

FIG. 10 is a flowchart illustrating an embodiment of re-extracting matching information by verifying matching information based on reference data, and FIG. 11 is a flowchart of another embodiment of re-extracting matching information by verifying matching information based on reference data to be.

According to the random sampling method according to FIG. 10, the matching information verification unit 110 randomly samples feature geometries for a sample among reference data estimated from the second geometrically-transformed aerial observation data and ground observation data. Matching information related to the sample feature geometry for which the mutually spaced amount of the feature geometries is minimum is calculated (S1005).

Next, the matching information verification unit 110 re-extracts the matching information of the geometric transformation model as a second geometric transformation consisting of matching information related to the sample feature geometry, and transmits the re-extracted to the data fusion unit 112 (S1010).

Specifically, the matching information verification unit 110 samples a plurality of aerial and image observation data sharing the same estimated feature geometry for each sample. These sampling sets are acquired in plurality for different standard feature geometries. At the same time, the reference data having the feature geometry estimated to be the same as the sample feature geometry are interlocked to be selected. Subsequently, when the aerial and ground observation data and reference data that share the same estimated sample feature geometries are arranged according to the second geometric-transformed geometric information, the sample feature geometries and reference data of the aerial and ground observation data We select the sample feature geometry of the observed data whose mutual separation is determined to be the minimum in the feature geometry of. This sample feature geometry is re-selected as reference data, and matching information related thereto is re-extracted.

As another embodiment, according to the predicted error analysis method according to FIG. 11, the matching information verification unit 110 minimizes the amount of separation between the reference data estimated to be the same from the second geometrically transformed aerial observation data and the ground observation data. The probability value is calculated (S1105).

The predicted error probability between the same estimated reference data is analyzed based on the minimum separation amount used when calculating the maximum value (S1110).

The most probable value is calculated by a predetermined analysis model equation, and some parameters among a plurality of parameters may be applied to this model equation.

By calculating the calculated maximum value as described above, geometric correction of aerial and ground observation data using the reference data is performed, and the time limit correction is the result of applying the reference data to the geometric transformation model and the geometric error of the observation data corresponding to the reference data. This is a process of analyzing a vehicle and calculating a parameter whose geometric error amount is minimal or optimized as a maximum value.

The maximum value may be at least one of error information of an external geometry and an internal geometry. In addition, the maximum probable value may be calculated as at least one of error information of geometric shape information generated by at least one of the above-described information and point cloud data related information, geocoding image data related information, object information, and map related information.

Subsequently, when the expected error according to the expected error probability exceeds the allowable error (Y in S1115), the matching information verification unit 110 is a constraint condition of the matching information extraction process according to the geometric transformation model applied to the second geometric transformation. To change (S1120).

In relation to the change of the constraint condition, a process of sampling and extracting geometric shape information estimated as different feature geometry from aerial observation data and the ground observation data as new reference data, and geometric transformation generated by the matching information extraction unit 108 Processing of removing reference data that is judged as abnormal data from the model, processing of changing at least some parameters applied to the geometric transformation model, and processing of changing or analyzing at least some parameters used in the analysis model equation applied at the time of calculating the maximum value At least one of the processes of changing the model expression may be performed. The matching information verification unit 110 may control the similarity analysis unit 106 and the matching information extraction unit 108 to perform the above-described processing, and according to the constraint condition after performing the processing, the matching information extraction unit 108 May re-extract the matching information of the geometric transformation model through the second geometric transformation.

Next, the data fusion unit 112 registers the second geometrically-transformed aerial observation data and the ground observation data as fusion data of the same coordinate system defined in the geometric transformation model based on the re-extracted matching information.

12 is a diagram illustrating the fusion of aerial observation data and ground observation data.

Various information belonging to aerial observation data and ground observation data are registered for each same location in the same coordinate system, and as shown in FIG. 12, it is possible to accurately grasp specific location information and object information with high-precision location data among aerial observation data and ground observation data. Not only can it be possible, but it is also useful for realizing spatial information composed of various information.

The components that make up the system 100 shown in Fig. 1 or the steps according to the embodiments shown in Figs. 3, 10, and 11 may be recorded on a computer-readable recording medium in the form of a program that realizes its function. I can. Here, the computer-readable recording medium refers to a recording medium that can be read by a computer by storing information such as data or programs by electrical, magnetic, optical, mechanical, or chemical action. Examples of such recording media that can be separated from a computer include portable storage, flexible disks, magneto-optical disks, CD-ROMs, CD-R/Ws, DVDs, DATs, and memory cards. In addition, as recording media fixed to mobile devices and computers, there are solid state disks (SSDs), hard disks, and ROMs.

Further, in the above, even if all the constituent elements constituting the embodiments of the present invention are described as being combined and operated as one, the present invention is not necessarily limited to these embodiments. That is, within the scope of the object of the present invention, all the constituent elements may be selectively combined and operated in one or more. In addition, although all of the components may be each implemented as one independent hardware, some or all of the components are selectively combined to provide a program module that performs some or all functions combined in one or a plurality of hardware. It may be implemented as a computer program having.

Although the present invention has been described in detail through exemplary embodiments above, a person of ordinary skill in the art to which the present invention pertains can make various modifications to the above-described embodiments without departing from the scope of the present invention. I will understand. Therefore, the scope of the present invention is limited to the described embodiments and should not be determined, and should be determined by all changes or modifications derived from the claims and the concept of equality as well as the claims to be described later.

Claims (10)

A geometric conversion unit for processing a first geometric conversion on at least one of aerial observation data and ground observation data;
A similarity analysis unit configured to perform a similarity analysis between the aerial observation data and the ground observation data subjected to the first geometric transformation on the at least one of the at least one of the above and set reference data that are mutually estimated to be the same from the aerial observation data and the ground observation data;
Based on the reference data, a second geometric transformation for at least one of the aerial geometric information of the aerial observation data for which a first geometric transformation has been processed for at least one of the observation data and the ground geometric information of the ground observation data A matching information extracting unit configured to extract matching information of a geometric transformation model between aerial observation data and ground observation data; And
A data fusion unit for registering the aerial observation data and the ground observation data, which are second geometrically transformed based on the matching information, as fusion data of the same coordinate system,
When the observation detection accuracy is lower than a detection threshold at at least one of the aerial observation data and some of the ground observation data, the similarity analysis unit aviation position data and the detection of the aerial observation data lower than the observation detection threshold. Aerial observation data and ground for filtering the aerial observation data and the ground observation data corresponding to a position lower than the detection threshold so as not to set the reference data by referring to the ground position data of the ground observation data lower than a threshold. A system of fusion between observational data. The method of claim 1,
When the first geometric transformation is processed on one of the aerial observation data and the ground observation data, the observation data subjected to the first geometric transformation is the position data of the unprocessed observation data and the observation direction of the observation data. The second geometric transformation is processed with reference to the attitude data defining the, and the second geometric transformation refers to the position data and the attitude data of the observation data in which the first geometric transformation is not processed, and the first geometric transformation is processed into the observation data. Is done about,
When the first geometric transformation is processed on all of the aerial observation data and the ground observation data, the first geometric transformation is an observation of the aerial attitude data and the ground observation data defining an observation direction of the aerial observation data. Aerial observation performed on the aerial observation data and the ground observation data, which are processed by referring to attitude data different from the ground attitude data defining a direction, and with reference to the different attitude data, all of which the first geometric transformation has been performed A system of fusion between data and ground observation data. The method of claim 1,
When at least one of the aerial observation data and the ground observation data is obtained from an image sensor, the similarity analysis unit analyzes a color change based on pixel coordinates among geocoding image data information belonging to the observation data, and When the similarity analysis is performed to set the reference data by extracting geometric shape information estimated to be, and at least one of the aerial observation data and the ground observation data acquires the observation data from a 3D survey sensor, the similarity degree The analysis unit is a fusion system between aerial observation data and ground observation data for setting the reference data by analyzing at least one of a geometric shape gradient and a laser intensity gradient of point cloud data belonging to the observation data to extract the geometric shape information . delete The method of claim 1,
The matching information extraction unit performs a second geometric transformation on the geometric information of the observation data higher than the observation detection threshold at a location where any one of the aerial observation data and the ground observation data is filtered, and the geometric transformation model A fusion system between aerial observation data and ground observation data to extract matching information. The method of claim 1,
A fusion system between aerial observation data and ground observation data, further comprising a matching information verification unit for re-extracting the matching information by verifying the matching information based on the reference data. The method of claim 6,
The matching information verification unit randomly samples sample feature geometries from among the reference data estimated to be the same from the second geometrically transformed aerial observation data and the ground observation data, so that the mutually spaced amount of the sample feature geometries is minimized. Calculate matching information related to the sample feature geometry, and re-extract the matching information of the geometric transformation model with a second geometric transformation consisting of matching information related to the sample feature geometry,
The data fusion unit is a fusion system between aerial observation data and ground observation data for registering the aerial observation data and the ground observation data, which are second geometrically transformed based on the re-extracted matching information, as fusion data of the same coordinate system. The method of claim 6,
The matching information verification unit calculates a most probable value that minimizes the amount of separation between the second geometrically transformed aerial observation data and the reference data estimated to be the same from the ground observation data, and calculates the maximum value. Analyzing the predicted error probability between the reference data estimated to be the same based on the minimum separation amount used at the time,
When the predicted error according to the predicted error probability exceeds the allowable error, the process of sampling and extracting geometric shape information estimated by different feature geometries from the aerial observation data and the ground observation data as new reference data, the matching A process of removing reference data determined as abnormal data from the geometric conversion model generated by the information extraction unit, a process of changing at least some parameters applied to the geometric conversion model, and an analysis model equation applied at the time of calculating the maximum value Controlling the similarity analysis unit and the matching information extraction unit to perform at least one of processing of changing at least some parameters used for or changing the analysis model equation,
According to the constraint condition after performing the processing, the matching information extracting unit re-extracts the matching information of the geometric transformation model by the second geometric transformation,
The data fusion unit is a fusion system between aerial observation data and ground observation data for registering the aerial observation data and the ground observation data, which are second geometrically transformed based on the re-extracted matching information, as fusion data of the same coordinate system. The method of claim 1,
When the aerial observation data is obtained from any one of an image sensor and a 3D survey sensor, and the ground observation data is obtained from a sensor different from the sensor of the aerial observation data, observation obtained from the 3D survey sensor The data uses virtual image data converted from point cloud data, the position data of the observation data related to the 3D survey sensor is defined as the position data of the virtual image data, and the attitude data of the observation data is the virtual image data. A fusion system between aerial observation data and ground observation data, which is defined as the viewing direction of the image data. The method of claim 1,
The geometric transformation model uses external geometric data consisting of position data and attitude data defining an observation direction, and internal geometric data calculated based on a geometric parameter defined in a sensor set to generate the observation data. model), the aviation geometric information includes at least aviation position data and aviation attitude data, and the ground geometric information includes at least ground position data and ground attitude data,
When the geometric transformation model is a replacement sensor model using a replacement model parameter other than the external geometric data, the aerial geometry information and the ground geometry information are respectively an aviation replacement model parameter and a ground replacement model parameter Convergence system between aerial observation data and ground observation data comprising a.

Images