KR101931819B1 - Digital map gereration system for determining target object by comparing image information and aerial photograph data, and obtaining 3-dimensional coordination of target object using information obtained by camera - Google Patents

Abstract

(여기서, a_m 은 영상 중심점에서 곡선의 현에 내린 수선의 길이를 나타내고, R은 영상 대각선의 길이를 나타내고, m = 1~M , M은 선의 개수를 나타내며, ψ는 사용자 파라미터를 나타냄)에 의해 획득되는 필터링 모듈; 상기 제 1 보정 그룹과 상기 제 2 보정 그룹이 제거된 영상 정보에 기초하여, 영상의 휘어짐 정도를 나타내는 매개변수를 추출하는 매개변수 추출 모듈; 상기 매개변수를 이용하여 상기 영상 정보의 보정을 수행하는 영상보정 모듈을 포함하고, 상기 데이터 처리 모듈은, 상기 대상 객체의 3차원 좌표를 획득하되, 이 중 Z축 좌표값은 좌측 카메라로부터 획득된 영상의 Y좌표(Y_l)와 우측 카메라로부터 획득된 영상의 Y좌표(Y_r)의 평균값, 초점거리(f) 및 상기 대상 객체까지의 Y축상 거리(Yo)에 기초하여 다음 수학식

을 이용하여 획득되는 것을 특징으로 하는 수치지도 제작 시스템을 제공한다.The present invention relates to a digital map production system for distorting and correcting image data using parameters and comparing the corrected image data with aerial photograph data, comprising: an acquired image acquisition module for acquiring image information from a plurality of cameras; A GPS receiving module for receiving GPS information from a satellite and obtaining current position information of the camera; An inertia measurement module for obtaining horizontal information of the camera using a plurality of sensors; An image processing module for correcting the image information based on the image information, the current position information, and the horizontal information; A control module for controlling the image information, current position information and horizontal information to be transmitted to the image processing module; And a buffer for storing image information; An aerial photograph database for receiving and storing the image information from the mobile mapping device or receiving and storing aerial photograph data from an external database; A data processing module for determining a target object by comparing the image information obtained from the mobile mapping device with the aerial photograph data obtained from the aerial photograph database, and performing three-dimensional space modeling using the determined target object information; And a communication module for receiving request information for a spatial modeling service from a user, transmitting the request information to the mobile mapping device, the aerial photograph database, and the data processing module, and transmitting the three-dimensional spatial data obtained from the data processing module to a user; Wherein the image processing module detects a curve from the image information received from the captured image acquisition module to set a first correction group, removes the first correction group from the image information, The second correction group is set as a second correction group on the three-dimensional coordinates from the center of the image information from which the correction group has been removed, and the second correction group is removed from the image information from which the first correction group is removed, The value is calculated by the following equation

(Where a _m Wherein R is the length of the diagonal line of the image, m = 1 to M, M is the number of lines, and? Is a user parameter), the filtering module ; A parameter extracting module for extracting a parameter indicating a degree of warping of the image based on the image information from which the first correction group and the second correction group have been removed; Wherein the data processing module obtains three-dimensional coordinates of the target object, wherein the Z-axis coordinate value is obtained from the left camera On the basis of the Y-coordinate distance (Y_l) of the image and the Y-coordinate distance (Y_r) of the image obtained from the right camera, the focal length (f)

And a digital map generation system for generating a digital map.

Description

TECHNICAL FIELD [0001] The present invention relates to a digital map generation system for determining a target object by comparing image information and aerial image data and acquiring three-dimensional coordinates of the target object using camera acquisition information. , AND OBTAINING 3-DIMENSIONAL COORDINATION OF TARGET OBJECT USING INFORMATION OBTAINED BY CAMERA}

The present invention relates to a digital map production system that compares image information and aerial photograph data to determine a target object and acquires three-dimensional coordinates of the target object using camera acquisition information.

The 3D space modeling method using existing aerial surveying is available only to the photographed part of the acquired building, and it has been forcibly extended to the end of the building. Since the image has been forcibly expanded, not only the visibility is significantly lowered but also the accurate coordinate information about the object can not be obtained, and it is difficult to provide more realistic image data to the user even after completing the modeling work.

In addition, in the conventional space modeling method, an excessive labor cost is generated by using data constructed manually by field survey, and errors of geographical information are frequently generated due to such manual operation, and it is difficult to correct and update In order to overcome these drawbacks, geographic information data on buildings and road facilities are being constructed using the latest surveying equipment such as mobile mapping system (MMS). However, it is difficult to acquire precise coordinate information due to distortion of the measured image, and the like. In addition, since the accuracy of the information is lacking in the urban area, there are many problems in building a geographic information database .

In addition, the mobile mapping system and terminals using geographical information will transmit and receive massive amounts of data using a communication system in the future in establishing geographic information data on buildings and road facilities using the mobile mapping system. Therefore, it is necessary to efficiently manage the data transmission amount in order to efficiently transmit and receive such a large amount of image data.

The present invention intends to acquire more accurate three-dimensional space coordinate information by correcting a warping phenomenon of an image through a filtering process after acquiring an image obtained through a mobile measurement equipment (MMS).

In addition, the present invention intends to improve the efficiency of signal processing by applying an adaptive filtering process to a frame of a photographed image.

In addition, the present invention intends to improve an image by applying a post-processing process using a high-frequency characteristic to a photographed image.

In addition, the present invention intends to perform finer image correction by performing filtering using a weighted threshold value when extracting parameters for image correction.

In addition, the present invention intends to perform more precise image correction by redundantly eliminating error elements when extracting parameters for image correction.

Also, the present invention compares the image information of the target object with the common point vector information generated from the aerial photograph, and obtains the coordinate information of the target object to thereby calculate the more accurate three-dimensional space coordinate information.

(여기서, a_m 은 영상 중심점에서 곡선의 현에 내린 수선의 길이를 나타내고, R은 영상 대각선의 길이를 나타내고, m = 1~M , M은 선의 개수를 나타내며, ψ는 사용자 파라미터를 나타냄)에 의해 획득되는 필터링 모듈; 상기 제 1 보정 그룹과 상기 제 2 보정 그룹이 제거된 영상 정보에 기초하여, 영상의 휘어짐 정도를 나타내는 매개변수를 추출하는 매개변수 추출 모듈; 상기 매개변수를 이용하여 상기 영상 정보의 보정을 수행하는 영상보정 모듈을 포함하고, 상기 데이터 처리 모듈은, 상기 대상 객체의 3차원 좌표를 획득하되, 이 중 Z축 좌표값은 좌측 카메라로부터 획득된 영상의 Y좌표(Y_l)와 우측 카메라로부터 획득된 영상의 Y좌표(Y_r)의 평균값, 초점거리(f) 및 상기 대상 객체까지의 Y축상 거리(Yo)에 기초하여 다음 수학식

을 이용하여 획득되는 것을 특징으로 하는 수치지도 제작 시스템을 제공한다.The present invention relates to a digital map production system for distorting and correcting image data using parameters and comparing the corrected image data with aerial photograph data, comprising: an acquired image acquisition module for acquiring image information from a plurality of cameras; A GPS receiving module for receiving GPS information from a satellite and obtaining current position information of the camera; An inertia measurement module for obtaining horizontal information of the camera using a plurality of sensors; An image processing module for correcting the image information based on the image information, the current position information, and the horizontal information; A control module for controlling the image information, current position information and horizontal information to be transmitted to the image processing module; And a buffer for storing image information; An aerial photograph database for receiving and storing the image information from the mobile mapping device or receiving and storing aerial photograph data from an external database; A data processing module for determining a target object by comparing the image information obtained from the mobile mapping device with the aerial photograph data obtained from the aerial photograph database, and performing three-dimensional space modeling using the determined target object information; And a communication module for receiving request information for a spatial modeling service from a user, transmitting the request information to the mobile mapping device, the aerial photograph database, and the data processing module, and transmitting the three-dimensional spatial data obtained from the data processing module to a user; Wherein the image processing module detects a curve from the image information received from the captured image acquisition module to set a first correction group, removes the first correction group from the image information, The second correction group is set as a second correction group on the three-dimensional coordinates from the center of the image information from which the correction group has been removed, and the second correction group is removed from the image information from which the first correction group is removed, The value is calculated by the following equation

(Where a _m denotes the length of the waterline on the curve at the center of the image, R denotes the length of the image diagonal, m = 1 to M, M denotes the number of lines, and ψ denotes the user parameter) A filtering module obtained by the filtering module; A parameter extracting module for extracting a parameter indicating a degree of warping of the image based on the image information from which the first correction group and the second correction group have been removed; Wherein the data processing module obtains three-dimensional coordinates of the target object, wherein the Z-axis coordinate value is obtained from the left camera On the basis of the Y-coordinate distance (Y_l) of the image and the Y-coordinate distance (Y_r) of the image obtained from the right camera, the focal length (f)

And a digital map generation system for generating a digital map.

According to the present invention, the curve set as the first correction group is obtained by searching a segment existing in an image, calculating a rate of change and an angle of each pixel in the segment, and calculating a pixel having a rate of change higher than a predetermined threshold, Sets pixels having the greatest change rate in a window of a predetermined size as a group and searches for the next pixel through the window with the pixel as a starting point, And is formed by connecting.

Also, in the present invention, the inertia measurement module may log data at intervals of 100 Hz.

In the present invention, the GPS receiving module may log data at intervals of 1 Hz.

Also, in the present invention, the communication module may include a local area network (LAN), a wireless local area network (WLAN), a wireless broadband (Wibro), a world interoperability for microwave access (Wimax), and a high speed downlink packet access .

The present invention can acquire more accurate three-dimensional space coordinate information by correcting the warping phenomenon of an image through filtering after acquiring an image acquired through a mobile measurement equipment (MMS).

Further, in the present invention, more detailed image correction can be performed by performing filtering using a weighted threshold value when extracting parameters for image correction.

In addition, the present invention can perform more precise image correction by redundantly eliminating error elements when extracting parameters for image correction.

Also, the present invention compares the image information of the target object with the common point vector information generated from the aerial photograph, and obtains the coordinate information of the target object, so that more accurate three-dimensional space coordinate information can be calculated.

In addition, the present invention can improve the efficiency of signal processing by applying an adaptive filtering process to a frame of a photographed image.

In addition, the present invention can perform image enhancement by applying a post-processing process using a high-frequency characteristic to a photographed image.

Fig. 1 shows a schematic block diagram of an automatic three-dimensional space modeling system to which the present invention is applied.
2 is a schematic block diagram of an image capture module 110 according to an embodiment of the present invention.
3A is a schematic block diagram of an image processing module 120 according to an embodiment of the present invention.
FIG. 3B is a schematic internal block diagram of the post-processing unit 127 according to an embodiment to which the present invention is applied.
4A illustrates a coordinate system for explaining a method of performing image correction by defining a first correction group and a second correction group according to an embodiment of the present invention.
FIG. 4B illustrates a coordinate system for explaining a weighted threshold calculation process for image correction according to an embodiment of the present invention. Referring to FIG.
5 is a schematic block diagram of the data processing module 500 according to an embodiment of the present invention.
FIG. 6 is a diagram illustrating a method of modeling three-dimensional coordinate information using coordinate information of a target object according to an embodiment of the present invention. Referring to FIG.
FIG. 7 is a flowchart for explaining a three-dimensional space modeling method according to an embodiment to which the present invention is applied.
8 is a flowchart for explaining a method of correcting an image obtained from a camera for three-dimensional space modeling according to an embodiment of the present invention.
9 is a flowchart for explaining a method for acquiring coordinate information of a target object from a corrected image for three-dimensional space modeling according to an embodiment of the present invention.
10 is a flowchart illustrating a method of applying an adaptive filtering process to a frame of a photographed image according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The above and other objects,

And the structure and operation of the present invention described with reference to the drawings are described as an embodiment, and the technical idea and the core structure and operation of the present invention are not limited thereby.

In addition, the term used in the present invention is intended to include,

, But the specific case is explained by using the terminology arbitrarily selected by the applicant. In such a case, the meaning is clearly stated in the detailed description of the relevant part, so it should be understood that the name of the term used in the description of the present invention should not be simply interpreted and that the meaning of the corresponding term should be understood and interpreted .

Space Modeling System is an information system that integrates geographical data occupying spatial position and related property data. It collects, stores, updates, processes, analyzes and outputs various types of geographical information efficiently. It can be defined as the total organization of hardware, software, geographical data, and human resources used for. Various applications of spatial information have become easier according to the use of the geographic information system. Therefore, it is necessary to use the mobile survey data efficiently in order to provide clear and realistic geographical information in a rapidly changing city center. Hereinafter, embodiments in which more accurate spatial modeling information is generated by using the mobile surveying data efficiently will be described.

FIG. 1 is a schematic block diagram of a spatial information generating system according to an embodiment of the present invention.

1, a spatial information generation system 10 according to an embodiment of the present invention includes a mobile mapping device 100, a communication module 200, an update module 300, an aerial photograph database 400, The mobile mapping apparatus 100 includes an image pickup module 110, an image processing module 120, an inertia measurement module 130, a GPS reception module 140, a control module 150, Buffer 160 may be configured.

Here, the spatial model may mean an object occupying a space in a three-dimensional image, for example, a road facility, a building, a road facility, and an auxiliary facility. The spatial model information may refer to the location, distance, area, and property information of the object, i.e., a road facility, a building, a road facility, and an auxiliary facility.

The mobile mapping apparatus 100 can measure a road facility, a building, a road-use facility, and an auxiliary facility in a target area while moving the route in the target area. The mobile mapping device 100 refers to a moving and measurable device, for example, a car or a plane equipped with a measurable camera.

The photographed image acquisition module 110 may include a left-eye image camera and a right-eye image camera. The photographed image acquisition module 110 may acquire stereo images by photographing the road facilities, buildings, road facilities, and auxiliary facilities using a left- can do. 2 is a schematic block diagram of an image capture module 110 according to an embodiment of the present invention.

The captured image acquisition module 110 may include an image sensing unit 111, a video coding unit 112, a buffer 113, and a data output unit 114.

The image sensing unit 111 may acquire a stereo image by photographing a road facility, a building, a road facility, and an auxiliary facility using a left eye image camera and a right eye image camera. The image signal output from the image sensing unit 111 may be an analog electric signal including wavelength information of RGB. However, for convenience of explanation, the image sensing unit 111 may include an analog-to-digital converter to output RGB raw image data (RGB RAW IMAGE DATA). The RGB raw image data output from the image sensing unit 111 may be stored in the buffer 113 and input to the video coding unit 112.

The video coding unit 112 may perform efficient signal processing by compressing and transmitting the received image information. In an embodiment to which the present invention is applied, the video coding unit 112 may include an adaptive filtering unit (not shown), and the adaptive filtering unit performs adaptive filtering on each frame, It can be possible. The adaptive filtering unit may generate a filter coefficient through the following process.

For example, let w [n] be the predicted pixel value of the current frame, let x [n] be the predicted pixel value of the filtered current frame, and let the filter coefficient be a _i . In this case, assuming that w [n] is an input value of the adaptive filtering unit and that an N-th order filter is used, the predicted pixel value x [n] of the filtered current frame is expressed by Equation Can be expressed.

Assuming that s [n] is the pixel value of the original frame and the difference between x [n] and s [n] is e [n], a filter coefficient a _i that minimizes e [n] have. The filter coefficient a _i can be expressed by the following equation (2).

The above E {e ² [n]} can be expressed by the following equation (3).

If the differential equation (3) with a _i to obtain the filter coefficients a _i to a minimum the E {e ² [n]} as shown in the following equation (4).

If w [n] and s [n] are fixed, then R _W [m] and R _ws [m] are the cross correlation between w [n] and w [n] (5) " (5) "

Equation (4) can be expressed as Equation (6) using Equation (5).

The value of a _i , which makes Equation (6) 0, is obtained, which is shown in Equation (7).

From the above equation (7), the filter coefficient a _i can be obtained.

In this case, the video coding unit 112 may output the generated filter coefficient to the data output unit 120. In this case, the filter coefficient a _i obtained through the above process is applied to the current frame, To the image processing module 120 or the data processing module via the image processing module 114. [ However, when the application of the filter coefficient does not significantly improve the efficiency of the signal processing, the filter coefficient may not be applied to the current frame by using a flag indicating whether or not the filter coefficient is applied without transmitting the filter coefficient. In this case, the video coding unit 112 may transmit the flag information to the image processing module 120 or the data processing module through the data output unit 114.

In addition, the captured image acquisition module 110 needs to correct the acquired image due to the camera shake at the time of shooting due to the shake of the mobile mapping device 100, or a bowing problem of the camera lens itself. Therefore, in the embodiment to which the present invention is applied, a method of estimating a correction parameter for correcting such an image and a new threshold value calculating method using weighting are proposed. This will be described in more detail in Figs. 4A and 4B.

The captured image acquisition module 110 acquires the left eye image and the right eye image acquired from the camera mounted in the mobile mapping device 100 and acquires the left eye image and the right eye image from the GPS receiving module 140, The current position information of the left eye image and the right eye image can be obtained using the position information. The moving path of the mobile mapping apparatus 100 can be generated from the current position information of the left eye image and the right eye image. That is, the movement path may be determined from the position information of the mobile mapping apparatus 100, and the left eye image and the right eye image acquired from the camera may be matched for each movement path.

The inertia measurement module 130 can acquire the horizontal information of the mobile mapping apparatus 100 by receiving sensing information using multiple sensors, and can perform more detailed image correction using the horizontal information. In an embodiment to which the present invention is applied, the inertia measurement module 130 logs data at intervals of 100 Hz.

The GPS receiving module 140 may receive the GPS information from the satellites and obtain the current position information of the mobile mapping device 100. [ The mobile mapping device 100 acquires location information of a vehicle in a city center through the GPS receiving module 140 and transmits the location information of the vehicle through an IMU (Inertial Measurement Unit) By obtaining horizontal information, more accurate 3D space modeling can be performed. In an embodiment to which the present invention is applied, the GPS receiving module 140 logs data at intervals of 1 Hz.

The image processing module 120 calculates parameters by using the horizontal information obtained from the inertia measurement module 130 and the position information obtained from the GPS receiving module 140 for correction of the acquired image. The parameter calculation method to which the present invention is applied is that a straight line in a 3D image is a straight line even if it is converted into a 2D image. That is, more accurate 3D space modeling can be performed by detecting lines that are not straight lines of the 3D image, excluding them, and then performing image correction. In addition, in the process of calculating the parameters, faster 3D space modeling can be performed by excluding lines out of the threshold. A detailed description of these embodiments is given in Figs. 4A and 4B.

The control module 150 outputs the left eye image and the right eye image obtained from the captured image acquisition module 110, the horizontal information obtained from the inertia measurement module 130, and the position information obtained from the GPS reception module 140, respectively To the image processing module 120, and controls the image processing module 120 to calculate parameters using the image information, the horizontal information, the position information, and the like. The control module 150 controls the synchronization between the sensed image acquisition module 110, the inertia measurement module 130 and the GPS reception module 140. In an embodiment to which the present invention is applied, The inertial measurement module 130, and the GPS reception module 140, respectively, based on the GPS time obtained from the GPS receiver 140, have.

The buffer 160 stores the image information of the left eye image and the right eye image obtained from the sensed image acquisition module 110 in a buffer and provides the requested image information when the image information is requested from the outside. The buffer 160 may be composed of a memory and a memory control module. The memory 160 may be rearranged through a log analysis or the like to provide a faster spatial data service. For example, it is possible to determine the received request information through the log analysis, that is, frequently requested video information, and process the video information in a different manner accordingly. If the received request information is frequently requested video information, the video information may be cached in a memory in units of layers and updated automatically. On the other hand, if the image information is not frequently requested, the corresponding image information is deleted from the memory, or if the image information is not previously requested, the corresponding image information is updated to the table information and a corresponding code is generated and stored have.

The communication module 200 transmits the request information to the mobile mapping apparatus 100, the update module 300, the aerial photograph database 400, and the data processing module 500 when receiving the request information for the spatial modeling service from the user. If the received request information requires video information of the mobile mapping apparatus 100, the mobile mapping apparatus 100 transmits the request information to the mobile mapping apparatus 100, and transmits the request information from the mobile mapping apparatus 100 It is possible to receive corresponding spatial data. The communication module 200 manages communication between a user terminal (not shown) and an external terminal (not shown) and between the mobile mapping apparatus 100 and the data processing module 500, May support a variety of communication standards such as local area network (LAN), wireless LAN (WLAN), wireless broadband (WIBRO), world interoperability for microwave access (WIMAX), high speed downlink packet access (HSDPA)

The update module 300 may update the data stored in the buffer 160 in the mobile mapping apparatus 100 with the modified data through the image processing module 120 or the data processing module 500 or the like. Also, the data stored in the aerial photograph database 400 can be updated with the modified data through the image processing module 120 or the data processing module 500. [

The aerial photograph database 400 includes information on the area, length, and location of the road facilities, buildings, road facilities, and other facilities. The aerial photograph database 400 receives data from the mobile mapping device 100, And can receive the aerial photograph data from the database.

The aerial photograph database 400 may include a memory and a memory control module. The aerial photograph database 400 may provide spatial data service more rapidly by rearranging memories through log analysis and the like. For example, it is possible to determine through the log analysis whether the received request information is frequently requested aerial photograph data, and to process the aerial photograph data accordingly. If the received request information is frequently requested aerial photograph data, the aerial photograph data may be cached in a memory in units of layers and automatically updated. On the other hand, if the data is not the frequently requested aerial photograph data, the corresponding aerial photograph data is deleted from the memory, or if the aerial photograph data is not previously requested, the corresponding aerial photograph data is updated to the table information, Can be created and stored.

The data processing module 500 generates 3D space modeling information using the data obtained from the mobile mapping apparatus 100 and the aerial photographing database 400. The data processing module 500 may include an object extraction module 501, an aerial photograph extraction module 502, an object positioning module 503, a coordinate comparison module 504, and a three-dimensional modeling module 505 . This will be described in more detail in FIG.

3A is a schematic block diagram of an image processing module 120 according to an embodiment of the present invention.

The image processing module 120 may include a parameter extraction module 121, a filtering module 123, an image correction module 125, and a post-processing unit 127.

Most cameras using a lens have a problem that a straight line is bent into a curved line, and a phenomenon occurs in which the image is expanded or contracted radially from the bent center. If the deflection phenomenon is not properly corrected, serious problems may occur in viewing 3D images or in digital image analysis.

In order to solve this problem, it is possible to perform mathematical modeling of the bending phenomenon and to perform compensation by estimating parameters for determining the degree of bending and inversely converting the parameters. Then, curves in the image are detected, and non-straight lines on the 3D coordinates are defined as a first correction group, and correction is performed. Furthermore, in one embodiment of the present invention, lines that are not a curve on the 3D coordinates and that interfere with parameter estimation can be defined and removed as a second correction group.

4A and 4B illustrate a coordinate system for explaining a method of performing image correction by defining the first correction group and the second correction group, And a coordinate system for explaining a weighted threshold calculation process for image correction.

Referring to FIG. 4A, when a basic image is denoted by R, a convex warped image is denoted by V, and a concave warped image is denoted by C, a coordinate of a point that is not warped with respect to the center point A is O = (X, Y) Assuming that the coordinates of the bent point is Oc = (Xc, Yc), the relationship between the two coordinates can be expressed by the following equations (8) and (9). Gi denotes a parameter reflecting the degree of warping of the image, and when Gi > 0, it indicates convex warpage, and Gi < 0 indicates concave warpage.

Here, a represents the distance from the center point A of the image to the coordinate Oc of the convex curved image, and a is represented by Equation 10 below.

As an embodiment to which the present invention is applied, a method of calculating the parameter will be described.

The parameter extraction module 121 first searches for a segment existing in the image, and calculates the rate of change and angle of each pixel in the segment. The segment can be detected by grouping the pixels of the same straight line into one group. A pixel having a rate of change higher than a given threshold value is set as a starting point and a pixel having the largest rate of change in a predetermined window is set as a group and the next pixel is searched again through the window using the pixel as a starting point. At this time, the window of the predetermined size may use 3x3, 4x4 or 5x5 window.

The curve may be formed by connecting the segments from the group consisting of the detected pixels through the above process. At this time, the distance between the end points of the two segments, the angle between the two segments, and the vertical distance of the two segments are calculated. If the calculated values are smaller than the preset threshold value, the segments are regarded as one line have.

A line that is connected to the segments and is regarded as a curve on a 3D coordinate may be set as a first correction group, and the first correction group may be removed in an image.

For example, if one target line of the F curves is referred to as Lm (m = 1 to M) and Nm pixels constituting Lm are denoted by Pn = (Xn, Yn) (n = 1 to Nm) The parameter Gi when the Nm number of pixels are maximally straight can be obtained. The calculated Gi can also be applied to other curves to obtain Pn. Pn is defined as an error En of each pixel, and when the pixels whose absolute magnitudes exceed the preset threshold value? _M exceed a preset percentage in one curve, they are set as the first correction group, The first correction group may be removed from the image.

Through this process, it is possible to calculate the parameter Gi which causes the first correction group to be the maximum straight line among the removed lines.

The filtering module 123 can define and filter the lines serving as the error element in the parameter calculation as the second correction group even though the curve is not a curve on the 3D coordinates. For lines located near the center point, you can narrow the range to which the parameter is applied by setting more precise thresholds. For example, referring to FIG. 4B, when it is determined that the error of each pixel exceeds a predetermined threshold value, a weight is applied to the threshold value according to the distance from the center of the image to the curve Lm, Can be determined.

The distance from the image center point A to the curve Lm can be defined as the length a _m of the perpendicular line drawn on the straight line F and the weight can be normalized to a range between 0 and 1. At this time, a _m Can be defined by Equation (11) below.

If the weight for normalization is 0, a _m Is 0 when the line passes the center of the image, and when the weight is 1, a _m This is the distance from the center of the image to the diagonal vertex. Represents the new threshold value δ _{m_ new} applying a weight to the equation shown in equation (12) below.

Here, R denotes the length of the image diagonal, m = 1 to M, M denotes the number of lines, and? Is a user parameter.

It is possible to calculate the new threshold value δ _{m_ new} by weighting the critical values depending on the distance of the line from the center of the image as described above, a more precise image correction them by obtaining more accurate parameter Gi through is possible.

The image correction module 125 may correct the image warping phenomenon using parameters obtained through the above process. The image corrected through the image correction module 125 is transmitted to the post-processing unit 127. The post-processing unit 127 can perform image enhancement by acquiring an image processing function based on the high-frequency component of the target image .

FIG. 3B is a schematic internal block diagram of the post-processing unit 127 according to an embodiment to which the present invention is applied.

The post-processing unit 127 may include a domain converting unit 127-1, a high-frequency filtering unit 127-2, a domain inverting unit 127-3, and an inverse filtering unit 127-4.

As an embodiment to which the present invention is applied, the post-processing unit 127 can perform image enhancement by acquiring an image processing function based on a high-frequency component of a target image. Various embodiments may be applied to obtain the image processing function. For example, the high-frequency filtering unit 127-2 can obtain a high-frequency component of an image to be improved. The image processing function can be obtained based on the high-frequency component. As a specific example, an image processing function can be obtained in consideration of inverse filtering in the YUV domain. At this time, domain conversion may be required, which may be performed by the domain conversion unit 127-1. Also, in order to output it, it is necessary to perform inverse transformation again, which can be performed in the domain inversion unit 127-3.

In another embodiment to which the present invention is applied, an image processing function may be obtained using inverse filtering through an inverse filtering unit 127-4 in the RGB domain.

For example, assuming that the image input through the captured image acquisition module is output through the mobile mapping device 100, each image is represented by a one-dimensional matrix, Let x, y, and D denote the system functions in the form of a cyclic Soffit matrix, as shown in Equation (13).

In this case, the acquired image is y, the object image for improvement is x, and the system function by the point spread function estimated using the optical transfer function is D. Therefore, when the transfer function of the lens is known accurately, the original input image can be obtained using the inverse matrix of the system function as shown in Equation (14).

Where H is the matrix inverse of D and is the matrix representation of the inverse point spread function estimated by the inverse Fourier transform of the inverse optical transfer function.

A target image to be post-processed can be divided into a high-frequency component and a low-frequency component of the image as shown in Equation (15).

In addition, since the image obtained through the mobile mapping apparatus 100 can be regarded as an image in which only low-frequency components are left by the lens, the low-frequency component can be considered as Equation (16).

The high frequency component can be obtained from the acquired image through high frequency filtering. For example, a high frequency component can be obtained for an arbitrary channel among input images. At this time, an arbitrary high-frequency filter can be used, and the arbitrary high-frequency filter can be expressed by the following Equation (17).

Accordingly, the image obtained by the image to be estimated can be expressed as Equation (18) using an arbitrary high-frequency filter.

In this case, considering the inverse filtering relation between x and y, Equation (18) can be expressed as Equation (19).

Therefore, the high-frequency filter can be obtained from the estimated inverse filter as: &quot; (20) &quot;

The image processing function for the arbitrary channel can be obtained based on the high frequency component obtained using the above equations. The obtained image processing function is expressed by the following equation (21).

Here, x '(i, j) represents a reconstructed image of a high frequency at a (i, j) pixel position, and HF (i, j) represents a high frequency component of an image acquired based on the high frequency filter obtained previously. Also, E {y (i, j)} may mean the average of the acquired image y. Equation (21) may be applied to different methods for edges, patterns, and flat regions. For example, in the case of the edge region, since the high frequency component is large, the lost high frequency component can be added while maintaining the acquired image. In the case of the flat area, a high frequency component can be added to the average of the acquired image to prevent the influence of noise, which is a side effect of image improvement, from becoming large. Since the pattern region includes the flat region and the edge region, it is necessary to adjust the high frequency component. Adding excessively high frequency components increases the noise in the flat region, and if the high frequency component is added to the small region, the edge improvement in the edge region may not be achieved. Therefore, the pattern region can add a high-frequency component to the middle of the edge and flat regions. For each high-frequency component, it is possible to restore to a more natural image by adjusting using parameters?,?,? As described above, the improved image can be restored by using the image processing function.

The mobile mapping apparatus 100 corrects and improves the image through the above process, and the corrected and improved image is transmitted to the data processing module 500.

5 is a schematic block diagram of the data processing module 500 according to an embodiment of the present invention.

The data processing module 500 may include an object extraction module 501, an aerial photograph extraction module 502, an object positioning module 503, a coordinate comparison module 504, and a three-dimensional modeling module 505 .

The object extraction module 501 may extract a desired object from the corrected image obtained from the mobile mapping apparatus 100 and obtain coordinate information of the object. A search area may be set to extract the target object, and a target object may be extracted from the corrected image according to the search area. At this time, the target object can use each pixel RGB value of the corrected image. For example, a pixel having the same characteristic can be detected by checking eight integer pixel values adjacent to the target pixel. If there are a plurality of target objects extracted through the above process, an object having the closest distance from the previous position of the target object can be selected. If no target object is extracted, the object can be selected by the user's input .

The object position determination module 503 can obtain the coordinate information of the target object by determining the center point of the target object extracted from the object extraction module 501. [ The center point can be determined by the following equation (22).

Here, x and y represent the coordinates of the center point, n is the number of pixels in the x-axis direction, and N is the number of pixels in the y-axis direction.

The aerial photograph extraction module 502 can extract the aerial photograph coordinate information corresponding to the corresponding image from the aerial photograph database 400. [ At this time, the aerial photograph extracting module 502 may extract the corresponding image corresponding to the position information from the aerial photograph database 400 according to the position information obtained from the GPS receiving module 140. [

The coordinate comparison module 504 can check whether the objects are matched by comparing the coordinate information of the object obtained from the object positioning module 503 with the aerial photograph coordinate information obtained from the aerial photograph extraction module 502 have. For example, matching points can be identified by extracting common points unchanged in size and rotation from images having different directions and sizes. An aerial photograph image extracted from the aerial photograph database 400 may be passed through a Gaussian filter having various dispersion values and an error image may be generated by sequentially subtracting the passed images according to a scale. This can be performed by Equation 23 below.

Where D (x, y, σ) represents the function of the image passed through the Gaussian filter and G (x, y, σ) represents the Gaussian filter. Respectively. The position of the pixel corresponding to the extreme value (maximum and minimum values) can be extracted from the generated error image. Here, the pixel position corresponding to the extreme value (maximum and minimum values) is a case where the selected pixel value is larger or smaller than a total of 26 pixel values, that is, eight pixels adjacent to the selected pixel value and nine pixels of the scaled upper and lower images . That is, pixels corresponding to the extreme values (maximum and minimum values) can be extracted as a common point. Noise removal and strong edge components are removed from the extracted common points, and direction information and size are set for the remaining common points. The direction information and the size are determined by the following equations (24) and (25), respectively.

A 16x16 block is set around the common point, and the direction of change is measured for each pixel in the block. The change direction is set to be proportional to the change slope, and the larger the change slope, the larger the probability distribution in the change direction. The probability distribution for the change direction is divided into eight directions and stored, which is expressed in a 128-dimensional vector form with respect to a 16x16 block. The common point vector information of the aerial photographic images thus obtained and the common point vector information of the corrected image are compared to check whether the objects are matched. The coordinate information of the target object can be extracted based on the matching check result.

As another embodiment to which the present invention is applied, filtering may be used for the object identification. First, a boundary region of an object in an image is identified, and a target object can be identified using the geometric characteristics of the boundary region. For example, since a building, a signboard, a roadside tree, and the like have respective geometric characteristics, a target object can be identified by extracting and comparing only the points of the corner of the building, the signboard, .

The three-dimensional modeling module 505 can model the three-dimensional coordinate information using the coordinate information of the target object. FIG. 6 is a diagram illustrating a method of modeling three-dimensional coordinate information using coordinate information of a target object according to an embodiment of the present invention. Referring to FIG.

Referring to FIG. 6, assuming that two left and right cameras shoot an object in order to model three-dimensional coordinate information, the distance between the interior angle of the triangle and the left and right cameras is calculated using Equations 26 to 33 below. .

The three-dimensional coordinates (X_o, Y_o, Z_o) of the target object O can be obtained by the following Equations (31) to (33), where Z_o coordinate is obtained by multiplying the Y_l coordinate of the image obtained from the left camera and the Y_l coordinate The average value of the coordinates, the focal length information, and the Y-axis distance to the object.

3D spatial modeling can be performed using the three-dimensional coordinates (X_o, Y_o, Z_o) of the target object O obtained by the equations (31) to (33).

FIG. 7 is a flowchart for explaining a three-dimensional space modeling method according to an embodiment to which the present invention is applied.

First, the captured image acquisition module 110 in the mobile mapping apparatus 100 includes a left eye image camera and a right eye image camera, and a left eye image and a right eye image can be obtained from the cameras, respectively (S710). The current position information of the cameras in the mobile mapping apparatus 100, that is, the obtained left and right eye images, is acquired from the GPS receiving module 140, and the horizontal information of the camera is acquired from the inertia measurement module 130 (S720).

The parameter extraction module 121 in the image processing module 120 may extract parameters for image correction using the horizontal information and the current position information of the camera in operation S730. At this time, the filtering module 123 may perform filtering using the weighted threshold value at the time of extracting the parameters (S740). The image correction module 125 may perform image correction using parameters that have been performed until filtering (S750).

The object extraction module 501 in the data processing module 500 receives the corrected image from the mobile mapping device 100 and extracts the target object in operation S760. Then, the object position determination module 503 can obtain the coordinate information of the target object by determining the center point of the extracted target object (S770).

The coordinate comparison module 504 can check whether the matching is performed by comparing the coordinate information of the obtained object and the coordinate information of the corresponding aerial photograph (S780).

The three-dimensional modeling module 505 can perform three-dimensional space modeling using the coordinate information of the target object (S790).

8 is a flowchart for explaining a method of correcting an image obtained from a camera for three-dimensional space modeling according to an embodiment of the present invention.

First, a curve may be detected from the image information obtained from the captured image acquisition module 110 (S810). The first correction group may be set to non-straight lines on the three-dimensional coordinates of the detected curves (S820). The lines corresponding to the set first correction group may be removed from the image information (S830).

In step S840, the first correction group may be detected from the center of the three-dimensional coordinate image, and the second correction group may be set to a second correction group. At this time, in order to reset a more precise threshold value, a new threshold value can be reset by applying a weight according to the distance from the image center point to the curve. And the second correction group can be set based on the reset threshold value. The lines corresponding to the set second correction group may be removed from the image information (S850).

The parameter may be extracted from the image from which the second correction group is removed (S860). The image correction may be performed by performing inverse transformation using the extracted parameters (S870).

9 is a flowchart for explaining a method for acquiring coordinate information of a target object from a corrected image for three-dimensional space modeling according to an embodiment of the present invention.

The image information of the target object obtained from the object position determination module 503 can be extracted (S910). Then, the coordinate information of the aerial photograph corresponding to the corrected image can be obtained from the aerial photograph database 400 (S920). remind

It is possible to confirm whether the objects are matched by comparing the image information of the target object with the aerial photograph coordinate information. For example, matching points can be identified by extracting common points unchanged in size and rotation from images having different directions and sizes. The aerial photographic image is passed through a Gaussian filter, and an error image is generated by sequentially subtracting the passed images according to a scale (S930).

The positions of pixels corresponding to extreme values (maximum and minimum values) from the generated error images can be extracted and set as common points (S940). Common point vector information can be generated by removing noise and strong edge components from the extracted common points and setting direction information and size for the remaining common points (S950).

By comparing the common point vector information of the aerial photographic images thus obtained and the common point vector information of the corrected image, it is possible to confirm whether or not the objects are matched. The coordinate information of the target object may be obtained based on the matching result (S960).

10 is a flowchart illustrating a method of applying an adaptive filtering process to a frame of a photographed image according to an embodiment of the present invention.

The spatial information generation system to which the present invention is applied can receive an RGB video signal through the image sensing unit 111 (S1010). Here, the RGB image signal may be a stereo image obtained by photographing a road facility, a building, a road facility, and an auxiliary facility using a left eye image camera and a right eye image camera.

The spatial information generation system acquires the predicted pixel value x [n] of the current frame of the received RGB video signal and the pixel value s [n] of the original frame through the video coding unit 112 (S1020). The difference value e [n] can be obtained using the predicted pixel value x [n] of the current frame and the pixel value s [n] of the original frame in operation S1030.

The video coding unit 112 may obtain a filter coefficient a _i that minimizes the difference e [n] (S1040). The video coding unit 112 can perform filtering by applying the obtained filter coefficients, and the filtered video signals are stored in the buffer 113 and transmitted to the data output unit 114. [ When the filter coefficient is not applied, a flag indicating whether or not the filter coefficient is applied may be used without transmitting the filter coefficient. By performing adaptive filtering through such a process, efficient signal processing can be performed.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined by the appended claims. , Substitution or addition, or the like.

Claims (4)

A digital map production system for distorting and correcting image data using parameters and comparing the corrected image data with aerial photograph data,
A captured image acquisition module for acquiring image information from a plurality of cameras; A GPS receiving module for receiving GPS information from a satellite and obtaining current position information of the camera; An inertia measurement module for obtaining horizontal information of the camera using a plurality of sensors; An image processing module that corrects the obtained image information based on the obtained image information, the current position information, and the horizontal information; A control module for controlling the transmission of the acquired image information, current position information and horizontal information to the image processing module; And a buffer for storing the corrected image information;
An aerial photograph database for receiving and storing the corrected image information from the mobile mapping device or receiving and storing aerial photograph data from an external database;
A data processing module for determining a target object by comparing the corrected image information obtained from the mobile mapping device with aerial photograph data obtained from the aerial photograph database and performing three-dimensional space modeling using the determined target object information, ; And
A communication module for receiving request information for a spatial modeling service from a user, transmitting the request information to the mobile mapping device, the aerial photograph database, and the data processing module, and transmitting the three-dimensional spatial data obtained from the data processing module to a user;
&Lt; / RTI &gt;
The image processing module includes:
Detecting a curve from the acquired image information received from the captured image acquisition module to set a first correction group, removing the first correction group from the acquired image information, And the second correction group is removed from the image information in which the first correction group is removed, and the threshold value is set to the following formula

(Where a _m denotes the length of the waterline on the curve at the center of the image, R denotes the length of the image diagonal, m = 1 to M, M denotes the number of lines, and ψ denotes the user parameter) A filtering module obtained by the filtering module;
A parameter extracting module for extracting a parameter indicating a degree of warping of the image based on the image information from which the first correction group and the second correction group have been removed;
An image correction module for performing correction of the obtained image information using the parameter,
/ RTI &gt;
The data processing module includes:
The Z coordinate value of the object is obtained from the Y coordinate (Y_l) of the image obtained from the left camera, the Y coordinate (Y_r) of the image obtained from the right camera, and the focal distance f And a Y-axis distance (Yo) to the target object,

Is obtained by using the digital map data. delete delete delete

Images