KR101179108B1 - System for determining 3-dimensional coordinates of objects using overlapping omni-directional images and method thereof - Google Patents

Abstract

Disclosed are a system and method for determining three-dimensional coordinates of an object using overlapping omnidirectional images according to the present invention.
A system for determining three-dimensional coordinates of an object using an overlapping omnidirectional image according to the present invention includes image information obtaining means for obtaining an overlapping omnidirectional image; Location information acquisition means for obtaining location information and attitude information of the image information acquisition means; And an image point vector calculated on a camera coordinate system based on the overlapped omnidirectional image, the position information and attitude information, and an image coordinate value of a conjugate point with respect to a preset target point. A user terminal which derives an equation of two or more straight lines in which a ground point exists, and determines a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of the derived two or more linear equations; Include.
Through this, the present invention can improve the accuracy of the three-dimensional coordinates for the target point.

Description

System and method for determining three-dimensional coordinates of an object using superimposed omnidirectional image {SYSTEM FOR DETERMINING 3-DIMENSIONAL COORDINATES OF OBJECTS USING OVERLAPPING OMNI-DIRECTIONAL IMAGES AND METHOD THEREOF}

The present invention relates to a three-dimensional coordinate determination method, in particular, by using the overlapping omnidirectional image that can determine the three-dimensional coordinates of the target point by calculating the intersection of each straight line derived from the conjugate point selected from the two overlapping omnidirectional image A system and method for determining three-dimensional coordinates of an object.

Recently, researches based on the convergence of various sensors mounted on various platforms have been actively conducted for faster and more economical construction of high quality spatial information. A system for acquiring spatial information using a system equipped with a multi-sensor, such as a camera and GPS / INS, is being developed using an unmanned aero vehicle (UAV) as a platform. Spatial information acquisition system using aircraft is vulnerable to texture acquisition of building walls in urban areas, so it is required to utilize the system using vehicle like mobile mapping system. Mobile mapping refers to acquiring information on a feature such as roads, traffic facilities, or buildings around a road using a moving vehicle. In addition, a mobile mapping system equipped with a multi-sensor has been developed, and a research has been conducted to track a moving object and extract only a specific object required by using a mobile mapping system.

Recently, attempts have been made to acquire spatial information by mounting omni-directional cameras on mobile mapping systems. The mobile mapping system equipped with the omnidirectional camera calculates the position of the feature by using the overlapped images according to the vehicle traveling direction. In this case, the baseline is longer than that of the frame camera, so the result can be generated with high accuracy. In relation to this, there was a research on optical flow based feature recognition through the use of a mobile mapping system equipped with an omnidirectional camera and an odometer.

The omnidirectional camera refers to a camera capable of acquiring image information in all directions from the photographing point because the field of view is 360 °. Conventional omnidirectional cameras have been obtained by omnidirectional image by inputting image data reflected through a hemispherical mirror into a single CCD, but these omnidirectional camera images inevitably have many distortions due to the operation method using a hemisphere mirror. It contains them.

Efforts have been made to correct the distortion in the omnidirectional image using the angle between the center and the image of the projection formed on the CCD sensor, but the distortion could not be completely removed.

Recently, omnidirectional cameras using multiple cameras have appeared, rather than using mirrors. Among them, the ladybug omni-directional camera acquires images through six cameras shooting different ranges. Six images obtained from each camera are generated as a single omnidirectional image having image information in all directions through image processing. The omnidirectional image generated in this way has less distortion and higher spatial resolution than conventional omnidirectional mirror images.

All-round cameras have been used in various fields. In the field of mobile robot system, a research was conducted to detect the position and direction of the robot by using tracking vertex and line features, and to determine the movement path by using it. A method of calculating the direction of the robot using the optical floor vector of the omnidirectional image has also been proposed. In addition, research has been carried out to recognize and track objects by projecting omnidirectional images onto a plane.

Since the mobile mapping system is configured by combining various types of sensors, the performance of the entire system is determined by the performance of the sensors. Existing omnidirectional cameras with ground mobile mapping systems have been used mainly for road view services, which typically detect the movement of objects or provide only raw raw images.

Therefore, to solve the problems of the prior art, an object of the present invention is to calculate the intersection of each straight line derived from the conjugate point selected from the two overlapping omnidirectional images to determine the three-dimensional coordinates of the target point To provide a system and method for determining the three-dimensional coordinates of an object by using a.

However, the objects of the present invention are not limited to those mentioned above, and other objects not mentioned can be clearly understood by those skilled in the art from the following description.

In order to achieve the above objects, a system for determining three-dimensional coordinates of an object using an overlapping omnidirectional image according to an aspect of the present invention includes image information obtaining means for obtaining an overlapping omnidirectional image; Location information acquisition means for obtaining location information and attitude information of the image information acquisition means; And an image point vector calculated on a camera coordinate system based on the overlapped omnidirectional image, the position information and attitude information, and an image coordinate value of a conjugate point with respect to a preset target point. A user terminal which derives an equation of two or more straight lines in which a ground point exists, and determines a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of the derived two or more linear equations; It may include.

The user terminal may calculate an image coordinate value of a conjugate point of the target point from each of the omnidirectional images when an arbitrary target point is selected from at least two abutted omnidirectional images.

Preferably, the user terminal may calculate an image point vector connecting the image points at the projection center of the omnidirectional camera on the camera coordinate system based on the image coordinate values of the conjugate points.

로 정의하고, 여기서, 상기 ^GP는 지상 좌표계로 표현된 지상점의 좌표를 의미하며 카메라의 투영 중심, 영상점과 지상점을 잇는 상기 직선 상에 존재하고, 상기 λ는 상기 직선 상에 한 점을 특정하기 위한 변수를 나타내며, 상기

는 카메라 좌표계로 표현된 벡터를 지상좌표계로 변환하기 위한 회전행렬을 나타내며, 상기 ^Cn는 카메라 좌표계로 표현되 영상점 벡터를 나타내며, 상기 ^GO_C는 지상 좌표계로 표현된 카메라의 투영 중심을 나타낼 수 있다.Preferably, the user terminal derives an equation of a straight line having a ground point represented by the ground coordinate system using the image point vector represented by the camera coordinate system, wherein the equation of the straight line is

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Can be.

Preferably, the user terminal uses the equation of two or more straight lines respectively derived from the conjugate points selected from two or more overlapping images, and calculates the intersection point where the sum of the squares of the distances between the straight lines is the minimum. After calculating the ground point coordinate values for each straight line, the distance between the ground point coordinate values is compared with the preset threshold value, and as a result of the comparison, the distance between the ground point coordinate values is larger than the preset threshold value. If it is small, the average of the ground coordinates can be obtained and the three-dimensional coordinates of the target point can be determined by the average of the coordinate values of the two ground points.

According to another aspect of the present invention, a system for determining three-dimensional coordinates of an object by using an overlapping omnidirectional image includes selecting a target point from each of the omnidirectional images when an arbitrary target point is selected from at least two received omnidirectional images. A conjugate point calculator for calculating an image coordinate value of the conjugate point; A vector calculator configured to calculate an image point vector on a camera coordinate system based on the calculated coordinate values of the conjugate points; A formula derivation unit for deriving an equation of two or more straight lines having a ground point on a ground coordinate system using the calculated image point vector; And a three-dimensional coordinate determination unit configured to determine a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of the derived two or more linear equations.

Preferably, the vector calculator may calculate an image point vector connecting the image points at the center of the projection of the image information obtaining unit on the camera coordinate system based on the calculated image coordinate values of the conjugate points.

로 정의하고, 여기서, 상기 ^GP는 지상 좌표계로 표현된 지상점의 좌표를 의미하며 카메라의 투영 중심, 영상점과 지상점을 잇는 상기 직선 상에 존재하고, 상기 λ는 상기 직선 상에 한 점을 특정하기 위한 변수를 나타내며, 상기

는 카메라 좌표계로 표현된 벡터를 지상좌표계로 변환하기 위한 회전행렬을 나타내며, 상기 ^Cn는 카메라 좌표계로 표현되 영상점 벡터를 나타내며, 상기 ^GO_C는 지상 좌표계로 표현된 카메라의 투영 중심을 나타낼 수 있다.Preferably, the equation derivation unit derives an equation of a straight line having a ground point represented by the ground coordinate system using the image point vector represented by the camera coordinate system, the equation of the straight line is

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Can be.

Preferably, the coordinate determination unit calculates the intersection point of which the sum of the squares of the distances between the straight lines is the minimum using the equation of two or more straight lines respectively derived from the conjugate points selected from two or more overlapping images. After calculating the ground point coordinate values for each straight line, the distance between the coordinate values of the ground point is compared with the preset threshold value, and as a result of the comparison, the distance between the coordinate values of the ground point is the preset threshold value. If it is smaller, the average of the coordinate values of the two ground points can be obtained and the three-dimensional coordinate value of the target point can be determined by the average of the coordinate values of the two ground points.

In addition, the system for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image according to the present invention further comprises an information collecting unit for receiving the overlapping omnidirectional image, position information and attitude information from the image information obtaining means and the position information obtaining means. It may include.

According to another aspect of the present invention, a method for determining three-dimensional coordinates of an object by using an overlapping omnidirectional image includes: (a) obtaining, by the image information acquiring means, the overlapping omnidirectional image; (b) obtaining, by the position information obtaining means, position information and attitude information of the image information obtaining means; (c) calculating, by the user terminal, an image point vector on a camera coordinate system based on the overlapping omnidirectional image, the position information and attitude information, and an image point coordinate value of a conjugate point with respect to a preset target point; (d) deriving, by the user terminal, an equation of a straight line in which a ground point exists in a ground coordinate system using the calculated image point vector in the camera coordinate system; And (e) determining a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point using the equation of the straight line in which the user terminal is derived.

Preferably, in the step (c), if an arbitrary target point is selected from at least two abutted omnidirectional images, an image coordinate value of a conjugate point with respect to the target point may be calculated from each omnidirectional image.

Preferably, the step (c) may calculate an image point vector connecting the image points at the center of the projection of the image information acquisition means on the camera coordinate system based on the image coordinate values of the conjugate points.

로 정의하고, 여기서, 상기 ^GP는 지상 좌표계로 표현된 지상점의 좌표를 의미하며 카메라의 투영 중심, 영상점과 지상점을 잇는 상기 직선 상에 존재하고, 상기 λ는 상기 직선 상에 한 점을 특정하기 위한 변수를 나타내며, 상기

는 카메라 좌표계로 표현된 벡터를 지상좌표계로 변환하기 위한 회전행렬을 나타내며, 상기 ^Cn는 카메라 좌표계로 표현되 영상점 벡터를 나타내며, 상기 ^GO_C는 지상 좌표계로 표현된 카메라의 투영 중심을 나타낼 수 있다.Preferably, step (d) derives an equation of a straight line having a ground point represented by the ground coordinate system using an image point vector represented by the camera coordinate system, wherein the equation of the straight line is

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Can be.

Preferably, the step (e) is calculated by calculating the intersection point of which the sum of the squares of the distances between the straight lines is the minimum using the equation of two or more straight lines respectively derived from the conjugate points selected from the two or more overlapping images. After calculating the ground point coordinate values for each straight line using the intersection point, the distance between the coordinate values of the ground point is compared with the preset threshold, and as a result of the comparison, the distance between the coordinate values of the ground point is If it is smaller than the set threshold value, the average of the coordinate values of the two ground points can be obtained, and the three-dimensional coordinate value of the target point can be determined based on the average of the coordinate values of the two ground points.

According to another aspect of the present invention, a method for determining three-dimensional coordinates of an object by using an overlapping omnidirectional image includes: (a) when an arbitrary target point is selected from at least two received omnidirectional images, from each omnidirectional image; Calculating image coordinate values of the conjugate point with respect to the target point; (b) calculating an image point vector on a camera coordinate system based on the calculated image coordinate value of the conjugate point; (c) deriving an equation of two or more straight lines having a ground point on a ground coordinate system using the calculated image point vector on the camera coordinate system; And (d) determining a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of the derived two or more straight line equations.

Preferably, the step (b) may calculate an image point vector connecting the image points at the center of the projection of the image information acquisition means on the camera coordinate system based on the calculated image coordinate values of the conjugate points.

로 정의하고, 여기서, 상기 ^GP는 지상 좌표계로 표현된 지상점의 좌표를 의미하며 카메라의 투영 중심, 영상점과 지상점을 잇는 상기 직선 상에 존재하고, 상기 λ는 상기 직선 상에 한 점을 특정하기 위한 변수를 나타내며, 상기

는 카메라 좌표계로 표현된 벡터를 지상좌표계로 변환하기 위한 회전행렬을 나타내며, 상기 ^Cn는 카메라 좌표계로 표현되 영상점 벡터를 나타내며, 상기 ^GO_C는 지상 좌표계로 표현된 카메라의 투영 중심을 나타낼 수 있다.Preferably, step (c) derives an equation of a straight line having a ground point represented by the ground coordinate system using the image point vector represented by the camera coordinate system, wherein the equation of the straight line is

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Can be.

Preferably, the step (d) is performed by calculating the intersection point of which the sum of the squares of the distances between the straight lines is the minimum using the equation of the two or more straight lines respectively derived from the conjugate points selected from the two or more overlapping images. After calculating the ground point coordinate values for each straight line using the intersection point, the distance between the coordinate values of the ground point is compared with the preset threshold, and as a result of the comparison, the distance between the coordinate values of the ground point is If it is smaller than the set threshold value, the average of the coordinate values of the two ground points can be obtained, and the three-dimensional coordinate value of the target point can be determined based on the average of the coordinate values of the two ground points.

In addition, the method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image according to the present invention includes (a-1) an omnidirectional image superimposed from the image information obtaining means and the position information obtaining means before step (a); The method may further include receiving location information and attitude information.

Through this, the present invention can determine the three-dimensional coordinates of the target point by calculating the intersection of each straight line derived from the conjugate point selected from the two overlapping omnidirectional image, it is possible to improve the accuracy of the three-dimensional coordinates for the target point It works.

1 is an exemplary view showing a schematic configuration of a system according to an embodiment of the present invention.
2 is an exemplary diagram illustrating a conceptual diagram of a definition of a coordinate system according to an exemplary embodiment of the present invention.
3 is an exemplary view showing a method for determining three-dimensional coordinates according to an embodiment of the present invention.
4 is an exemplary diagram illustrating a relationship between a ground point and an image point on a camera coordinate system according to an exemplary embodiment of the present invention.
5 is an exemplary diagram illustrating a relationship between each coordinate system and a ground point according to an exemplary embodiment of the present invention.
6 is an exemplary view for explaining a principle of determining three-dimensional coordinates according to an embodiment of the present invention.
7 is an exemplary view showing a detailed configuration of the user terminal 130 shown in FIG.
8 is an exemplary view showing the appearance of a omnidirectional camera according to an embodiment of the present invention.
9 is an exemplary view illustrating a process of generating an omnidirectional image according to the present invention.
10 is an exemplary view showing a sensor incorporating a omnidirectional camera and a GPS / INS sensor according to an embodiment of the present invention.
11 is an exemplary view showing a mobile mapping system completed by mounting an integrated sensor in a vehicle according to an embodiment of the present invention.
12 is an exemplary view illustrating an acquisition path and a ground point group of an image according to an embodiment of the present invention.
13 is an exemplary view showing an omnidirectional image according to an embodiment of the present invention.
14 is an exemplary view showing a checkpoint for each region according to an exemplary embodiment of the present invention.
15 is an exemplary view showing ground points and inspection points determined from an omnidirectional image according to an embodiment of the present invention.
16 is an exemplary view showing a relationship between a camera position and a checkpoint position according to an embodiment of the present invention.
17 is an exemplary view illustrating ground points and inspection points determined from an omnidirectional image according to an embodiment of the present invention.

Hereinafter, a system and method for determining three-dimensional coordinates of an object using an overlapping omnidirectional image according to an embodiment of the present invention will be described with reference to FIGS. 1 to 17. It will be described in detail focusing on the parts necessary to understand the operation and action according to the present invention. Like reference numerals in the drawings denote like elements throughout the specification.

The present invention proposes a method that can determine the three-dimensional coordinates of the target point by calculating the intersection of each straight line derived from the conjugate point selected from the two overlapping omnidirectional image.

The proposed method can be largely composed of the process of setting the coordinate system, collecting data, performing internal expression, performing external expression, and determining the target point coordinates. The first step is to check the configuration of the mounted sensor, define a three-dimensional coordinate system that is individually defined on the sensor and the ground, and establish the relationship between the coordinate systems. The second process collects the omnidirectional camera image, the camera's external expression elements and calculates the coordinate values of the conjugate points. In the third and fourth processes, internal and external expressions are performed to express a straight line in which a three-dimensional candidate ground point exists that can be projected to one image point on the three-dimensional object coordinate system. The internal expression is a process of expressing a straight line of a candidate ground point corresponding to one image point in a camera coordinate system, and the external expression is a process of converting a straight line expressed in a camera coordinate system into a ground coordinate system through a relationship between the camera and the ground. The final process determines the three-dimensional coordinates of the target point by calculating the intersection of each straight line derived from the conjugate point selected from two superimposed images.

1 is an exemplary view showing a schematic configuration of a system according to an embodiment of the present invention.

As shown in FIG. 1, the system according to the present invention includes image information acquiring means 110 such as two sensors such as an omni-directional camera and a GPS / INS (Global Positioning System / Inertial Navigation System). ) May be implemented by mounting location information acquisition means 120 such as a sensor, and a user terminal 130. The mounted sensors can output raw sensor data based on a three-dimensional coordinate system defined independently for each individual sensor. The omnidirectional camera 110 is a camera for capturing images of surrounding omnidirectional, ie, front, rear, left, and right directions at one time, and is arranged such that a plurality of camera modules face the front direction. Therefore, the system may acquire the surrounding image, that is, the superposed omnidirectional image, through the omnidirectional camera 110.

In addition, the system may acquire position information, attitude information, and the like through the GPS / INS sensor 120 as an external expression element of the omnidirectional camera 110. The location information and attitude information obtained through the GPS / INS sensor 120 may be location information and attitude information of the GPS / INS sensor 120 itself, but the GPS / INS sensor 120 may be the same as the omnidirectional camera 110. Since the vehicle is mounted on the vehicle, the location information and the attitude information of the omnidirectional camera 110 may be the location information and the attitude information of the vehicle.

The user terminal 130 may receive an omnidirectional image from the omnidirectional camera 110, and receive position information and attitude information through the GPS / INS sensor 120. In addition, the user terminal 130 may calculate an image coordinate value of a conjugate point with respect to a target object or a target point in two overlapping omnidirectional images. Here, the conjugate point may be a concept encompassing a conjugate point in two or more overlapping omnidirectional images as well as a conjugate point in two overlapping omnidirectional images.

The user terminal 130 may determine a 3D coordinate value for the target viewpoint based on the input omnidirectional image, position information and attitude information, and image coordinate values of the conjugate point. For example, the user terminal 130 may be a concept including a smartphone, a notebook, a navigation, and the like.

In addition, the user terminal 130 first checks the configuration of the mounted sensor in order to determine the three-dimensional coordinate value for the target point, and then defines a three-dimensional coordinate system that is individually defined on the sensor and the ground, and the relationship between each coordinate system It should be established with reference to FIG.

2 is an exemplary diagram illustrating a conceptual diagram of a definition of a coordinate system according to an exemplary embodiment of the present invention.

As shown in FIG. 2, the 3D coordinate system according to the present invention may be defined as a camera coordinate system (CCS), a GPS / INS coordinate system (ICS), a ground coordinate system (GCS), or the like. For the mutual fusion of sensor data, the relationship between the camera coordinate system, the GPS / INS coordinate system, and the ground coordinate system must be precisely established.

In order to reduce the errors that may occur in the process of establishing the relationship between the camera coordinate system, the GPS / INS coordinate system, and the ground coordinate system, the X and Y axes of the camera coordinate system and the GPS / INS coordinate system are set to be parallel to each other, and the Z axis should be matched with each other. Can be. Here, the X axis in each coordinate system may represent a traveling direction of the vehicle, the Y axis may represent a horizontal right angle direction of the vehicle traveling direction, and the Z axis may represent a vertical right angle direction of the vehicle traveling direction.

3 is an exemplary view showing a method for determining three-dimensional coordinates according to an embodiment of the present invention.

As shown in FIG. 3, the user terminal according to the present invention may receive an overlapping omnidirectional image, location information, and attitude information from the omnidirectional camera and the GPS / INS sensor (S310). The user terminal may calculate an image coordinate value of a conjugate point of the target point from each of the omnidirectional images when a certain target point is selected from at least two received omnidirectional images.

Next, the user terminal may calculate an image point vector connecting the image points at the center of the projection of the camera on the camera coordinate system based on the calculated image coordinate values of the conjugate points (S330).

4 is an exemplary diagram illustrating a relationship between a ground point and an image point on a camera coordinate system according to an exemplary embodiment of the present invention.

As shown in FIG. 4, the relationship between an arbitrary image point on a spherical surface and a ground point projected on the spherical surface of the camera coordinate system according to the present invention is illustrated in FIG. The origin of the camera coordinate system corresponds to the center of the projection of the camera, and according to the principle of the center projection, the center of the projection, the image point, and the ground point exist on one straight line.

A vector that forgets the image point at the center of the projection, that is, the image point vector ^C n, can be expressed by the following Equation 1 by the principle of the three-dimensional polar coordinate system. The image point vector has the same direction as the ground point vector that forgets the ground point at the center of the projection. Thus, the ground point may exist on a straight line with the direction of the image point vector ^C n at the center of the projection.

[Equation 1]

Here, α may represent a horizontal angle and β may represent a vertical angle. Such horizontal angle α and vertical angle β will be described with reference to Fig. (B). Figure (b) is an image of the spherical image of Figure (a) unfolded into a plane.

The coordinate of the image point is expressed as (r, c) based on the upper left of the image, where r may mean the number of each row and c may mean the number of each column. The relationship between the horizontal angle α and the vertical angle β representing the coordinates of the image point and the position on the spherical surface is shown in Equation 2 below.

&Quot; (2) "

That is, the present invention may set the image point vector ^C n by performing internal expression in the second process.

Next, the user terminal may derive an equation of a straight line having a ground point represented by the ground coordinate system using the image point vector represented by the camera coordinate system (S340).

5 is an exemplary diagram illustrating a relationship between each coordinate system and a ground point according to an exemplary embodiment of the present invention.

As shown in Fig. 5, the relationship between the camera coordinate system, the GPS / INS coordinate system, the ground coordinate system, and the ground point is shown. The coordinate vector ^G P on the ground coordinate system of the ground point projected to one image point is the sum of the vector λ ^G n connecting the ground point at the center of the projection and the vector ^G O _C connecting the center of the projection at the origin of the ground coordinate system. As shown in the following equation (3).

&Quot; (3) "

Here, λ may mean a variable.

In this case, the following Equation 4 can be obtained by rotation-converting the image point vector obtained from the internal expression of the direction vector ^G n of the ground point represented by the ground coordinate system.

&Quot; (4) "

는 지상 좌표계와 카메라 좌표계 간의 회전행렬을 나타내며,

는 카메라 좌표계와 GPS/INS 좌표계 간의 회전행렬을 나타내며,

는 지상 좌표계와 GPS/INS 좌표계 간의 회전행렬을 나타낼 수 있다.here,

Represents the rotation matrix between the ground and camera coordinates,

Represents the rotation matrix between the camera coordinate system and the GPS / INS coordinate system.

May represent a rotation matrix between the ground coordinate system and the GPS / INS coordinate system.

In this case, R may represent attitude information of the camera, and O may represent location information of the camera.

In addition, by using the offset between the origin of the GPS / INS coordinate system and the projection center of the camera, ^G O _C , which is the projection center of the camera, can be obtained by the following Equation 5.

[Equation 5]

Using the equations [4] and [Equation 5] obtained in this way to summarize the ^G P as shown in [Equation 6].

&Quot; (6) "

As a result, Equation 6 is a formula representing a straight line in which candidate ground points projected to one image point exist in a ground coordinate system.

로 설정할 수 있는데, I₃은 단위 벡터이기 때문에 [수학식 4]와 [수학식 5]로부터 다음의 [수학식 7]을 얻을 수 있다.If we assume that the camera coordinate system and the GPS / INS coordinate system

Since I ₃ is a unit vector, the following [Equation 7] can be obtained from [Equation 4] and [Equation 5].

[Equation 7]

Applying Equation 7 to Equation 6 above, Equation 8 can be obtained.

[Equation 8]

Next, the user terminal may calculate a three-dimensional coordinate value for the target point by using the equation of the two or more straight lines derived (S350). Specifically, the user terminal calculates the intersection point of which the sum of the squares of the distances between the straight lines is the minimum using the equation of the two or more straight lines respectively derived from the conjugate points selected from the two or more overlapping images. After calculating the ground point coordinate values for each straight line, the distance between the coordinate values of the ground point can be compared with a preset threshold. If the distance between the coordinate values of the ground point is greater than the preset threshold value, the user terminal may determine that it is a matching error.

On the other hand, if the distance between the coordinates of the ground points is smaller than the preset threshold, the user terminal calculates the average of the coordinate values of the two ground points as the average of the coordinate values of the two ground points. Three-dimensional coordinate values for the target point can be calculated.

The principle of determining the three-dimensional coordinates of the target point by calculating the intersection of the equation of each straight line derived from the conjugate point selected from the two overlapping images will be described with reference to FIG. 6.

6 is an exemplary view for explaining a principle of determining three-dimensional coordinates according to an embodiment of the present invention.

That is, the ground points P1 and P2 projected on the conjugate points of two superimposed images may be expressed by Equation 9 represented by the direction vector and the starting point, respectively, using Equation 8.

[Equation 9]

In Equation 9, since the ground points P1 and P2 are conjugate points, they exist on the same straight line and represent the same point, and the following Equation 10 can be derived using Equation 9.

&Quot; (10) "

If Equation 10 is expressed as a matrix, it can be expressed as Equation 11 below.

&Quot; (11) "

In order to obtain the unknown ξ by using the method of least squares, Equation 11 can be expressed as Equation 12 below.

[Equation 12]

를 [수학식 9]에 적용하여 다음의 [수학식 13]을 구할 수 있다.Unknown

Apply to Equation 9 to obtain the following Equation 13.

&Quot; (13) "

간의 거리

를 보여주고 있다. 만약 두 점

사이의 거리가 임계값보다 크면, 정합오류로 판단하게 된다. 반면 두 점

사이의 거리가 임계값보다 작으면 두 점의 중점

를 산출하여 이를 3차원 좌표로 결정하게 된다.Referring to Figure 6, the estimated ground point

Distance between

Is showing. If two points

If the distance between them is larger than the threshold, it is determined as a matching error. While two points

Midpoint of two points if distance between them is less than threshold

It is determined by calculating the three-dimensional coordinates.

는 두 점

의 평균값일 수 있다.For example, two points

Two points

It may be an average value of.

At this time, the present invention determines the three-dimensional coordinates of the target point by calculating the intersection of the equation of the straight line derived from the conjugate point selected from the two overlapping images, but is not necessarily limited to this conjugate point selected from at least two overlapping images You can also apply

7 is an exemplary view showing a detailed configuration of the user terminal 130 shown in FIG.

As shown in FIG. 7, the user terminal 130 according to the present invention includes an information collecting unit 131, a conjugate point calculating unit 132, a vector calculating unit 133, a mathematical induction unit 134, and a coordinate determining unit 135. ) And the like.

The information collecting unit 131 may receive an overlapping omnidirectional image, location information, and attitude information from the image information obtaining unit and the location information obtaining unit. The conjugate point calculator 132 may calculate an image coordinate value of the conjugate point of the target point from each of the omnidirectional images when an arbitrary target point is selected from the at least two received omnidirectional images.

The vector calculator 133 may calculate an image point vector connecting the image points at the center of the projection of the camera on the camera coordinate system based on the calculated image coordinate values of the conjugate points.

The equation inducing unit 134 may derive an equation of two or more straight lines having a ground point on the ground coordinate system using the calculated image point vector.

The three-dimensional coordinate determination unit 135 may determine the three-dimensional coordinate value for the target point by using the intersection point calculated by calculating the intersection of the derived equations of the two or more straight lines. Specifically, the three-dimensional coordinate determination unit 135 uses the equation of two or more straight lines respectively derived from the conjugate points selected from two or more overlapping images to determine the intersection point of which the sum of the squares of the straight lines is the minimum sum. After calculating the ground point coordinate values for each straight line by using the calculated intersection point, the distance between the coordinate values of the ground point may be compared with a preset threshold. If the distance between the coordinate values of the ground point is larger than the preset threshold value, the 3D coordinate determination unit 135 may determine that it is a matching error.

On the other hand, if the distance between the coordinates of the ground points is smaller than the preset threshold, the 3D coordinate determination unit 135 calculates the average of the coordinate values of the two ground points and calculates the coordinates of the two ground points. The average of the values may determine a three-dimensional coordinate value for the target point.

Hereinafter, a result of experimenting a method for determining 3D coordinates of an object according to an embodiment of the present invention will be described with reference to FIGS. 8 to 17.

Sensor system

8 is an exemplary view showing the appearance of a omnidirectional camera according to an embodiment of the present invention.

As shown in Figure 8, the specification of the omni-directional camera used in the present invention is shown in the following [Table 1]. That is, the camera has a field of view (FOV) of 360 degrees through six CCDs having a spatial resolution of 1600 x 1200. The output image has a resolution of 5400 x 2700 including a horizontal 360 Hz and a vertical 180 Hz.

[Table 1]

9 is an exemplary view illustrating a process of generating an omnidirectional image according to the present invention.

As shown in FIG. 9, the process of acquiring six individual images through an omnidirectional camera and generating one finally integrated image is shown. First, Bayer-tiled Raw format image is acquired through 6 CCDs, then compressed into JPEG format and transmitted to PC (Personal Computer) through Firewire. It decompresses the image transmitted from the PC, converts the raw image into RGB color image, and delivers it to the graphic card. Rectification process to eliminate barrel distortions on graphics cards, projection process to convert image texture to 2D or 3D coordinate system, and blending to connect images acquired through each CCD to each other Finally, one stitched image is generated.

GPS / INS sensor is equipped with omnidirectional camera to acquire external expression elements of omnidirectional camera image. The sensor used is a sensor suitable for a mobile system such as a mobile mapping system. It consists of an accelerometer that measures acceleration in three axes and an Inertial Measurement Unit (IMU) with a built-in gyroscope that measures the angular velocity of three rotary axes. GPS uses the GMS (GNSS Azimuth Measurement Subsystem), which not only measures the position of three-dimensional absolute coordinates but also corrects the error of the INS using two GPS antennas. A system that measures the relative position vector between two GPS antennas using GPS carriers and continuously compensates for errors in the INS.

Table 2 below shows the accuracy of position and attitude measured by the GPS / INS sensor used.

[Table 2]

10 is an exemplary view showing a sensor incorporating a omnidirectional camera and a GPS / INS sensor according to an embodiment of the present invention, Figure 11 is a mobile mapping system completed by mounting the integrated sensor according to an embodiment of the present invention in a vehicle It is an exemplary figure to show.

10 to 11, an omnidirectional camera is mounted on the upper end of the integrated sensor, and an INS is mounted below, and two GPS antennas are mounted in the vehicle travel direction (X-axis direction). The completed integrated sensor is considerably longer in the vertical direction, and there is a risk of collision when passing through the tunnel and the lower part of the bridge.

Experimental data

12 is an exemplary view showing an image acquisition path and a ground point group according to an embodiment of the present invention, and FIG. 13 is an exemplary view showing an omnidirectional image according to an embodiment of the present invention.

Referring to FIG. 12, the image data obtained through the experiment was an image having a resolution of 5400 * 2700 having an average capacity of 1MB and used the Jpeg compression method. The place where the images were acquired was obtained by moving from east to west on the map around the road near Osan City Hall, Gyeonggi-do. The road on which the vehicle travels is a two- or three-way one-way street, with nearby parks and low-rise buildings on and around the fifth floor, and high-rise apartments and malls at some distance. Two pairs of overlapping images were selected along the length of the baseline in three groups A, B, and C, which had previously secured a checkpoint. Also, in group C, two pairs of superimposed images with similar baselines were selected to analyze the relationship between the object position and the camera's mutual position and accuracy. An example of the selected omnidirectional image can be seen in FIG. 13. The omnidirectional image includes omnidirectional information in a range corresponding to 360 ° in the horizontal direction and 180 ° in the vertical direction, and the center of the image represents the driving direction of the vehicle.

[Table 3] below shows the external expression elements and the baseline distance indicating the position / posture of the omnidirectional camera at the time of acquiring the individual images measured from the GPS / INS data. In each region, the short-term ship group A1, B1, C1 has a baseline of about 2 m and the long-term ship group A2, B2, C2 has a baseline of 16-18 m. Groups C3 and C4 were selected to analyze the relationship between the object and the camera's mutual position and accuracy, and have a baseline of 12 m.

[Table 3]

In order to verify the accuracy of the ground point determined from the omnidirectional image through the proposed method, three-dimensional absolute coordinates of the main points that can be easily identified in the image using GPS and total station were used as test points.

14 is an exemplary view showing a checkpoint for each region according to an exemplary embodiment of the present invention.

As shown in Fig. 14, area A includes four check points as warning signs and area B includes three check points as road signs. Finally, area C contains three checkpoints with directional arrows on the road.

Results and analysis

15 is an exemplary view showing ground points and inspection points determined from an omnidirectional image according to an embodiment of the present invention.

As shown in FIG. 15, the ground point determined from the omnidirectional image is compared with the inspection point measured by the GPS and the total station through the proposed method. Polygons (green, L1-Ln) connecting the ground points determined through the long-term line image (green, L1-Ln) rather than polygons connecting the ground points determined through the short-line image (blue, C1- More similar to Cn).

To examine this quantitatively, the error of ground point coordinates was calculated and summarized in the following [Table 4].

[Table 4]

Assuming the checkpoint coordinates are true, the difference from the ground coordinates was calculated and the RMSE was calculated from several tens of cm to about 1 m. In the case of long-term lines, the similarity of polygons appeared to be high, but the RMSE for absolute coordinates was not significantly different according to the length of the baseline. Although the polygons connecting the ground points and check points were observed to be quite similar, it is considered that the ground point coordinates contain a considerable amount of system error, considering that the RMSE of the absolute coordinate values is large. This systematic error can be eliminated through three-dimensional movement and rotational transformation between ground and inspection points. Alternatively, the external expression element of the image may be removed by estimating one or two ground control points. In order to determine the magnitude of the error still remaining after the systematic error is removed, the three-dimensional method is determined by considering the three-axis rotation angle and the amount of movement between the ground point and the inspection point after fixing the scale. The relative error amount of the coordinates was calculated.

[Table 5] below shows the error of the absolute coordinate between the inspection point and the ground point, and the unit is [m].

[Table 5]

[Table 6] shows the relationship between the check points acquired through the total station and the GPS and the ground points estimated by the proposed methodology through three movement transformation coefficients and three rotation transformation coefficients.

TABLE 6

[Table 7] below shows the ground point error calculated from the six conversion coefficients, and the unit is [m].

[Table 7]

It can be seen that the RMSE of relative coordinate error without systematic error is significantly reduced compared to the RMSE of absolute coordinate error including systemic error. Based on these results, the three-dimensional movement and rotation conversion coefficients between the checkpoints obtained by precise measurement and the adjusted ground point are estimated to eliminate the systematic error. It can be seen that the relative coordinates can be extracted with an accuracy of about several cm.

16 is an exemplary view showing a relationship between a camera position and a checkpoint position according to an embodiment of the present invention.

By selecting an overlapping image with a baseline that is not too short for the distance to the target object and maintaining the proper ratio of distance to the baseline, two relatively low collinearity equations can be obtained, from which Three-dimensional coordinates can be determined precisely.

However, as in the case of the group C3 in FIG. 16, when the target points are distributed in the traveling direction (x-axis direction) of the vehicle, even when the base line is long, the straight lines connecting to one target point at each projection center appear quite similar. Good correlations cannot be obtained because of the large correlation between collinear conditions. Therefore, not only the ratio of the distance to the base line but also the mutual position between the object and the camera are predicted to be important factors in determining the 3D coordinates. In order to verify this, the overlapping images of which the baseline lengths are kept substantially the same and have different mutual positions, that is, group C4 of FIG. 16 were selected. In the experiment of the present invention, only the image according to the position of the camera is different, and the three-dimensional absolute coordinates and errors of the ground points were obtained by equalizing all the experimental conditions such as the length of the base line and the inspection point used. As shown in FIG. 16, group C3 is a case where a checkpoint is not located between two cameras based on the X axis, and C4 is a case where a checkpoint group is located between two cameras.

17 is an exemplary view illustrating ground points and inspection points determined from an omnidirectional image according to an embodiment of the present invention.

As shown in FIG. 17, the inspection point and the ground point determined from the superimposed images of the two groups are shown. It can be seen that the ground points of the group C4 (green, C4) are more similar to the checkpoints (blue, P) in shape, size and direction than the ground points of the group C3 (red, C3). In the case of C3, the correlation of each collinear condition is high, whereas in the case of C4, the correlation is very low, so an independent collinear condition can be obtained.

[Table 8] shows the three-dimensional absolute coordinates of the checkpoint and the ground point and their error quantitatively and the unit is [m].

[Table 8]

Table 8 shows that group C4 is about twice as accurate as C3.

[Table 9] below shows the results of experiments under the environment of groups C3 and C4 after removing the system error, and the unit is [m].

TABLE 9

As in the accuracy verification experiment according to the baseline length, the accuracy of the relative coordinates excluding the system error is higher than that of the absolute coordinates including the system error. Through these experiments, it was found that the model of the target object can be extracted more accurately by using the superimposed images located in front and behind the checkpoint.

A mathematical model for calculating three-dimensional coordinates of a target object using omnidirectional camera superimposition images and GPS / INS information was established and verified by applying it to the measured data. The three-dimensional coordinates of the ground point were determined by deriving a pair of straight lines defined on the corresponding ground coordinate system from the pair of conjugate points acquired from the two images and calculating their intersection points. The accuracy was verified by comparing the test points precisely measured by GPS and total station. If the length of the baseline and the mutual position between the camera and the object are properly set, it can be seen that the relative coordinates of the object or the three-dimensional model of the object can be extracted with an accuracy of several cm. This means that it is possible to extract key objects such as road components, road signs, traffic facilities and buildings around the road from the omnidirectional camera image to a level that can be fully utilized.

The absolute coordinates of the target point show accuracy of about several centimeters to about 1 m and include systematic errors that can be represented by simple geometric models. This error may be considerably eliminated by more accurately setting the relationship between the camera coordinate system and the GPS / INS coordinate system and performing omnidirectional camera calibration.

Various modifications and variations can be made by those skilled in the art to determine the three-dimensional coordinates of an object using the omnidirectional image according to the present invention, and those skilled in the art to which the present invention pertains. This will be possible. Therefore, the embodiments disclosed in the present invention are not intended to limit the scope of the present invention but to limit the scope of the technical idea of the present invention. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within the equivalent scope should be interpreted as being included in the scope of the present invention.

110: image information acquisition means
120: location information acquisition means
130: user terminal
131: Information collecting section
132: conjugate point calculation unit
133: vector calculator
134: formula guide
135: 3D coordinate determination unit

Claims (20)

Image information acquiring means for acquiring the superposed omnidirectional image;
Location information acquisition means for obtaining location information and attitude information of the image information acquisition means; And
The image point vector calculated on the camera coordinate system is calculated based on the overlapped omnidirectional image, the position information and attitude information, and the image coordinate value of the conjugate point with respect to the preset target point. A user terminal for deriving an equation of two or more straight lines in which a point exists and determining a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of the derived two or more linear equations;
System for determining the three-dimensional coordinates of the object using an overlapping omnidirectional image comprising a. The method according to claim 1,
The user terminal comprises:
Determining three-dimensional coordinates of an object by using an overlapping omnidirectional image, wherein an image coordinate value of a conjugate point for the target point is calculated from each omnidirectional image when an arbitrary target point is selected in at least two abutment omnidirectional images. System. The method of claim 2,
The user terminal comprises:
And an image point vector for connecting the image points at the projection center of the omnidirectional camera on the camera coordinate system based on the image coordinate values of the conjugate points. The method according to claim 1,
The user terminal comprises:
Using the image point vector represented by the camera coordinate system to derive an equation of a straight line in which the ground point represented by the ground coordinate system exists,
The equation of the straight line is an equation

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. System for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image, characterized in that. The method according to claim 1,
The user terminal comprises:
Ground point for each straight line using the intersection point calculated by calculating the intersection point of which the sum of squares of the distances between the straight lines is the minimum using equations of two or more straight lines respectively derived from conjugate points selected from two or more overlapping images. After calculating the coordinate values, the distance between the ground point coordinate values is compared with the preset threshold value, and if the distance between the ground point coordinate values is smaller than the preset threshold value, the ground point coordinate value A system for determining three-dimensional coordinates of an object using an overlapping omnidirectional image, wherein the average of the two points is determined to determine a three-dimensional coordinate value of the target point by averaging the coordinate values of the two ground points. A conjugate point calculator configured to calculate an image coordinate value of a conjugate point of the target point from each of the omnidirectional images when an arbitrary target point is selected from the at least two received omnidirectional images;
A vector calculator configured to calculate an image point vector on a camera coordinate system based on the calculated coordinate values of the conjugate points;
A formula derivation unit for deriving an equation of two or more straight lines having a ground point on a ground coordinate system using the calculated image point vector; And
A three-dimensional coordinate determination unit configured to determine a three-dimensional coordinate value for the target point by using the intersection point calculated by calculating an intersection point of two or more derived straight line equations;
System for determining the three-dimensional coordinates of the object using an overlapping omnidirectional image comprising a. The method of claim 6,
The vector calculation unit,
The 3D coordinates of the object are determined using an overlapping omnidirectional image, which calculates an image point vector connecting the image points at the center of the projection of the image information acquisition means on the camera coordinate system based on the calculated image coordinate values of the conjugate points. System for doing so. The method of claim 6,
The formula induction unit,
Using the image point vector represented by the camera coordinate system to derive an equation of a straight line in which the ground point represented by the ground coordinate system exists,
The equation of the straight line is an equation

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. System for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image, characterized in that. The method of claim 6,
The coordinate determination unit,
Ground point for each straight line using the intersection point calculated by calculating the intersection point of which the sum of squares of the distances between the straight lines is the minimum using equations of two or more straight lines respectively derived from conjugate points selected from two or more overlapping images. After calculating the coordinate values, the distance between the coordinate values of the ground point is compared with the preset threshold value, and as a result of the comparison, if the distance between the coordinate values of the ground point is smaller than the predetermined threshold value, the two ground points A system for determining three-dimensional coordinates of an object using an overlapping omnidirectional image, wherein the coordinate values of the target points are determined by averaging the coordinate values of the two ground points. The method of claim 6,
An information collecting unit which receives the omnidirectional image, position information, and attitude information superimposed from the image information obtaining unit and the position information obtaining unit;
The system for determining the three-dimensional coordinates of the object using an overlapping omnidirectional image further comprising. (a) obtaining, by the image information acquiring means, an overlapping omnidirectional image;
(b) obtaining, by the position information obtaining means, position information and attitude information of the image information obtaining means;
(c) calculating, by the user terminal, an image point vector on a camera coordinate system based on the overlapping omnidirectional image, the position information and attitude information, and an image point coordinate value of a conjugate point with respect to a preset target point;
(d) deriving, by the user terminal, an equation of a straight line in which a ground point exists in a ground coordinate system using the calculated image point vector in the camera coordinate system; And
(e) determining, by the user terminal, a three-dimensional coordinate value for the target point using the intersection point calculated by calculating an intersection point using the equation of the straight line derived;
Method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image comprising a. 12. The method of claim 11,
The step (c)
Determining three-dimensional coordinates of an object by using an overlapping omnidirectional image, wherein an image coordinate value of a conjugate point of the target point is calculated from each omnidirectional image when an arbitrary target point is selected from at least two abutment images. Way. 12. The method of claim 11,
The step (c)
Determining three-dimensional coordinates of an object using an overlapping omnidirectional image, characterized by calculating an image point vector connecting an image point at the center of the projection of the image information acquisition means on a camera coordinate system based on the image coordinate value of the conjugate point. Way. 12. The method of claim 11,
The step (d)
Using the image point vector represented by the camera coordinate system to derive an equation of a straight line in which the ground point represented by the ground coordinate system exists,
The equation of the straight line is an equation

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image, characterized in that. 12. The method of claim 11,
In step (e),
Ground point for each straight line using the intersection point calculated by calculating the intersection point of which the sum of squares of the distances between the straight lines is the minimum using equations of two or more straight lines respectively derived from conjugate points selected from two or more overlapping images. After calculating the coordinate values, the distance between the coordinate values of the ground point is compared with the preset threshold value, and as a result of the comparison, if the distance between the coordinate values of the ground point is smaller than the predetermined threshold value, the two ground points A method for determining three-dimensional coordinates of an object using an overlapping omnidirectional image, characterized by obtaining an average of coordinate values of and determining a three-dimensional coordinate value of a target point by averaging the coordinate values of the two ground points. (a) calculating an image coordinate value of a conjugate point of the target point from each of the omnidirectional images when an arbitrary target point is selected from the at least two received omnidirectional images;
(b) calculating an image point vector on a camera coordinate system based on the calculated image coordinate value of the conjugate point;
(c) deriving an equation of two or more straight lines having a ground point on a ground coordinate system using the calculated image point vector on the camera coordinate system; And
(d) determining a three-dimensional coordinate value for the target point using the intersection point calculated by calculating an intersection point of the two or more linear equations derived;
Method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image comprising a. 17. The method of claim 16,
The step (b)
The 3D coordinates of the object are determined using an overlapping omnidirectional image, which calculates an image point vector connecting the image points at the center of the projection of the image information acquisition means on the camera coordinate system based on the calculated image coordinate values of the conjugate points. How to. 17. The method of claim 16,
The step (c)
Using the image point vector represented by the camera coordinate system to derive an equation of a straight line in which the ground point represented by the ground coordinate system exists,
The equation of the straight line is an equation

Wherein, ^G P is the coordinate of the ground point represented by the ground coordinate system and is present on the straight line connecting the projection center of the camera, the image point and the ground point, and λ is one point on the straight line. Represents a variable for specifying

Denotes a rotation matrix for converting a vector expressed in a camera coordinate system into a ground coordinate system, ^C n denotes an image point vector expressed in a camera coordinate system, and ^G O _C denotes a projection center of a camera expressed in a ground coordinate system. Method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image, characterized in that. 17. The method of claim 16,
The step (d)
Ground point for each straight line using the intersection point calculated by calculating the intersection point of which the sum of squares of the distances between the straight lines is the minimum using equations of two or more straight lines respectively derived from conjugate points selected from two or more overlapping images. After calculating the coordinate values, the distance between the coordinate values of the ground point is compared with the preset threshold value, and as a result of the comparison, if the distance between the coordinate values of the ground point is smaller than the predetermined threshold value, the two ground points A method for determining three-dimensional coordinates of an object by using an omnidirectional image, characterized in that for obtaining the average of the coordinate values of the two ground points to determine the three-dimensional coordinate value for the target point. 17. The method of claim 16,
(a-1) receiving an overlapping omnidirectional image, position information, and attitude information from the image information obtaining unit and the position information obtaining unit before the step (a);
Method for determining the three-dimensional coordinates of the object using the overlapping omnidirectional image further comprising.

Images