KR101866804B1 - Method and apparatus for calculating surface curve of transparent material - Google Patents

Abstract

The present invention relates to a method for calculating a surface curvature of a transparent material and an apparatus thereof. The method for calculating a surface curvature of a transparent material comprises a step in which a distortion camera coordinate calculation unit calculates a distortion observation point camera coordinate based on a transparent material transmission image including an arbitrary observation point located on an arbitrary observation plane; a step in which a non-distortion camera coordinate calculation unit calculates a non-distortion observation point camera coordinate based on a transparent material removal image including an arbitrary observation point photographed after removing a transparent material; a step in which a curved surface parameter determination unit determines a plurality of curved surface parameters based on a difference between the distortion observation point camera coordinate and the non-distortion observation point camera coordinate; and a step in which a transparent material surface curvature calculation unit calculates a surface curvature of the transparent material by applying the determined curved surface parameters to a pre-stored transparent material curved surface model.

Description

METHOD AND APPARATUS FOR CALCULATING SURFACE CURVE OF TRANSPARENT MATERIAL

The present invention relates to a method and an apparatus for calculating a transparent material curvature value for calculating a curvature value of a transparent material based on an image captured through a transparent material.

2. Description of the Related Art In recent years, various camera products installed inside a vehicle to monitor the outside of a vehicle have been developed and used in order to provide various functions related to driving assistance of a vehicle and autonomous driving of the vehicle.

Various camera products installed inside the vehicle and monitoring the outside of the vehicle transmit transparent materials including the glass of the vehicle to monitor the outside of the vehicle, so that images captured by the various camera products described above are distorted.

The distortion caused by the transparent material is determined by the geometry of the camera, the transparent material, and the subject. Conventionally, a separate three-dimensional position measuring device is used in addition to the camera to correct such distortion.

However, the use of such a separate three-dimensional position measuring instrument requires additional time and effort to match the coordinate system between the camera and the three-dimensional position measuring instrument as well as the additional cost to provide the three-dimensional position measuring instrument There is a problem.

Korean Patent Publication No. 10-2001-0017356 (Mar.

SUMMARY OF THE INVENTION The object of the present invention is to solve the above-mentioned problems, and it is an object of the present invention to provide an apparatus and a method for correcting distortion of an image by transmitting a transparent material through an observation point located on an arbitrary observation plane, And a plurality of curved surface parameters are determined on the basis of the difference of the point camera coordinates, and then the curved surface value of the transparent material is calculated based on the determined plurality of curved surface parameters.

The problems to be solved by the present invention are not limited to the above-mentioned problem (s), and another problem (s) not mentioned can be clearly understood by those skilled in the art from the following description.

According to another aspect of the present invention, there is provided a method of calculating a transparent material curve value for calculating a curve value of a transparent material, the method comprising the steps of: Calculating a distorted observation point camera coordinate based on a transparent material transmission image including an arbitrary observation point located on the transparent material; Calculating a plurality of curved surface parameters based on the difference between the distortion observation point camera coordinates and the non-distortion observation point point camera coordinates; Value calculation unit applies a plurality of determined curved surface parameters to a previously stored transparent material curved surface model, And calculating a curved surface value of the workpiece.

According to one embodiment, the step of calculating the distortion observation point camera coordinates may comprise the step of calculating the distortion observation point camera coordinates based on the previously stored transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) And calculating a distortion observation point camera coordinate based on an intersection between the optical refraction path and an arbitrary observation plane.

For example, the step of determining the plurality of curved surface parameters may include a condition that the difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates is minimized, and a condition that the amount of at least one line segment disposed in advance on the first surface of the transparent material Determining a plurality of curved surface parameters such that the camera coordinate distance between at least one first end point and the second end point corresponding to each of the end points is equal to the length of at least one line segment, respectively.

According to one embodiment, a plurality of curved surface parameters are determined based on Equation (13) below.

&Quot; (13) "

는 k번째 관찰점에 대한 비왜곡 관찰점 카메라 좌표,

는 k번째 관찰점에 대한 왜곡 관찰점 카메라 좌표,

는 k번째 관찰점의 인클라이네이션(inclination) 각도 좌표,

는 k번째 관찰점의 아지무스(azimuth) 각도 좌표,

는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제1 끝점 카메라 좌표,

는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제2 끝점 카메라 좌표, L_i는 i번째 선분의 길이를 의미한다.S ^* is a set of curved surface parameters including a plurality of determined curved surface parameters, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,

Is the non-distortion observation point camera coordinate for the kth observation point,

Is the distortion observation point camera coordinates for the kth observation point,

Is the inclination angle coordinate of the kth observation point,

Is the azimuth angle coordinate of the kth observation point,

The first end camera coordinates of the i-th line segment disposed on the first surface of the transparent material,

Is the coordinates of the second end point camera of the i-th line segment arranged on the first side of the transparent material, and L _i is the length of the i-th line segment.

For example, the transparent material surface model is defined based on Equation (4) below.

&Quot; (4) "

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표, a_x는 제1 곡면 파라미터, a_y는 제2 곡면 파라미터, a_z는 제3 곡면 파라미터 및 a_r은 제4 곡면 파라미터를, A_m,n은 제1 곡면 파라미터 행렬의 m행 n열, B_m,n은 제2 곡면 파라미터 행렬의 m행 n열을 의미한다.r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

A _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, and a _r is a fourth curved surface parameter, and A _{m, n} is a M rows and n columns of the first curved surface parameter matrix, B _{m, n} means m rows and n columns of the second curved surface parameter matrix.

For example, a method for calculating a transparent material curve value according to an embodiment of the present invention is a method for calculating a transparent material curve value based on a plurality of determined curved surface parameters and a transparent material transmitted image captured with at least one camera, And calculating the corrected observation point camera coordinates by calculating the corrected observation point camera coordinates.

According to one embodiment, when at least one camera is a single camera, calculating the corrected viewing point camera coordinates may include calculating, based on the determined plurality of curved surface parameters, an arbitrary observation located on an arbitrary observation plane from one camera Calculating a corrected observation point camera coordinate by calculating a distortion observation point camera coordinate based on an intersection of the light refraction path and an arbitrary observation plane;

For example, when at least one camera includes a first camera and a second camera, the step of calculating the corrected observation point camera coordinates may include calculating the corrected observation point camera coordinates based on a plurality of first camera curved surface parameters Calculating a first camera photographic refraction path that is a photorefractive path from the first camera to an arbitrary observation point on the basis of the plurality of curved surface parameters for the first camera, Calculating a second camera photographic refraction path that is a photorefractive path from the first camera photoreflection path to an arbitrary observation point, calculating the distortion observation point camera coordinates based on the intersection of the first camera photoreflection path and the second camera photoreflection path, And calculating point camera coordinates.

For example, in a method of calculating a transparent material curve value according to an embodiment of the present invention, an image coordinate correction unit calculates a correction coordinate value of a corrected observation point camera coordinate based on a coordinate relationship between a camera coordinate and a two- Into image coordinates.

For example, if a plurality of second camera curved surface parameters are not determined, the method further comprises the step of determining a plurality of second camera curved surface parameters, wherein the step of determining the plurality of second camera curved surface parameters comprises: Calculating a first camera curved surface value by applying a camera curved surface parameter to a transparent curved surface model, calculating a second estimated camera curved surface value by applying a plurality of second camera curved surface parameter estimates to a transparent curved surface model, Calculating an imaginary first camera curved surface value corresponding to the estimated second camera curved surface value based on the relationship between the first camera coordinate and the second camera coordinate, calculating a difference between the first camera curved surface value and the virtual first camera curved surface value Determining a plurality of second camera curved surface parameter estimation values as a plurality of second camera curved surface parameters, .

According to another aspect of the present invention, there is provided a transparent material curvature value calculating device for calculating a curvature value of a transparent material, A distortion camera coordinate calculation unit for calculating a distortion observation camera coordinate based on a transparent material transmission image including an observation point, a distortion camera coordinate calculation unit for calculating a distortion camera coordinate value based on a transparent material removal image based on a transparent material removal image, A non-distorted camera coordinate calculation unit for calculating point camera coordinates, a curved surface parameter determination unit for determining a plurality of curved surface parameters based on the difference between the distortion observation camera coordinates and the non-distorted observation point camera coordinates, and a plurality of determined curved surface parameters A transparent material curved surface value calculating unit for calculating the curved surface value of the transparent material by applying the curved surface model to the transparent material curved surface model It includes.

According to one embodiment, the distorted camera coordinate calculator calculates a refractive index of the light from the center of the camera to an arbitrary observation point based on a previously stored transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) And the distortion observation point camera coordinates are calculated based on the intersection point between the optical refraction path and the arbitrary observation plane.

For example, the curved surface parameter determination unit may determine that the difference between the non-distorted observation point camera coordinate and the distortion observation point camera coordinate is minimized and the condition that the distances between the non- A plurality of curved surface parameters are determined so as to satisfy all the conditions that the camera coordinate distance between the at least one first end point and the second end point is equal to the length of at least one line segment.

According to one embodiment, a plurality of curved surface parameters are determined based on Equation (13) below.

&Quot; (13) "

는 k번째 관찰점에 대한 비왜곡 관찰점 카메라 좌표,

는 k번째 관찰점에 대한 왜곡 관찰점 카메라 좌표,

는 k번째 관찰점의 인클라이네이션(inclination) 각도 좌표,

는 k번째 관찰점의 아지무스(azimuth) 각도 좌표,

는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제1 끝점 카메라 좌표,

는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제2 끝점 카메라 좌표, L_i는 i번째 선분의 길이를 의미한다.S ^* is a set of curved surface parameters including a plurality of determined curved surface parameters, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,

Is the non-distortion observation point camera coordinate for the kth observation point,

Is the distortion observation point camera coordinates for the kth observation point,

Is the inclination angle coordinate of the kth observation point,

Is the azimuth angle coordinate of the kth observation point,

The first end camera coordinates of the i-th line segment disposed on the first surface of the transparent material,

Is the coordinates of the second end point camera of the i-th line segment arranged on the first side of the transparent material, and L _i is the length of the i-th line segment.

For example, the transparent material surface model is defined based on Equation (4) below.

&Quot; (4) "

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표, a_x는 제1 곡면 파라미터, a_y는 제2 곡면 파라미터, a_z는 제3 곡면 파라미터 및 a_r은 제4 곡면 파라미터를, A_m,n은 제1 곡면 파라미터 행렬의 m행 n열, B_m,n은 제2 곡면 파라미터 행렬의 m행 n열을 의미한다.r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

A _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, and a _r is a fourth curved surface parameter, and A _{m, n} is a M rows and n columns of the first curved surface parameter matrix, B _{m, n} means m rows and n columns of the second curved surface parameter matrix.

For example, the transparent material surface value calculating device according to the embodiment of the present invention calculates the distortion observation point camera coordinates based on the determined plurality of curved surface parameters and the transparent material transmission image captured with at least one camera, And a camera coordinate correcting unit for calculating a camera coordinate correcting unit.

According to one embodiment, when at least one camera is a single camera, the camera coordinate correction unit may be configured to calculate, based on the determined plurality of curved surface parameters, a light refraction path from one camera to an arbitrary observation point located on an arbitrary observation plane And calculates the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the intersection of the optical refraction path and the arbitrary observation plane.

For example, when at least one camera includes a first camera and a second camera, the camera coordinate correcting unit may obtain, from the first camera, based on a plurality of first camera curved surface parameters that are a plurality of curved surface parameters for the first camera Calculating a first camera photographic refraction path that is a photorefractive path toward an arbitrary observation point and calculating a first camera photorefractive path from the second camera to an arbitrary observation point based on a plurality of second camera surface parameters that are a plurality of curved surface parameters for the second camera A second camera photographic refraction path that is a photorefractive path is calculated, and the distortion observation point camera coordinates are calculated based on the intersection of the first camera photoreflection path and the second camera photorefractive path to calculate corrected viewing point camera coordinates.

According to one embodiment, the transparent material curvature value calculating device further includes an image coordinate correcting unit for converting the corrected observation point camera coordinates into corresponding corrected two-dimensional image coordinates on the basis of the coordinate relationship between the previously set camera coordinate and the two- .

For example, when a plurality of second camera curved surface parameters are not determined, the camera coordinate correction unit applies a plurality of predetermined first camera curved surface parameters to the transparent material curved surface model to calculate a first camera curved surface value, Calculating an estimated second camera curvature value by applying the camera curved surface parameter estimation value to the transparent material curved surface model, calculating a virtual second curvature value based on a relationship between the first camera coordinate and a second camera coordinate set in advance, A plurality of second camera curved surface parameter estimates are determined as a plurality of second camera curved surface parameters so as to minimize the difference between the first camera curved surface value and the virtual first camera curved surface value.

According to the present invention, based on the difference between the camera coordinates of the distortion observation point camera, which is obtained by transmitting the transparent material, and the coordinates of the non-distortion observation point camera, which are obtained by removing the transparent material, from an arbitrary observation point located on an arbitrary observation plane, It is possible to calculate the curved surface value of the transparent material on the basis of the determined plurality of curved surface parameters and correct the distortion of the photographed image by transmitting the transparent material through only the image captured by the camera.

BRIEF DESCRIPTION OF DRAWINGS FIG. 1 is a block diagram illustrating a transparent material curvature value calculating apparatus according to an embodiment of the present invention. FIG.
2 is a flowchart illustrating a method of calculating a transparent material curve value according to an embodiment of the present invention.
FIG. 3 is a flowchart for explaining a step of calculating the distortion observation point camera coordinates in the transparent material curve value calculating method according to the embodiment of the present invention.
FIG. 4 is a flowchart for explaining a step of calculating the distortion observation point camera coordinates in the transparent material curve value calculating method according to the embodiment of the present invention.
5 is a flowchart for explaining a step of calculating corrected viewing point camera coordinates according to the first embodiment of the present invention.
6 is a flowchart for explaining a step of calculating corrected viewing point camera coordinates according to a second embodiment of the present invention.
7 is a diagram for explaining the relationship between camera coordinates and image coordinates in a transparent material surface value calculating method and apparatus according to an embodiment of the present invention.
8 is a view for explaining a transparent material surface model in a transparent material surface value calculating method and apparatus according to an embodiment of the present invention.
FIGS. 9A and 9B are diagrams for explaining a step of calculating the distortion observation point camera coordinates in the transparent material surface value calculating method and apparatus according to the embodiment of the present invention. FIG.
10 is a diagram for explaining the step of calculating the corrected viewing point camera coordinates according to the first embodiment of the present invention.
11 is a diagram for explaining the step of calculating the corrected observation point camera coordinates according to the second embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings in order to facilitate a person skilled in the art to easily carry out the technical idea of the present invention. . In the drawings, the same reference numerals are used to designate the same or similar components throughout the drawings. In the following description of the present invention, a detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

Hereinafter, a method and an apparatus for calculating a transparent material surface value according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

Before describing a method and an apparatus for calculating a transparent material curved surface value according to an embodiment of the present invention, a conventional technique for describing the relationship between camera coordinates and image coordinates will be described with reference to FIG.

7 is a diagram for explaining the relationship between camera coordinates and image coordinates in a transparent material surface value calculating method and apparatus according to an embodiment of the present invention.

7, when the first observation point W1 is photographed with respect to the center C of the camera, the first observation point W1 is imaged on the image photographed by the camera as shown in the first image coordinate I1 Similarly, when the second observation point W2 is photographed based on the center C of the camera, the second observation point W2 is expressed as the second image coordinate I2 in the image photographed by the camera.

)와 같은 3차원 구면 좌표로 나타나게 된다.In this case, the image coordinates for a specific observation point are represented by two-dimensional coordinates such as (x ₁ , x ₂ ), and the camera coordinates for a specific observation point are (r,

) As the three-dimensional spherical coordinates.

Hereinafter, a conventional technique for converting image coordinates, which are two-dimensional coordinates included in an image taken by a camera, into camera coordinates, which are three-dimensional coordinates, will be described.

In the prior art, camera parameters are required to convert image coordinates obtained through a camera to camera coordinates, and these camera parameters include internal parameters including focus, principal point, etc., and external parameters related to conversion between the camera coordinate system and the actual coordinate system It is divided.

In this case, the relationship between the image coordinate, which is a two-dimensional coordinate, and the camera coordinate, which is a three-dimensional coordinate, can be expressed by the following equation (1).

[Equation 1]

Here, x denotes a two-dimensional image coordinate, K denotes an internal parameter matrix, R denotes a rotation matrix, t denotes a motion vector, w denotes a three-dimensional coordinate, and λ denotes a proportional coefficient.

Further, when the three-dimensional coordinate w is the camera coordinate w _c , the rotation matrix R is an identity matrix and the motion vector t is a zero vector.

)를 계산하기 위해서는, 상술한 특정 관찰점이 이미 좌표 정보를 알고 있는 특정 관찰 평면 상에 위치하는 것을 가정해야 한다.In this case, the 3D camera coordinates for a given two-dimensional image coordinates of _{(x 1, x 2) (} r, θ,

), It is to be assumed that the above-mentioned specific observation point is already located on a specific observation plane which already knows the coordinate information.

)가 산출된다.That is, if (x ₁ , x ₂ ), which is two-dimensional image coordinates of a specific observation point located on a specific observation plane already known to coordinate information, is converted into three-dimensional camera coordinates based on the above- (R,?,

) Is calculated.

Here, the camera coordinates may mean three-dimensional coordinates of a specific point with respect to the center of the camera.

를 포함하도록 산출되게 되며, 그 좌표는 (

, θ,

)로 산출되게 된다.On the other hand, if (x ₁ , x ₂ ), which is a two-dimensional image coordinate of a specific observation point without assuming a specific observation plane, is transformed into three-dimensional camera coordinates based on the above-described expression (1) Direction vector

, And the coordinates are calculated to include (

,?,?

).

, θ,

)로 변환하는 경우에서도, 방향에 대한 정보는 얻을 수 있음을 확인할 수 있으며, 이는 특정 2차원 영상 좌표에 해당하는 3차원 카메라 좌표는 카메라의 광 경로의 직선 상에만 존재하기 때문이다.That is, without assuming a specific observation plane, the two-dimensional image coordinates (x ₁ , x ₂ ) of a specific observation point are converted into three-dimensional camera coordinates

,?,?

), It can be seen that information on the direction can be obtained even if the three-dimensional camera coordinates corresponding to the specific two-dimensional image coordinates exist only on the straight line of the optical path of the camera.

)로 결정될 수 있다.On the other hand, even when a specific observation plane is not assumed, when a plurality of cameras are used, the three-dimensional camera coordinates of a specific observation point are (r,

). ≪ / RTI >

)가 산출될 수 있으며, 이는 특정 관찰 평면을 전제하지 않더라도 복수 개의 광 경로의 교점을 산출할 수 있는 경우 3차원 카메라 좌표는 특정한 카메라 좌표로 결정될 수 있기 때문이다.That is, when each of the two-dimensional image coordinates of a specific observation point photographed by the plurality of cameras is converted into each of the three-dimensional camera coordinates, the three-dimensional camera coordinates are calculated based on the intersection of the optical paths from each of the plurality of cameras to the specific observation point. The coordinates (r,?,

) Can be calculated because if the intersection point of a plurality of optical paths can be calculated without assuming a specific observation plane, the three-dimensional camera coordinate can be determined to a specific camera coordinate.

In other words, in the conventional technique, a camera calibration is performed to calculate the internal parameters and the external parameters described above, and this process is performed by observing the image coordinates of a point at which the actual coordinates are known.

At this time, since the point on the specific observation plane is observed, the coordinates of the observation point can be calculated, and the error between the calculated coordinate and the actual coordinate is minimized to determine the internal and external parameters.

The detailed description of the related art relating to the relationship between the camera coordinates and the image coordinates and the mutual conversion between the camera coordinates and the image coordinates will be omitted.

Now, a transparent material curvature value calculating apparatus according to an embodiment of the present invention will be described with reference to FIG.

BRIEF DESCRIPTION OF DRAWINGS FIG. 1 is a block diagram illustrating a transparent material curvature value calculating apparatus according to an embodiment of the present invention. FIG.

1, the transparent material curvature value calculating apparatus 100 according to the embodiment of the present invention includes a distortion camera coordinate calculating unit 110, a non-distorted camera coordinate calculating unit 120, a curved surface parameter determining unit 130 and a transparent material curvature value calculation unit 140.

According to an exemplary embodiment, the transparent material surface value calculation apparatus 100 may further include a camera coordinate correction unit 150 or an image coordinate correction unit (not shown), but the present invention is not limited thereto.

According to one embodiment, the transparent material curve value calculation device 100 may mean a transparent material curve value calculation device 100 for calculating a curve value of a transparent material.

For example, the transparent material curvature value calculating device 100 may mean a transparent material curvature value calculating device for correcting distortion of an image taken through a transparent material.

The distortion camera coordinate calculation unit 110 may calculate the distortion observation point camera coordinates based on the transparent material transmission image including any observation point located on an arbitrary observation plane.

According to one embodiment, the distorted camera coordinate calculation section 110 may calculate, based on a plurality of surface parameter estimates and a previously stored transparent material surface model, in addition to the transparent material transmission image including any observation point located on any observation plane The distortion observation point camera coordinates may be calculated, but the present invention is not limited thereto.

Here, the plurality of curved surface parameter estimates may mean a plurality of curved surface parameters that have not yet been determined, and the plurality of curved surface parameter estimates may include each of the plurality of curved surface parameters in an unknown form.

For example, the distorted camera coordinate calculation unit 110 may calculate an arbitrary reference point based on the center of the camera based on the transparent material transmitted image, which is an imaging image including an arbitrary observation point positioned on an arbitrary observation plane photographed through a transparent material, Which is the camera coordinate with respect to the observation point of the distortion observation point camera.

The non-distorted camera coordinate calculation unit 120 removes the transparent material and calculates the unoriented observation point camera coordinates based on the transparent material removed image including any observed observation point.

For example, the non-distorted camera coordinate calculation unit 120 calculates a non-distorted camera coordinate value based on the center of the camera based on the transparent material removal image, which is a photographed image including an arbitrary observation point positioned on an arbitrary observation plane photographed, It is possible to calculate the non-distorted observation point camera coordinates which is the camera coordinates for an arbitrary observation point.

At this time, the distortion camera coordinate calculation unit 110 may receive the transparent material transmission image from the camera 10, and the non-distorted camera coordinate calculation unit 120 may receive the transparent material removal image from the camera 10. [

In FIG. 1, the camera 10 is shown as one person, but this is for convenience of explanation. The number of cameras receiving the various images by the transparent material surface value calculating device 100 according to the embodiment of the present invention is But is not limited to dogs.

The curved surface parameter determination unit 130 determines a plurality of curved surface parameters based on the difference between the distortion observation point camera coordinates and the non-distortion observation point camera coordinates.

At this time, the plurality of curved surface parameter estimation values described above by the curved surface parameter determination unit 130 can be determined by a plurality of specific curved surface parameters.

The transparent material curvature value calculation unit 140 calculates a curvature value of the transparent material by applying a plurality of determined curvature parameters to a previously stored transparent curvilinear surface model.

For example, the distorted camera coordinate calculation unit 110 calculates the optical axis of the optical path from the center of the camera to an arbitrary observation point based on the previously stored transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) And the distortion observation point camera coordinates can be calculated based on the intersection point between the optical refraction path and the arbitrary observation plane.

For example, the distorted camera coordinate calculation unit 110, the aforementioned transparent material surface models, in addition to the thickness of the transparent material (d ₀₎ and refractive index (n _w) further based on the estimated value of the plurality of curved surface parameters random from the center of the camera The distortion-observation-point camera coordinates may be calculated based on the intersection between the optical refraction path and the arbitrary observation plane, but the present invention is not limited thereto.

)에 기초하여 카메라의 중심(C)을 기준으로 하는 입사 교점(X)에 대한 카메라 좌표인 입사 교점 카메라 좌표(

)를 산출하고, 투명 소재의 두께(d₀), 내부 굴절각(j) 및 내부 굴절 방향 벡터(

)에 기초하여 입사 교점(X)을 기준으로 하는 투명 소재의 제2 면과 광 경로의 교점인 굴절 교점(Y)에 대한 3차원 좌표인 입사 교점 기준 굴절 교점 3차원 좌표(

)를 산출하고, 굴절 교점(Y)을 기준으로 하는 상기 임의의 관찰점(W)에 대한 3차원 좌표인 굴절 교점 기준 관찰점 3차원 좌표(

)와 입사 교점 카메라 좌표(

)의 비례 관계를 나타내는 비례 계수(λ)에 기초하여 굴절 교점 기준 관찰점 3차원 좌표(

)를 산출하고, 입사 교점 카메라 좌표(

), 입사 교점 기준 굴절 교점 3차원 좌표(

) 및 굴절 교점 기준 관찰점 3차원 좌표(

)의 벡터합에 기초하여 왜곡 관찰점 카메라 좌표를 산출한다.For example, the distorted camera coordinate calculation unit 110 calculates the distortion camera coordinate calculation unit 110 based on the distance r between the center C of the camera and the incident intersection X that is the intersection of the first surface of the transparent material and the optical path, The incident direction vector (X)

), Which is a camera coordinate for an incident intersection X based on the center C of the camera,

), And calculates the thickness d _o of the transparent material, the internal refraction angle j and the internal refraction direction vector

Dimensional coordinate of the intersection of the intersection of the second surface of the transparent material and the optical path on the basis of the incident intersection X based on the three-

), And calculates three-dimensional coordinates of the arbitrary observation point (W) with reference to the refraction intersection (Y) as three-dimensional coordinates

) And intersection camera coordinates (

) Based on the proportional coefficient (?) Indicating the proportional relationship of the refraction intersection reference observation point 3-dimensional coordinates

), And calculates the incidence intersection camera coordinates (

), Incidence intersection reference refraction intersection three-dimensional coordinates (

) And refraction intersection reference observation point three-dimensional coordinates (

) Of the distortion-observation-point camera.

For example, the thickness (d ₀ ) of the transparent material may be constant, but the present invention is not limited thereto.

Here, the first surface of the transparent material may mean a surface on which the light path starting from the center C of the camera is incident on the transparent material, and the second surface of the transparent material may have a light path refracted by the transparent material, It can mean a plane that goes out of the transparent material to point to the observation point.

At this time, the first surface of the transparent material may be a surface including the incident intersection X, and the second surface of the transparent material may include a surface including the refraction intersection Y.

), 투명 소재의 두께(d₀), 내부 굴절각(j) 및 내부 굴절 방향 벡터(

)에 기초하여 산출된다.For example, the distortion observation point camera coordinate is a camera coordinate of an incident intersection X that is an intersection of a first surface of a transparent material and a light path on the basis of a proportional coefficient (?) And a center C of the camera, location(

), The thickness (d ₀ ) of the transparent material, the internal refraction angle (j) and the internal refraction direction vector

).

For example, the plurality of curved surface parameters may include a condition for minimizing the difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates, and a condition for minimizing the difference between both end points of at least one line segment disposed in advance on the first surface of the transparent material And the camera coordinate distance between the at least one first end point and the second end point corresponding to the first end point is equal to the length of at least one line segment, respectively.

More specifically, the curved surface parameter determination unit 130 determines whether the difference between the non-distorted observation point camera coordinate and the distortion observation point camera coordinate is minimized or not, A plurality of curved surface parameters can be determined so as to satisfy all the conditions that both the coordinate distance between the first end camera coordinates and the second end point camera coordinates, which are the camera coordinates of the end points of the arranged line segments, are equal to the length of the line segment.

In this case, the line segment previously arranged on the first surface of the transparent material may mean various linear materials including bars arranged in advance on the first surface of the transparent material, and the present invention is not limited to a specific line segment.

), 제1 곡면 파라미터(a_x), 제2 곡면 파라미터(a_y), 제3 곡면 파라미터(a_z) 및 제4 곡면 파라미터(a_r)를 포함한다.For example, the transparent material surface model may include a surface definition function (" A ") including a first surface parameter matrix A and a second surface parameter matrix B

, A first curved surface parameter (a _x ), a second curved surface parameter (a _y ), a third curved surface parameter (a _z ), and a fourth curved surface parameter (a _r ).

The camera coordinate correcting unit 150 calculates the corrected observation point camera coordinate by calculating the distortion observation point camera coordinates based on the determined plurality of curved surface parameters and the transparent material transmission image captured by at least one camera.

For example, the camera coordinate correction unit 150 corrects the distortion observation point camera coordinates calculated based on the transparent material transmission image photographed by at least one camera based on a plurality of determined curved surface parameters and curved surface values, The camera coordinates can also be calculated.

According to one embodiment, when at least one camera is a single camera, the camera coordinate correcting unit 150 corrects the camera coordinate correction unit 150 based on a plurality of determined curved surface parameters, from one camera to an arbitrary observation point located on an arbitrary observation plane And calculating the distortion observation point camera coordinates on the basis of the intersection of the optical refraction path and the arbitrary observation plane, the corrected observation observation point camera coordinates can be calculated.

According to another embodiment, when at least one camera includes a first camera and a second camera, the camera coordinate correction unit 150 corrects a plurality of first camera curved surface parameters, which are a plurality of curved surface parameters for the first camera, Based on a plurality of second camera curved surface parameters, which are a plurality of curved surface parameters for the second camera, from the second camera, based on the first camera curved surface parameters, A second camera photographic refraction path that is a photorefractive path toward an arbitrary observation point is calculated and the distortion observation point camera coordinates are calculated based on the intersection of the first camera photoreflection path and the second camera photoreflection path, The coordinates can be calculated.

The image coordinate correcting unit (not shown) converts the corrected observation point camera coordinates into the corresponding corrected two-dimensional image coordinates based on the coordinate relationship between the camera coordinates and the two-dimensional image coordinate set in advance.

According to one embodiment, when a plurality of second camera curved surface parameters are not determined, a plurality of second camera curved surface parameters may be calculated based on the following equation (16).

&Quot; (16) "

은 복수의 제1 카메라 곡면 파라미터에 기초하여 산출된 제1 카메라의 중심(C₁)을 기준으로 하는 투명 소재의 제1 면과 제1 카메라 광 굴절 경로의 k 번째 교점인 k번째 입사 교점(X_k)의 카메라 좌표,

는 복수의 곡면 파라미터 추정값에 기초하여 산출된 제2 카메라(C₂)의 중심을 기준으로 하는 투명 소재의 제1 면과 제2 카메라 광 굴절 경로의 k 번째 교점인 k번째 입사 교점(X_k)의 카메라 좌표를 의미한다.S ₂ is a set of second camera curved surface parameters including a plurality of calculated second camera curved surface parameters, S is a set of curved surface parameter estimation values including a plurality of curved surface parameter estimates, R is a position of the center of the first camera, T is a translation vector between a position of the center of the first camera and a position of the center of the second camera, S ₁ is a rotation vector between the center of the first camera surface parameter and the first camera surface parameter, Camera surface parameter sets,

(X), which is the kth intersection of the first camera photorefractive path and the first surface of the transparent material, which is based on the center (C ₁ ) of the first camera calculated based on the plurality of first camera curved surface parameters, _k ), < / _RTI >

A first surface and a second k-th intersection point in the k-th incidence of the camera photorefractive path intersection of the transparent material which on the basis of the center of the second camera (C ₂₎ is calculated based on a plurality of surfaces parameter estimate (X _k) Quot; camera coordinates "

A more detailed description of each configuration of the transparent material surface value calculation device 100 according to the embodiment of the present invention will be described later with reference to FIG. 2 to FIG. 10, and redundant description will be omitted.

Referring now to FIG. 2, a method of calculating a two-dimensional material surface value according to an embodiment of the present invention will be described.

2 is a flowchart illustrating a method of calculating a transparent material curve value according to an embodiment of the present invention.

2, the transparent material curve value calculating method according to the embodiment of the present invention includes a distortion observation camera coordinate calculation step S210, a non-distortion observation observation point camera coordinate calculation step S230, a plurality of curved surface parameters (S250), and calculating a curved surface value of the transparent material (S270).

For example, the transparent material curve value calculating method according to the embodiment of the present invention may mean a method of calculating a transparent material curve value for calculating a curve value of a transparent material.

For example, the method of calculating a transparent material curve value according to an embodiment of the present invention may mean a method of calculating a transparent material curve value for correcting distortion of an image photographed through a transparent material.

Hereinafter, in order to facilitate understanding of the embodiment of the present invention, the transparent material surface model used in step S270 will be described first, but the present invention is not limited to the description order.

A transparent material surface model will now be described with reference to Fig.

8 is a view for explaining a transparent material surface model in a transparent material surface value calculating method and apparatus according to an embodiment of the present invention.

) 3차원 좌표로 표현될 수 있다.8, each point located in the transparent material is represented by (x, y, z) three-dimensional coordinates, or (r,?,

) Three-dimensional coordinates.

), 구면 좌표계의 아지무스(azimuth) 각도 좌표(

)로 표현될 수 있기 때문에, 투명 소재 곡면 모델은 x, y 및 z 상호간의 관계로 정의되거나, r, θ 및

상호간의 관계로 정의될 수 있다.That is, each of the points located in the transparent material can be represented by x-axis direction distance, y-axis direction distance and z-axis direction distance based on a specific origin, and similarly, The distance coordinate (r) of the coordinate system, the inclination angle coordinate of the spherical coordinate system (

), The azimuth angle coordinate of the spherical coordinate system (

), The transparent material surface model is defined as the relationship between x, y and z, or r, &thetas; and

Can be defined as a mutual relationship.

For example, since glass windows, which are transparent materials used as vehicle windows, have a uniform thickness and a sufficiently smooth surface, a transparent material curved surface model for a glass window can be obtained by using the thickness value of the transparent material and the inner surface model inner surface model), and a transparent material surface model is defined as a bivariate function in a spherical coordinate system having a camera as an origin.

&Quot; (2) "

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표,

는 곡면 정의 함수, a_x는 제1 곡면 파라미터, a_y는 제2 곡면 파라미터, a_z는 제3 곡면 파라미터 및 a_r은 제4 곡면 파라미터를 의미한다.In this case, r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

Is the azimuth angle coordinate of the spherical coordinate system,

A _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, and a _r is a fourth curved surface parameter.

)는 곡면을 정의하기 위한 임의의 정의 함수를 의미할 수 있으며, 예컨대, 곡면 정의 함수(

)가

에 대한 테일러 시리즈(Taylor Series)와

에 대한 변형된 푸리에 시리즈(Fourier Series)를 사용하여 정의된 임의의 함수인 경우, 곡면 정의 함수(

)는 아래 수학식 3과 같이 표현될 수 있다.Here, the surface definition function (

) May refer to any definition function for defining a surface, such as a surface definition function (

)end

With Taylor Series (Taylor Series)

For any function defined using a modified Fourier Series for a surface definition function (< RTI ID = 0.0 >

) Can be expressed by the following equation (3).

&Quot; (3) "

는 곡면 정의 함수,

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표, A_m,n은 제1 곡면 파라미터 행렬의 m행 n열, B_m,n은 제2 곡면 파라미터 행렬의 m행 n열을 의미한다.here,

Is a surface definition function,

Is the inclination angle coordinate of the spherical coordinate system,

Denotes an azimuth angle coordinate of the spherical coordinate system, A _{m, n} denotes an m row and n column of the first curved surface parameter matrix, and B _{m and n} denotes an m row and n column of the second curved surface parameter matrix.

At this time, the first curved surface parameter a _x , the second curved surface parameter a _y , the third curved surface parameter a _z , and the fourth curved surface parameter a _r , the first curved surface parameter matrix A, A case in which the value of each parameter matrix B is not determined and remains in the form of a variable is defined as a surface parameter estimation value, and a case where each value of the above-described surface parameter is determined is defined as a determined surface parameter.

At this time, the present invention is not limited to a specific surface defining function, and the first surface parameter matrix and the second surface parameter matrix may have arbitrary sizes.

)는, 구면 좌표계의 인클라이네이션(inclination) 각도 좌표(

) 및 구면 좌표계의 아지무스(azimuth) 각도 좌표(

)를 변수로 하되, 제1 곡면 파라미터 행렬(A)과 제2 곡면 파라미터 행렬(B)를 포함하는 함수를 의미할 수 있다.That is, in the above example,

) Is the inclination angle coordinate of the spherical coordinate system (

) And the azimuth angle coordinate of the spherical coordinate system (

) May be a function including a first curved surface parameter matrix A and a second curved surface parameter matrix B as variables.

), 구면 좌표계의 아지무스(azimuth) 각도 좌표(

) 상호 간의 관계식으로 표현되되, 제1 곡면 파라미터(a_x), 제2 곡면 파라미터(a_y), 제3 곡면 파라미터(a_z), 제4 곡면 파라미터(a_r), 제1 곡면 파라미터 행렬(A) 및 제2 곡면 파라미터 행렬(B) 각각의 파라미터 값에 따라 의미하는 곡면이 달라지는 일반화된 곡면식을 의미할 수 있다.In other words, through the expressions (2) and (3), the transparent material curve model according to one embodiment is obtained by calculating the distance coordinate (r) of the spherical coordinate system and the inclination angle coordinate

), The azimuth angle coordinate of the spherical coordinate system (

(A _x ), a second curved surface parameter (a _y ), a third curved surface parameter (a _z ), a fourth curved surface parameter (a _r ), a first curved surface parameter matrix ( A) and the second curved surface parameter matrix (B), respectively.

를 만족하며, 평면성 및 곡면성을 쉽게 가지도록 설정될 수 있다.For example, the transparent material surface model

, And can be set to have easy planarity and curvature.

가 되어 상술한 투명 소재 곡면 모델은 평면을 의미하게 되며, 상술한 복수의 파라미터를 다르게 조정하면,

가 되어 상술한 투명 소재 곡면 모델은 구면을 의미하게 될 수 있다.For example, by adjusting the plurality of curved surface parameters described above in the transparent material surface model,

The above-mentioned transparent material curve model means plane, and if the above-mentioned plurality of parameters are adjusted differently,

And the above-mentioned transparent material surface model may mean spherical surface.

Substituting Equation (3) for the surface definition function into Equation (2) for the transparent material surface model, Equation (4) below can be derived.

&Quot; (4) "

Here, the detailed description of each variable included in Equation (4) is the same as Equation (2) and Equation (3), and a duplicate explanation will be omitted.

For convenience of explanation, the first curved surface parameter (a _x ), the second curved surface parameter (a _y ), the third curved surface parameter (a _z ), the fourth curved surface parameter (a _r ) (2) or (4) by defining a set including both the first curved surface parameter matrix A and the second curved surface parameter matrix B as a curved surface parameter set S, .

&Quot; (5) "

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표, S는 복수의 곡면 파라미터를 포함하는 곡면 파라미터 집합,

는 투명 소재 곡면 모델을 의미한다.In this case, r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

Azimuth angle coordinate of the spherical coordinate system, S is a set of curved surface parameters including a plurality of curved surface parameters,

Means a transparent material surface model.

Now, with reference to FIG. 2, a method of calculating a transparent material curve value according to an embodiment of the present invention will be described.

Step S210 may mean that the distorted camera coordinate calculation unit 110 calculates the distortion observation point camera coordinates based on the transparent material transmitted image including any observation point located on an arbitrary observation plane.

According to one embodiment, in operation S210, the distorted camera coordinate calculation unit 110 may calculate a plurality of curved surface parameter estimates and a previously stored transparent material curved surface model, in addition to the transparent material transmitted image including any observation point located on an arbitrary observation plane, But it should be understood that the present invention is not limited thereto.

For example, in step S210, the distortion camera coordinate calculation unit 110 calculates the distortion of the camera based on the transparent material transmission image, which is an imaging image including an arbitrary observation point positioned on an arbitrary observation plane photographed through transparent material, Which is a camera coordinate for an arbitrary observation point based on the center of the viewpoint camera.

)를 산출할 수 있다.For example, in step S210, the distorted camera coordinate calculation unit 110 calculates a light refraction path passing through the transparent material from the center C of the camera to an arbitrary observation point W located on an arbitrary observation plane P The camera position of the distortion observation point camera, which is the relative camera coordinate of the arbitrary observation point W with respect to the center C of the camera

) Can be calculated.

Now, with reference to FIG. 3, FIG. 4, FIG. 9A and FIG. 9B simultaneously, step S210 will be described in more detail.

As shown in FIG. 3, step S210 may include a light refraction path calculation step S211 and a distortion observation point camera coordinate calculation step S212 from the camera to an arbitrary observation point.

Step S211 may be a step of calculating a light refraction path from the center of the camera to the arbitrary observation point based on a previously stored transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) of the transparent material have.

The step S211 is a step of calculating, based on the above-described transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) of the transparent material, in addition to the estimated values of a plurality of curved surface parameters, , But the present invention is not limited thereto.

Step S212 may mean calculating the distortion observation point camera coordinates based on an intersection between the optical refraction path and an arbitrary observation plane.

Now, the steps S211 and S212 will be described in more detail through the steps S215, S216, S217 and S218 shown in FIG.

As shown in FIG. 4, step S210 includes steps of calculating intersection intersection camera coordinates S215, incidence intersection reference refraction intersection three-dimensional coordinate calculation step S216, refraction intersection reference observation point three-dimensional coordinate calculation step S217, View camera coordinate calculation step S218 based on the vector sum of the intersection camera coordinates, the incidence intersection reference refraction intersection three-dimensional coordinates, and the refraction intersection reference observation point three-dimensional coordinates.

)는 카메라의 중심(C)을 기준으로 하는 투명 소재의 제1 면과 광 경로의 교점인 입사 교점(X)에 대한 카메라 좌표인 입사 교점 카메라 좌표(

), 입사 교점(X)을 기준으로 하는 투명 소재의 제2 면과 광 경로의 교점인 굴절 교점(Y)에 대한 3차원 좌표인 입사 교점 기준 굴절 교점 3차원 좌표(

) 및 굴절 교점(Y)을 기준으로 하는 임의의 관찰점(W)에 대한 3차원 좌표인 굴절 교점 기준 관찰점 3차원 좌표(

)의 벡터합을 통해 산출될 수 있다.As shown in FIG. 9B, the distortion observation point camera coordinates ((X, Y, Z)), which is a camera coordinate indicating an arbitrary observation point W positioned on an arbitrary observation plane P by transmitting a transparent material with respect to the center C of the camera

Is an intersection camera coordinate (x, y), which is the camera coordinate of the incident intersection X, which is the intersection of the first surface of the transparent material and the optical path with respect to the center C of the camera

), A three-dimensional coordinate of the intersection of the second surface of the transparent material with respect to the incident intersection X and the refractive intersection Y that is the intersection of the optical path and the three-dimensional coordinates of the incident intersection reference intersection

) And refraction intersection reference observation point three-dimensional coordinates (three-dimensional coordinates of an arbitrary observation point W based on the refraction intersection Y)

). ≪ / RTI >

)는 S211단계의 산출 대상이 되는 카메라로부터 임의의 관찰점에 이르는 광 굴절 경로를 의미할 수 있으며, 그 결과 S211 단계의 광 굴절 경로는 상술한, 입사 교점 카메라 좌표(

), 입사 교점 기준 굴절 교점 3차원 좌표(

) 및 교점 기준 관찰점 3차원 좌표(

)의 벡터합을 의미하게 된다.In addition, the above-mentioned distortion observation camera coordinates (

) May mean a light refraction path from the camera to be calculated to an arbitrary observation point in step S211. As a result, the light refraction path in step S211 is the incident intersection camera coordinates

), Incidence intersection reference refraction intersection three-dimensional coordinates (

) And intersection reference observation point 3-dimensional coordinates (

). ≪ / RTI >

)는 상술한 입사 교점 카메라 좌표(

), 입사 교점 기준 굴절 교점 3차원 좌표(

) 및 굴절 교점 기준 관찰점 3차원 좌표(

)를 산출함으로써 산출될 수 있다.That is, in step S210, the distortion observation camera coordinates

) Is the intersection of the incident intersection camera coordinates (

), Incidence intersection reference refraction intersection three-dimensional coordinates (

) And refraction intersection reference observation point three-dimensional coordinates (

). ≪ / RTI >

)를 산출하는 단계를 의미할 수 있다.For example, in step S215 shown in FIG. 4, the incident intersection camera coordinates (

). ≪ / RTI >

)에 기초하여 카메라의 중심(C)을 기준으로 하는 입사 교점(X)에 대한 카메라 좌표인 입사 교점 카메라 좌표(

)를 산출하는 단계를 의미할 수 있다.In step S215, the distance r between the center C of the camera and the first intersection of the optical path and the incident intersection X of the transparent material and the incident direction X from the center C of the camera to the incident intersection X vector(

), Which is a camera coordinate for an incident intersection X based on the center C of the camera,

). ≪ / RTI >

)를 산출하는 단계를 의미할 수 있다.For example, in step S216 shown in FIG. 4, the incidence intersection reference refraction intersection three-dimensional coordinates (

). ≪ / RTI >

)에 기초하여 입사 교점(X)을 기준으로 하는 투명 소재의 제2 면과 광 경로의 교점인 굴절 교점(Y)에 대한 3차원 좌표인 입사 교점 기준 굴절 교점 3차원 좌표(

)를 산출하는 단계를 의미할 수 있다.In step S216, the thickness d _o of the transparent material, the internal refraction angle j, and the internal refraction direction vector

Dimensional coordinate of the intersection of the intersection of the second surface of the transparent material and the optical path on the basis of the incident intersection X based on the three-

). ≪ / RTI >

)를 산출하는 단계를 의미할 수 있다.For example, in step S217 shown in FIG. 4, the three-dimensional coordinates of the refraction intersection reference viewing point

). ≪ / RTI >

)와 입사 교점 카메라 좌표(

)의 비례 관계를 나타내는 비례 계수(λ)에 기초하여 굴절 교점 기준 관찰점 3차원 좌표(

)를 산출하는 단계를 의미할 수 있다.In step S217, the three-dimensional coordinates of the arbitrary observation point W with reference to the refraction intersection Y as refraction intersection reference observation point three-dimensional coordinates

) And intersection camera coordinates (

) Based on the proportional coefficient (?) Indicating the proportional relationship of the refraction intersection reference observation point 3-dimensional coordinates

). ≪ / RTI >

), 입사 교점 기준 굴절 교점 3차원 좌표(

) 및 굴절 교점 기준 관찰점 3차원 좌표(

)의 벡터합에 기초하여 왜곡 관찰점 카메라 좌표를 산출하는 단계를 의미할 수 있다.For example, in step S218 shown in FIG. 4, the incident intersection camera coordinates (

), Incidence intersection reference refraction intersection three-dimensional coordinates (

) And refraction intersection reference observation point three-dimensional coordinates (

) Of the distortion-observation-point camera coordinates.

At this time, the vector sum calculated in step S218 means the intersection between the optical refraction path in step S212 and an arbitrary observation plane.

The process of calculating the above-described distortion observation point camera coordinates is expressed by Equation (6) below.

That is, the distortion observation point camera coordinates can be calculated based on Equation (6) below.

&Quot; (6) "

및

는 왜곡 관찰점 카메라 좌표,

는 구면 좌표계의 인클라이네이션(inclination) 각도 좌표,

는 구면 좌표계의 아지무스(azimuth) 각도 좌표, S는 복수의 곡면 파라미터 추정값을 포함하는 곡면 파라미터 추정값 집합, λ는 비례 계수,

는 카메라의 중심(C)을 기준으로 하는 투명 소재의 제1 면과 광 경로의 교점인 입사 교점(X)에 대한 카메라 좌표인 입사 교점 카메라 좌표, d₀는 투명 소재의 두께, j는 내부 굴절각,

는 내부 굴절 방향 벡터를 의미한다.

And

Distortion observation point Camera coordinates,

Is the inclination angle coordinate of the spherical coordinate system,

Is an azimuth angle coordinate of a spherical coordinate system, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,? Is a proportional coefficient,

Is incident in the camera coordinates of the first surface and the incident point of intersection (X) intersecting point of the optical path of transparent material which on the basis of the center (C) of the camera intersection camera coordinates, d ₀ is the thickness of the transparent material, j is the internal refraction angle ,

Denotes an internal refraction direction vector.

Now, the steps S215, S216, S217, and S218 will be described with reference to FIG. 9A and the above-described Equation 6.

)는

로 표현될 수도 있으며, 이는, 왜곡 관찰점 카메라 좌표가 구면 좌표계의 인클라이네이션(inclination) 각도 좌표(

) 및 구면 좌표계의 아지무스(azimuth) 각도 좌표(

)의 2변수 함수로 표현될 수 있음을 의미한다.As expressed in Equation (6), the distortion observation point camera coordinates (

)

This is because the distortion observation point camera coordinates may be expressed by the inclination angle coordinates of the spherical coordinate system (

) And the azimuth angle coordinate of the spherical coordinate system (

) Can be expressed as a two-variable function.

)는 곡면 파라미터 추정값 집합(S)에 포함된 제1 곡면 파라미터(a_x), 제2 곡면 파라미터(a_y), 제3 곡면 파라미터(a_z), 제4 곡면 파라미터(a_r), 제1 곡면 파라미터 행렬(A) 및 제2 곡면 파라미터 행렬(B) 각각의 파라미터 값에 따라 의미하는 좌표 지점이 달라지는 일반화된 카메라 좌표식을 의미한다.In addition, the distortion observation point camera coordinates (

(A _x ), a second curved surface parameter (a _y ), a third curved surface parameter (a _z ), a fourth curved surface parameter (a _r ) included in the set of curved surface parameter estimation values S, Means a generalized camera coordinate expression in which the coordinate points that are determined according to the parameter values of the curved surface parameter matrix A and the second curved surface parameter matrix B vary.

)는 수학식 6 상에서 1×

의 형태로 표현되었다.At this time, the incident intersection camera coordinates (

) Is expressed by 1 x

In this case.

)는 카메라의 중심(C)과 투명 소재의 제1 면과 광 경로의 교점인 입사 교점(X)간의 거리(r) 및 카메라의 중심(C)으로부터 입사 교점(X)으로 향하는 입사방향벡터(

)의 곱으로 표현될 수 있으며, 이를 수학식으로 표현하면 아래 수학식 7과 같다.As shown in FIG. 9A, the incident intersection camera coordinates (

Is defined by the distance r between the center C of the camera and the first face of the transparent material and the incident intersection X that is the intersection of the optical path and the incident direction vector from the center C of the camera to the incident intersection X

), And this can be expressed by the following equation (7). &Quot; (7) "

&Quot; (7) "

는 입사 교점 카메라 좌표, r은 카메라의 중심(C)과 입사 교점(X)간의 거리,

는 카메라의 중심(C)으로부터 입사 교점(X)으로 향하는 입사방향벡터를 의미한다.here,

R is the distance between the center of the camera (C) and the incident intersection (X)

Means an incident direction vector directed from the center C of the camera to the incident intersection X.

)는 수학식 6 상에서

의 형태로 표현되었으며, 도 9a에 도시된 바와 같이, d₀는 투명 소재의 두께를 의미하고, 내부 굴절각(j)는 입사 교점(X)와 굴절 교점(Y)를 연결하는 직선과 굴절 교점(Y)의 법선 간의 사이각을 의미하고, 내부 굴절 방향 벡터(

)는 입사 교점 기준 굴절 교점 3차원 좌표(

)의 방향 벡터를 의미한다.In addition, the incident intersection reference refraction intersection three-dimensional coordinates (

) ≪ RTI ID = 0.0 >

9A, d ₀ denotes the thickness of the transparent material, and the internal refraction angle j is a sum of a straight line connecting the incident intersection X and the refraction intersection Y, Y), and the internal refractive direction vector (

) Is a three-dimensional coordinate

).

At this time, the internal refraction angle j can be determined by Snell's law based on the refractive index n _w of the transparent material, and the internal refraction angle j can be determined based on Equation 8 below.

&Quot; (8) "

Where i is the incident angle, n _w is the refractive index of the transparent material, j is the internal refraction angle, and f is the external refraction angle.

9A, the incident angle i means an angle between a straight line connecting the center C of the camera and the incident intersection X and a normal line of the incident intersection X, (f) can mean the angle between the straight line connecting the refraction point Y and the arbitrary observation point W and the normal line of the refraction point Y.

Here, the incident angle (i) can be calculated based on the following equation (9).

&Quot; (9) "

는 카메라의 중심(C)으로부터 입사 교점(X)으로 향하는 입사방향벡터,

는 입사 교점(X)에서의 법선 벡터를 의미한다.In this case, i is an incident angle,

The incident direction vector from the center C of the camera to the incident intersection X,

Denotes a normal vector at the incident intersection X.

)를 산출하기 위한 내부 굴절 방향 벡터(

)는 아래 수학식 10에 기초하여 산출될 수 있다.On the other hand, in step S216, the three-dimensional coordinates

The internal refractive direction vector < RTI ID = 0.0 >

) Can be calculated based on Equation (10) below.

&Quot; (10) "

은 내부 굴절 방향 벡터, n_w는 투명 소재 굴절률,

는 카메라의 중심(C)으로부터 입사 교점(X)으로 향하는 입사방향벡터, j는 내부 굴절각, i는 입사각,

는 입사 교점(X)에서의 법선 벡터를 의미한다.here,

Is an internal refractive index vector, n _w is a refractive index of a transparent material,

Is an incidence direction vector directed from the center C of the camera to the incidence intersection X, j is an internal refraction angle, i is an incidence angle,

Denotes a normal vector at the incident intersection X.

는 아래 수학식 11에 기초하여 산출될 수 있다.At this time, the normal vector at the incident intersection X commonly used in the equations (9) and (10)

Can be calculated based on Equation (11) below.

&Quot; (11) "

는 입사 교점(X)에서의 법선 벡터, r은 카메라의 중심(C)과 입사 교점(X)간의 거리,

는 투명 소재 곡면 모델을 의미한다.

Is the normal vector at the incidence intersection X, r is the distance between the center C of the camera and the incidence intersection X,

Means a transparent material surface model.

)이 왜곡 관찰점 카메라 좌표를 정의하기 위해 활용되기 때문에, 왜곡 관찰점 카메라 좌표(

)는 곡면 파라미터 추정값 집합(S)에 포함된 제1 곡면 파라미터(a_x), 제2 곡면 파라미터(a_y), 제3 곡면 파라미터(a_z), 제4 곡면 파라미터(a_r), 제1 곡면 파라미터 행렬(A) 및 제2 곡면 파라미터 행렬(B) 각각의 파라미터 값에 따라 의미하는 좌표 지점이 달라지는 일반화된 카메라 좌표식으로 산출될 수 있게 된다.That is, in Expression (11), the transparent material surface model

) Is utilized to define the distortion observation point camera coordinates, the distortion observation point camera coordinates (

(A _x ), a second curved surface parameter (a _y ), a third curved surface parameter (a _z ), a fourth curved surface parameter (a _r ) included in the set of curved surface parameter estimation values S, It can be calculated by a generalized camera coordinate expression in which the coordinate points that are different according to the parameter values of the curved surface parameter matrix A and the second curved surface parameter matrix B are changed.

)는 수학식 6 상에서 λ×

의 형태로 표현되었다.On the other hand, the three-dimensional coordinates of the refraction intersection reference viewing point calculated in step S217

) Is expressed as < EMI ID =

In this case.

)와 굴절 교점 기준 관찰점 3차원 좌표(

)는 서로 평행하기 때문이며, 그 결과, 입사 교점 카메라 좌표(

)와 굴절 교점 기준 관찰점 3차원 좌표(

) 상호 간의 비례 계수(λ)를 산출하면, 굴절 교점 기준 관찰점 3차원 좌표(

)를 산출할 수 있게 된다.This is because the external refraction angle f and the incident angle i are the same and are on one plane,

) And refraction intersection reference observation point three-dimensional coordinates (

) Are parallel to each other, and as a result, the incident intersection camera coordinates (

) And refraction intersection reference observation point three-dimensional coordinates (

) Is calculated, the three-dimensional coordinates of the refraction intersection reference observation point (

) Can be calculated.

At this time, the proportional coefficient? Can be calculated by the following equation (12), which is a linear equation for obtaining the intersection of the observation plane P and the optical path.

&Quot; (12) "

조건을 만족하는 모든 점,

는 카메라의 중심(C)을 기준으로 한 Z의 카메라 좌표,

는 카메라의 중심(C)을 기준으로 한 굴절 교점(Y)의 카메라 좌표,

는 카메라의 중심(C)을 기준으로 한 입사 교점(X)의 카메라 좌표, λ는 비례 계수를 의미한다.W is an arbitrary observation point, P is an arbitrary observation plane, Z is

All points that satisfy the condition,

The camera coordinates of Z with reference to the center C of the camera,

The camera coordinates of the refraction intersection Y with reference to the center C of the camera,

Is the camera coordinate of the incidence intersection X with reference to the center C of the camera, and? Is a proportional coefficient.

Here, the description of the concrete method of calculating the solution of the linear equation corresponding to the equation (12) is omitted since it is known in the prior art.

), 구면 좌표계의 인클라이네이션(inclination) 각도 좌표(

) 및 구면 좌표계의 아지무스(azimuth) 각도 좌표(

)는 상술한 종래의 기술에 대한 수학식 1에 기초하여 산출될 수 있다.On the other hand, the incidence direction vector < RTI ID = 0.0 >

), Inclination angle coordinates of the spherical coordinate system (

) And the azimuth angle coordinate of the spherical coordinate system (

) Can be calculated based on Equation (1) for the conventional technique described above.

), 투명 소재의 두께(d₀), 내부 굴절각(j) 및 내부 굴절 방향 벡터(

)에 기초하여 산출될 수 있다.In other words, the distortion observation point camera coordinates are the camera coordinates for the incident intersection X, which is the intersection of the optical path and the first surface of the transparent material with respect to the center C of the camera, Camera coordinates (

), The thickness (d ₀ ) of the transparent material, the internal refraction angle (j) and the internal refraction direction vector

). ≪ / RTI >

With continued reference to FIG. 2, step S230 will now be described.

Step S230 may mean that the non-distorted camera coordinate calculation section 120 calculates the unoriented observation point camera coordinates based on the transparent material removal image including the arbitrary observation point photographed by removing the transparent material have.

More specifically, in step S230, the non-distorted camera coordinate calculation unit 120 calculates a non-distorted camera coordinate based on the transparent material removal image, which is a photographed image including an arbitrary observation point located on any photographed observation plane, And may be a step of calculating the camera coordinates of the non-distorted observation point camera, which is the camera coordinate of an arbitrary observation point based on the center of the camera.

At this time, the method of removing the transparent material in step S230 and calculating the camera coordinates of the non-distorted observation point camera as the camera coordinates with respect to any observation point can be performed based on Equation (1) for the conventional technique described above, A detailed description thereof will be omitted.

With continued reference to FIG. 2, step S250 will continue.

)와 비왜곡 관찰점 카메라 좌표(

)의 차이에 기초하여 복수의 곡면 파라미터를 결정하는 단계를 의미할 수 있다.In operation S250, the curved surface parameter determination unit 130 determines whether or not the distortion observation point camera coordinates

) And non-distortion observation point camera coordinates (

) Of the plurality of curved surface parameters.

In this case, each of the plurality of curved surface parameter estimation values may be determined as a specific curved surface parameter value through step S250.

For example, in step S250, the plurality of curved surface parameters include a condition for minimizing the difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates, And the coordinate distance between the first end-point camera coordinate and the second end-point camera coordinate, which are the camera coordinates of both end points of the segment, is equal to the length of the line segment.

For example, in step S250, the condition for minimizing the difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates and the condition for making the difference between the both ends of at least one line segment disposed in advance on the first surface of the transparent material Determining a plurality of curved surface parameters such that the camera coordinate distance between the at least one first end point and the second end point is equal to the length of at least one line segment, respectively.

For example, in step S250, a plurality of curved surface parameters may be determined based on Equation (13) below.

&Quot; (13) "

는 카메라의 중심(C)을 기준으로 하는 k번째 관찰점에 대한 비왜곡 관찰점 카메라 좌표,

는 카메라의 중심(C)을 기준으로 하는 k번째 관찰점에 대한 왜곡 관찰점 카메라 좌표,

는 k번째 관찰점의 인클라이네이션(inclination) 각도 좌표,

는 k번째 관찰점의 아지무스(azimuth) 각도 좌표,

는 카메라의 중심(C)을 기준으로 하는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제1 끝점 카메라 좌표,

는 카메라의 중심(C)을 기준으로 하는 투명 소재의 제1 면 상에 미리 배치된 i번째 선분의 제2 끝점 카메라 좌표,

는 복수의 곡면 파라미터로 표현된 i번째 선분의 제1 끝점 카메라 좌표 및 제2 끝점 카메라 좌표 간의 좌표 거리, L_i는 i번째 선분의 길이를 의미한다.S ^* is a set of curved surface parameters including a plurality of determined curved surface parameters, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,

Distortion observation point camera coordinates for the k-th observation point with respect to the center C of the camera,

Is a distortion observation point camera coordinate for the kth observation point with respect to the center C of the camera,

Is the inclination angle coordinate of the kth observation point,

Is the azimuth angle coordinate of the kth observation point,

Coordinates of the first end point camera of the i-th line segment disposed in advance on the first surface of the transparent material with reference to the center C of the camera,

Coordinates of the second end point camera of the i-th line segment arranged on the first surface of the transparent material with reference to the center C of the camera,

Is the coordinate distance between the first end camera coordinate and the second end point camera coordinate of the i-th line segment expressed by a plurality of curved surface parameters, and L _i is the length of the i-th line segment.

)와 왜곡 관찰점 카메라 좌표(

)의 차이가 최소가 되도록 하는 관계식 이외에도 추가적이 관계식이 더 필요할 수 있다.For example, in step S250, a _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, a _r is a fourth curved surface parameter, a first curved surface parameter matrix A, In order to determine all of the plurality of curved surface parameters including the curved surface parameter matrix B, the non-distorted observation point camera coordinates (

) And distortion observation point camera coordinates (

) May be minimized, but this additional relationship may be necessary.

In this case, in step S250, the curved surface parameter determination unit 130 uses the coordinates of the first end point camera coordinates and the coordinates of the second end point camera, which are camera coordinates of the end points of at least one segment segment arranged in advance on the first surface of the transparent material Thereby further generating additional relational expressions for determining a plurality of curved surface parameters.

To this end, each of the first end-point camera coordinates and the second end-point camera coordinates, which are camera coordinates with respect to both end points of at least one line segment disposed on the first surface of the transparent material, are respectively represented by a plurality of curved surface parameters on the spherical coordinate system .

)를 복수의 곡면 파라미터를 포함하도록 표현하면

와 같이 표현될 수 있으며, 마찬가지로 제2 끝점 카메라 좌표(

)를 복수의 곡면 파라미터를 포함하도록 표현하면

와 같이 표현될 수 있다.To this end, the coordinates of the first end point camera (

) Is expressed to include a plurality of curved surface parameters

And similarly, the coordinates of the second end point camera (

) Is expressed to include a plurality of curved surface parameters

Can be expressed as

및

각각은 구면 좌표계 상에서의 i 번째 선분의 제1 끝점 카메라 좌표의 방향 성분으로 상술한 수학식 1 및 종래의 기술에 의하여 산출될 수 있으며, 마찬가지로

및

각각은 구면 좌표계 상에서의 i 번째 선분의 제2 끝점 카메라 좌표의 방향 성분으로 상술한 수학식 1 및 종래의 기술에 의하여 산출될 수 있다.At this time,

And

Each can be calculated by the above-described Equation 1 and the conventional technique as the direction component of the first end point camera coordinate of the i-th line segment on the spherical coordinate system,

And

Each can be calculated by the above-described equation (1) and the conventional technique as the direction component of the second end point camera coordinate of the i-th line segment on the spherical coordinate system.

는 상술한 투명 소재 곡면 모델에 의해 산출된 복수의 곡면 파라미터로 표현된 구면 좌표계 상에서의 i 번째 선분의 제1 끝점 카메라 좌표의 거리 성분을 의미할 수 있으며, 마찬가지로

는 상술한 투명 소재 곡면 모델에 의해 산출된 복수의 곡면 파라미터로 표현된 구면 좌표계 상에서의 i 번째 선분의 제2 끝점 카메라 좌표의 거리 성분을 의미할 수 있다.Furthermore,

May mean the distance component of the coordinates of the first end point camera of the i-th line segment on the spherical coordinate system expressed by a plurality of curved surface parameters calculated by the transparent material curved surface model described above,

May refer to a distance component of a second end point camera coordinate of an i-th line segment on a spherical coordinate system represented by a plurality of curved surface parameters calculated by the transparent material curve model described above.

As a result, a relational expression can be derived such that the coordinate distance between the coordinates of the first end point camera coordinates represented by the plurality of curved surface parameters and the coordinates of the second end point camera coordinates expressed with the plurality of curved surface parameters is equal to the length of the actual line segment stored in advance.

For example, when the number of segments is insufficient to determine the plurality of curved surface parameters described above in step S250, the curved surface parameter determination unit 130 may derive a relational expression among a plurality of curved surface parameters, A plurality of curved surface parameters can be estimated by utilizing an error (Trial and Error) method.

For example, when a sufficient number of segments are used to determine all of the plurality of curved surface parameters described above in step S250, the curved surface parameter determination unit 130 may calculate the values of the plurality of curved surface parameters.

Step S270 will be described with reference to FIG.

In step S270, the transparent material curvature value calculation unit 140 may calculate the curvature value of the transparent material by applying the plurality of determined curvature parameters to the previously stored transparent curvilinear surface model.

), 제1 곡면 파라미터(a_x), 제2 곡면 파라미터(a_y), 제3 곡면 파라미터(a_z) 및 제4 곡면 파라미터(a_r)를 포함한다.As described above, the transparent material surface model includes a surface defining function (A) including a first surface parameter matrix A and a second surface parameter matrix B

, A first curved surface parameter (a _x ), a second curved surface parameter (a _y ), a third curved surface parameter (a _z ), and a fourth curved surface parameter (a _r ).

In step S250, the first curved surface parameter matrix A, the second curved surface parameter matrix B, the first curved surface parameter a _x , the second curved surface parameter a _y , the third curved surface parameter a _z , When each value of the four curved surface parameters a _r is determined to be a specific value, the transparent material curved surface value calculating unit 140 assigns a plurality of curved surface parameter values to the transparent material curved surface model of Equation (2) or (4) The curved surface value of the transparent material can be calculated.

,

상호간의 관계식을 의미할 수도 있다.Here, the curved surface value of the transparent material is calculated by the following equation: r < RTI ID = 0.0 >

,

It may mean a mutual relationship.

,

상호간의 관계식에서 특정

,

에 대한 특정 r 값을 의미할 수도 있다.Here, the curved surface value of the transparent material is calculated by the following equation: r < RTI ID = 0.0 >

,

In a mutual relationship,

,

Lt; RTI ID = 0.0 > r < / RTI >

For convenience of explanation, the above-described expression (4) will be described below, and the transparent material surface model can be defined based on the following equation (4).

&Quot; (4) "

Here, the detailed description of each variable included in Equation (4) is the same as Equation (2) and Equation (3), and a duplicate explanation will be omitted.

Although not shown in the drawing, in the transparent material curvature value calculating method according to the embodiment of the present invention, after step S270, the camera coordinate correcting unit 150 calculates the curvature radius of the transparent material transmission image captured by at least one camera (Not shown) of calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the corrected observation point camera coordinates.

)가 비왜곡 관찰점 카메라 좌표(

)와 최대한 근사하도록 결정되었기 때문에, 보정 관찰점 카메라 좌표를 산출하는 단계(미도시)에서 왜곡 관찰점 카메라 좌표(

)에 결정된 복수의 곡면 파라미터를 대입하는 경우, 보정 관찰점 카메라 좌표가 산출될 수 있다.In step S250, the plurality of curved surface parameters include the distortion observation point camera coordinates

) Is the non-distorted observation point camera coordinates (

(Not shown) in the step of calculating the corrected observation point camera coordinates, the distortion observation point camera coordinates

), The corrected observation point camera coordinates can be calculated.

That is, if the above-mentioned distortion observation point camera coordinates are calculated on the basis of the plurality of curved surface parameters determined in step S250 in the step of calculating the corrected observation observation camera coordinates (not shown), the calculated distortion observation camera coordinates are corrected Can be determined with observation point camera coordinates because the determined plurality of curved surface parameters have already been determined so that the distortion eye point camera coordinates and the anti-distortion eye point camera coordinates are as close as possible to each other.

5 and 10, in the step (not shown) of calculating the corrected observation point camera coordinates, the distortion observation point camera coordinates for any observation point photographed by one camera are corrected to obtain the correction observation point camera coordinates The method for calculating the value is described below.

5 is a flowchart for explaining a step of calculating corrected viewing point camera coordinates according to the first embodiment of the present invention.

10 is a diagram for explaining the step of calculating the corrected viewing point camera coordinates according to the first embodiment of the present invention.

As shown in Fig. 5, when at least one camera is a single camera, the step of calculating the corrected observation point camera coordinates (not shown) calculates a light refraction path from one camera to an arbitrary observation point Step S510 and calculating the corrected observation point camera coordinates (S520).

Step S510 may mean calculating a light refraction path from one camera to an arbitrary observation point located on an arbitrary observation plane based on the determined plurality of curved surface parameters.

As described above, in order to calculate the camera coordinates for an arbitrary observation point using one camera, it is assumed that the arbitrary observation point is located on an arbitrary observation plane that knows the coordinate information.

At this time, in step S510, the camera coordinate correcting unit 150 substitutes the plurality of curved surface parameters determined in step S250 in the above-described equations (6) to (12) to calculate the incident intersection X ) And the refraction cross point Y to reach any observation point W located on an arbitrary observation plane P can be calculated.

)를 산출하기 때문에, 왜곡 관찰점 카메라 좌표(

)는 복수의 곡면 파라미터의 값에 따라 의미하는 좌표 지점이 달라지는 임의의 관찰점에 대한 일반화된 카메라 좌표식인데 비하여, S510 단계에서 산출하는 광 굴절 경로는 S250 단계 및 S270 단계에서 각각 결정된 복수의 곡면 파라미터 및 곡면값을 이용하여 산출되기 때문에, 투명 소재를 제거하고 산출된 임의의 평면 관찰 평면 상에 위치한 임의의 관찰점에 대한 비왜곡 관찰점 카메라 좌표에 최대한 근사하는 광 굴절 경로로써 산출될 수 있다.The difference between step S210 and step S510 is that in step S210, the distorted observation point camera coordinates (

), The distortion observation point camera coordinates (

Is a generalized camera coordinate formula for an arbitrary observation point at which a coordinate point corresponding to a value of a plurality of curved surface parameters is changed. On the other hand, the light refraction path calculated in step S510 is a plurality of curved surfaces determined in steps S250 and S270 Can be calculated as a light refraction path that is closest to the non-distortion observation point camera coordinates for any observation point located on any of the calculated plane observation planes, by removing the transparent material and calculating it using the parameter and the surface value .

Step S520 may mean calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the intersection of the optical refraction path and the arbitrary observation plane.

For example, in step S520, the camera coordinate correction unit 150 may perform a vector sum operation on the optical refraction path, and as a result, the corrected observation point camera coordinates can be calculated.

In addition, although not shown in the drawing, a method for calculating a transparent material curve value according to an embodiment of the present invention is a method for calculating a transparent material curve value based on a coordinate relationship between a camera coordinate set in advance and a two- (Not shown) converting the point camera coordinates into the corresponding corrected two-dimensional image coordinates.

In other words, the step (not shown) of transforming the corrected two-dimensional image coordinates into the corrected two-dimensional image coordinates is performed such that the image coordinate correcting unit (not shown) corrects the corrected viewing point camera coordinates generated in step S520 based on the relationship between the camera coordinates and the two- (Not shown) of transforming the corrected two-dimensional image coordinates into two-dimensional corrected image coordinates corresponding to the observation point camera coordinates.

As shown in Fig. 10, the two-dimensional image coordinate (a) before correction can be converted into the corrected two-dimensional image coordinate (b).

For example, the relationship between the camera coordinates and the two-dimensional image coordinates used in the step of converting to the corrected two-dimensional image coordinates (not shown) may be stored in advance in the image coordinate correction unit (not shown).

For example, the relationship between the camera coordinates and the two-dimensional image coordinates used in the step of converting to the corrected two-dimensional image coordinates (not shown) may refer to a conventional technique as described in Equation 1, do.

6 and 11, in the step of calculating the corrected observation point camera coordinates (not shown), the distortion observation point camera coordinates for any observation point photographed by the two cameras are corrected to obtain the corrected observation point camera coordinates The method for calculating the value is described below.

6 is a flowchart for explaining a step of calculating corrected viewing point camera coordinates according to the second embodiment of the present invention.

11 is a diagram for explaining the step of calculating the corrected observation point camera coordinates according to the second embodiment of the present invention.

As shown in FIG. 6, when at least one camera includes the first camera and the second camera, the step of calculating the corrected viewing point camera coordinates (not shown) A first camera light refraction path calculation step S610, a second camera light refraction path calculation step S630 from the second camera to an arbitrary observation point, and an intersection point between the first camera light refraction path and the second camera light refraction path (S650) based on the corrected observation point camera coordinates.

Step S610 is a step of calculating a first camera light refraction path that is a light refraction path from the first camera to an arbitrary observation point on the basis of a plurality of first camera surface parameters, which are a plurality of curved surface parameters for the first camera can do.

For the step S610, a plurality of first camera curved surface parameters, which are a plurality of curved surface parameters for the first camera, may be required, and a plurality of first camera curved surface parameters may be used for the first camera in steps S210, S230, And performing step S270.

Further, a method of calculating the first camera photographic refraction path, which is a photorefractive path from the first camera to an arbitrary observation point, can be calculated by performing the above-described step S510 for the first camera.

Step S630 is a step of calculating a second camera light refraction path that is a light refraction path from the second camera to an arbitrary observation point based on a plurality of second camera curved surface parameters, which are a plurality of curved surface parameters for the second camera can do.

For the step S630, a plurality of second camera curved surface parameters may be required for the second camera, and a plurality of second camera curved surface parameters may be required for the second camera in steps S210, S230, S250 And performing step S270.

Further, a method of calculating the second camera photographic refraction path, which is a photorefractive path from the second camera to an arbitrary observation point, can be calculated by performing the above-described step S510 for the second camera.

Step S650 may mean calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the intersection of the first camera light refraction path and the second camera light refraction path.

At this time, since the distortion observation point camera coordinates calculated in step S650 are calculated based on the determined plurality of first camera curved surface parameters and the determined plurality of second camera curved surface parameters, the distortion observation point camera coordinates mean the corrected observation point camera coordinates .

For example, when at least one camera includes the first camera and the second camera, the step of calculating the corrected observation point camera coordinates (not shown) corrects the intersection of the first camera light refraction path and the second camera light refraction path Since the observation point camera coordinates can be calculated, the corrected observation point camera coordinates can be calculated without assuming a separate observation plane.

For example, in the case where at least one camera includes the first camera and the second camera, the distortion observation point camera coordinates include coordinates of the first camera light refraction path and the second camera light refraction path, Parameter or a camera coordinate for any observation point expressed as a plurality of second camera surface parameters.

Referring now to FIG. 11, when at least one camera includes a first camera and a second camera, a method of calculating the corrected observation point camera coordinates in step S650 will be described.

In step S650, the corrected observation point camera coordinates can be calculated based on the following equations (14) and (15).

&Quot; (14) "

&Quot; (15) "

는 제1 카메라의 중심(C₁)을 기준으로 하는 제2 카메라의 중심(C₂)의 카메라 좌표,

는 제1 카메라의 중심(C₁)을 기준으로 하는 투명 소재의 제2 면과 제1 카메라 광 굴절 경로의 교점인 제1 굴절 교점(Y₁)의 카메라 좌표, λ₁은 제1 비례 계수,

는 제1 카메라 중심(C₁)을 기준으로 하는 투명 소재의 제1 면과 제1 카메라 광 굴절 경로의 교점인 제1 입사 교점(X₁)의 카메라 좌표,

는 제2 카메라의 중심(C₂)을 기준으로 하는 투명 소재의 제2 면과 제2 카메라 광 굴절 경로의 교점인 제2 굴절 교점(Y₂)의 카메라 좌표, λ₂는 제2 비례 계수,

는 제2 카메라의 중심(C₂)을 기준으로 하는 투명 소재의 제1 면과 제2 카메라 광 굴절 경로의 교점인 제2 입사 교점(X₂)의 카메라 좌표,

는 임의의 원점(O)을 기준으로 하는 임의의 관찰점(W)의 3차원 좌표,

는 임의의 원점(O)을 기준으로 하는 제1 굴절 교점(Y₁)의 3차원 좌표,

는 임의의 원점(O)을 기준으로 하는 제2 굴절 교점(Y₂)의 3차원 좌표를 의미한다.

Is the camera coordinates of the center (C ₂ ) of the second camera relative to the center (C ₁ ) of the first camera,

Of the camera coordinate, λ ₁ of the first camera transparent second surface and the first of the first refractive intersection (Y ₁₎ the intersection of the camera light refraction path of a material on the basis of the center (C ₁₎ of a first proportional coefficient,

Camera coordinates of the first incident intersection X _{1, which} is the intersection of the first surface of the transparent material with respect to the first camera center C ₁ and the first camera light refraction path,

A second camera, the center camera coordinate, λ ₂ is a second proportion coefficient of the transparent second surface and the second of intersection of the camera photorefractive path second refractive intersection point of the material (Y ₂₎ that, based on the (C ₂₎ of,

Camera coordinates of the second incident intersection X _{2, which} is the intersection of the first surface of the transparent material with respect to the center C ₂ of the second camera and the second camera light refraction path,

Dimensional coordinates of an arbitrary observation point W with respect to an arbitrary origin O,

Dimensional coordinate of the first refractive intersection Y ₁ with respect to an arbitrary origin O,

Means a three-dimensional coordinate of a second refractive intersection Y ₂ with respect to an arbitrary origin O as a reference.

The above equations (14) and (15) can be generated through the geometric relationship of each point shown in FIG.

)는 제1 카메라의 중심(C₁)에서 시작하여 제1 굴절 교점(Y₁), 임의의 관찰점(W) 및 제2 굴절 교점(Y₂)을 거쳐 제2 카메라의 중심(C₂)으로 도달하는 경로의 벡터합과 동일하다는 의미이다.Referring to Equation (14), Equation (14) is the camera coordinate of the center (C ₂ ) of the second camera relative to the center (C ₁ ) of the first camera

) Is first refractive intersection (Y _1), through the arbitrary observation point (W) and the second refractive intersection (Y ₂₎ the center of the second camera (C _2), starting from the center (C ₁₎ of the first camera, Is the same as the vector sum of the paths that reach

)는 제1 카메라의 중심(C₁)을 기준으로 하는 투명 소재의 제1 면과 제1 카메라 광 굴절 경로의 교점인 제1 입사 교점(X₁)의 카메라 좌표(

)와 제1 입사 교점(X₁)을 기준으로 하는 투명 소재의 제2 면과 제1 카메라 광 굴절 경로의 교점인 제1 굴절 교점(Y₁)의 3차원 좌표(

)의 벡터합 연산으로 산출될 수 있다.At this time, camera coordinates of a first refracting point (Y ₁ ) intersecting the second surface of the transparent material with respect to the center (C ₁ ) of the first camera and the first camera light refraction path

Is the camera coordinate of the first incident intersection X _{1, which} is the intersection of the first surface of the transparent material with respect to the center C ₁ of the first camera and the first camera light refraction path

) And the three-dimensional coordinates of the first incident point of intersection (transparent second surface and the first point of intersection of the camera's optical path, the refractive first refractive intersection point of the material (Y ₁₎ that, based on the X ₁₎ (

) Can be calculated by a vector sum operation.

)는 제1 카메라의 중심(C₁)을 기준으로 하는 제1 입사 교점(X₁)의 카메라 좌표(

)와 평행하므로, 제1 비례 계수(λ₁)을 활용하여

으로 표현될 수 있다.In addition, three-dimensional coordinates (x, y, z) of arbitrary observation points W based on the first refractive intersection Y ₁

) Of the camera coordinates of the first camera, the first incident point of intersection (X ₁₎ which on the basis of the center (C ₁₎ of the (

), So that the first proportional coefficient? ₁ is utilized

. ≪ / RTI >

) 및 제1 입사 교점(X₁)을 기준으로 하는 제1 굴절 교점(Y₁)의 3차원 좌표(

) 각각은 제1 카메라에 대하여 상술한 수학식 6 내지 수학식 11을 적용하여 산출될 수 있다.At this time, camera coordinates (X ₁ , X ₂ ) of the first incident intersection X ₁ based on the determined first camera curved surface parameter and the center C ₁ of the first camera corresponding to the determined first curved surface value

) And three-dimensional coordinates of the first incident point of intersection (first refractive intersection (Y ₁₎ that, based on the X ₁₎ (

Can be calculated by applying Equations (6) to (11) to the first camera.

)와 제1 카메라의 중심(C₁)을 기준으로 하는 제1 입사 교점(X₁)의 카메라 좌표(

) 간의 비례 관계를 알 수가 없기 때문이다.However, the first proportional coefficient? ₁ can not be calculated even by applying Equation (12). This is because, in the case where at least one camera includes the first camera and the second camera, The three-dimensional coordinates of an arbitrary observation point W based on the first refractive intersection Y ₁

) And the camera coordinates of the first incident intersection (X ₁ ) based on the center (C ₁ ) of the first camera (

) Can not be known.

)는 제2 카메라의 중심(C₂)을 기준으로 하는 투명 소재의 제1 면과 제2 카메라 광 굴절 경로의 교점인 제2 입사 교점(X₂)의 카메라 좌표(

)와 제2 입사 교점(X₂)을 기준으로 하는 투명 소재의 제2 면과 제2 카메라 광 굴절 경로의 교점인 제2 굴절 교점(Y₂)의 3차원 좌표(

)의 벡터합 연산으로 산출될 수 있다.Likewise, camera coordinates of the second refractive intersection Y _{2, which} is the intersection of the second surface of the transparent material with respect to the center C ₂ of the second camera, and the second camera light refraction path

Is the camera coordinate of the first incidence point (X ₂ ), which is the intersection of the first surface of the transparent material with respect to the center (C ₂ ) of the second camera, and the second camera light refraction path

) And the three-dimensional coordinates of the second intersection is incident (the transparent second surface and the second intersection point of a camera bends light path intersecting point of the second refractive material (Y ₂₎ that, based on the X ₂₎ (

) Can be calculated by a vector sum operation.

)는 제2 카메라의 중심(C₂)을 기준으로 하는 제2 입사 교점(X₂)의 카메라 좌표(

)와 평행하므로, 제2 비례 계수(λ₂)을 활용하여

으로 표현될 수 있다.In addition, three-dimensional coordinates (x, y, z) of arbitrary observation point W based on the second refractive intersection Y ₂

Is the camera coordinate of the second incidence intersection X ₂ based on the center C ₂ of the second camera

), So that the second proportional coefficient [lambda] ₂ is utilized

. ≪ / RTI >

) 및 제2 입사 교점(X₂)을 기준으로 하는 제2 굴절 교점(Y₂)의 3차원 좌표(

) 각각은 제2 카메라에 대하여 상술한 수학식 6 내지 수학식 11을 적용하여 산출될 수 있다.At this time, the camera coordinates (X ₂ , X ₂ ) of the second incident intersection X ₂ based on the determined second camera curved surface parameters and the center C ₂ of the second camera corresponding to the determined second curved surface value

) Of the second refraction intersection (Y ₂ ) with respect to the second incidence point (X ₂ )

) Can be calculated by applying the above-described Equations (6) to (11) to the second camera.

)와 제2 카메라의 중심(C₂)을 기준으로 하는 제2 입사 교점(X₂)의 카메라 좌표(

) 간의 비례 관계를 알 수가 없기 때문이다.However, the second proportional coefficient? ₂ can not be calculated even by applying Equation (12). This means that, in the case where at least one camera includes the first camera and the second camera, The three-dimensional coordinates of an arbitrary observation point W based on the second refractive intersection Y ₂

) Of the second incident intersection (X ₂ ) with the center (C ₂ ) of the second camera as a reference

) Can not be known.

)는 제1 카메라의 중심(C₁)과 제2 카메라의 중심(C₂)간의 미리 설정된 위치 관계에 기초하여 산출될 수 있다.For example, in the formula (14), the camera coordinates of the center (C ₂ ) of the second camera relative to the center (C ₁ ) of the first camera

) It can be calculated based on a preset positional relationship between the first center of the camera (C ₁₎ and a second center of the camera (C _2).

)는 스테레오 카메라의 파라미터를 활용하여 산출될 수 있다.According to one embodiment, the camera coordinates of the center (C ₂ ) of the second camera relative to the center (C ₁ ) of the first camera in (14)

) Can be calculated using the parameters of the stereo camera.

In order to calculate the first proportional coefficient (λ ₁ ) and the second proportional coefficient (λ ₂ ) utilized in the expression (14), a separate relation including the first proportional coefficient λ ₁ and the second proportional coefficient λ ₂ And the relational expression is the same as the above-mentioned expression (15).

), 임의의 원점(O)을 기준으로 하는 제1 굴절 교점(Y₁)의 3차원 좌표(

), 임의의 원점(O)을 기준으로 하는 제2 굴절 교점(Y₂)의 3차원 좌표(

) 각각은 상술한 수학식 1를 비롯한 종래의 기술을 활용하여 산출할 수 있으며, 그 구체적인 설명은 생략한다.For example, the three-dimensional coordinates of an arbitrary observation point W based on an arbitrary origin O utilized in Equation (15)

), Three-dimensional coordinates of the first inflection point of intersection (Y ₁₎ which on the basis of any origin (O) (

), The three-dimensional coordinates of the second refraction intersection (Y ₂ ) with respect to an arbitrary origin (O)

Can be calculated using conventional techniques including the above-described Equation (1), and a detailed description thereof will be omitted.

)는 임의의 원점(O)을 기준으로 하는 제1 굴절 교점(Y₁)의 3차원 좌표(

) 및 제1 굴절 교점(Y₁)을 기준으로 하는 임의의 관찰점(W)의 3차원 좌표(

)의 벡터합과 같으며, 임의의 원점(O)을 기준으로 하는 임의의 관찰점(W)의 3차원 좌표(

)는 임의의 원점(O)을 기준으로 하는 제2 굴절 교점(Y₂)의 3차원 좌표(

) 및 제2 굴절 교점(Y₂)을 기준으로 하는 임의의 관찰점(W)의 3차원 좌표(

)의 벡터합과 같다는 것이다.The above-described expression (15) means that the three-dimensional coordinates of an arbitrary observation point W based on an arbitrary origin O

) Is three-dimensional coordinates of the first inflection point of intersection (Y ₁₎ which on the basis of any origin (O) (

) And an arbitrary observation point W based on the first refractive intersection Y ₁

), And the three-dimensional coordinates of an arbitrary observation point W based on an arbitrary origin O (

) Is a three-dimensional coordinate of the second refractive intersection Y ₂ based on an arbitrary origin O (

) And the arbitrary observation point W based on the second refractive intersection Y ₂

). ≪ / RTI >

In step S650, the camera coordinate correcting unit 150 corrects the first proportional coefficient? ₁ and the second proportional coefficient? ₂ based on the intersection of the first camera light refraction path and the second camera light refraction path, The distortion observation camera coordinates can be calculated by calculating the equations (14) and (15), which are linear equations, and the calculated distortion observation camera coordinates can be determined as the correction observation point camera coordinates.

), 상기 제1 카메라의 중심(C₁)을 기준으로 하는 상기 투명 소재의 제2 면과 상기 제1 카메라 광 굴절 경로의 교점인 제1 굴절 교점(Y₁)의 카메라 좌표(

), 제1 비례 계수(λ₁), 상기 제1 카메라 중심(C₁)을 기준으로 하는 상기 투명 소재의 제1 면과 상기 제1 카메라 광 굴절 경로의 교점인 제1 입사 교점(X₁)의 카메라 좌표(

), 상기 제2 카메라의 중심(C₂)을 기준으로 하는 상기 투명 소재의 제2 면과 상기 제2 카메라 광 굴절 경로의 교점인 제2 굴절 교점(Y₂)의 카메라 좌표(

), 제2 비례 계수(λ₂), 상기 제2 카메라의 중심(C₂)을 기준으로 하는 상기 투명 소재의 제1 면과 상기 제2 카메라 광 굴절 경로의 교점인 제2 입사 교점(X₂)의 카메라 좌표(

), 임의의 원점(O)을 기준으로 하는 상기 임의의 관찰점(W)의 3차원 좌표(

), 상기 임의의 원점(O)을 기준으로 하는 상기 제1 굴절 교점(Y₁)의 3차원 좌표(

) 및 상기 임의의 원점(O)을 기준으로 하는 상기 제2 굴절 교점(Y₂)의 3차원 좌표(

)에 기초하여 산출될 수 있다.In other words, the corrected viewing point camera coordinates are the camera coordinates of the center (C ₂ ) of the second camera with respect to the center (C ₁ ) of the first camera

) Camera coordinates of a first refracting point (Y ₁ ), which is an intersection of a second surface of the transparent material with respect to the center (C ₁ ) of the first camera, and the first camera light refraction path

), A first proportional coefficient (λ ₁ ), a first incident intersection (X ₁ ) intersecting the first surface of the transparent material with respect to the first camera center (C ₁ ) and the first camera light refraction path, Camera coordinates (

) Camera coordinates of a second refracting point (Y ₂ ), which is an intersection of the second surface of the transparent material and the second camera light refraction path, with respect to the center (C ₂ ) of the second camera

), And the second proportion coefficient (λ _2), the first surface of the transparent material which on the basis of the center (C ₂₎ of the second camera and the second camera photorefractive intersection of the second incident point of intersection of the path (X ₂ ) Camera coordinates (

), Three-dimensional coordinates of the arbitrary observation point W with respect to an arbitrary origin O (

), The three-dimensional coordinates of the first refraction intersection (Y ₁ ) based on the arbitrary origin (O)

) And the three-dimensional coordinates of the second refractive intersection (Y ₂ ) based on the arbitrary origin (O)

). ≪ / RTI >

In addition, although not shown in the drawings, in the method of calculating the transparent material curve value according to the embodiment of the present invention, the image coordinate correction unit (not shown) converts the corrected observation point camera coordinates generated in step S650 into camera coordinates and two- (Not shown) of transforming the corrected two-dimensional image coordinates into two-dimensional image coordinates corresponding to the corrected observation point camera coordinates based on the relationship between the corrected observation point camera coordinates.

For example, the relationship between the camera coordinates and the two-dimensional image coordinates used in the step of converting to the corrected two-dimensional image coordinates (not shown) may be stored in advance in the image coordinate correction unit (not shown).

For example, the relationship between the camera coordinates and the two-dimensional image coordinates used in the step of converting to the corrected two-dimensional image coordinates (not shown) may refer to a conventional technique as described in Equation 1, do.

For example, a plurality of first camera curved surface parameters are determined in steps S610 and S630, but a plurality of second camera curved surface parameters may not be determined.

In this case, the camera coordinate correcting unit 150 corrects the plurality of second camera curved surface parameters based on the positional relationship between the center C ₁ of the first camera and the center C ₂ of the second camera, The camera surface parameters can be estimated.

In other words, when a plurality of second camera curved surface parameters are not determined, the transparent material curved surface value calculation method according to the embodiment of the present invention includes a step of the camera coordinate correction unit 150 determining a plurality of second camera curved surface parameters (Not shown).

The step (not shown) of determining the plurality of second camera curved surface parameters includes a step (not shown) of calculating a first camera curved surface value by applying a plurality of predetermined first camera curved surface parameters to the transparent curved surface model, A step (not shown) of calculating an estimated second camera curvature value by applying a plurality of second camera curved surface parameter estimation values to a transparent material curved surface model, calculating a second camera curved surface value based on a relationship between the first camera coordinate and the second camera coordinate, (Not shown) calculating a virtual first camera curved surface value corresponding to the camera curved surface value, calculating a virtual camera curved surface value corresponding to the first camera curved surface value (Not shown) of determining a parameter estimation value as a plurality of second camera curved surface parameters.

In a step (not shown) of calculating a first camera curved surface value by applying a plurality of predetermined first camera curved surface parameters to the transparent material curved surface model, the camera coordinate correcting unit 150 corrects a plurality of first The first camera curved surface value, which is a transparent material curved surface value based on the first camera, can be calculated using the camera curved surface parameter.

At this time, a concrete method of calculating the transparent material curved surface value based on the first camera is the same as in step S270 described above, so duplicate explanation is omitted.

In the step (not shown) of calculating the estimated second camera curvature values by applying a plurality of second camera curved surface parameter estimation values to the transparent material curved surface model, the camera coordinate correction unit 150 calculates a plurality of curved surface parameters estimated for the second camera The second camera curved surface parameter estimation value can be calculated by applying the second camera curved surface parameter estimation values to the transparent material curved surface model.

Here, the estimated second camera curved surface value may mean a transparent material curved surface value based on the second camera generated based on the plurality of second uncorrected camera curved surface parameter estimation values. As a result, Value may not be determined to be a specific value but may be a form of a function that takes a plurality of second camera curved surface parameter estimation values as a variable, but the present invention is not limited thereto.

In a step (not shown) of calculating a virtual first camera curved surface value corresponding to the estimated second camera curved surface value based on the relationship between the first camera coordinates and the second camera coordinates set in advance, the camera coordinate correction unit 150 The virtual first camera curved surface value described above can be calculated by performing coordinate conversion on the estimated second camera curved surface value into coordinates based on the first camera based on the coordinate relationship between the set first camera coordinate and the second camera coordinate .

That is, the virtual first camera curved surface value is obtained by converting the estimated second camera curved surface value determined based on the plurality of second camera curved surface parameter estimation values to the curved surface value based on the first camera, .

(Not shown) of determining the plurality of second camera curved surface parameter estimates as a plurality of second camera curved surface parameters to minimize the difference between the first camera curved surface value and the virtual first camera curved surface value, The controller 150 determines a plurality of second camera curved surface parameter estimates as a plurality of second camera curved surface parameters that minimize the difference between the first camera curved surface value and the virtual first camera curved surface value.

That is, the camera coordinate correction unit 150 corrects the first camera curvature value, which is a curved surface value based on the first camera generated by coordinate conversion of the estimated second camera curvature value estimated based on the second camera, A plurality of second camera curved surface parameter estimates may be determined as a plurality of second camera curved surface parameters so that the difference of the first camera curved surface values calculated based on one camera is minimized.

)를 산출한 뒤, 제1 카메라의 중심(C₁)과 제2 카메라의 중심(C₂) 상호 간의 관계를 나타내는 제1 카메라의 중심의 위치와 제2 카메라의 중심의 위치 간의 회전(Rotation) 행렬(R), 제1 카메라의 중심의 위치와 제2 카메라의 중심의 위치 간의 이동(Translation) 벡터(T)에 기초하여, 제1 카메라의 중심(C₁)을 기준으로 하는 복수의 굴절 교점(X_k)의 카메라 좌표(

)를 변형하여 제2 카메라의 중심(C₂)을 기준으로 하는 복수의 굴절 교점(X_k)의 추정 카메라 좌표(

)를 산출하고, 제2 카메라의 중심(C₂)을 기준으로 하는 복수의 굴절 교점(X_k)의 카메라 좌표(

)를 산출한 뒤, 제2 카메라의 중심(C₂)을 기준으로 하는 복수의 굴절 교점(X_k)의 추정 카메라 좌표(

)와 제2 카메라의 중심(C₂)을 기준으로 하는 복수의 굴절 교점(X_k)의 카메라 좌표(

)의 차이가 최소가 되도록 하는 복수의 곡면 파라미터를 복수의 제2 카메라 곡면 파라미터로 결정할 수 있다.The camera coordinate correcting unit 150 corrects the camera coordinate values of the plurality of refraction intersections X _k based on the center C ₁ of the first camera based on the plurality of first camera curved surface parameters, location(

And then calculates the rotation between the center position of the first camera and the position of the center position of the second camera showing the relationship between the center C ₁ of the first camera and the center C ₂ of the second camera, A plurality of refractive intersections R 1 and R 2 based on the center C ₁ of the first camera on the basis of a matrix R and a translation vector T between a position of the center of the first camera and a position of the center of the second camera. (X _k )

) Of the second camera to estimate camera coordinates of a plurality of refraction intersections X _k based on the center C ₂ of the second camera

, And calculates the camera coordinates of a plurality of refraction intersections X _k based on the center C ₂ of the second camera

And then calculates estimated camera coordinates of a plurality of refraction intersections X _k based on the center C ₂ of the second camera

(X _k ) with respect to the center (C ₂ ) of the first camera and the center (C ₂ ) of the second camera

Can be determined as a plurality of second camera curved surface parameters.

At this time, each of the rotation matrix R, which is a matrix indicating the positional relationship between the two cameras, and the translation vector T, which is a vector indicating the positional relationship between the two cameras, Technology, and a detailed description thereof will be omitted.

In other words, if a plurality of second camera curved surface parameters are not determined in steps S610 and S630, a plurality of second camera curved surface parameters may be calculated based on the following equation (16).

&Quot; (16) "

은 복수의 제1 카메라 곡면 파라미터에 기초하여 산출된 제1 카메라의 중심(C₁)을 기준으로 하는 투명 소재의 제1 면과 제1 카메라 광 굴절 경로의 k 번째 교점인 k번째 입사 교점(X_k)의 카메라 좌표,

는 복수의 임의의 곡면 파라미터에 기초하여 산출된 제2 카메라(C₂)의 중심을 기준으로 하는 투명 소재의 제1 면과 제2 카메라 광 굴절 경로의 k 번째 교점인 k번째 입사 교점(X_k)의 카메라 좌표를 의미한다.S ₂ is a set of second camera curved surface parameters including a plurality of calculated second camera curved surface parameters, S is a set of curved surface parameter estimation values including a plurality of curved surface parameter estimates, R is a position of the center of the first camera, T is a translation vector between a position of the center of the first camera and a position of the center of the second camera, S ₁ is a rotation vector between the center of the first camera surface parameter and the first camera surface parameter, Camera surface parameter sets,

(X), which is the kth intersection of the first camera photorefractive path and the first surface of the transparent material, which is based on the center (C ₁ ) of the first camera calculated based on the plurality of first camera curved surface parameters, _k ), < / _RTI >

Is a second camera (C ₂₎ the transparent material first surface and a second of the k-th intersection point of a camera photorefractive path k th incident point of intersection (X _k of which relative to the center of the calculation based on a plurality of arbitrary curved surface parameters ) Of the camera.

While the present invention has been described in connection with what is presently considered to be practical exemplary embodiments, it is to be understood that the invention is not limited to the disclosed embodiments, but many variations and modifications may be made without departing from the scope of the present invention. It will be understood that the invention may be practiced.

10: Camera
100: Transparent material surface value calculating device
110: distortion camera coordinate calculation unit
120: Non-distorted camera coordinate calculation unit
130: Curved surface parameter determination unit
140: Transparent material curve value calculating section
150: Camera coordinate correction unit

Claims (20)

A transparent material curved surface value calculating method for calculating a curved surface value of a transparent material,
Calculating a distortion observation camera coordinate based on a transparent material transmission image including an arbitrary observation point located on an arbitrary observation plane obtained through a transparent material;
Calculating a non-distorted observation point camera coordinate based on a transparent material removal image including the arbitrary observation point photographed by removing the transparent material;
Determining a plurality of curved surface parameters based on a difference between the distortion observation point camera coordinates and the non-distortion observation point camera coordinates; And
Wherein the transparent material curvature value calculating unit includes calculating the curvature value of the transparent material by applying the determined plurality of curvature parameters to a previously stored transparent material curvature model,
Wherein the determining of the plurality of curved surface parameters comprises:
And a condition for minimizing a difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates, and a condition for setting at least one of the at least one segment corresponding to each of both end points of the at least one line segment arranged on the first surface of the transparent material Wherein the plurality of curved surface parameters are determined so as to satisfy all the conditions that the camera coordinate distance between the first end point and the second end point is equal to the length of the at least one line segment. The method according to claim 1,
Wherein the step of calculating the distortion observation point camera coordinates comprises:
Calculating a light refraction path from the center of the camera to said arbitrary observation point based on a previously stored transparent material surface model, a thickness (d ₀ ) and a refractive index (n _w ) of said transparent material; And
And calculating the distortion observation point camera coordinates based on an intersection between the optical refraction path and the arbitrary observation plane. delete The method according to claim 1,
Wherein the plurality of curved surface parameters are determined based on Equation (13) below.
&Quot; (13) "

S ^* is a set of curved surface parameters including a plurality of determined curved surface parameters, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,

Is the non-distortion observation point camera coordinate for the kth observation point,

Is the distortion observation point camera coordinates for the kth observation point,

Is the inclination angle coordinate of the kth observation point,

Is the azimuth angle coordinate of the kth observation point,

The first end camera coordinates of the i-th line segment disposed on the first surface of the transparent material,

Is the coordinate of the second end point camera of the i-th line segment arranged in advance on the first surface of the transparent material, and L _i is the length of the i-th line segment. The method according to claim 1,
Wherein the transparent material surface model is defined based on Equation (4) below.
&Quot; (4) "

r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

A _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, and a _r is a fourth curved surface parameter, and A _{m, n} is a M rows and n columns of the first surface parameter matrix, B _{m, n} mean m rows and n columns of the second surface parameter matrix. The method according to claim 1,
Further comprising calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the plurality of curved surface parameters determined by the camera coordinate correction unit and the transparent material transmission image captured by the at least one camera, How to calculate a transparent material surface value. The method according to claim 6,
If the at least one camera is a single camera,
The step of calculating the corrected observation point camera coordinates includes:
Calculating a light refraction path from the one camera to the arbitrary observation point located on the arbitrary observation plane based on the determined plurality of curved surface parameters; And
And calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on an intersection of the optical refraction path and the arbitrary observation plane. The method according to claim 6,
If the at least one camera comprises a first camera and a second camera,
The step of calculating the corrected observation point camera coordinates includes:
Calculating a first camera light refraction path that is a light refraction path from the first camera to the arbitrary observation point based on a plurality of first camera curved surface parameters that are a plurality of curved surface parameters for the first camera;
Calculating a second camera photographic refraction path that is a photorefractive path from the second camera to the arbitrary observation point based on a plurality of second camera curved surface parameters that are a plurality of curved surface parameters for the second camera;
And calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the intersection of the first camera light refraction path and the second camera light refraction path. 9. The method according to any one of claims 7 to 8,
Wherein the image coordinate correction unit further comprises a step of converting the corrected viewing point camera coordinates into corresponding corrected two-dimensional image coordinates based on a coordinate relationship between the camera coordinates and the two-dimensional image coordinate set in advance. 9. The method of claim 8,
If the plurality of second camera surface parameters are not determined,
Further comprising determining a plurality of second camera curved surface parameters,
Wherein determining the plurality of second camera curved surface parameters comprises:
Calculating a first camera curved surface value by applying a plurality of predetermined first camera curved surface parameters to the transparent curved surface model;
Applying a plurality of second camera curved surface parameter estimation values to the transparent material curved surface model to calculate an estimated second camera curved surface value;
Calculating an imaginary first camera curved surface value corresponding to the estimated second camera curved surface value based on a relationship between a first camera coordinate set and a second camera coordinate set in advance;
Determining the plurality of second camera curved surface parameter estimates as the plurality of second camera curved surface parameters to minimize the difference between the first camera curved surface value and the virtual first camera curved surface value, Method of calculating a curved surface value. A transparent material curvature value calculating device for calculating a curvature value of a transparent material,
A distortion camera coordinate calculator for calculating a distortion observation point camera coordinate based on a transparent material transmission image captured at an arbitrary observation point located on an arbitrary observation plane with a transparent material interposed therebetween;
A non-distorted camera coordinate calculation unit for calculating the coordinates of the non-distorted observation point camera based on the transparent material removal image including the arbitrary observation point photographed by removing the transparent material;
A curved surface parameter determination unit for determining a plurality of curved surface parameters based on a difference between the distortion observation camera coordinates and the non-distortion observation point camera coordinates; And
And a transparent material curvature value calculation unit for calculating the curvature value of the transparent material by applying the determined plurality of curvature parameters to a previously stored transparent material curvature model,
Wherein the curved surface parameter determination unit comprises:
And a condition for minimizing a difference between the non-distorted observation point camera coordinates and the distortion observation point camera coordinates, and a condition for setting at least one of the at least one segment corresponding to each of both end points of the at least one line segment arranged on the first surface of the transparent material Wherein the plurality of curved surface parameters are determined so as to satisfy all the conditions that the camera coordinate distance between the first end point and the second end point is equal to the length of the at least one line segment. 12. The method of claim 11,
The distortion camera coordinate calculator calculates,
Calculating a light refraction path from the center of the camera to the arbitrary observation point based on a previously stored transparent material surface model, the thickness (d ₀ ) and the refractive index (n _w ) of the transparent material, And calculates the distortion observation point camera coordinates on the basis of an intersection point between the observation plane and the observation plane. delete 12. The method of claim 11,
Wherein the plurality of curved surface parameters are determined based on Equation (13) below.
&Quot; (13) "

S ^* is a set of curved surface parameters including a plurality of determined curved surface parameters, S is a set of curved surface parameter estimates including a plurality of curved surface parameter estimates,

Is the non-distortion observation point camera coordinate for the kth observation point,

Is the distortion observation point camera coordinates for the kth observation point,

Is the inclination angle coordinate of the kth observation point,

Is the azimuth angle coordinate of the kth observation point,

The first end camera coordinates of the i-th line segment disposed on the first surface of the transparent material,

Is the coordinate of the second end point camera of the i-th line segment arranged in advance on the first surface of the transparent material, and L _i is the length of the i-th line segment. 12. The method of claim 11,
Wherein the transparent material surface model is defined based on Equation (4) below.
&Quot; (4) "

r is the distance coordinate of the spherical coordinate system,

Is the inclination angle coordinate of the spherical coordinate system,

A _x is a first curved surface parameter, a _y is a second curved surface parameter, a _z is a third curved surface parameter, and a _r is a fourth curved surface parameter, and A _{m, n} is a M rows and n columns of the first surface parameter matrix, B _{m, n} mean m rows and n columns of the second surface parameter matrix. 12. The method of claim 11,
Further comprising a camera coordinate correcting unit for calculating the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on the determined plurality of curved surface parameters and the transparent material transmission image captured with at least one camera, Value calculating device. 17. The method of claim 16,
If the at least one camera is a single camera,
Wherein the camera coordinate correction unit comprises:
Calculating a light refraction path from the one camera to the arbitrary observation point located on the arbitrary observation plane based on the determined plurality of curved surface parameters, and calculating, based on the intersection of the light refraction path and the arbitrary observation plane And calculates the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates. 17. The method of claim 16,
If the at least one camera comprises a first camera and a second camera,
Wherein the camera coordinate correction unit comprises:
A first camera light refraction path that is a light refraction path from the first camera to the arbitrary observation point on the basis of a plurality of first camera curved surface parameters that are a plurality of curved surface parameters for the first camera, Calculating a second camera light refraction path that is a light refraction path from the second camera to the arbitrary observation point based on a plurality of second camera curved surface parameters that are a plurality of curved surface parameters for the camera, And calculates the corrected observation point camera coordinates by calculating the distortion observation point camera coordinates based on an intersection of the path and the second camera light refraction path. 19. A method according to any one of claims 17 to 18,
Further comprising an image coordinate correcting unit for converting the corrected observation point camera coordinates into corresponding corrected two-dimensional image coordinates on the basis of a coordinate relationship between a preset camera coordinate and a two-dimensional image coordinate. 19. The method of claim 18,
If the plurality of second camera surface parameters are not determined,
Wherein the camera coordinate correction unit comprises:
Applying a plurality of predetermined first camera curved surface parameters to the transparent material curved surface model to calculate a first camera curved surface value and applying a plurality of second camera curved surface parameter estimates to the transparent curved surface model to obtain an estimated second camera curved surface value Calculates a virtual first camera curved surface value corresponding to the estimated second camera curved surface value based on a relationship between a first camera coordinate and a second camera coordinate set in advance, Wherein the second camera curved surface parameter estimating unit determines the plurality of second camera curved surface parameter estimates as the plurality of second camera curved surface parameters so that the difference between the first camera curved surface values of the first camera curved surface values is minimized.

Images