KR100933304B1 - An object information estimator using the single camera, a method thereof, a multimedia device and a computer device including the estimator, and a computer-readable recording medium storing a program for performing the method. - Google Patents

Abstract

An object information estimator using a single camera and a method thereof are disclosed. The method for estimating object information may include separating an image of an object from an input image input from a camera; And calculating the image of the object in three-dimensional space and calculating at least one of the position and the height of the object based on the result of the history and outputting the calculation result. There is an effect of accurately estimating the information (at least one of the height and position of the object).

Camera, object, projective

Description

An object information estimator using a single camera, a method thereof, a multimedia device and a computer device including the estimator, and a computer-readable recording medium recording a program for performing the method. apparatus including said estimator and computer apparatus, and computer-readable record medium recording program for performing said method}

The present invention relates to an object information estimating technique, and more particularly, to an object information estimator and a method for estimating object information using a single camera.

The technology of detecting object information such as the position or height of an object with only a single camera is important because of its applicability to recently received perceptual interface, smart video surveillance system, and ubiquitous computing. It is emphasized day by day.

For example, the intelligent video surveillance apparatus detects object information such as key height, location, and walking characteristics of a user even at a distance without additional equipment such as RFID and GPS, and recognizes and tracks the user based on the detected object information. Can be.

In general, object information of a user based on an image may be detected based on three-dimensional geometric information from a two-dimensional image. Therefore, in order to detect the object information, an accurate calculation process of three-dimensional coordinate values using a projection relationship between a two-dimensional image and a three-dimensional space is essential.

However, in the projective relationship defined by the projective camera model, the object information is distorted because a scale value applied when a point on the 2D coordinate system of the image is back-projected into 3D space is not defined. Can be.

The single image-based three-dimensional information reconstruction technique according to the related art is implemented by using geometric clues such as the vertical-horizontal relationship of the environmental structures existing in the image, or the vanishing point and the vanishing line.

These methods estimate the vanishing point and the vanishing line in the image using the horizontal and vertical components obtained from the image, calculate the transforms to restore them to their original positions on infinity, and then perform the restoration of the geometric information based on this.

In order to perform the above process, geometric conditions for vertical and horizontal components in a 3D space projected as an image must be acquired in advance.

Papers related to related technologies (D. Leibowitz, A. Criminisi, and A. Zisserman, "Creating Architectural Models from Images," Proc. EuroGraphics'99 vol. 18, No. 3, Sep. 1999, A. Criminisi, I Reid, and A. Zisserman, "Single View Metrology," Int'l J. Computer Vision, vol. 40, no. 2, pp. 123-148, 2000). Techniques for estimating relative heights of objects in an image are disclosed.

The relative height estimation of the object disclosed in this paper is to calculate the transform to restore the vanishing point and the coordinates of the vanishing line in the image to the original position of the three-dimensional space, and then use it to restore the affine structure of the three-dimensional space. .

And the relative height of the target object is calculated using the property that the relative ratio of the length is preserved in the space of the affine structure.

The relative height measurement method of the target object does not require a separate camera calibration process, and has an advantage of relatively simple implementation.

However, in the method of measuring the relative height of the target object, it may be difficult to accurately estimate the vanishing point and the vanishing line due to the influence of noise in the image and the limitation of the image resolution itself.

Location information of the object located on the reference plane can be obtained by using the projective relationship between the image and the reference plane. However, since this method is based on the two-dimensional projective relationship between the image and the reference plane, the three-dimensional coordinates of points that do not exist on the reference plane generally cannot be restored.

As an alternative to this, there is a method of restoring geometric information in three-dimensional space by using a plurality of cameras from the image, but the method includes the complexity of calculating the correspondence relationship between the plurality of images, the ambiguity generated when calculating the correspondence point in the three-dimensional space, and Difficulties in restoring the geometric information may occur due to uneven image quality and characteristics.

In addition, the method has a difficulty in correcting all the plurality of cameras accurately.

Accordingly, the technical problem to be achieved by the present invention is to provide an object information estimator and a method for estimating object information using a single camera.

In addition, the technical problem to be achieved by the present invention is to provide an object information estimator and a method for correcting the error of the object information caused by inaccurate camera calibration.

The object information estimation method for achieving the technical problem comprises the steps of separating the image of the object from the input image input from the camera; And inverting the image of the object in a three-dimensional space and calculating at least one of a position and a height of the object based on the inverted result and outputting a calculation result.

The method for estimating object information may further include calibrating the camera before separating an image of the object.

(여기서, 상기 K는 상기 정칙행렬, 상기 f_x는 상기 입력 영상의 x축 방향의 스케일 값, 상기 f_y는 상기 입력 영상의 y축 방향의 스케일 값, 상기 s는 상기 입력 영상에 대한 비틀림 파라미터(skew parameter), 및 상기 (x₀, y₀)는 상기 입력 영상의 주점(principal point))를 계산하는 단계를 포함 할 수 있다.The calibrating the camera may be a regular matrix corresponding to a camera calibration matrix having an internal parameter of the camera as an element.

Where K is the regular matrix, f _x is the scale value in the x-axis direction of the input image, f _y is the scale value in the y-axis direction of the input image, and s is a torsion parameter for the input image. (skew parameter) and (x ₀ , y ₀ ) may include calculating a principal point of the input image.

The separating of the image of the object may include: generating a difference image corresponding to a difference between an image including the object and a reference image not including the object; And binarizing the difference image to output the binarized image as an image of a separated object.

The outputting of the binarized image as an image of the separated object may include comparing the pixel values of the difference image with a reference value and outputting the binarized image based on a comparison result. .

Calculating at least one of the position and the height of the object includes generating an inverted ray corresponding to the inverted result,

,

(여기서, 상기 L은 상기 역사영 광선, 상기 P는 N*N(N은 자연수)부분행렬, 상기

는 상기 카메라의 사영행렬로서 상기

, 상기 λ는 상기

에 대한 스케일 변수, 상기 C는 상기 카메라의 중심 좌표로서 상기

, 상기

는 상기 L과 상기 객체의 영상이 교차하는 어느 하나의 교점, 및 상기 D는 상기 역사영 광선에 대한 무한 원점)와 상응하고,The history zero rays,

,

Where L is the inverse light ray, P is N * N (N is a natural number) partial matrix, and

Is a projection matrix of the camera

, Wherein λ is

For the scale variable, wherein C is the center coordinate of the camera

, remind

Is an intersection of the L and the image of the object, and D is an infinite origin for the inverse light ray),

At least one of the position and the height of the object may be calculated based on the inverse light ray.

The inversion beam may correspond to at least one straight line.

(여기서, 상기 M₀ The position of the object corresponds to a first intersection point at which the inverse light ray and the reference plane in the three-dimensional space intersect, and the first intersection point is

Where M ₀

는 상기 L과 상기 객체의 영상이 교차하는 어느 하나의 교점)일 수 있다.Is the first intersection point, λ ₀ is a scale variable for M ₀ , and

May be any one intersection point at which L and the image of the object intersect each other.

(여기서, 상기 π₀는 상기 기준면으로서 상기

는 상기 π₀상의 좌표들 중에서 상기

과 대응되는 좌표)일 수 있다.remind

(Wherein π ₀ is the reference plane

Is among the coordinates on the π ₀

And coordinates corresponding to "

The height of the object corresponds to a distance between the position of the object and the height point of the object, wherein the height point may intersect the straight line perpendicular to the reference plane in the three-dimensional space and the inverse zero ray.

(여기서, 상기

는 상기 기준면에 수직인 직선, 상기

는 상기 역사영 광선과 상기 3차원 공간상의 기준면이 교차하는 교점, 상기 D_v는 상기

에 대한 무한 원점, 상기 μ는 M_h(단, M_h는 상기

와 상기 L과의 교점)에 대한 스케일 변수)와 상응할 수 있다.The straight line perpendicular to the reference plane is

Where

Is a straight line perpendicular to the reference plane, the

Is an intersection point at which the historical zero ray and the reference plane in the three-dimensional space intersect, and D _v is the

For the infinite origin, where μ is M _h , where M _h is

And an intersection point with L).

The method for estimating object information may include detecting a first homography between the reference plane and the ideal reference plane based on a reference plane and an ideal reference plane inverted on the three-dimensional space; And correcting at least one of the position and the height based on the first homography, calculating at least one of the corrected position and the corrected height, and outputting a calculation result.

(여기서, 상기

는 상기 교정된 위치, 및 상기

는 상기 제1 호모그래피, 및 상기

는 교정되기 전의 위치)일 수 있다.The corrected position is,

Where

Is the calibrated position, and

Is the first homography, and

May be a position before calibration).

를 지나고 상기 이상적인 기준면에 수직인 직선일 수 있다.The calibrated height corresponds to a distance between the calibrated position and a calibrated height point of the object, wherein the calibrated height point corresponds to a first intersection point at which a first straight line and a second straight line intersect each other. The first straight line converts a first intersection point at which the inverse light ray corresponding to the result of the image of the object being inverted on the reference plane and the reference plane to any one point on the ideal reference plane based on the first homography. The second intersection and the straight line passing through the center of the camera, the second straight line,

It may be a straight line passing through and perpendicular to the ideal reference plane.

The object information estimator for achieving the technical problem is an object image separation unit for separating the image of the object from the input image input from the camera; And an object information estimator configured to invert the image of the object in a three-dimensional space and calculate at least one of a position and a height of the object based on the inverted result and output a calculation result.

The object information estimator may further include a camera calibrator configured to calibrate the camera and output a calibrated image as the input image.

(여기서, 상기 K는 상기 정칙행 렬, 상기 f_x는 상기 입력 영상의 x축 방향의 스케일 값, 상기 f_y는 상기 입력 영상의 y축 방향의 스케일 값, 상기 s는 상기 입력 영상에 대한 비틀림 파라미터(skew parameter), 및 상기 (x₀, y₀)는 상기 입력 영상의 주점(principal point))를 계산하여 출력할 수 있다.The camera calibration unit is a regular matrix corresponding to a camera calibration matrix having an internal parameter of the camera as an element.

Where K is the regular matrix, f _x is a scale value in the x-axis direction of the input image, f _y is a scale value in the y-axis direction of the input image, and s is a twist with respect to the input image A skew parameter and (x ₀ , y ₀ ) may be calculated by outputting a principal point of the input image.

The object image separator may include: a difference image generator configured to generate a difference image corresponding to a difference between an image including the object and a reference image not including the object; And an object image output unit configured to binarize the difference image to output the binarized image as an image of a separated object.

The object image output unit may compare the pixel values constituting the difference image with a reference value and output the binarized image based on a comparison result.

,

(여기서, 상기 L은 상기 역사영 광선, 상기 P는 N*N(N은 자연수)부분행렬, 상기

는 상기 카메라의 사영행렬로서 상기

, 상기 λ는 상기

에 대한 스케일 변수, 상기 C는 상기 카메라의 중심 좌표로서 상기

, 상기

는 상기 L과 상기 객체의 영상이 교차하는 어느 하나의 교점, 및 상기 D는 상기 역사영 광선에 대한 무한 원점)와 상응하고,The object information estimator generates an inverse light ray corresponding to the inversed light, and the inverse light is

,

Where L is the inverse light ray, P is N * N (N is a natural number) partial matrix, and

Is a projection matrix of the camera

, Wherein λ is

For the scale variable, wherein C is the center coordinate of the camera

, remind

Is an intersection of the L and the image of the object, and D is an infinite origin for the inverse light ray),

The object information estimating unit may calculate at least one of the position and the height of the object based on the historical zero ray and output a calculation result.

The inversion beam may correspond to at least one straight line.

(여기서, 상기 M₀는 상기 제1 교점, 상기 λ₀는 상기 M₀에 대한 스케일 변수, 및 상기

는 상기 L과 상기 객체의 영상이 교차하는 어느 하나의 교점)일 수 있다.The position of the object corresponds to a first intersection point at which the inverse light ray and the reference plane in the three-dimensional space intersect, and the first intersection point is

(Wherein M ₀ is the first intersection point, λ ₀ is a scale variable for M ₀ , and

May be any one intersection point at which L and the image of the object intersect each other.

(여기서, 상기 π₀는 상기 기준면으로서 상기

는 상기 π₀상의 좌표들 중에서 상기

과 대응되는 좌표)일 수 있다.remind

(Wherein π ₀ is the reference plane

Is among the coordinates on the π ₀

And coordinates corresponding to "

The height of the object corresponds to a distance between the position of the object and the height point of the object, wherein the height point is a second intersection point at which the straight line perpendicular to the reference plane in the three-dimensional space and the inverse zero ray intersect. May correspond to

(여기서, 상기

는 상기 기준면에 수직인 직선, 상기

는 상기 호모그래피와 상기 3차원 공간상의 기준면이 교차하는 교점, 상기 D_v는 상기

에 대한 무한 원점, 상기 μ는 M_h(단, M_h는 상기

와 상기 L과의 교점)에 대한 스케일 변수)와 상응할 수 있다.The straight line perpendicular to the reference plane,

Where

Is a straight line perpendicular to the reference plane, the

Is an intersection where the homography and the reference plane in the three-dimensional space intersect, and D _v is the

For the infinite origin, where μ is M _h , where M _h is

And an intersection point with L).

The object information estimating unit detects a first homography between the reference plane and the ideal reference plane based on the reference plane and the ideal reference plane inverted on the three-dimensional space and based on the detected first homography, the position and the At least one of the heights may be calibrated and at least one of the calibrated position and the calibrated height may be calculated and the calculation result may be output.

The object information estimator may be implemented in a multimedia device.

The object information estimator may be implemented in a computer device.

As described above, the object information estimator and the method according to the present invention have the effect of estimating the information (at least one of the height and the position of the object) of the object using only a single camera.

 The object information estimator and the method according to the present invention have the effect of estimating the information of the object more accurately by correcting the error of the object information caused by the incorrect camera calibration.

In addition, the object information estimator and method according to the present invention has an effect that can be used to provide accurate object information by using a single camera-based monitoring, control, or security system.

In order to fully understand the present invention, the operational advantages of the present invention, and the objects achieved by the practice of the present invention, reference should be made to the accompanying drawings which illustrate preferred embodiments of the present invention and the contents described in the accompanying drawings. Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements.

1 is a diagram illustrating an object information estimator according to an embodiment of the present invention, FIG. 2 is a diagram for describing object information output from the object information estimator of FIG. 1, and FIG. 3 is a diagram illustrating object information of the object information estimator of FIG. 1. It is a figure for explaining the output process.

FIG. 4 is a diagram for describing a process of outputting an object image by the object image output unit of FIG. 1, and FIG. 5 is a diagram for describing a geometric relationship between a marker generated in the object image output unit of FIG. 1 and an input image. It is for the drawing. 1 to 5, the object information estimator 10 separates an image BI of an object from an input image input from the camera 5 and history the separated object image BI in three-dimensional space. Based on the zeroed and inverted results, at least one of the position LO and the height HG of the object may be calculated and a calculation result may be output.

The camera 5 may include a lens implemented as a CCD (Charge Coupled Device) image sensor or a CMOS image sensor (CIS) to photograph the subject and output the input image.

The camera 5 may be a digital camera or a video camera, and may be a camera implemented in a multimedia device such as a portable terminal, a notebook computer, a desktop computer, or a portable multimedia computer (PMP).

The object information estimator 10, which may be implemented inside the camera 5, may include a camera calibrator 12, an object image separator 14, and an object information estimator 20.

The object information estimator 10 may be implemented in a multimedia device such as a notebook computer, a PDA, a mobile phone, or a digital camera, or may be implemented in a computer device.

Each of the components 12 to 20 constituting the object information estimator 10 or the object information estimator 10 may be implemented in hardware (H / W) and / or software (S / W).

The camera calibrator 12 may output the calibrated image as the input image by calibrating the camera.

For example, the camera calibration unit 12 may calculate a regular matrix corresponding to a camera calibration matrix having an internal parameter of the camera 5 as an element based on Equation 1.

(여기서, 상기 K는 상기 정칙행렬, 상기 f_x는 상기 입력 영상의 x축 방향의 스케일 값, 상기 f_y는 상기 입력 영상의 y축 방향의 스케일 값, 상기 s는 상기 입력 영상에 대한 비틀림 파라미터(skew parameter), 및 상기 (x₀, y₀)는 상기 입력 영상의 주점(principal point)이다.)

Where K is the regular matrix, f _x is the scale value in the x-axis direction of the input image, f _y is the scale value in the y-axis direction of the input image, and s is a torsion parameter for the input image. (skew parameter), and (x ₀ , y ₀ ) are principal points of the input image.)

The camera calibration unit 12 is a camera calibration technique proposed by Zhang (Z. Zhang, "Flexible New Technique for Camera Calibration," IEEE Trans.Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 1330-1334, Nov. 2000.

Zhang's camera calibration technique is an image of absolute conic (IAC) w projected using an invariance to isometric transformation, one of the features of absolute points on an infinite plane. After calculating, the K can be obtained from the relationship of w ⁻¹ = kk ^T , and thus a detailed description thereof will be omitted.

The regular matrix K calculated by the camera calibrating unit 12 is used when the object information estimating unit 20 calculates object information (eg, at least one of the position LO and the height HG of the object). The calibration error generated by the camera 5 can be compensated, which will be described later.

The camera calibrator 12 may output a reference image RI (FIG. 3A) as a basis for generating the difference image DI by the difference image generator 16.

The reference image RI is an image in which the object BI is not included among the input images input through the camera 5, and the difference image generating unit 16 includes an image in which the object BI is included (FIG. b), a differential image DI corresponding to a difference between the SI) and the reference image RI that does not include the object BI may be generated (c) of FIG. 3.

In addition, the camera calibrator 12 generates a marker MK on the three-dimensional space and serves as a reference for detecting the object information among a plurality of planes constituting the three-dimensional space. It may be disposed on the reference plane π ₀ .

The marker MK may be a coordinate system or a direction of the coordinate system which is a reference for setting a reference coordinate system on the reference plane π ₀ or calculating an error correction matrix of a camera.

The object image separator 14 separates an image of an object from an input image input from the camera 5.

The object image separator 14 may include a difference image generator 16 and an object image output unit 18. The difference image generator 16 may generate a difference image DI corresponding to a difference between the image SI including the object BI and the reference image RI without the object BI.

The object image output unit 18 may binarize the difference image DI to output the binarized image CI as an image of a separated object.

In more detail, the object image output unit 18 may compare the pixel values constituting the difference image DI with a reference value and output the binary image CI based on a comparison result.

For example, the object image output unit 18 may output the binarized image CI based on Equation 2.

는 차분 영상(DI)을 구성하는 픽 셀들 중에서 상기 I_D(x,y)에 대응되는 픽셀의 값으로, 상기 I_R(x,y)는 객체(BI)가 포함된 영상(SI)에서의 픽셀 값이고, I_Ci(x,y)는 상기 객체(BI)가 포함되지 않은 기준영상(RI)에서의 픽셀 값이고, 상기 R_th는 미리 정해진 기준 값이다.)Here, I _D (x, y) is any one pixel among pixels constituting the binarized image CI,

Is a value of a pixel corresponding to the I _D (x, y) among the pixels constituting the differential image DI, and the I _R (x, y) is an image SI including the object BI. A pixel value, I _Ci (x, y) is a pixel value in the reference image RI without the object BI, and R _th is a predetermined reference value.)

)이 상기 기준 값(Rth)보다 큰 경우 제1 픽셀 값(예컨대, 255)을 상기 픽셀의 값으로서 출력하고, 상기 대응되는 픽셀 값(

)이 상기 기준 값(Rth)보다 작은 경우 제2 픽셀 값(예컨대, 0)을 상기 픽셀의 값으로서 출력할 수 있다.That is, the object image output unit 18 may have a corresponding pixel value among pixels constituting the difference image DI.

Is greater than the reference value Rth, a first pixel value (eg, 255) is output as the pixel value, and the corresponding pixel value (

) Is smaller than the reference value Rth, a second pixel value (eg, 0) may be output as the pixel value.

In addition, the object image output unit 18 may obtain a final object image after performing a morphology operation on the binarized image CI.

The object image output unit 18 may remove an area of a pixel having a predetermined size or less from among pixels constituting the image of the object extracted by the morphology operation as noise.

, 제2 좌표는

)로 각각 계산될 수 있다.For example, the binarized image CI output by the object image output unit 18 includes a binarized object CK, as shown in FIG. 4, and coordinate values of the binarized object for information estimation of the object CK. Are coordinates obtained by dividing the horizontal width (w) and the vertical width (h) of the object area by two (eg, the first coordinate is

, The second coordinate is

Can be calculated respectively.

The object information estimator 20 inverts the image CI of the binary object output from the object image output unit 18 in a three-dimensional space, and positions the object CK based on the result of the inversion. At least one of LO) and height HG may be calculated and a calculation result may be output.

The object information estimating unit 20 may generate an inverted ray of light as shown in Equation 3 corresponding to the result of the inversion of the object, and the object information estimator 20 may generate the object based on the inversion ray of light. At least one of the position LO and the height HG of CK may be calculated and a calculation result may be output.

, (여기서, 상기 L은 상기 역사영 광선, 상기 P는 N*N(N은 자연수)부분행렬, 상기 는 상기 카메라의 사영행렬로서 상기 , 상기 λ는 상기 에 대한 스케일 변수, 상기 C는 상기 카메라의 중심 좌표로서 상기 , 상기 는 상기 L과 상기 객체(CK)의 영상이 교차하는 어느 하나의 교점, 및 상기 D는 상기 역사영 광선에 대한 무한 원점)와 상응할 수 있다.

Where L is the inverse light beam, P is an N * N (N is a natural number) submatrix, is a projection matrix of the camera, λ is a scale variable for the C, and C is the camera As the center coordinates of the, may be any one intersection point of the L and the image of the object (CK), and the D may correspond to the infinite origin for the inverse light ray.

When the object information estimator 20 inverts the image CI of the object output from the object image output unit 18 in a three-dimensional space in detail as follows.

,

라고 할 때, 상기

과 상기

의 사영 관계는 p*q(예컨대, 3 X 4) 카메라 행렬에 의해서 다음의 수학식 4와 같이 정의될 수 있다.Homogeneous coordinate system coordinates for one point M = (X, Y, Z) and one point m = (x, y) in the three-dimensional space, respectively

,

When said,

And said

The projective relationship of may be defined as Equation 4 by p * q (eg, 3 × 4) camera matrix.

는 사영행렬이고, 상기 λ는 상기

에 대한 스케일 변수이고, 상기 R은 카메라(5)의 회전변위에 의한 행렬(예컨대, 3*3 행렬)이고, 상기 r_i는 상기 R의 i번째 열(column)을 나타내고, 상기 t는 카메라(5)의 이동을 의미하는 이동 벡터 (translation vector, 예컨대, 3*1 행렬)이고, 상기 K는 카메라의 내부 파라미터(intrinsic parameter)를 원소로 갖는 카메라 교정 행렬(camera calibration matrix)을 나타내는 정칙 행렬 (non-singular matrix, 예컨대,(3*3 행 렬), 수학식 1)이고, 상기 수학식 4에서 첨자 "~ "는 동차 좌표계 좌표를 의미한다.)Where

Is a projective matrix, and λ is

Is a scale variable with respect to R, wherein R is a matrix (e.g., a 3 * 3 matrix) by rotational displacement of the camera 5, wherein r _i represents an i-th column of R, and t is a camera ( 5) is a translation vector (e.g., a 3 * 1 matrix), which means the movement of K, and K is a regular matrix representing a camera calibration matrix having an intrinsic parameter of the camera as an element. non-singular matrix, e.g., (3 * 3 matrix), Equation 1), and the subscript "~" in Equation 4 means homogeneous coordinates.)

The object information estimator 20 may map the image CI of the two-dimensional object to the reference plane π ₀ existing in the three-dimensional space according to Equation 4, and the result of the inversion is two-dimensional. It can be defined as a homography (homography) between the image and the reference plane.

를 계산할 수 있다.The object information estimator 20 is provided between the four different points existing on the reference plane π ₀ and an image CI of the object for them to generate the inverse light beam corresponding to the inversed result. Represents the history of the image BI of the two-dimensional object using homography

Can be calculated.

이 상기 객체의 영상(BI)의

로 사영되었다고 할 때,

및

로 구성된 행렬

와

사이의 사영관계는 수학식 5와 같다.For example, the reference surface is also assumed that an XY plane, and on the plane (π ₀₎ is a three-dimensional (or three-dimensional reference coordinate system), such as 4, four points on the (π ₀₎

Of the image BI of this object

When projected to

And

Matrix

Wow

The projective relationship therebetween is shown in Equation 5.

(여기서, 상기 p₁는 사영행렬( )의 각각의 열 벡터이다.)

(Where p ₁ is each column vector of the projection matrix ().)

의 좌표 지정시 상기 기준면(π₀) 상에 위치한 마커(MK)의 네 꼭지점을 이용하여 지정할 수 있다.The object information estimating unit 20

When designating the coordinates of, may be specified using the four vertices of the marker (MK) located on the reference plane (π ₀ ).

의 좌표를 상기 기준면(π₀) 상에 위치한 마커(MK)의 네 꼭지점으로 지정한 경우 상기 수학식 5는 수학식 6과 같이 정리될 수 있다.The object information estimating unit 20

When the coordinate of is specified as four vertices of the marker MK located on the reference plane π ₀ , Equation 5 may be summarized as in Equation 6.

(Where (x _i , y _i ) represents the i th vertex coordinate in the object image CK, r _ij represents the elements of the i row j columns of the rotation matrix, t _x , t _y , and t _z Each is an element of the motion vector t)

)를 이용하여 상기 r_ij 및 t를 계산하고 계산결과를 출력할 수 있다.The object information estimating unit 20 has four vertices of the equation (5) and the marker (MK) (Fig. 4).

R _ij using And t and output the calculation result.

In addition, the object information estimator 20 converts the third column vector of R from the orthogonal condition of the rotation matrix to the external r ₃ of r ₁ and r ₂ . = r ₁ xr ₂ , where r ₃ is the third column vector of R.

분해한 다음, 회전 행렬을 최적의 근사값인

로 근사화시킬 수도 있다.The object information estimating unit 20 may calculate and output a rotation matrix R through Equation 6, and when the rotation matrix R does not satisfy an orthogonal condition, the paper G. Golub and C Loan, "Matrix Computations: 3rd ed.," Johns Hopkins Univ. Press, 1996.), converts the rotation matrix (R) using SVD (singular value decomposition) to

After decomposition, the rotation matrix is approximated

It can also be approximated by

를 수학식 7과 같이 계산할 수 있다.Therefore, when the object information estimating unit 20 assumes that the length of one side of the marker MK is w _m ,

Can be calculated as in Equation 7.

를 수학식 7과 같이 계산함으로써, 3차원 기준 좌표계의 원점은 한 변의 길이가 w_m인 정사각형의 중심, 즉, 마커(MK)의 중심에 위치할 수 있으며, 상기 w_m은 역사영될 3차원 기준 좌표계에 대한 스케일을 결정짓는 역할을 할 수 있다.That is, the object information estimating unit 20

By calculating as in Equation 7, the origin of the three-dimensional reference coordinate system can be located at the center of the square, that is, the length of one side of w _m , that is, the center of the marker (MK), wherein w _m is the three-dimensional to be mapped It may serve to determine the scale with respect to the reference coordinate system.

That is, the object information estimation unit (20) has at least one point (e.g., m _h, m _0), the history spirit beams (L _1, in the three-dimensional space L which is located in the image (CI) binarization as shown in FIG. 5 ₂ ) history can be done.

As shown in FIG. 5, the inversion light rays L ₁ and L ₂ include at least one point (eg, m _h , m ₀ ) positioned at the center coordinate C of the camera 5 and the binarized image CI. Passing.

라고 정의하면, 상기 카메라(5)의 중심의 좌표 C는 다음의 수학식 8과 같다.Camera matrix using 3 * 3 submatrix P

In this case, the coordinate C of the center of the camera 5 is expressed by the following expression (8).

The object information estimator 20 may include the inverse zero rays L ₁ and L connecting the center C of the camera and each of at least one point (eg, m _h , m ₀ ) positioned in the binarized image CI. ₂₎ the history of zero light (L _1, L ₂₎ can be determined from the orientation of each of the point at infinity (point at infinity).

) 를 수학식 9를 이용하여 계산할 수 있다.In addition, the object information estimating unit 20 is at least one point on the two-dimensional plane (π ₀ ) that is mapped to the three-dimensional space is located at the center coordinate (C) and the binarized image (CI) Since it forms a straight line with (m, e.g., m _h , m ₀ ), the infinite zero point for the homography (L ₁ , L ₂ )

) Can be calculated using Equation 9.

That is, the object information estimating unit 20 uses the inverse zero light beam (L in Equation 3, for example, L ₁ and L _{2 in} FIG. 6), which is a zeroing result of the point m using Equations 8 and 9. Can be generated.

The location LO and the object of the object based on a result of the object information estimator 20 inverting the image CI of the object output from the object image output unit 18 in three-dimensional space. At least one of the height HG of the object may be detected.

The object information estimator 20 detects a second intersection M ₀ at which an inverse light ray (eg, L1 of FIG. 6) and the reference plane π ₀ on the three-dimensional space intersect and detect the detected second intersection point ( M ₀ ) may be output as the location LO of the object CK.

The object information estimator 20 may calculate the second intersection point M ₀ as shown in Equation 10 and output a calculation result.

는 상기 L과 상기 객체의 영상이 교차하는 어느 하나의 교점)Where M ₀ Is the second intersection point, λ ₀ is a scale variable for M ₀ , and

Is the intersection of any one of the L and the image of the object)

The object information estimator 20 may calculate the λ ₀ as shown in Equation 11 and output a calculation result.

는 상기 π₀상의 좌표들 중에서 상기

과 대응되는 좌표)(Wherein π ₀ is the reference plane

Is among the coordinates on the π ₀

And corresponding coordinates)

The process of the object information estimator 20 detecting the location LO of the object CK will be described in detail with reference to FIG. 6 as follows.

Said by the object information estimation unit 20 is history as a linear MoMh of three-dimensional objects (CK, a straight line m _o m _h) as shown in FIG zero and calculate the Mo on the reference plane (π ₀₎ on the basis of the history of zero results The location LO of the object CK may be detected.

For example, the object information estimating unit 20 calculates the coordinate value of the Mo (Equation 10), which is the intersection point of the _first inverse light ray L ₁ and the XY plane that is the reference plane π ₀ on the world coordinate system WC, to the object. It can output as the position LO of (CK).

If it is assumed that the first intersection point Mo is always located on the reference plane π ₀ , Equation 12 is satisfied.

Equation 11 is calculated from Equation 12, and the object information estimating unit 20 calculates λ ₀ , which is a calculation result of Equation 11, and Equation 10 from the first inverse projected light beam L ₁ and The first intersection Mo of the reference plane π ₀ can be detected as the position of the object CK.

In addition, the object information estimating unit 20 may output the distance between the position Mo of the object CK and the height point of the object CK as the height HG of the object CK. . The height point may be a second intersection point (Mh) where a straight line (Lh) perpendicular to the reference plane (π ₀ ) on the three-dimensional space and the second inversion zero light beam (L ₂ ) intersect.

For example, the object information estimator 20 may detect a straight line Lh perpendicular to the reference plane π ₀ through Equation 13.

(여기서, 상기 는 상기 기준면에 수직인 직선, 상기 는 상기 제1 역사영 광선(L1)와 상기 3차원 공간상의 기준면이 교차하는 교점, 상기 D_v는 상기 에 대한 무한 원점, 상기 μ는 M_h(여기서, M_h는 상기 와 제2 역사영 광선(L2)와의 교점)에 대한 스케일 변수, 및 첨자 "~ "는 동차 좌표계 좌표)

(Wherein, is a straight line perpendicular to the reference plane, the is the intersection point of the first inversion zero light beam L1 and the reference plane in the three-dimensional space, D _v is the infinite origin for the, μ is M _h Where M _h is the intersection of the second inverse light ray L2 and the subscript "~" is the coordinate of the homogeneous coordinate system)

The process of the object information estimator 20 detecting the height HG of the object CK will be described in detail with reference to FIG. 6 as follows.

In FIG. 6, the height HG in the three-dimensional space of the object CK is a distance between the first intersection Mo and the second intersection Mh.

The second intersection point Mh is an intersection point of a straight line Lh perpendicular to the reference plane π ₀ and the second inversion zero light beam L2, and L ₂ and L _h are each represented by Equation 14.

The object information estimating unit 20 has a second intersection point Mh which is an intersection point of the straight line Lh perpendicular to the reference plane π ₀ from the equations 14 and 15 and the second inverse light ray L2. Can be detected.

In addition, the object information estimator 20 arranges Equation 15 as shown in Equation 16, calculates values of λ and μ, and calculates the calculated λ and μ by substituting the calculated Equation 14 into an object ( CK) height HG can be calculated.

(Wherein m _i , c _i , d _hi , and d _vi are elements of the i th row of Mo, C, D _h , and D _v , respectively.)

In addition, the object information estimator 20 is based on the reference plane (eg, π in FIG. 7) and the ideal reference plane (eg, π _{0 in} FIG. 7) that is mapped on the three-dimensional space. ) Detects a first homography between the ideal reference plane (π ₀ ) and corrects at least one of the position (LO) and the height (HG) based on the detected first homography, and the corrected position And at least one of the corrected heights and output the calculation result.

For example, the object information estimator 20 may output the corrected position by calculating Equation 17.

(여기서, 상기 는 상기 교정된 위치, 및 상기 는 제1 호모그래피, 및 상기 는 교정되기 전의 위치)

(Wherein the is the corrected position, and is the first homography, and is the position before being corrected)

)를 검출하는 과정을 도 7과 도 8을 통하여 상세히 설명하면 다음과 같다.The corrected position of the object information estimator 20

) Will be described in detail with reference to FIGS. 7 and 8 as follows.

If the camera calibration is not performed correctly, serious reprojection errors may occur during the history of the object image (CI).

For example, an incorrect camera matrix may distort the reference coordinate system, which may distort the calculation result of the position and height of the object CK.

FIG. 7 is a diagram for explaining an error of object information generated due to incorrect camera calibration. Referring to FIG. 7, in FIG. 7A, π ₀ is an accurate reference plane (ie, an ideal reference plane) for the camera, and π ＇is an incorrectly detected reference plane (ie, an actually detected reference plane) due to camera calibration errors. The case where an error occurs in the relative position detection between the reference plane and the camera.

FIG. 7B is an example of a side view of FIG. 7A, and shows that serious errors may occur in calculating the position and height of an object when the same image is inverted on different reference planes.

That is, the actual height of the object CK should always be a constant value regardless of the position of the object CK. However, when the geometric relationship of the reference plane is distorted as shown in FIG. 7B, as the position of the object CK moves away from the camera, the calculated height value may gradually decrease, and the position of the object CK may occur. It can also be inaccurate.

FIG. 8 is a diagram for describing a process of correcting an error of object information by the object information estimator of FIG. 1. Referring to FIG. 8, assuming an ideal case where no error exists when vertex coordinates x ₁ to x ₄ of the marker acquired in the two-dimensional coordinate system of the object image CI are mapped in three-dimensional space, an intersection with a reference plane Should be X ₁ to X ₄ in FIG.

However, if a camera calibration error occurs, the calculated intersection point is the detected reference plane (π ＇). Can be detected in four points, that is, X ₁ 'to X _4' on.

The coordinates of each of X ₁ ＇to X ₄ 이다 are the intersections of the center C of the camera and the straight lines passing through x ₁ to x ₄ and the detected reference plane π＇.

,

)을 수학식 18과 같이 계산할 수 있다.The object information estimating unit 20 uses coordinates of four vertices of the marker MK having a length w _m of one side based on the ideal vertex coordinate values X ₁ to X ₄ specified using the marker MK. Constructed matrix (

,

) Can be calculated as shown in Equation 18.

에 기초하여 계산된 마커(MK)의 꼭지점 좌표와 이상적인 좌표 값 사이의 제1 호모그래피(Hp)를 수학식 19를 통하여 계산할 수 있다.The object information estimating unit 20

The first homography Hp between the vertex coordinate and the ideal coordinate value of the marker MK calculated based on may be calculated through Equation 19.

는 상기 X₁＇ 내지 상기 X₄＇로 구성된 행렬이며, 상기 Hp는 제3 호모그래피이다.)Where

Is a matrix composed of X ₁ ′ to X ₄ ′ and Hp is a third homography.)

)가

인 경우 제1 호모그래피(Hp)에 기초하여 교정된 위치(

)를

과 같이 계산하여 출력할 수 있다.Therefore, the object information estimating unit 20 detects the position on the detected reference plane π ＇.

)end

If the corrected position based on the first homography (Hp) (

)

It can be calculated and printed as follows.

In addition, the object information estimator 20 may output the distance between the calibrated position and the calibrated height point of the object as the height of the calibrated object CK.

를 지나고 상기 이상적인 기준면(π_o)에 수직인 직선(L_h)과 교차하는 제3 교점(M_h)과 상응한다.The corrected height point is a third inverse light beam (L ₂ ＇in FIG. 8 (b)) corresponding to the result inverted on the reference plane (π＇ in FIG. 8 (b)) in which the image CK of the object is detected. ) And a second intersection point M＇c at which the reference plane π ＇intersects is converted to a second intersection point Mc on the ideal reference plane π _o based on the first homography Hp. The intersection L ₂ and the straight line L ₂ passing through the center C of the camera and the

Corresponds to a third intersection point M _h passing through and intersecting a straight line L _h perpendicular to the ideal reference plane π _o .

The height output process of the calibrated object CK of the object information estimator 20 will now be described in detail with reference to FIG. 8.

The object information estimating unit 20 calculates the first intersection point M＇c where the third inverse light ray L ₂ 와 intersects with the reference plane π 을 through Equation 20 and outputs a calculation result. .

,

,

Where λ _C is the scale variable for M ＇ _C

The object information estimator 20 may convert the first intersection point M＇c into a second intersection point McC on the ideal reference plane π ₀ using Equation 21.

)에 상응하는 객체의 교정된 높이를 검출할 수 있다.The object information estimating unit 20 is the center of the camera (C) And an intersection of the straight line L ₂ passing through the second intersection Mc and the straight line L _h passing through the position M ₀ of the object CK and perpendicular to the ideal reference plane π _o using Equations 15 and 16: calculating calculates the coordinates of the calculation result and the M M _h from _h Difference between and M ₀ (

The corrected height of the object corresponding to

9A and 9B are graphs illustrating an effect of improving the error of object information by the object information estimator of FIG. 1. FIG. 9A and 9B illustrate changes in the position of an object in a laboratory environment in which conditions for lighting and background information are limited. A graph showing a result comparison between an accuracy analysis and an error correction process of an object information estimation method according to an embodiment.

1 to 9B, in the experiment of the object information estimation 10, a bar having a length of 30 cm is positioned in front of the camera 5, and then 10 cm in the front and rear directions of the camera 5, and the horizontal direction of the left and right sides. It was performed by calculating the error between the actual position (LO) and height (HG) of the object (CK) and the measured value while changing the position by 25cm.

Also, for setting the reference coordinate system, the length of one side is w _m A square marker of = 30 cm was used. The x- axis of each graph of FIGS. 9A and 9B means the distance in the front-rear direction between the camera 5 and the object CK, and each curve is the distance in the left-right direction between the camera 5 and the object CK. The absolute value of the error from the actual height and position for the change is shown.

9A and 9B are graphs showing the output of the object information estimator 10 when the error correction process is not performed, and FIGS. 9B and 9B show the error correction process. In this case, it is a graph showing the output of the object information estimator 10.

 If the error correction is not performed (Fig. 9a (a) and (b)), a very large error occurs, but it can be seen that the error occurrence is significantly reduced through the error correction process.

In addition, it can be seen that the error increases as the distance between the object and the camera increases. In particular, it can be seen that the distance in the left and right directions from the camera also affects the calculation result.

The cause of the error is presumed to be due to the limitation of the image resolution itself and the error component of the process of calculating the zero-coordinate due to the noise present in the image.

However, when the error correction is performed ((a) and (b) of FIG. 9B), the range of the measured error is relatively small considering the distance between the camera 5 and the object CK and the height of the object. Can be considered.

That is, the method for estimating object information according to an embodiment of the present invention is somewhat affected by the distance between the camera and the object and the position of the object, but it can be confirmed that no serious error occurs in the result.

FIG. 11 is a diagram for describing a first experiment process of evaluating the accuracy of object information output from the object information estimator of FIG. 1, and FIG. 12 is a graph showing the results of the first experiment of FIG. 11.

FIG. 13 is a diagram for describing a second experiment process of evaluating the accuracy of the object information output from the object information estimator of FIG. 1, and FIG. 14 is a graph showing the results of the second experiment of FIG. 13.

FIG. 15 is a diagram for describing a third experiment process of evaluating the accuracy of the object information output from the object information estimator of FIG. 1, FIG. 16 is a graph showing a result of the third experiment, and FIG. 17 is a diagram. 11, 13, and 15 are tables for analyzing the experimental results of each.

1 to 17, in the experiment of FIG. 10, the accuracy and error analysis of the object information estimation method for the change of the position of the pedestrian moving in the real environment was performed.

After setting the moving paths (paths 1 to 3) of the pedestrians on the reference plane as shown in FIG. 10 (a), the height and the moving trajectory of the pedestrians moving along them are calculated and the accuracy of the proposed technique is evaluated therefrom. do. Figure 10 (b) is a top view (bird's eye view) observed from the top of the experimental process of Figure (a).

In the course of the experiment, the pedestrian's moving path is designated by attaching markers for the path designation in each rectangular area having a size of 280cm X 270cm, and the pedestrian moves back and forth along the designated path. For the experiment, a square marker with a length of w _m = 60cm was used and it is assumed that the pedestrian in the image is always coplanar with the reference plane on which the marker is located.

(A) to (c) of FIG. 11 show a reference coordinate system and a pedestrian calculated from a marker as a result of calculating the position and key height of each pedestrian moving along the path (paths 1 to 3) determined according to the experiment of FIG. Show a vector pointing to a location.

12 (a) shows the result of the height calculation of the object information estimator 10, (b) is a position calculation result of the object information estimator 10, a bird's eye view, the movement path of the pedestrian, It shows the position and principal ray of the camera.

In general, the height of a pedestrian has a characteristic of changing periodically due to the influence of the length of the pedestrian and the shape of the pedestrian, and the minimum value when the length of the pedestrian is the maximum and the maximum value when the length of the pedestrian is the minimum. The average value of the heights calculated during walking is regarded as the height of the pedestrian.

The actual height of the pedestrians participating in the experiment was 185cm, the average height of the calculated height was about 186cm, and the standard deviation was 2.15cm ~ 2.56cm.

In addition, for accurate error analysis, the calculated pedestrian's movement path was rotated by -1 degrees around the origin to match the reference coordinate system calculated from the marker. Results are corrected using the homography used.

12 and 17, the object information estimation method according to the embodiment of the present invention is affected by the distance and the relative position change between the camera and the pedestrian, but the height and It can be confirmed that the position calculation results do not cause serious errors.

The second experiment of FIG. 13 and the third experiment of FIG. 15 are diagrams showing an experimental process and results for a pedestrian moving in an arbitrary direction, and the results of FIG. 14 (a) and (b) and FIG. The height of the pedestrian, the position of the pedestrian, the distance of the pedestrian, etc. can be confirmed from a) and (b), and the table of FIG. Can be.

18 is a flowchart illustrating a method of estimating object information according to an embodiment of the present invention. 1 to 7 and 18, the camera calibrating unit 12 calibrates the camera 5 (S10), and the object image separating unit 14 is configured from an input image input from the camera 5. The image BI of the object is separated (S12).

The object information estimator 20 inverts the image BI of the object in a three-dimensional space and calculates at least one of the position LO and the height HG of the object based on the inverted result. The calculation result is output (S14).

Between the object information estimation unit 20 is the reference plane which is history reflected in the three-dimensional space (π ') and the ideal reference plane (π ₀₎ and the reference plane (π based on ") and the ideal reference plane (π ₀₎ Detects a first homography Hp and corrects at least one of the position LO and the height HG based on the detected first homography Hp, and among the corrected positions and corrected heights as a result of the calibration. At least one is calculated and a calculation result is output (S16).

The invention can also be embodied as computer readable code on a computer readable recording medium. The computer-readable recording medium includes all kinds of recording devices in which data that can be read by a computer system is stored.

Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, and the like. The program code for performing the object information estimation method according to the present invention may be a carrier wave. It may also be transmitted in the form of (for example, transmission via the Internet).

The computer readable recording medium can also be distributed over network coupled computer systems so that the computer readable code is stored and executed in a distributed fashion. And functional programs, codes and code segments for implementing the present invention can be easily inferred by programmers in the art to which the present invention belongs.

Although the present invention has been described with reference to one embodiment shown in the drawings, this is merely exemplary, and those skilled in the art will understand that various modifications and equivalent other embodiments are possible therefrom. Therefore, the true technical protection scope of the present invention will be defined by the technical spirit of the appended claims.

BRIEF DESCRIPTION OF THE DRAWINGS In order to better understand the drawings cited in the detailed description of the invention, a brief description of each drawing is provided.

1 illustrates an object information estimator according to an embodiment of the present invention.

FIG. 2 is a diagram for describing object information output from the object information estimator of FIG. 1.

3 is a diagram for describing a process of outputting object information in the object information estimator of FIG. 1.

 FIG. 4 is a diagram for describing a process of outputting an object image by the object image output unit of FIG. 1.

FIG. 5 is a diagram for describing a geometric relationship between a marker generated in the object image output unit of FIG. 1 and an input image.

FIG. 6 is a diagram for describing a process of the object information estimator of FIG. 1 performing a history of an image of the object in a three-dimensional space.

FIG. 7 is a diagram for explaining an error of object information generated due to incorrect camera calibration.

FIG. 8 is a diagram for describing a process of correcting an error of object information by the object information estimator of FIG. 1.

9A and 9B are graphs illustrating an effect of improving the error of object information by the object information estimator of FIG. 1.

FIG. 10 is a diagram for describing the accuracy of object information output from the object information estimator of FIG. 1.

FIG. 11 is a diagram for describing a first experimental process of evaluating the accuracy of object information output from the object information estimator of FIG. 1.

FIG. 12 is a graph showing the results of the first experiment of FIG. 11.

FIG. 13 is a diagram for describing a second experimental process of evaluating the accuracy of object information output from the object information estimator of FIG. 1.

FIG. 14 is a graph showing the results of the second experiment of FIG. 13.

FIG. 15 is a diagram for describing a third experimental process of evaluating the accuracy of object information output from the object information estimator of FIG. 1.

FIG. 16 is a graph showing the results of the third experiment of FIG. 15.

17 is a table for analyzing the experimental results of each of FIGS. 11, 13, and 15.

18 is a flowchart illustrating a method of estimating object information according to an embodiment of the present invention.

Claims (27)

Separating an image of an object from an input image input from a camera; And Inverting the image of the object in three-dimensional space and calculating at least one of the position and height of the object inverted using the reference coordinate system in the three-dimensional space and outputting a calculation result Estimation method. The method of claim 1, wherein the object information estimation method comprises: Before the step of separating the image of the object, And calibrating the camera. The method of claim 2, wherein calibrating the camera, A regular matrix corresponding to a camera calibration matrix having the internal parameters of the camera as an element

Where K is the regular matrix, f _x is the scale value in the x-axis direction of the input image, f _y is the scale value in the y-axis direction of the input image, and s is a torsion parameter for the input image. (skew parameter), and (x ₀ , y ₀ ) is a principal point (pricipal point) of the input image. The method of claim 1, wherein the separating of the image of the object comprises: Generating a difference image corresponding to a difference between an image including the object and a reference image not including the object; And Binarizing the difference image and outputting the binarized image as an image of a separated object. The method of claim 4, wherein the outputting of the binarized image as an image of a separated object comprises: Comparing each pixel value constituting the difference image with a reference value, if the pixel value is larger than the reference value, a first pixel value is output; and if the pixel value is smaller than the reference value, a second pixel value is output. And outputting the binarized image . The method of claim 1, wherein calculating at least one of the position and the height of the object comprises: Generating an inverse beam of light corresponding to the inversed result; The history zero rays,

,

Where L is the inverse light ray, P is N * N (N is a natural number) partial matrix, and

Is a projection matrix of the camera

, Wherein λ is

For the scale variable, wherein C is the center coordinate of the camera

, remind

Is an intersection of the L and the image of the object, and D is an infinite origin for the inverse light ray), The location of the object corresponds to a first intersection of the inverse light ray and the reference plane in three-dimensional space, and the height of the object corresponds to a distance between the location of the object and the height point of the object Information estimation method . The method of claim 6, The first intersection point,

Where M ₀ Is the first intersection point, λ ₀ is a scale variable for M ₀ , and

Is an intersection of the intersection of the L and the image of the object. 8. The method of claim 7, wherein

(Wherein π ₀ is the reference plane on the three-dimensional space

Is among the coordinates on the π ₀

And coordinates corresponding to the object information estimation method. The method of claim 6, The height point, And a second intersection point at which a straight line perpendicular to the reference plane in the three-dimensional space intersects the inverse light ray. The method of claim 9, wherein the straight line perpendicular to the reference plane on the three-dimensional space ,

Where

Is a straight line perpendicular to the reference plane in the three-dimensional space ,

Is an intersection point at which the historical zero ray and the reference plane in the three-dimensional space intersect, and D _v is the

For the infinite origin, where μ is M _h , where M _h is

And a scale variable for the intersection with L). The method of claim 1, wherein the object information estimation method comprises: Detecting a first homography between a first reference plane and a second reference plane on the three-dimensional space ; And At least one of the position and the height is corrected by using the position coordinates of the object on the second reference plane obtained by using the first homography and the height coordinates perpendicular to the position coordinates, and among the corrected positions and corrected heights. Calculating at least one of the at least one and outputting a result of the calculation. The method of claim 11, wherein the corrected position,

Where

Is the calibrated position, and

Is the first homography, and

Where is the position before correction). The method of claim 12, wherein the corrected height is, Corresponds to the distance between the calibrated position and the calibrated height point of the object, The corrected height point is, Corresponds to a first intersection point at which the first and second straight lines intersect, The first straight line, A first intersection point at which the first inverse light beam corresponding to the result of the image of the object being inverted on the first reference plane and the first reference plane intersects any one on the second reference plane based on the first homography. A second intersection converted into a point and a straight line passing through the center of the camera, The second straight line, remind

And a straight line perpendicular to the second reference plane . A computer-readable recording medium having recorded thereon a program for performing the object information estimating method according to claim 1. An object image separator to separate an image of an object from an input image input from a camera; And An object including an object information estimator for rewriting the image of the object in three-dimensional space and calculating at least one of the position and height of the object, which is inverted using the reference coordinate system in three-dimensional space, and outputting a calculation result. Information estimator. The method of claim 15, wherein the object information estimator, And a camera calibrator configured to calibrate the camera and output the calibrated image as the input image. The method of claim 16 , wherein the camera calibration unit, A regular matrix corresponding to a camera calibration matrix having the internal parameters of the camera as an element

Where K is the regular matrix, f _x is the scale value in the x-axis direction of the input image, f _y is the scale value in the y-axis direction of the input image, and s is a torsion parameter for the input image. (skew parameter) and (x ₀ , y ₀ ) is an object information estimator for calculating and outputting a principal point of the input image. The method of claim 15, wherein the object image separator, A difference image generation unit configured to generate a difference image corresponding to a difference between an image including the object and a reference image not including the object; And And an object image output unit configured to binarize the difference image to output the binarized image as an image of a separated object. The method of claim 18, wherein the object image output unit, Comparing each pixel value constituting the difference image with a reference value, if the pixel value is larger than the reference value, a first pixel value is output; and if the pixel value is smaller than the reference value, a second pixel value is output. An object information estimator for outputting the binarized image . The method of claim 15, wherein the object information estimating unit, To generate an inverse light beam corresponding to the inversed result, The history zero rays,

,

Where L is the inverse light ray, P is N * N (N is a natural number) partial matrix, and

Is a projection matrix of the camera

, Wherein λ is

For the scale variable, wherein C is the center coordinate of the camera

, remind

Is an intersection point of the L and the image of the object, and D is an infinite origin for the inverse light ray), The location of the object corresponds to a first intersection of the inverse light ray and the reference plane in three-dimensional space, and the height of the object corresponds to a distance between the location of the object and the height point of the object Information estimator . The method of claim 20, The first intersection point,

Where M ₀ Is the first intersection point, λ ₀ is a scale variable for M ₀ , and

Is an intersection of the intersection of the L and the image of the object. The method of claim 21, wherein

(Wherein π ₀ is the reference plane on the three-dimensional space

Is among the coordinates on the π ₀

Object information estimator. The method of claim 21, The height point, An object information estimator corresponding to a second intersection point at which a straight line perpendicular to the reference plane in the three-dimensional space intersects the inverse light ray. The method of claim 23, wherein the straight line perpendicular to the reference plane on the three-dimensional space ,

Where

Is a straight line perpendicular to the reference plane in the three-dimensional space ,

Is an intersection point at which the historical zero ray and the reference plane in the three-dimensional space intersect, and D _v is the

For the infinite origin, where μ is M _h , where M _h is

And a scale variable for the intersection with L). The method of claim 15, wherein the object information estimating unit, Detecting a first homography between a first reference plane and a second reference plane mapped on the three-dimensional space and using the detected first homography, the position coordinates of the object on the second reference plane and the position coordinates perpendicular to the position coordinates An object information estimator for correcting at least one of the position and the height, calculating at least one of the corrected position and the corrected height, and outputting a calculation result through a height coordinate . A multimedia device comprising the object information estimator according to claim 15. A computer device comprising the object information estimator of claim 15.

Images