KR101776638B1 - Apparatus and Method for Estimation of Spatial information of multiple objects - Google Patents

Abstract

The present invention relates to a method for estimating spatial information of a plurality of objects. The method for estimating spatial information of a plurality of objects according to the present invention includes: an image input step of receiving images about the plurality of objects; a step of acquiring feature point coordinates of the plurality of objects to be tracked from the inputted image; a step of calculating at least three-dimensional normalized feature point coordinates using the acquired feature point coordinates and preset parameter information of a camera; and a step of estimating the spatial information of the object using the three-dimensional normalized feature point coordinates. According to the present invention, the spatial information of the plurality of objects can be estimated from the image photographed by a single camera.

Description

[0001] Apparatus and method for estimating spatial information of a plurality of objects [0002]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an apparatus and method for estimating spatial information of a plurality of objects, and more particularly, to a spatial relationship between a camera and a plurality of objects, for example, .

In a vision-based system, it is important to grasp the spatial position and distance of an object existing in the working environment, and it is important to estimate the velocity of a moving object. For example, in order to estimate the position of an object on a three-dimensional plane by using a photographing device such as a camera, distance information of the object is required. As a method for this, a stereo vision method of calculating distances using a trigonometric method has been conventionally used. However, in the case of the stereo vision method, two cameras are required, and a precise calibration procedure between the two cameras is required. However, there has been a problem that a large error occurs in the position estimation even by a small calibration error.

Also, there is a method of acquiring distance information to each point of an object included in a photographed image by using a separate sensor such as a laser sensor or an ultrasonic sensor, but there is a disadvantage that the price of a separately provided sensor is expensive .

An improved method has since been introduced for monocular cameras. For example, in the conventional method, the distance information between the camera and the object and the size of the object are inputted and set. When the distance between the camera and the object is calculated by calculating the degree of change of the size of the object in the image as the camera or the object moves, However, there is a limit in that information about the distance between the camera and the object and the size of the object must be input in advance. In addition, it is difficult to apply real-time tracking of an object because it is possible to estimate only the trajectory of a simple object, and batch processing that requires a large amount of image data is essential.

A method for estimating the structure and motion (SaM, Structure and Motion) of a given scene containing an object has been extensively studied in the field of vision-based control. This structure and motion estimation method can be used for visual servoing, tracking and manipulation of UGV and UAV based on the vision system. In particular, many studies on 3D reconstruction in motion environment of various cameras / objects using monocular camera have been performed. However, there is a problem in practical use such that the distance and the speed of the object are not estimated at the same time, or the moving area of the camera and the object is limited.

When a camera that is not allowed to panning or tilting is attached to a moving object such as a UGV or a UAV, the movement of the vehicle can be used to obtain the speed of the camera. However, there is a limitation in that the camera speed must be calculated when the camera can have mobility independently of the movement of the moving object or when the movement of the moving object is difficult to be calculated.

US 6,999,612

An object of the present invention is to provide an apparatus and a method for estimating spatial information of a plurality of objects in a plurality of objects and cameras having mobility under a small conditional restriction on a speed and a moving method of a camera and a plurality of objects.

In particular, the present invention provides a method of analyzing an object photographed image when a camera and a plurality of objects have mobility, and estimating spatial information at a higher speed using a nonlinear observer and a scale factor.

According to an aspect of the present invention, there is provided a method for estimating a plurality of object space information, comprising: inputting an image of a plurality of objects; extracting feature points of a plurality of objects Calculating normalized minutiae coordinates of at least three dimensions on the basis of coordinates of the obtained minutiae points and parameter information of a camera set in advance, and calculating minutiae point coordinates of the plurality of objects using the three- .

The step of estimating the spatial information of the plurality of objects includes calculating the components of the normalized minutiae coordinates as a component, defining the state vectors and the system models for describing the spatial information of the object, The spatial information of a plurality of objects can be estimated from the estimation of an unmeasurable vector by dividing the vector into a possible vector and a non-measurable vector, and the spatial information can be estimated based on the speed of the objects, the relative speed of the objects with respect to the camera, And scale factor information, which is a constant other than 0, as an element for adjusting the scale of the state vector related to the velocity of the object and the distance of the object.

The step of estimating the spatial information of the plurality of objects may include the steps of defining system models including a state vector as position information for describing spatial information of a plurality of objects and including a measurable vector and an unmeasurable vector, Defining nonlinear estimators for estimating nonmeasurable vectors of nonlinear observers using systematic models and observer errors defined by the differences between the estimates; Estimating the non-measurable vector of the nonlinear observer using the nonlinear estimators, and estimating the unknown spatial information using the estimation of the non-measurable vector.

The system models of the plurality of objects include differentials for the state vectors and are elements for adjusting the scale of the state vector related to the components of the state vectors, the speed of the plurality of objects, the speed of the camera, and the distance of a plurality of objects And a scale factor that is a constant other than 0, for example.

The step of estimating the non-measurable vector includes defining an observer error defined by the difference between the system model and the estimate, defining a filtered observer error defined as the observer error filtered through a low-pass filter, And defining an estimation equation of the non-measurable vector using the observer error.

Meanwhile, in the unknown spatial information estimating step, the unknown spatial information is a scale factor that is a constant that is a non-zero constant that controls the scale of state vectors related to the relative speed of the objects with respect to the camera and the distance of a plurality of objects, Estimating a non-linear observer using non-measurable vectors, estimating a non-zero scale factor as an element for adjusting a scale of a state vector associated with a distance of a plurality of objects using the estimated non-linear observers Estimating a relative speed of the plurality of objects with respect to the camera using the estimated scale factor.

Nonlinear estimators can also be defined to estimate non-measurable vectors using non-measurable vectors and observer errors included in nonlinear observers, which are derivatives of non-measurable vectors contained in nonlinear observers, A distance condition of a plurality of objects, and a presumed variable, which is a given or unmeasured variable, in which the distance condition of a plurality of objects is a distance condition of a plurality of objects, The present invention can be applied to a case where the distances from the camera to the plurality of objects are different and the distances from the cameras of the plurality of objects are given and the relative speed between the plurality of objects is zero.

The step of calculating the coordinates of the normalized feature points on the dimension may calculate the coordinates of the normalized feature points on the three dimensional plane by calculating the coordinates of the feature points and the intrinsic calibration matrix of the camera defined by the parameter information of the camera, The coordinates are the coordinates of the pixels related to the vector of the feature points in the image space to which the objects belong, and the normalized three-dimensional space coordinates of the feature points are set in the three-dimensional spatial coordinate system with the camera position as the origin, And is a normalized coordinate based on the set photographing direction.

In the present invention, an image may be captured by a camera moving at a speed and an angular velocity according to a predetermined value or function.

The method of estimating a plurality of object space information according to yet another embodiment of the present invention is a method of estimating a plurality of object space information of a first object and a second object, Acquiring normalized first and second minutiae coordinates of at least three dimensions on the basis of the acquired first and second minutiae coordinates and parameter information of a camera set in advance, A first state vector and a second state vector for describing the spatial information of the first and second objects are defined, and the first and second state vectors are respectively measured Defining a first system model and a second system model including a possible vector and an unmeasurable vector, using the given spatial information from the first and second system models The method comprising: defining first and second nonlinear observers; using first and second observer errors defined as the differences between the first and second system models and their respective estimates, Estimating a non-measurable vector of the first and second nonlinear observers using the defined first and second nonlinear estimators, and estimating the non-measurable vector of the first and second nonlinear observers using the defined first and second nonlinear estimators, And estimating the first and second nonlinear observers using the first and second nonlinear observers.

Estimating scale factors that are non-zero constants as elements for adjusting a scale of state vectors associated with a distance of a plurality of objects using the estimated first and second nonlinear observers; And estimating a relative speed with respect to the camera. Or estimating the first and second system models using the estimated first and second nonlinear observers.

The plurality of object space information estimating apparatuses according to an embodiment of the present invention include an image input unit for receiving images of a plurality of objects, a minutiae point coordinate acquisition unit for acquiring minutiae coordinates of a plurality of objects to be tracked from the input image, A normalized coordinate calculation unit for calculating coordinates of at least three-dimensionally normalized minutiae points by using the coordinates of the obtained minutiae points and the parameter information of the camera set in advance, the coordinates of the plurality of object minutiae points using the coordinates of the three- And a spatial information estimating section for estimating information.

Here, the spatial information estimating unit may include state vectors as positional information for describing spatial information of a plurality of objects as a constituent component of the calculation of the components of the coordinates of the normalized minutiae points, and may include a measurable vector and an unmeasurable vector A nonlinear observer defining unit that defines nonlinear observers defined as a derivative of the system models, a system model defining unit that defines system models including the observer errors defined by the difference between the system models and the estimates, A nonlinear estimator defining nonlinear estimators for estimating non-measurable vectors of nonlinear observers, an unmeasurable vector defining unit for estimating non-measurable vectors of nonlinear observers using nonlinear estimators, and estimates of unmeasurable vectors W, can be implemented by including a spatial information estimation unit for estimating the spatial information of the image of the unknown.

Wherein the unknown spatial information estimator is a scale factor that is a non-zero constant that controls the scale of state vectors related to the relative speed of the plurality of objects with respect to the camera and the distance of the objects, Estimating nonlinear observers and system models using a vector and adjusting a scale of state vectors associated with a distance of a plurality of objects from the estimated system models, Estimating the velocity of the object, estimating distance information and state vectors using the estimated scale factor, and estimating velocity vectors of the object. In the nonlinear estimator definition unit, the nonlinear estimator may measure the velocity included in the nonlinear observer Is a derivative of an impossible vector, and a nonlinear observer The observer error, and the positive constant diagonal matrices of the object, which are included in the estimate matrix.

The normalized coordinate calculation unit may calculate the coordinates of the normalized minutiae points on the three-dimensional plane by calculating coordinates of the minutiae points and a correction matrix unique to the camera defined by the parameter information of the camera. Which is a coordinate of a pixel related to the feature points, on the screen of the image of the camera, and the normalized coordinate calculator calculates the coordinate of the camera in the 3D spatial coordinate system having the camera position as the origin, The normalized coordinates on the three-dimensional coordinates, which are the coordinates obtained, can be calculated.

According to the method and apparatus for estimating spatial information of a plurality of objects of the present invention, spatial information of a plurality of objects can be estimated from an image photographed using a single camera when the camera and a plurality of objects have mobility at the same time.

According to the method of estimating spatial information of a plurality of objects of the present invention, spatial information of a plurality of objects can be estimated quickly by analyzing the captured image and using a nonlinear observer.

1 is a block diagram of a plurality of object space information estimation apparatuses according to an embodiment of the present invention.
2 is a reference diagram for explaining a coordinate system utilized in the present invention.
3 is a flowchart of a method for estimating a plurality of object space information according to an exemplary embodiment of the present invention.
4 is a detailed flowchart of spatial information estimation as a method of estimating a plurality of object space information according to another embodiment of the present invention.
5 is a flow chart showing in detail the step of estimating unknown spatial information.
FIG. 6 is a flow chart showing in detail a step of estimating an unmeasurable vector.
7 is a flowchart illustrating a method of estimating spatial information of a first object and a second object according to an embodiment of the present invention.
8 is a flowchart illustrating a method of estimating spatial information of a first object and a second object according to an embodiment of the present invention Figure 9 is a flowchart illustrating a method of estimating spatial information of a first object and a second object according to an embodiment of the present invention Fig.
FIG. 10 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
FIG. 11 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention.
12 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
FIG. 13 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention.
FIG. 14 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
FIG. 15 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
16 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention.
FIG. 17 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
18 shows motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
FIG. 19 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.
20 is a block diagram of a plurality of object space estimation apparatuses according to an embodiment of the present invention.
FIG. 21 is a block diagram of a plurality of object space estimation apparatuses according to an embodiment of the present invention.

In order to fully understand the present invention, operational advantages of the present invention, and objects achieved by the practice of the present invention, reference should be made to the accompanying drawings and the accompanying drawings which illustrate preferred embodiments of the present invention.

BEST MODE FOR CARRYING OUT THE INVENTION Hereinafter, the present invention will be described in detail with reference to the preferred embodiments of the present invention with reference to the accompanying drawings. However, the present invention can be implemented in various different forms, and is not limited to the embodiments described. In order to clearly describe the present invention, parts that are not related to the description are omitted, and the same reference numerals in the drawings denote the same members.

Throughout the specification, when an element is referred to as " including " an element, it does not exclude other elements unless specifically stated to the contrary. The terms "part", "unit", "module", "block", and the like described in the specification mean units for processing at least one function or operation, And a combination of software.

In Structure and Motion (SaM) prediction referred to in this embodiment, a structure is referred to in the study of tracking the position and motion of an object based on image capturing. For example, , Relative Euclidean distance in Y, Z coordinates. Motion can refer to the motion of the object relative to the camera.

Also, as a condition for tracking the position and motion, the angular velocity of the camera can be obtained by using epipolar geometry from two consecutive images, so that only the linear velocity of the camera can be considered.

In order to estimate the position (i.e., distance) and motion (i.e., linear velocity) of an object, the dynamics of some measurable states can be considered. Considering more general camera and object motion, existing nonlinear observers for SaM estimation may not be used. Conventional methods based on algebraic equations using existing least squares or optimization methods can not be effectively utilized and are particularly disadvantageous for high-speed motion, even though constraints are imposed on the motion of the camera and the object. Therefore, a robust nonlinear observer is needed that can maintain robustness even if there is interference such as uncertain factors that vary with pixel noise and time of image.

In the present invention, each of the camera and the plurality of objects includes one of three motions of a static motion having a linear velocity vector of zero size, a dynamic motion having a non-zero linear velocity vector, and a general motion of the dynamic or the static motion Lt; / RTI > SaM for a plurality of objects is represented using SaM dynamics for a robust nonlinear observer according to the cases described below using information between two objects.

Feature motion can be induced by the camera staring at the object in the image plane. When moving both the camera and a plurality of objects, the feature points of the object also move in the camera frame. Corresponding points on a continuous camera image are shown and can be calculated by feature-tracking techniques.

1 is a block diagram of a plurality of object space information estimation apparatuses according to an embodiment of the present invention. The apparatus for estimating spatial information of a plurality of objects according to an embodiment of the present invention may include an image input unit 100, a minutiae coordinate acquisition unit 200, a normalized coordinate calculation unit 300, and a spatial information estimation unit 400 have. The plurality of object space information tracking devices can analyze images of a plurality of objects photographed by a camera and estimate spatial information given or not measured from a given condition in a system composed of a single camera and a plurality of objects. The camera referred to in the present invention is not limited in its kind and may be any device capable of image capturing.

The image input unit 100 can receive images of a plurality of objects. According to an embodiment of the present invention, the image input unit 100 may receive images captured by a camera moving at a speed and an angular velocity according to preset values or functions. According to another embodiment of the present invention, the camera can photograph an object at a predetermined time interval on the move.

The minutiae coordinate acquiring unit 200 can acquire minutiae coordinates of the plurality of objects to be traced from the input image. According to an embodiment of the present invention, the minutiae coordinate acquiring unit 200 can acquire minutiae coordinates, which are the coordinates of pixels related to the minutiae on the screen of an image for a plurality of objects.

The normalized coordinate calculation unit 300 may calculate at least three-dimensional normalized minutia point coordinates using the obtained minutia coordinate and parameter information of the camera set in advance. According to an embodiment of the present invention, the normalized coordinate calculation unit 300 calculates coordinates of the characteristic point and the intrinsic calibration matrix of the camera, which are defined by the parameter information of the camera, and calculates the three-dimensional normalized minutiae coordinates .

According to another embodiment of the present invention, the normalized coordinate calculation unit 300 calculates a normalized coordinate of a three-dimensional spatial coordinate system having a camera as its origin, The coordinates can be calculated.

The spatial information estimation unit 400 may estimate the spatial information of the plurality of objects using the three-dimensional normalized minutiae coordinates. The spatial information estimating unit 400 includes a system model defining unit 410, a nonlinear observer defining unit 420, a nonlinear estimating unit defining unit 430, an unmeasurable vector defining unit 440, an unknown spatial information estimating unit 450, . ≪ / RTI >

The system model defining unit 410 may define a system model including a measurable vector and a non-measurable vector as a model for describing spatial information of a plurality of objects.

The nonlinear observer defining unit 420 may define a nonlinear observer defined as a derivative of the system model using the defined system model.

The nonlinear estimator defining unit 430 may define a nonlinear estimator for estimating the non-measurable vector of the nonlinear observer using the system model and the observer error.

According to an embodiment of the present invention, in the nonlinear estimator defining unit 430, the nonlinear estimator may be defined as a derivative of an unmeasurable vector included in the nonlinear observer, and the nonmeasurable vector included in the nonlinear observer, And may be defined to estimate spatial information of the plurality of objects using observer error and positive constant diagonal matrices.

The non-measurable vector defining unit 440 can define a non-measurable vector of the non-linear observer using a non-linear estimator. The unmeasurable vector that is defined can be measured in the experiment or included in the system model as a non-given value.

The unknown spatial information estimating unit 450 can estimate the unknown spatial information by using the estimation of the unmeasurable vector.

According to an embodiment of the present invention, in the unknown spatial information estimating unit 450, the unknown spatial information includes an element for adjusting the scale of the state vectors related to the relative speed of the objects with respect to the camera, Estimating the nonlinear estimator, the nonlinear observer, and the system model using an estimated non-measurable vector and calculating a scale of a state vector associated with a distance of a plurality of objects from the estimated system model, A scale factor that is a non-zero constant and a plurality of object velocities, and estimates the distance information and the state vector using the estimated scale factor.

2 is a reference diagram for explaining a coordinate system utilized in the present invention.

는 초기시간 t0에서 카메라의 위치를 기준으로 하는 직교 좌표계이다. 여기서

는 카메라의 촬영 방향으로(예컨대 카메라 렌즈 면의 법선 방향) z* 축이 설정될 수 있고, z* 축과 직교하도록 x*, y* 축이 설정될 수 있다. 여기서 x*, y*, z* 축의 방향은 필요에 따라 위에서 설명한 방향으로부터 소정의 각도를 가지도록 회전된 방향으로 설정될 수도 있다. 또한 도 2에서

는 초기시간 t0으로부터 소정의 시간 동안 카메라가 이동한 후의 카메라(CM)를 위치를 기준으로 하는 직교 좌표계이다. 여기서

좌표계에서도 마찬가지로 x, y, z의 3축 방향이 설정될 수 있다. 이때 카메라의 이동은 회전행렬 R(t)와 평행이동벡터

로 표현될 수 있고,

와

간의 좌표계 간 변환 역시 상기 회전행렬 및 평행이동벡터로 표현될 수 있다. 또한 여기서 카메라와 객체의 3차원 공간 상에서의 위치를 나타내는 좌표계로서 관성 프레임(inertial frame or inertial reference frame)이 정의될 수 있다. 또한 여기서 카메라가 촬영한 영상의 평면 공간 πi가 도 2에 도시된 바와 같이 정의될 수 있다. 여기서 i는 카메라가 촬영한 i번째 프레임에 해당하는 영상을 지칭한다. 여기서 영상 공간 는

좌표계에 따라 정의될 수 있고,

좌표계에서 z 축 좌표 값이 일정한 값을 가지도록 z 축에 수직한 평면 공간으로 정의될 수 있다. 예를 들어 영상 공간 의

좌표계에서의 z 축 값은 1로 설정될 수 있고, 필요에 따라 다른 값으로 설정될 수도 있다.2,

Is an orthogonal coordinate system based on the position of the camera at an initial time t0. here

The z * axis can be set in the photographing direction of the camera (e.g., the normal direction of the camera lens surface), and the x *, y * axes can be set so as to be orthogonal to the z * axis. Here, the directions of the x *, y *, and z * axes may be set in a rotated direction so as to have a predetermined angle from the above-described direction, if necessary. 2,

Is an orthogonal coordinate system based on the position of the camera (CM) after the camera has moved for a predetermined time from the initial time t0. here

Likewise, three coordinate axes x, y and z can be set in the coordinate system. At this time, the movement of the camera corresponds to the rotation matrix R (t)

, ≪ / RTI >

Wow

The transformation between the coordinate systems can also be expressed by the rotation matrix and the translation vector. In addition, an inertial frame or an inertial reference frame can be defined as a coordinate system indicating the position of the camera and the object in the three-dimensional space. Here, the plane space pi of the image taken by the camera here can be defined as shown in Fig. Here, i denotes an image corresponding to the i-th frame captured by the camera. Here,

Can be defined according to a coordinate system,

And can be defined as a plane space perpendicular to the z-axis so that the z-axis coordinate value in the coordinate system has a constant value. For example,

The z-axis value in the coordinate system can be set to 1, and may be set to another value if necessary.

3 is a flowchart of a method for estimating a plurality of object space information according to an exemplary embodiment of the present invention. A method for estimating a plurality of object space information according to the present invention includes the steps of inputting an image (S100), acquiring coordinates of characteristic points (S200), calculating coordinates of three-dimensional normalized characteristic points (S300) (S400). The spatial information referred to in the present invention may include at least one piece of information selected from the group consisting of position, motion, distance, and velocity information of a plurality of objects.

The step of inputting an image (S100) can receive an image of a plurality of objects. According to an embodiment of the present invention, in step S100 of inputting an image, the image may be photographed by a camera moving at a speed and an angular velocity according to a preset value or function.

The step of acquiring the minutiae point coordinates (S200) may acquire the minutiae point coordinates of the plurality of objects to be traced from the input image. At this time, in defining or measuring the angular velocity according to the rotation movement of the camera, the angular velocity of the camera can be defined or measured according to the angle at which the camera rotates about a fixed point. For example, the angular velocity of a camera can be measured by an inertial measurement unit (IMU).

In step S300 of calculating the coordinates of the three-dimensional normalized minutiae points, at least three-dimensional normalized minutiae point coordinates can be calculated using the coordinates of the obtained minutiae points and the parameter information of the camera set in advance.

According to an embodiment of the present invention, the minutiae coordinate is a coordinate of a pixel associated with a vector of minutiae on the image space pi for the plurality of objects, and the normalized three-dimensional spatial coordinate of the minutiae is the position of the camera In the three-dimensional spatial coordinate system having the origin, one axis may be set to the photographing direction of the camera, and the coordinates may be normalized based on the set photographing direction.

및

로 정의한다. 이때, 도 2 에서,

는 초기 시간 t0에서의 초기 위치에서의 카메라에 결합되고,

는 가 초기시간으로부터 회전 행렬

및 변환 벡터

를 통해 변환된 직교 좌표를 의미한다. 카메라로부터 촬영된 특징점의 직교 좌표는

이 된다. 카메라 프레임

에서 정규화된 직교 좌표계

는

가 된다.To derive a camera-object relative motion model, the two Cartesian coordinates with respect to the camera are

And

. At this time, in FIG. 2,

Is coupled to the camera at an initial position at an initial time t0,

From the initial time,

And transform vector

Which are transformed through the coordinate system. Cartesian coordinates of the feature points taken from the camera are

. Camera frame

Lt; RTI ID = 0.0 > orthogonal < / RTI &

The

.

The step of estimating the spatial information (S400) may estimate the spatial information of the plurality of objects using the three-dimensional normalized minutiae coordinates.

According to an embodiment of the present invention, the step of estimating spatial information of a plurality of objects (S400) may include defining a state vector and a system model for describing spatial information of a plurality of objects, The spatial information of the plurality of objects can be estimated from the estimation of the unmeasurable vector by distinguishing the vectors as impossible.

According to an embodiment of the present invention, the step of estimating spatial information of a plurality of objects (S400) includes a calculation of the components of the normalized minutiae coordinates as a component, and the state vectors for describing the spatial information of the object System models can be defined and spatial information of the plurality of objects can be estimated from the estimation of the unmeasurable vector by dividing the system models into measurable vectors and non-measurable vectors.

4 is a detailed flowchart of spatial information estimation as a method of estimating a plurality of object space information according to another embodiment of the present invention.

The spatial information estimation method in FIG. 4 will be described with reference to a coordinate space defined as shown in FIG. The object space information estimation method according to the present invention includes the steps of defining system models (S410), defining nonlinear observers (S420), defining nonlinear estimators (S430), estimating unmeasurable vectors (S440) And estimating unknown spatial information (S450).

Defining the system models (S410) may define system models including a measurable vector and a non-measurable vector as a model for describing spatial information of an object.

In one embodiment of the present invention, the plurality of system models each include a derivative with respect to the state vector including a component of a normalized minutia coordinate, and the constituent components of the state vector, the speed of the object , The speed of the camera, and the scale factor.

According to an embodiment of the present invention, the measurable vector included in the system models includes the first and second components of the state vector and the components of the velocity vector of the given camera, The vector may be represented by a scale factor that is a non-zero constant as an element that scales the third component of the state vector, the velocity vector components of the object, and the state vector associated with the distance of the object from the estimated system model .

According to an embodiment of the present invention, a system model of a plurality of objects includes a derivative with respect to the state vector including a component of the normalized minutia coordinate, The speed of the object, the speed of the camera, and the scale factor. According to another embodiment of the present invention, in the system model, the velocity condition of the object and the camera may be such that one component of the object velocity vector is zero and a given one component of the camera velocity vector is not zero.

. Y1(t), y2(t)는

으로부터 얻어질 수 있으며, 유클리드 상대 거리 x3(t)와 관련되므로, 거리 추정값을 얻기 위하여 추정되어야 한다. 단일 카메라가 두 개의 오브젝트를 촬영하는 경우를 가정하므로, 두 개의 오브젝트에 대하여 다음과 같이 변수를 정의할 수 있다. 아래첨자 a 와 b를 각각의 오브젝트에 대하여 이용하여, 각 오브젝트들의 상태 벡터들을 수학식 5a, 수학식 5b와 같이 정의한다.As an embodiment of the present invention, a first object, a second object, and a single camera can be described as follows. The state vector y is defined as follows.

. Y1 (t), y2 (t)

And is related to Euclidean relative distance x3 (t), so it must be estimated to obtain the distance estimate. Since it is assumed that a single camera shoots two objects, the following variables can be defined for two objects. The subscripts a and b are used for each object, and the state vectors of the respective objects are defined as shown in Equations (5a) and (5b).

에서의 특징점 좌표 qc는(qa는 a qb는 b)

과 같이 나타낼 수 있다. 이때,

이고,

는

좌표계의 원점으로부터

까지의 벡터이다. 상대 모션의 동역학은

과 같으며,

는

에서의 카메라 각속도 벡터이고, 비대칭(skew-symmetric) 행렬

는

와 같이 정의된다.In the same way,

The feature point coordinate qc (qa is a bb is b)

As shown in Fig. At this time,

ego,

The

From the origin of the coordinate system

Lt; / RTI > The kinematics of relative motion

Lt; / RTI >

The

And the skew-symmetric matrix < RTI ID = 0.0 >

The

Respectively.

에서의

에 상대적인 카메라 선속도가 두 개의 오브젝트들에 대하여 정의되므로, 카메라와 각 오브젝트들 간의 상대 속도는

과 같이 주어질 수 있다.

In

Since the relative camera linear velocity is defined for two objects, the relative velocity between the camera and each object is

As shown in FIG.

는

에서의

의 선속도로서,

와 같이 구해지고, R은

와 관성 기준 좌표계 간의 회전 매트릭스이고,

는

의 관성 좌표계에서의 선속도 벡터이고, 카메라 속도

는 카메라의 선속도 벡터이다. 따라서, 다음과 같은 가정을 도입할 수 있다.At this time,

The

In

As a linear velocity of

, And R is

And an inertia reference coordinate system,

The

Is a linear velocity vector in the inertial coordinate system of the camera,

Is the linear velocity vector of the camera. Therefore, the following assumptions can be introduced.

및 오브젝트 속도 벡터

는 유한하고(bounded) 연속적으로 미분 가능하고, 또한,

를 알 수 있는 경우, 카메라 각속도 벡터는 두 개의 연속적인 2차원 이미지로부터 등극선 기하를 통하여 추정할 수 있다. 따라서, 카메라의 선속도 추정에만 집중할 수 있다.

및

을 이용할 때, 일부 측정 가능한 상태 벡터

에 대한 동역학은 수학식 10과 같이 나타낼 수 있다.

의 값은

로부터 스케일 팩터를 통하여 구할 수 있으므로, 수학식 11을 0이 아닌 미지 상수

로부터 구할 수 있다),

는 시간에 따라 변하는

대신 구하고 거리 추정에 이용할 수 있다. 측정 가능한 벡터

으로 정의할 때, 수학식 10로부터 측정 가능한 상태들에 대한 SaM 모델은 수학식 12와 같이 정의 될 수 있고, 이때

이다. Camera speed vector

And object velocity vector

Is bounded and continuously differentiable, and further,

, The camera angular velocity vector can be estimated from two successive two-dimensional images through the isodose geometry. Therefore, it is possible to concentrate on estimating the linear velocity of the camera.

And

, Some measurable state vectors

Can be expressed as: < EMI ID = 10.0 >

The value of

Can be obtained from the scale factor,

, ≪ / RTI >

Varies with time

Instead, it can be used to estimate the distance. Measurable vector

, The SaM model for measurable states from equation (10) can be defined as in equation (12), and at this time

to be.

Defining non-linear observers (S420) may use a defined system model to define a nonlinear observer defined as a derivative of the system model.

Defining non-linear estimators (S430) may define a non-linear estimator for estimating the non-measurable vector of the non-linear observer using system models and observer errors.

According to an embodiment of the present invention, the nonlinear estimator may be defined as a derivative of an unmeasurable vector included in the nonlinear observer, and the non-linear vector included in the nonlinear observer, the observer error, and the positive constant diagonal matrixes To estimate spatial information of the object.

는

와 같다.

와

이 모두 추정되어야 하므로, 수학식 31은 수학식 32과 같이 다시 쓰여질 수 있다. 이때,

은 언제나 가역이다. 또한,

,

,

이 양의 상수

,

,

에 대하여 성립할 수 있다.

이므로,

의 추정은 후술하는 수학식 37과 같은 관측기로부터 얻어질 수 있으며,

에 대하여 수렴하고,

의 추정은 후술하는 것과 같이 구할 수 있다. 첫째로, 수학식 33를 이용하여

및

를 얻을 수 있고, 결합하여 수학식 34을 얻을 수 있다. 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지며, 카메라 속도와 오브젝트들의 속도는 다르고, 따라서,

는 완전히 0이 아닌 경우에, 수학식 34으로부터 얻은

의 추정을 이용하여,

의 추정을 수학식 33로부터 수학식 35와 같이 얻을 수 있다. 따라서,

의 추정은 수학식 33와 수학식 35를 이용하여 수학식 36와 같이 구할 수 있다.According to one embodiment of the present invention, i = 1, 2, 3

The

.

Wow

(31) can be rewritten as Equation (32). At this time,

Is always reversible. Also,

,

,

This positive constant

,

,

. ≪ / RTI >

Because of,

Can be obtained from an observer such as Equation (37)

Lt; / RTI >

Can be obtained as described below. First, using Equation 33,

And

(34) can be obtained. The two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different,

Lt; RTI ID = 0.0 > 34 < / RTI >

Lt; / RTI >

Can be obtained from the equation (33) as shown in the equation (35). therefore,

Can be obtained as shown in Equation (36) using Equations (33) and (35).

의 추정기는

를 정의함으로써 얻을 수 있다. 이때,

는 후술하는 수학식 37에서의 관측기로부터 얻어진다. 또한,

는 양의 유한 상수 대각 행렬

에 대해 정의된다. 그러면, 수학식 32에 대한 비선형 관측기는 수학식 37과 같이 구해지고,

는 수학식 38과 같이 얻어질 수 있다. 이때,

이고,

이다. 이때 오차 동역학은, 수학식 39 및 수학식 40와 같다.Based on the SaM model in equation (32)

The estimator of

. ≪ / RTI > At this time,

Is obtained from the observer in the following Expression (37). Also,

Is a positive finite constant diagonal matrix

Lt; / RTI > Then, the nonlinear observer for the equation (32) is obtained as shown in equation (37)

Can be obtained as shown in Equation (38). At this time,

ego,

to be. At this time, the error dynamics are expressed by Equations (39) and (40).

In step S440 of estimating the unmeasurable vector, non-linear estimators of the nonlinear observer can be estimated using nonlinear estimators. Herein, the measurable vector included in the system models according to an embodiment includes the first and second components of the state vectors, and the unmeasurable vector included in the system models includes the third component of the state vector, As a factor adjusting the scale of the state vector related to the relative speed of the second object with respect to the camera, the relative speed with respect to the camera of the second object, and the distance of the object.

Estimation of unknown spatial information (S450) may be performed using estimation of an impossible vector to estimate unknown spatial information, and details thereof will be described later with reference to FIG.

5 is a flow chart showing in detail the step of estimating unknown spatial information. The spatial information estimation method in FIG. 5 will be described with reference to a coordinate space defined as shown in FIG. The object space estimation method according to the present invention may include a step S441 of defining an observer error, a step S443 of defining a filtered observer error, and a step S445 of defining an estimation equation of an unmeasurable vector.

를

와 같이 정의할 수 있다.Defining the observer error (S441) may define the observer error defined by the difference between the system model and its estimate. According to one embodiment of the present invention, the observer error, which is the difference between the system model and its estimate,

To

Can be defined as follows.

는 양의 대각 행렬인

에 대하여

과 같이 정의될 수 있다.Defining the filtered observer error (S443) may define a filtered observer error that is defined as filtering the observer error through a low-pass filter. According to one embodiment of the invention, the filtered observer error

Is a positive diagonal matrix

about

Can be defined as follows.

Defining the estimation equation of the non-measurable vector (S445) may define an estimation equation of the non-measurable vector using the observer error and the filtered observer error.

에 대한 비선형 추정기는

는 양의 상수 대각 행렬

에 대하여 수학식 38과 같이 구해질 수 있다.According to one embodiment of the present invention, an unmeasurable vector

The nonlinear estimator for

A positive constant diagonal matrix

Can be obtained as shown in equation (38).

FIG. 6 is a flow chart showing in detail a step of estimating an unmeasurable vector. The spatial information estimation method in FIG. 6 will be described with reference to a coordinate space defined as shown in FIG.

The object space information estimation method according to the present invention may include estimating nonlinear observers (S451), estimating scale factors (S453), and estimating a relative velocity (S455). The unknown spatial information may be a scale factor that is a non-zero constant that controls the scale of state vectors related to the relative speed of the objects with respect to the camera and the distance of the plurality of objects.

In the step of estimating nonlinear observers (S451), the nonlinear observers can be estimated using the estimated non-measurable vectors. An unmeasurable vector can be estimated from the nonlinear estimator obtained as shown in Equation (37), and a nonlinear observer can be estimated by combining with Equation (36), and thus an estimation of the system as shown in Equation (31) can be obtained.

In step S453 of estimating the scale factors, a scale factor that is a non-zero constant may be estimated as an element for adjusting the scale of the state vector related to the distance of the plurality of objects using the estimated nonlinear observers. In this case, the system model may be estimated from the nonlinear estimator obtained as shown in Equation 36 and a scale factor may be estimated as shown in Equations 33 and 34 according to an embodiment.

In the step of estimating the relative speed (S455), the relative speed of the plurality of objects with respect to the camera can be estimated using the estimated scale factor. The estimation of the relative speed can be obtained by using the equation (34) as shown in the equation (35).

7 is a flowchart illustrating a method of estimating spatial information of a first object and a second object according to an embodiment of the present invention. According to an embodiment of the present invention, a spatial information estimation method for a first object and a second object includes the steps of obtaining normalized first and second minutia coordinates (S10), defining a first system model and a second system model (S30) defining a first nonlinear estimator and a second nonlinear estimator, defining a first nonlinear estimator and a second nonlinear estimator (S40), estimating an unmeasurable vector (S50), and Estimating a first nonlinear observer and a second nonlinear observer (S60).

In the step (S10) of obtaining the normalized first and second minutiae coordinates, the first and second minutiae points of the first object and the second minutiae of the second object are input from the input image, And acquire normalized first and second minutiae coordinates of at least three dimensions on the basis of the obtained first and second minutiae coordinates and parameter information of the camera set in advance. In this case, according to one embodiment, the minutiae coordinate is the coordinates of the pixel related to the minutiae point vector on the image space pi for the plurality of objects, and the normalized three-dimensional spatial coordinates of the minutiae point is the origin of the camera One axis in the three-dimensional spatial coordinate system is set to the photographing direction of the camera, and it can be regarded as a normalized coordinate with respect to the set photographing direction.

The step S20 of defining the first system model and the second system model includes a calculation of the components of the coordinates of the obtained first and second feature points as constituent components and sets the spatial information of the first and second objects as A first system model and a second system model can be defined that include a first state vector and a second state vector for describing the first and second state vectors and a vector capable of measuring each of the first and second state vectors and an unmeasurable vector .

및

을 이용할 때, 일부 측정 가능한 상태 벡터

에 대한 동역학은 수학식 10과 같이 나타낼 수 있다.

의 값은

로부터 스케일 팩터를 통하여 구할 수 있으므로, (수학식 11을 0이 아닌 미지 상수

로부터 구할 수 있다),

는 시간에 따라 변하는

대신 구하고 거리 추정에 이용할 수 있다. 측정 가능한 벡터

으로 정의할 때, 수학식 10로부터 측정 가능한 상태들에 대한 SaM 모델은 수학식 12와 같이 정의 될 수 있다.At this time, as an embodiment of the present invention, the first object, the second object,

And

, Some measurable state vectors

Can be expressed as: < EMI ID = 10.0 >

The value of

(Expression 11 is an unknown constant other than 0,

, ≪ / RTI >

Varies with time

Instead, it can be used to estimate the distance. Measurable vector

, The SaM model for measurable states from Equation (10) can be defined as Equation (12).

의 추정기는

를 정의함으로써 얻을 수 있다. 이때,

는 후술하는 수학식 37에서의 관측기로부터 얻어진다. 또한,

는 양의 유한 상수 대각 행렬

에 대해 정의된다. 그러면, 수학식 32에 대한 비선형 관측기는 수학식 37과 같이 구해지고,

는 수학식 38과 같이 얻어지고, 이때

이다.In the step S30 of defining the first nonlinear observer and the second nonlinear observer, the first and second nonlinear observers may be defined from the first and second system models using previously given spatial information. According to one embodiment, based on the SaM model in equation (32)

The estimator of

. ≪ / RTI > At this time,

Is obtained from the observer in the following Expression (37). Also,

Is a positive finite constant diagonal matrix

Lt; / RTI > Then, the nonlinear observer for the equation (32) is obtained as shown in equation (37)

Is obtained as shown in Equation (38), and at this time

to be.

In a step S40 of defining the first nonlinear estimator and the second nonlinear estimator, using the first and second observer errors defined as the differences between the first and second system models and their respective estimates, the first and second It is possible to define first and second nonlinear estimators for estimating an unmeasurable vector of a nonlinear observer. In this case, the non-linear estimator can be estimated from the nonlinear estimator obtained by the equation (37), and the nonlinear observer can be estimated by combining with the equation (36).

The step of estimating the non-measurable vector (S50) can estimate the unmeasurable vector of the first and second nonlinear observers using the defined first and second nonlinear estimators, and the first nonlinear observer and the second non- (S60) can estimate the first and second nonlinear observers using the estimated non-measurable vector. According to one embodiment, a non-measurable vector can be estimated from a nonlinear estimator obtained as in Equation 37 and combined with Equation 36 to estimate a nonlinear observer.

8 is a flowchart illustrating a method of estimating spatial information of a first object and a second object according to an embodiment of the present invention. The spatial information estimation method according to FIG. 8 may further include a step (S70) of estimating a scale factor and a step (S80) of estimating a relative speed in the spatial information estimation method in FIG.

In the step of estimating the scale factor (S70), scale factors, which are constants other than 0, can be estimated as factors for adjusting the scale of the state vectors related to the distance of the plurality of objects using the estimated first and second nonlinear observers have. In this case, the system model may be estimated from the nonlinear estimator obtained as shown in Equation 36 and a scale factor may be estimated as shown in Equations 33 and 34 according to an embodiment.

In the step of estimating the relative speed (S80), the relative speed of the objects with respect to the camera can be estimated from the estimated scale factors. According to an embodiment of the present invention, an estimation of the relative speed can be obtained by using the following equation (34) as shown in the following equation (35).

9 is a flowchart illustrating a spatial information estimation method for a first object and a second object according to an embodiment of the present invention. The spatial information estimation method according to FIG. 9 may further include a step (S90) of estimating the first and second system models in the spatial information estimation method in FIG. 7, wherein the first and second system models are estimated The first and second nonlinear observers can be used to estimate the first and second system models. According to an embodiment of the present invention, it is possible to estimate a non-measurable vector from a nonlinear estimator obtained by Equation (37) to be described later and to combine with Equation (36) to estimate a nonlinear observer, Can be obtained.

FIGS. 10 to 19 illustrate moving motion conditions of the camera and the first and second objects as an embodiment of the present invention. At this time, the types of motions that the camera and the objects can have include the above three motions of static, dynamic and general. Hereinafter, the above-described three motions will be described. In addition, since it is very difficult and impossible to estimate the distance and motion for a plurality of objects if relative information between two objects is not given, the following conditions are assumed. 1) the two objects are the same distance from the camera; Whether or not relative speeds between objects are given, the speed of cameras and objects must be limited; 2) the two objects have different distances from the camera; Whether or not relative distances and speed between objects are given, the motion of the camera and objects must be limited.

In Figs. 10 to 13, it is considered that the camera speed is partially or completely not given. This is distinct from the case where there is only one object. Here, the motion of the camera and the two objects has less restrictions. On the other hand, relative information between two objects is required.

는 완전히 0은 아닌 경우로서, 카메라 속도 벡터

는 완전히 주어지고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지는 경우에 있어서 카메라와 각각 다이나믹(A) 및 제너럴 모션(B)을 가지는 오브젝트들의 기하 구조를 도시한다.FIG. 10 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention. 10 assumes a situation in which the two objects have the same distance of motion from the camera and the camera looks down at the vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero, and the camera speed vector

And the geometry of the objects with dynamic (A) and general motion (B), respectively, given that one component of the relative velocity vector between the objects is given non-zero.

,

의 정보 와

를 결합함으로써 간단히 얻어질 수 있다. 따라서, 여기서 논의하지 않는다. 수학식 12의 SaM 모델을 수학식 13와 같이 변형할 수 있다.

및

가 추정 가능하므로, 수학식 13는 수학식 14과 같이 다시 쓰여질 수 있다. 이때, 측정 가능한 벡터

는

이고, 측정 불가능한 벡터

는 수학식 15와 같이 정의된다. 이때,

은 언제나 가역이다. 카메라 속도 벡터와 오브젝트들의 속도 벡터들은 유한하고, 연속적으로 미분 가능하므로,

,

,

이 양의 상수

,

,

에 대하여 성립할 수 있다.

이므로,

의 추정은 후술하는 수학식 21과 같이 디자인되는 비선형 관측기로부터 얻을 수 있다. 본문에 따라

에 대하여 수렴함이 보장되며,

의 추정은 수학식 15의

와

의 차로부터 수학식 16와 같이 얻을 수 있다.In the same case, the relative distance between two objects can be given instead of the relative speed, and the distance

,

Information and

Can be obtained easily. Therefore, it is not discussed here. The SaM model of Equation (12) can be modified as shown in Equation (13).

And

(13) can be rewritten as Equation (14). At this time,

The

And the non-measurable vector

Is defined as " (15) " At this time,

Is always reversible. Since the camera velocity vector and velocity vectors of objects are finite and continuously differentiable,

,

,

This positive constant

,

,

. ≪ / RTI >

Because of,

Can be obtained from a nonlinear observer designed as shown in Equation (21). According to the text

Is converged,

Lt; RTI ID = 0.0 > (15)

Wow

Can be obtained from the following equation (16).

의 0이 아닌 성분을, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어진 경우에 얻을 수 있으므로,

은 수학식 16로부터 수학식 17과 같이 구해진다. 수학식 17으로부터 구한

의 값을 이용할 때, 남은

의 미지 성분들과

는 수학식 16 및 수학식 15 를 이용하여 수학식 18 및 수학식 19과 같이 구해질 수 있다. 수학식 14에서의 SaM 모델에 기반하여,

에 대한 추정기는

를 정의함으로써 디자인될 수 있다. 이때,

는 측정 가능하고,

는 수학식 20와 같이 디자인된 관측기로부터 얻을 수 있다. 또한, 양의 유한한 상수 대각 행렬

로부터

정의된다. 따라서, 수학식 14에 대한 비선형 관측기는 수학식 20와 같이 정의되고,

는 수학식 21과 같이 얻을 수 있으며,

이다.

및

가 제한되므로,

,

이다. 수학식 20 및 수학식 21에서의 비선형 관측기를 이영하여, 오브젝트들의 거리와

및

의 미지 성분을 추정할 수 있다.

Can be obtained when one component of the relative velocity vector between the objects is given not to be 0,

Is obtained from the expression (16) through the expression (17). From equation (17)

, The remaining

Unknown components of

Can be obtained as shown in Equations (18) and (19) using Equations (16) and (15). Based on the SaM model in Equation 14,

The estimator for

Can be designed. At this time,

Lt; / RTI >

Can be obtained from an observer designed as shown in equation (20). Also, a positive finite constant diagonal matrix

from

Is defined. Hence, the nonlinear observer for Equation 14 is defined as Equation 20,

Can be obtained as shown in Equation (21)

to be.

And

As a result,

,

to be. The nonlinear observer in Equation (20) and Equation (21) is plotted, and the distance of the objects

And

Can be estimated.

행렬의 대각 성분은 수학식 24을 만족하고, 수학식 20 및 수학식 21에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

, 두 오브젝트들 간의 상대 속도

의 미지 성분 및

카메라와 오브젝트 간의 상대속도가 수학식 17,수학식 18 및 수학식 19로부터 구해질 수 있다.Further, for the distance and motion (SaM) equation (14), when i is 1,2,3,

The diagonal elements of the matrix satisfy Equation (24), and when using the nonlinear observer in Equations (20) and (21)

The

Lt; / RTI > like this

, The relative speed between the two objects

Unknown component of

The relative speed between the camera and the object can be obtained from the equations (17), (18) and (19).

의 하나 또는 두 개의 성분이 주어지는 경우를 고려하면,

의 주어진 성분수와 동일한 수의 오브젝트 속도

의 성분들을 다음과 같이 추정할 수 있다. SaM 방정식을 위하여, 대각 상수 행렬

가 i=1,2,3일 때 수학식 24을 만족하는 경우,

는

에 대하여 수학식 20,수학식 21의 비선형 관측기를 이용하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여,

,

및

의 미지 성분 뿐 아니라 오브젝트 속도

의 미지 성분도 추정될 수 있다. 따라서,

와

의 조합으로부터

의 미지성분에 대한 추정을 얻을 수 있다.Camera speed vector

Given the case of one or two components of < RTI ID = 0.0 >

The same number of object speeds

Can be estimated as follows. For the SaM equation, a diagonal constant matrix

When Equation 24 is satisfied when i = 1, 2, and 3,

The

Converges using the nonlinear observer of Equations (20) and (21). Using given components of camera speed,

,

And

Not only the unknown component of the object velocity

Can also be estimated. therefore,

Wow

From the combination of

Can be obtained.

FIG. 11 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention.

는 완전히 0은 아니다. 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지며, 카메라 속도 벡터

에 있어서, 두 개의 성분들이 주어진다. 각각의 오브젝트 속도 벡터

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되어야 한다.11 (a) shows a geometry in the following case for a camera and objects having dynamic (A) and general (B) motions, respectively. Both objects assume a situation in which the camera has a motion with the same distance from the camera and the camera looks down at a vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero. One component of the relative velocity vector between the objects is given instead of zero, and the camera velocity vector

, Two components are given. Each object velocity vector

, If the two components are unknown and the remainder of the given component is zero, both of them must be moved or maintained in a plane.

는 완전히 0은 아니고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어진다. 카메라 속도 벡터

에 있어서, 하나의 성분 만이 주어지며 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되어야 한다.11 (b) shows the geometry of the camera and the following cases for objects having dynamic (A) and general (B) motions, respectively. Both objects assume a situation in which the camera has a motion with the same distance from the camera and the camera looks down at a vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero, and one component of the relative velocity vector between the objects is given, not zero. Camera speed vector

, Only one component is given, and each object velocity vector

, If only one component is unknown and the remaining components are zero, then both objects must be moved or maintained along the line.

의 주어진 성분수와 동일한 수의 오브젝트 속도

의 성분들을 다음과 같이 추정할 수 있다. SaM 방정식을 위하여, 대각 상수 행렬

가 i=1,2,3일 때 수학식 24을 만족하는 경우,

는

에 대하여 수학식 20, 수학식 21의 비선형 관측기를 이용하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여,

,

및

의 미지 성분 뿐 아니라 오브젝트 속도

의 미지 성분도 추정될 수 있다. 다시 말해,

와

의 조합으로부터

의 미지성분에 대한 추정을 얻을 수 있다.In each of the other cases shown in Fig. 11,

The same number of object speeds

Can be estimated as follows. For the SaM equation, a diagonal constant matrix

When Equation 24 is satisfied when i = 1, 2, and 3,

The

Converges using the nonlinear observer of Equations (20) and (21). Using given components of camera speed,

,

And

Not only the unknown component of the object velocity

Can also be estimated. In other words,

Wow

From the combination of

Can be obtained.

및

의 미지 성분들을 다음 과 같이 추정할 수 있다. SaM 추정을 위하여, 상수 대각 행렬

가 i=1,2,3 에서 수학식 24을 만족하는 경우수학식 20, 수학식 21 에서의 비선형 관측기들을 이용하면

는

에 대하여 수렴한다. 카메라 속도 벡터의 주어진 성분들을 이용하면,

,

의 미지 성분 및

가 추정될 수 있다.In these cases, the distance of objects,

And

Can be estimated as follows. For SaM estimation, a constant diagonal matrix

When the nonlinear observers in Equation (20) and Equation (21) are used when Equation (24) is satisfied in i = 1,

The

Lt; / RTI > Using the given components of the camera velocity vector,

,

Unknown component of

Can be estimated.

는 완전히 0은 아니다. 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지며, 카메라는 스태틱 모션을 가진다.12 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention. 12 shows the geometry for the motion of the static camera and the objects having the dynamic (A) and general (B) motions, respectively, under the conditions described below. Both objects assume a situation in which the camera has a motion with the same distance from the camera and the camera looks down at a vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero. One component of the relative velocity vector between objects is given, not zero, and the camera has static motion.

의 미지 성분들 및

를 다음과 같이 추정할 수 있다. SaM 방정식 수학식 14에 대하여, I는 1,2,3인 때,

행렬의 대각 성분은 수학식 24을 만족하고, 수학식 20 및 수학식 21에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

,

의 미지 성분 및

가 수학식 17,수학식 18 및 수학식 25로부터 추정될 수 있다. 따라서, 수학식 19이 수학식 25로 간략히 써질 수 있음을 알 수 있다.In such a case, the distance of objects,

Unknown components of

Can be estimated as follows. SaM Equation For Equation 14, when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (24), and when using the nonlinear observer in Equations (20) and (21)

The

Lt; / RTI > like this

,

Unknown component of

Can be estimated from Equations (17), (18), and (25). Therefore, it can be seen that Equation (19) can be simply written as Equation (25).

는 완전히 0은 아니다. 또한 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의한다. 또한,

의 성분 중 하나는 0이 아니며 주어진다.FIG. 13 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention. 13 shows the geometry for motion of a camera having dynamic motion and objects having static (A) and general (B) motions, respectively, under the conditions described below. Both objects assume a situation in which the camera has a motion with the same distance from the camera and the camera looks down at a vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero. The camera also has dynamic motion, one of which is by static motion and the other by general motion. Also,

One of the components of < / RTI >

및 제너럴 오브젝트의 속도

를 다음과 같이 추정할 수 있다. SaM 방정식 수학식 14에 대하여, I는 1,2,3인 때,

행렬의 대각 성분은 수학식 24을 만족하고, 수학식 20 및 수학식 21에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

의 추정, 및

가 수학식 15로부터 수학식 26, 수학식 27과 같이 구해질 수 있다. 또한,

의 추정은 수학식 28과 같이 구해진다. 따라서, 스태틱 오브젝트에 대하여, 수학식 15는 수학식 29과 같이 쓰여질 수 있으므로,

및

은 수학식 29 및 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지는 것으로부터, 수학식 26, 수학식 27과 같이 구해질 수 있다. 한편, 제너럴 오브젝트에 대하여, 수학식 28로부터 추정될 수 있는

로부터 수학식 15는 수학식 30와 같이 쓰여질 수 있다.In such a case, the distance of the objects, the camera speed

And the speed of the general object

Can be estimated as follows. SaM Equation For Equation 14, when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (24), and when using the nonlinear observer in Equations (20) and (21)

The

Lt; / RTI > like this

, And

Can be obtained from Equation (15) to Equation (26) and Equation (27). Also,

Is obtained as shown in Equation (28). Therefore, for a static object, Equation (15) can be written as Equation (29)

And

(29) and the camera have dynamic motion, one of the objects is based on static motion and the other is based on general motion,

One of the components of " 0 " is not 0 and can be obtained from Equation (26) and Equation (27). On the other hand, with respect to the general object,

Equation (15) can be written as Equation (30).

를 추정하는 데 이용될 수 있다.

및

의 조건들 역시

이 완전히 미지인 경우와 같이 완화될 수 있다.Thus, a pair of dynamic camera / static objects is an unknown value of distance and camera speed, and

As shown in FIG.

And

The conditions of

Can be mitigated as if it were completely unknown.

14 to 19, it is assumed that two objects have different distances from the camera. When the objects are at the same distance from the camera, the camera speed is completely or partially unknown, and there are few restrictions on the motion of the camera and the two objects. A case where a relative distance between two objects is given and a case where a relative speed between two objects is given will be described later.

는 완전히 주어지고, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가진다. 카메라 속도와 오브젝트들의 속도는 달라야 한다. 따라서,

는 완전히 0이 아니고, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0이다.FIG. 14 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention. (A) and a second object (B) each having a dynamic motion and a camera having general motion under the following conditions. Camera speed vector

Are given completely, and the two objects have motion at different distances to the camera. Camera speed and object speed must be different. therefore,

Is not totally zero, the relative distance between the two objects is given, and the relative velocity between the objects is zero.

가 수학식 11과 같이 표현될 수 있으므로,

과

이

로부터 유지될 수 있으므로, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0 인 것은,

이 주어짐을 의미한다. 상대 속도 대신 오브젝트들 간의 상대 간격(distance)과 양자 간의 상대 거리(range) 모두 주어진 경우에, 거리는 정리 3과 오브젝트들 간의 기하적 관계를 고려하여 얻어질 수 있다. 이때, 수학식 12에서의 SaM 모델은 수학식 31과 같이 쓰여질 수 있다. 이때, i=1, 2, 3에서

는

와 같다.

와

이 모두 추정되어야 하므로, 수학식 31은 수학식 32과 같이 다시 쓰여질 수 있다. 이때,

은 언제나 가역이다. 또한,

,

,

이 양의 상수

,

,

에 대하여 성립한다.

이므로,

의 추정은 후술하는수학식 37과 같은 관측기로부터 얻어질 수 있으며,

에 대하여 수렴함은 후술하는 바에 의하여 증명되고,

의 추정은 후술하는 것과 같이 구할 수 있다. 첫째로, 수학식 33를 이용하여

및

를 얻을 수 있고, 결합하여 수학식 34을 얻을 수 있다. 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지며, 카메라 속도와 오브젝트들의 속도는 다르고, 따라서,

는 완전히 0이 아닌 경우에, 수학식 34으로부터 얻은

의 추정을 이용하여,

의 추정을 수학식 33로부터 수학식 35와 같이 얻을 수 있다. 따라서,

의 추정은 수학식 33와 수학식 35를 이용하여 수학식 36와 같이 구할 수 있다.In this case,

Can be expressed as Equation (11)

and

this

The relative distance between the two objects is given, and the relative velocity between the objects is zero,

Is given. Given the relative distance between objects and the relative distance between them instead of the relative velocity, the distance can be obtained by taking into account the geometric relationship between theorem 3 and objects. At this time, SaM model in Equation (12) can be written as Equation (31). At this time, i = 1, 2, 3

The

.

Wow

(31) can be rewritten as Equation (32). At this time,

Is always reversible. Also,

,

,

This positive constant

,

,

.

Because of,

Can be obtained from an observer such as Equation (37)

Is proved by the following,

Can be obtained as described below. First, using Equation 33,

And

(34) can be obtained. The two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different,

Lt; RTI ID = 0.0 > 34 < / RTI >

Lt; / RTI >

Can be obtained from the equation (33) as shown in the equation (35). therefore,

Can be obtained as shown in Equation (36) using Equations (33) and (35).

의 추정기는

를 정의함으로써 얻을 수 있다. 이때,

는 후술하는 수학식 37에서의 관측기로부터 얻어진다. 또한,

는 양의 유한 상수 대각 행렬

에 대해 정의된다. 그러면, 수학식 32에 대한 비선형 관측기는 수학식 37과 같이 구해지고,

는 수학식 38과 같이 얻어지고, 이때

이다.

와

가 제한되므로,

,

,

이다. 수학식 37 및 수학식 38로부터 정의되는 비선형 관측기를 이용하여, 오브젝트들의 거리 및

을 다음과 같이 추정할 수 있다.Based on the SaM model in equation (32)

The estimator of

. ≪ / RTI > At this time,

Is obtained from the observer in the following Expression (37). Also,

Is a positive finite constant diagonal matrix

Lt; / RTI > Then, the nonlinear observer for the equation (32) is obtained as shown in equation (37)

Is obtained as shown in Equation (38), and at this time

to be.

Wow

As a result,

,

,

to be. Using the nonlinear observer defined by equations (37) and (38), the distance of objects and

Can be estimated as follows.

행렬의 대각 성분은수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

및

이 수학식 35, 수학식 36로부터 추정된다. 정리하면,

는 수학식 34과같이 추정될 수 있다. 따라서,

는 수학식 35,

은 수학식 36와 같이 추정될 수 있다.SaM model For equation 32, when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this

And

Is estimated from Equations (35) and (36). In short,

Can be estimated as shown in Equation (34). therefore,

(35)

Can be estimated as shown in Equation (36).

FIG. 15 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.

에 있어서, 두 개의 성분들이 주어지고, 각각의 오브젝트 속도 벡터

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되어야 하고, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지며 카메라 속도와 오브젝트들의 속도는 달라야 한다. 따라서,

는 완전히 0이 아니다. 또한, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0인 경우에, 제너럴 모션을 가지는 카메라와 각각 다이나믹 모션을 가지는 제1 오브젝트(A) 및 제2 오브젝트(B)에 대하여 기하구조를 도시한 것이다.15 (a) shows the camera velocity vector

, Two components are given, and each object velocity vector

, If the two components are unknown and the remainder of the given component is zero then both objects must move or stay in the plane and the two objects have motion at different distances to the camera and the camera speed and the speed of the objects are different do. therefore,

Is not completely zero. In addition, when a relative distance between two objects is given, and a relative speed between the objects is 0, a camera having general motion and a first object (A) and a second object (B) Fig.

에 있어서, 하나의 성분 만이 주어지고, 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되어야 하며, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지며 카메라 속도와 오브젝트들의 속도는 달라야 한다. 따라서,

는 완전히 0이 아니다. 또한, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0인 경우에, 제너럴 모션을 가지는 카메라와 각각 다이나믹 모션을 가지는 제1 오브젝트(A) 및 제2 오브젝트(B)에 대하여 구조를 도시한 것이다.FIG. 15 (b)

, Only one component is given, and each object velocity vector

The two objects must have motion at different distances relative to the camera, and the camera speed and the speed of the objects Should be different. therefore,

Is not completely zero. In addition, when relative distances between the two objects are given and the relative speed between the objects is 0, the camera having the general motion and the first object (A) and the second object (B) FIG.

행렬의 대각 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여,

,

및

과

의 미지 성분을 추정할 수 있다. 카메라 모션이 스태틱인 경우를 가정한다. 이 경우에, 오브젝트들의 거리

와

에 대하여 추정할 수 있다. 또한, 수학식 32의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

및

는 수학식 35와 수학식 42로부터 추정될 수 있다. 따라서,

=0 과 수학식 36를 결합하여 수학식 42로 간략히 쓸 수 있다.In the case of Fig. 15, from the SaM equation of the equation (32), when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > Using given components of camera speed,

,

And

and

Can be estimated. It is assumed that the camera motion is static. In this case, the distance

Wow

. ≪ / RTI > From the SaM equation of the equation (32), when I is 1, 2, and 3, a constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this,

And

Can be estimated from Equations (35) and (42). therefore,

= 0 and Equation (36) can be concatenated into Equation (42) briefly.

16 illustrates motion conditions of a camera and a plurality of objects according to an exemplary embodiment of the present invention.

는 완전히 0이 아니고, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0인 경우에 카메라와 각각 다이나믹 모션을 가지는 제1 오브젝트(A) 및 제2 오브젝트(B)의 구조를 도시한 것이다.16 (a), the camera has a static motion, and the two objects have motions at different distances to the camera. Camera speed and object speed must be different. therefore,

(A) and the second object (B), each of which has a dynamic motion with the camera, when the relative distance between the two objects is zero and the relative speed between the objects is zero It is.

는 완전히 주어지고, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가진다. 카메라 속도와 오브젝트들의 속도는 달라야 한다. 따라서,

는 완전히 0이 아니고, 오브젝트들 간의 상대 거리가 주어진다. 또한, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 경우에, 카메라와 각각 다이나믹(A) 및 제너럴(B) 모션을 가지는 오브젝트들의 구조를 도시한 것이다.16 (b) shows the camera velocity vector

Are given completely, and the two objects have motion at different distances to the camera. Camera speed and object speed must be different. therefore,

Is not completely zero, and the relative distance between objects is given. Also, it shows the structure of the camera and the objects having the dynamic (A) and general (B) motions, respectively, given a part of the relative velocity between the objects.

와

가 모두 추정되어야 하므로, 수학식 43는 수학식 44과 같이 쓰여질 수 있다. 이때,

는

을 만족하는 측정 가능한 벡터이고, 측정 불가능한 벡터는

으로 정의되며, 수학식 45와 같이 쓰여질 수 있다. 수학식 45가 수학식 46와 같이 표현될 수 있으므로, 그 차는 수학식 47과 같다. 따라서,

는 수학식 47으로부터

및

는 오브젝트들 간의 상대 거리가 주어진다. 또한, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 가정으로부터 주어지므로, 수학식 48과 같이 얻어질 수 있다. 수학식 48에서의

를 이용하면,

는 수학식 46로부터 수학식 49와 같이 얻어질 수 있다. 이때, 주어지는

로부터

는 수학식 50과 같이 주어진다. 수학식 49, 수학식 50으로부터

의 추정을 이용하여,

의 추정은 수학식 45와 수학식 49, 수학식 50을 이용하여 수학식 51과 같이 구해질 수 있으며,

의 미지 성분은 수학식 51로부터 수학식 52과 같이 얻을 수 있다. 따라서, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용하여, 오브젝트들의 거리

,

및

의 미지 성분들을 다음과 같이 구할 수 있다.In the case of (a) and (b) above, the SaM model of Equation (12) can be written as Equation (43).

Wow

(43) can be written as shown in Equation (44). At this time,

The

, And the non-measurable vector is a measurable vector satisfying

, And can be written as shown in Equation (45). Since Equation 45 can be expressed as Equation 46, the difference is represented by Equation 47. [ therefore,

≪ RTI ID = 0.0 >

And

Is given a relative distance between the objects. In addition, since it is given from the assumption that a part of the relative velocity between the objects is given, it can be obtained as shown in equation (48). In Equation 48,

In this case,

Can be obtained from Equation (46) as Equation (49). At this time,

from

Is given by Equation (50). From equations (49) and (50)

Lt; / RTI >

Can be obtained by using Equation (45), Equation (49), and Equation (50) as Equation (51)

Can be obtained from the expression (51) to the expression (52). Thus, using the nonlinear observer in equations (37) and (38), the distance of objects

,

And

Can be obtained as follows.

의 성분은 수학식 41을 만족하고, 수학식 37 및 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

,

,

및

의 미지 성분을 수학식 49, 수학식 50, 수학식 51, 수학식 52로부터 추정할 수 있다.

는 전술한 수학식 49, 수학식 50에 따라 추정되고, 이에 따라,

및

의 미지 성분은 수학식 51 및 수학식 52에서와 같이 추정된다.From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation 41, and when using the nonlinear observer in Equations 37 and 38

The

Lt; / RTI > like this,

,

,

And

Can be estimated from Equation (49), Equation (50), Equation (51), and Equation (52).

Is estimated according to the above-described Equations 49 and 50,

And

Is estimated as shown in Equation (51) and Equation (52).

FIG. 17 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention.

에 있어서, 두 개의 성분들이 주어지고, 각각의 오브젝트 속도 벡터

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되어야 하며 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고 카메라 속도와 오브젝트들의 속도는 달라

는 완전히 0이 아니고, 오브젝트들 간의 상대 거리가 주어진다. 또한, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 경우에, 제너럴 모션을 가지는 카메라와 각각 다이나믹(A) 및 제너럴(B) 모션을 가지는 오브젝트들의 구조를 도시한 것이다.17 (a) shows the camera velocity vector

, Two components are given, and each object velocity vector

, If the two components are unknown and the remainder of the given component is 0, then both objects must move or stay in a plane, and the two objects have motion at different distances to the camera,

Is not completely zero, and the relative distance between objects is given. In addition, a camera having general motion and a structure of objects having dynamic (A) and general (B) motions, respectively, are shown when a part of the relative speed between objects is given.

에 있어서, 하나의 성분 만이 주어지고, 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되어야 하며 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고 카메라 속도와 오브젝트들의 속도는 달라야 하여,

는 완전히 0이 아니고, 오브젝트들 간의 상대 거리가 주어지고, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 경우에, 제너럴 모션을 가지는 카메라와 각각 다이나믹(A) 및 제너럴(B) 모션을 가지는 오브젝트들의 구조를 도시한 것이다.17 (b) shows the camera velocity vector

, Only one component is given, and each object velocity vector

, The objects of both are moved or maintained along a line, and the two objects have motion at different distances to the camera and the camera speed and the speed of the objects are Should be different,

(A) and general (B) motion of the camera having general motion and the motion of the object having motion of general (B) motion, respectively, when the relative distance between the objects is given, FIG.

의 하나 또는 두 개의 성분이 주어진 경우에,

의 주어진 성분 수와 동일한 수의

의 미지 성분을 추정할 수 있다.In the above cases (a) and (b), the camera speed vector

Given one or two components of < RTI ID = 0.0 >

The same number of components as the given number of components of

Can be estimated.

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용할 때,

및

의 미지 성분뿐만 아니라, 오브젝트 속도

의 미지성분 역시 동일한 성분 수만큼 추정할 수 있다. 따라서,

와

의 조합으로부터

의 추정을 얻을 수 있다.From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > When using given components of camera speed,

And

Not only the unknown component of the object velocity

Can be estimated by the same number of components. therefore,

Wow

From the combination of

Can be obtained.

가 추정될 수 있다. 위(a), (b)와 같은 경우들에 있어서, 오브젝트들의 거리,

및

와

의 미지 성분들을 추정할 수 있다. 수학식 32의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여

,

및

와

의 미지 성분을 추정할 수 있다.At this time,

Can be estimated. In cases such as (a) and (b) above, the distance of objects,

And

Wow

Can be estimated. From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > Using given components of camera speed

,

And

Wow

Can be estimated.

는 완전히 0이 아니며, 오브젝트들 간의 상대 거리가 주어진다. 또한, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 경우에 스태틱 모션을 가지는 카메라와 각각 다이나믹(A) 및 제너럴(B) 모션을 가지는 오브젝트들의 구조를 도시한 것이다.18 shows motion conditions of a camera and a plurality of objects according to an embodiment of the present invention. Figure 18 shows that the camera has static motion and both objects have motion at different distances relative to the camera. Camera speed and object speed must be different. therefore,

Is not completely zero, and the relative distance between the objects is given. Further, FIG. 5 shows the structure of a camera having static motion and objects having dynamic (A) and general (B) motions, respectively, when a part of the relative speed between objects is given.

의 미지 성분들 및

를 다음과 같이 구할 수 있다. 수학식 44의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

,

,

및

의 미지 성분에 대하여 수학식 49, 수학식 50, 수학식 53 및 수학식 52을 이용하여 추정할 수 있다. 따라서, vc=0 과 수학식 51를 결합하여 수학식 53으로 간략히 할 수 있다.In this case, the distance of objects,

Unknown components of

Can be obtained as follows. From the SaM equation of Equation (44), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this,

,

,

And

Can be estimated using Equation (49), Equation (50), Equation (53), and Equation (52) with respect to the unknown component of the input image. Therefore, it is possible to simplify the equation (53) by combining vc = 0 and equation (51).

의 성분 중 하나는 0이 아니며 주어진다. 또한, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가진다. 카메라 속도와 오브젝트들의 속도는 달라야 한다. 따라서,

는 완전히 0이 아니며, 오브젝트들 간의 상대 거리가 주어진다. 한편, 두 오브젝트들 간의 상대 속도는 일부 주어지는 경우에, 카메라와 각각 스태틱(A) 및 제너럴 모션(B)을 가지는 오브젝트를 도시한 것이다.FIG. 19 illustrates motion conditions of a camera and a plurality of objects according to an embodiment of the present invention. The camera has dynamic motion, one of which is by static motion and the other by general motion. Also,

One of the components of < / RTI > Also, the two objects have motions at different distances to the camera. Camera speed and object speed must be different. therefore,

Is not completely zero, and the relative distance between the objects is given. On the other hand, in the case where a relative speed between two objects is given, a camera and an object having static (A) and general motion (B), respectively.

는

과 같이 정의 될 수 있고, 오브젝트들의 거리,

의 미지 성분 및

를 추정하고 이를 기초로 수학식 33 대신 수학식 54와 같이 표현될 수 있다.In this case, in this case, Equation

The

And the distance of objects,

Unknown component of

And can be expressed by Equation (54) instead of Equation (33) on the basis of this.

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때는

에 대하여 수렴한다. 이와 같이

및

의 추정을 다음과 같이 구할 수 있다. 수학식 55, 수학식 56. 또한,

및

의 추정을 다음과 같이 구할 수 있다. 수학식 57, 수학식 58. 스태틱 오브젝트에 대하여, 수학식 54는 수학식 59과 같이 표현 될 수 있다. 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지는 것으로부터,

는 0이 아님을 알 수 있으므로,

및

는 수학식 59로부터 수학식 55및 수학식 56와 같이 구할 수 있다. 한편, 제너럴 오브젝트에 대하여, 수학식 54는 수학식 60과 같이 표현될 수 있다. 이때,

는

을 만족하며 0이 아니고 주어진다. 따라서,

는 수학식 60 로부터 수학식 57와 같이 구해지고,

는 수학식 58과 같이 추정될 수 있다.From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38) The

Lt; / RTI > like this

And

Can be obtained as follows. Equations (55) and (56) Furthermore,

And

Can be obtained as follows. Equation 57, Equation 58. For a static object, Equation 54 can be expressed as Equation 59. " (59) " The camera has dynamic motion, one of which is by static motion and the other by general motion,

Lt; RTI ID = 0.0 > 0 < / RTI >

Is not 0, and therefore,

And

Can be obtained from Equation (59) to Equation (55) and Equation (56). On the other hand, with respect to the general object, Equation (54) can be expressed as Equation (60). At this time,

The

Is satisfied and is not 0. therefore,

Is obtained from Expression (60) as Expression (57)

Can be estimated as shown in Equation (58).

20 is a block diagram of a plurality of object space estimation apparatuses according to an embodiment of the present invention.

20 includes a first camera unit 1010, a first object motion modeling unit 1020, a first subtractor 1030, a first nonlinear observer unit 1040, an estimator (not shown) 1051 to 1059), a relative distance information section 1060 between objects, and a first relative speed information section 1070.

The first camera unit 1010 includes a first camera unit 1011, a first camera unit 1013, a first camera unit 1015, a first camera unit 1017, -5 camera unit 1019. The camera-

The 1-1 camera unit 1011 is a camera having general motion and can acquire the coordinates of the minutiae point of the object included in the image of the object for which the spatial information is to be estimated. The 1-2 camera unit 1013 is a camera having static motion, and can acquire the coordinates of the minutiae point of the object included in the image of the object to which the spatial information is to be estimated. The 1-3 camera unit 1015 is a camera having general motion and can acquire the coordinates of the minutiae point of the object included in the image of the object for which the spatial information is to be estimated. The first to fourth camera unit 1017 is a camera having static motion, and can acquire the coordinates of the minutiae point of the object included in the image of the object to which the spatial information is to be estimated. The 1-5 camera unit 1019 can acquire coordinates of minutiae points of an object included in an image of an object for which spatial information is to be estimated, as a camera having a general or static motion. Each camera unit can transmit the coordinates of the minutiae point of the acquired object to the first object motion modeling unit 1020. [

The first object motion modeling unit 1020 receives the coordinates of the obtained minutiae as input, and generates a system model of an object having general motion. The first object motion modeling unit 1020 can generate a system model assuming that the first object and the second object have different distances from the camera.

The first object motion modeling unit 1020 may include a first-first SaM dynamic unit 1021, a first-second SaM dynamic unit 1023, and a 1-3 SaM dynamic unit 1025.

In the case of two objects having dynamic motion, the first-first SaM dynamic part 1021 is given the difference between the distances of the first object and the second object, and when the relative speed between the first and second objects is 0 You can create a system model.

In the case of two objects having dynamic and general motions, the first and second SaM dynamic components 1023 are given a relative distance between the first and second objects, and a system model when a relative speed between the two is given Can be generated.

In the case of two objects having static and general motions, the first and third SaM dynamic components 1025 are provided with a system model in the case where relative distances between the first and second objects are not given, Can be generated.

The first subtractor 1030 subtracts the estimates of the system model generated by the first object motion modeling unit 1020 and the system model estimated by the first nonlinear observer 1040, To the nonlinear observer (1040).

The first nonlinear observer 1040 designes the first nonlinear observer using the system model input from the first subtractor 1030 and the subtraction value of the estimated system model and the coordinate information of the object, It is possible to design an estimator of an unmeasurable vector.

The estimators 1051 to 1059 are able to receive estimates of unmeasurable vectors from the first nonlinear observer 1040 and calculate estimates of spatial information that can not be given or measured in advance.

The first estimator 1051 may estimate the distance of the objects from the camera and the relative velocities between the objects and the camera, given a relative distance between the objects.

The second estimator 1053 may estimate the distance of the objects from the camera and the velocity of the objects, given a relative distance between the objects.

The third estimator 1055 is given a relative distance between the objects, and when given a relative speed between the objects, the third estimator 1055 calculates the distance between the objects from the camera, the relative speed between the camera and the objects, Can be estimated.

The fourth estimator 1057 may estimate the distance of the objects from the camera, the speed of the objects, and the unknown component of the relative speed between the objects, given a relative distance between the objects and given a relative speed between the objects have.

The fifth estimator 1059 may estimate the distance of the objects from the camera, the camera speed, and the speed of the general object, if some relative speed between the objects is given.

The relative distance information unit 1060 between the objects can transmit the distance information between the objects and the camera, which are calculated or given in advance, to the first to fourth estimators 1051, 1053, 1055, and 1057. In this embodiment, a spatial information estimation method in which distance information between a camera and objects is given or measured is exemplified. Information on a relative distance between objects, which is a difference in distance from the camera, is inputted or calculated in advance, To the fourth estimating units 1051, 1053, 1055, and 1057, respectively.

The first relative speed information unit 1070 may transmit the relative speed information between the objects calculated or given in advance to the third through fifth estimators 1055, 1057, and 1059.

FIG. 21 is a block diagram of a plurality of object space estimation apparatuses according to an embodiment of the present invention.

21, the spatial estimating apparatus for a plurality of objects includes a second camera unit 1110, a second object motion modeling unit 1120, a second subtractor 1130, a second nonlinear observer unit 1140, 8 estimation units 1151, 1153, and 1155, and a second relative speed information unit 1160. [

The second camera unit 1110 may include a second-first camera unit 1111, a second-second camera unit 1113, and a second-third camera unit 1115.

The second-second camera unit 1113 includes a camera having static motion, and the second-third camera unit 1115 includes a camera having general motion. The second-second camera unit 1113 includes a camera having static motion. Each of the camera units may acquire coordinates of object feature points included in the images of the objects for which spatial information is to be estimated, through a camera having general, static, or dynamic motion.

The second object motion modeling unit 1120 may include a 2-1 SaM dynamic unit 1121 and a 2-2 SaM dynamic unit 1123. [

The 2-1 SaM dynamic section 1121 can generate a system model by assuming that two objects having the dynamic motion and the general motion respectively have the same distance from the camera. The 2-2 SaM dynamic section 1123 can generate a system model by assuming that two objects having static and general motions respectively have the same distance from the camera.

The 2-1 SaM dynamic section 1121 can generate a system model when given a part of the relative velocity between two objects having dynamic motion and general motion, respectively, and having a relative velocity other than zero.

The 2-2 SaM dynamic section 1123 can generate a system model when the relative speed between two objects having static and general motions is not given, respectively.

The second subtractor 1130 subtracts the estimates of the system model generated by the second object motion modeling unit 1120 and the system model estimated by the second nonlinear observer 1140, To the nonlinear observer 1140.

The second nonlinear observer 1140 designes the second nonlinear observer using the system model input from the second subtractor 1130 and the subtraction value of the estimated system model and the coordinate information of the object, It is possible to design an estimator of an unmeasurable vector.

The sixth to eighth estimators 1151, 1153, and 1155 can receive the estimates of the unmeasurable vectors from the second nonlinear observer 1140 and calculate the estimated values of the spatial information that can not be given or measured in advance.

The sixth estimator 1151 can estimate the distance of the objects from the camera, the unknown component of the relative velocity between the objects, and the relative velocities between the objects and the camera, if the relative velocity between the objects is not zero and a part is given.

The seventh estimator 1153 can estimate the distance of the objects from the camera, the unknown component of the relative velocity between the objects, and the velocity of the objects, if the relative velocity between the objects is not zero and a part is given.

The eighth estimator 1157 may estimate the distance of the objects from the camera, the speed of the camera, and the speed of the general object.

The second relative speed information unit 1160 may transmit the relative speed information between the calculated or given objects to the sixth to eighth estimation units 1155, 1157, and 1159.

및

로 정의한다. 이때,

는 초기 시간 t0에서의 초기 위치에서의 카메라에 결합되고,

는

가 초기시간으로부터 회전 행렬

및 변환 벡터

를 통해 변환된 직교 좌표를 의미한다. 카메라로부터 촬영된 특징점의 직교 좌표는Hereinafter, assumptions, mathematical expressions, and proofs for implementing the spatial information estimation method and apparatus according to the present invention will be described. To derive a camera-object relative motion model, the two Cartesian coordinates with respect to the camera are

And

. At this time,

Is coupled to the camera at an initial position at an initial time t0,

The

From the initial time,

And transform vector

Which are transformed through the coordinate system. Cartesian coordinates of the feature points taken from the camera are

에서 정규화된 직교 좌표계

는 수학식 2가된다.. Camera frame

Lt; RTI ID = 0.0 > orthogonal < / RTI &

&Quot; (2) "

Thus, the pixel coordinates in the image space are

이 이미지 공간에서 특징점의 좌표일 때, u,v,

는 일정하게 주어지고 가역적인 카메라 교정(calibration) 행렬이다. P와

를 이용하여

를 구할 수 있다. 따라서, SaM 모델을 구현하기 위하여, 상태 벡터 y를 다음과 같이 정의한다.. At this time,

In this image space, u, v,

Is a constant and a reversible camera calibration matrix. P and

Using

Can be obtained. Therefore, in order to implement the SaM model, the state vector y is defined as follows.

,

는

으로부터 얻어질 수 있으며, 상대 유클리드 거리

와 관련되므로, 이는 거리 추정값을 얻기 위하여 추정되어야 한다.

,

The

And the relative Euclidian distance

, It should be estimated to obtain the distance estimate.

Assuming that a single camera shoots two objects, we need to define the following variables for the two objects. Using the subscripts a and b for each object, the state vectors of the respective objects

.

에서의 특징점 좌표 qc는(qa는 a qb는 b) In the same way,

The feature point coordinate qc (qa is a bb is b)

이고,

는

좌표계의 원점으로부터

까지의 벡터이다. 상대 모션의 동역학은 As shown in Fig. At this time,

ego,

The

From the origin of the coordinate system

Lt; / RTI > The kinematics of relative motion

는

에서의 카메라 각속도 벡터이고, 비대칭(skew-symmetric) 행렬

와 같이 정의된다.Lt; / RTI >

The

And the skew-symmetric matrix < RTI ID = 0.0 >

Respectively.

에서의

에 상대적인 카메라 선속도가 두 개의 오브젝트들에 대하여 정의되므로, 카메라와 각 오브젝트들 간의 상대 속도는

In

Since the relative camera linear velocity is defined for two objects, the relative velocity between the camera and each object is

As shown in Fig.

는

에서의

의 선속도로서,

와 같이 구해지고, R은

와 관성 기준 좌표계 간의 회전 매트릭스이고,

는

의 관성 좌표계에서의 선속도 벡터이고, 카메라 속도

는 카메라의 선속도 벡터이다.At this time,

The

In

As a linear velocity of

, And R is

And an inertia reference coordinate system,

The

Is a linear velocity vector in the inertial coordinate system of the camera,

Is the linear velocity vector of the camera.

는 유한하고(bounded) 연속적으로 미분 가능하고, 또한,

를 알 수 있는 경우에, 카메라 각속도 벡터는 두개의 연속적인 2차원 이미지로부터 등극선 기하를 통하여 추정할 수 있다. 따라서, 카메라의 선속도 추정에만 집중한다.Thus, the camera velocity vector vc (t) and the object velocity vector

Is bounded and continuously differentiable, and further,

, The camera angular velocity vector can be estimated from two successive two-dimensional images through the isodose geometry. Therefore, we focus only on estimating the linear velocity of the camera.

및

을 이용할 때, 일부 측정 가능한 상태 벡터

에 대한 동역학은 다음과 같이 나타낼 수 있다.

And

, Some measurable state vectors

Can be expressed as follows.

의 값은

로부터 스케일 팩터를 통하여 구할 수 있으므로,

The value of

Can be obtained from the scale factor,

로부터 구할 수 있다),

는 시간에 따라 변하는

대신 구하고 거리 추정에 이용할 수 있다. 측정 가능한 벡터

으로 정의할 때, 수학식 10로부터 측정 가능한 상태들에 대한 SaM 모델은 수학식 12와 같이 정의 될 수 있다.To a nonzero unknown constant

, ≪ / RTI >

Varies with time

Instead, it can be used to estimate the distance. Measurable vector

, The SaM model for measurable states from Equation (10) can be defined as Equation (12).

Assuming that the SaM estimation for a plurality of objects is very difficult and impossible if relative information is not given between two objects, the following conditions are assumed. I) The two objects have the same distance from the camera; Whether or not relative speeds between objects are given, the speeds of the camera and objects must be limited; ii) the two objects have different distances from the camera; Whether or not relative distances and speed between objects are given, the motion of the camera and objects must be limited.

Here, consideration is given to the case where the camera speed is not given in part or all. This is distinct from the case where there is only one object. Here, the motion of the camera and the two objects has less restrictions. On the other hand, relative information between two objects is required.

When the objects have the same distance from the camera, various estimation methods of distance and relative information for the two objects are presented according to the following assumptions.

는 완전히 0은 아닌 경우를 가정한다.Both objects assume a situation in which the camera has a motion with the same distance from the camera and the camera looks down at a vehicle or objects located on the ground in the air. Also, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero.

,

는 수학식 11와 같이 0이 아닌 상수

,

에 따라

와 같이 나타낼 수 있으며, 두 오브젝트들은 카메라로부터 거리가 동일한 모션을 가지며, 카메라가 공중에서 평지에 위치하는 차량 또는 오브젝트들을 내려다보는 상황에서, 또한, 카메라 속도와 각 오브젝트들의 속도는 같지 아니하며, 카메라와 각 오브젝트들 간의 속도

는 완전히 0은 아닌 경우에, 다음 등식이 성립한다.

. 따라서,

및

및

및

의 관계 중 어느 관계이든 사용할 수 있다. 따라서

및

의 등식들을 사용하여 간략히 할 수 있다.Street

,

Is a constant other than 0, as shown in Equation (11)

,

Depending on the

And the two objects have the same distance from the camera and the camera is looking at a vehicle or objects located on the ground in the air and the camera speed and the speed of each object are not the same, The speed between each object

Is not completely zero, the following equation holds.

. therefore,

And

And

And

Or the like. therefore

And

Can be simplified using equations

Depending on whether the relative speed between the objects is given or not, the SaM of the objects is estimated differently.

If the relative speed between the objects is not zero and a part is given, the relative speed between the object and the camera and the relative speed between the camera and the object can be estimated as follows. For distance estimation, the following assumptions are introduced.

는 완전히 주어질 수 있고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어질 수 있다.Camera speed vector

Can be given completely, and one component of the relative velocity vector between the objects can be given instead of zero.

는 완전히 0은 아니며, 카메라 속도 벡터가 완전히 주어지고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어짐에 따라 제너럴 모션을 가질 수 있다.Since at least one object must have dynamic motion, even though the relative velocity between the objects must be a non-zero vector, other objects and cameras may have the same motion with distance from the camera, Assuming a situation in which the vehicle or objects are overlooked, the camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not completely zero, the camera speed vector is given completely, and one component of the relative velocity vector between the objects is not zero, and can have general motion as given.

는 완전히 0은 아닌 경우에 해당하지 않는 경우에, 카메라가 0의 상대 속도로 움직이는 오브젝트를 응시함으로써 더 이상의 정보를 얻을 수 없다. 따라서,

는 0이 아닌 벡터여야 한다.Therefore, even when one component of the relative velocity vector between the objects is given, rather than 0, the two objects have the same motion distance from the camera, and in a situation where the camera looks down at the vehicle or objects located on the ground in the air, The camera speed and the speed of each object are not the same, and the speed between the camera and each object

Is not totally nonzero, then the camera can not get any more information by staring at the moving object at a relative speed of zero. therefore,

Must be a non-zero vector.

는 완전히 0은 아닌 경우에 해당하기 위해서는, 수학식 10에서 보인 것 과 같이

일 것을 요구한다.On the other hand, in the situation where the two objects have the same motion distance from the camera and the camera looks down the vehicle or objects located on the ground in the air, the camera speed and the speed of each object are not the same, speed

In order to correspond to the case where the value is not completely 0,

.

,

의 정보 와

를 결합함으로써 간단히 얻어질 수 있다. 따라서, 여기서 논의하지 않는다.Also, the relative distance between two objects can be given instead of the relative speed, and the distance

,

Information and

Can be obtained easily. Therefore, it is not discussed here.

는 완전히 0은 아니며, 카메라 속도 벡터가 완전히 주어지고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지는 경우를 고려하였을 때, 수학식 12의 SaM 모델을The two objects have the same distance from the camera, and the camera assumes a situation in which the camera looks at a vehicle or objects lying on the ground in the air. The camera speed and the speed of each object are not the same,

Is not completely zero, and given that the camera velocity vector is completely given and one component of the relative velocity vector between the objects is given instead of zero, the SaM model of equation (12)

및

가 추정 가능하므로, 수학식 13는 Can be modified as shown in Equation (13).

And

Lt; RTI ID = 0.0 > (13) < / RTI &

는

이고, 측정 불가능한 벡터

는 Can be rewritten as. At this time,

The

And the non-measurable vector

The

은 언제나 가역이다. 카메라 속도 벡터와 오브젝트들의 속도 벡터들은 유한하고, 연속적으로 미분 가능하므로,

,

,

이 양의 상수

,

,

에 대하여 성립할 수 있다.(15) " At this time,

Is always reversible. Since the camera velocity vector and velocity vectors of objects are finite and continuously differentiable,

,

,

This positive constant

,

,

. ≪ / RTI >

이므로,

의 추정은 후술하는 수학식 21과 같이 디자인되는 비선형 관측기로부터 얻을 수 있다. 본문으로부터

에 대하여 수렴함이 보장되며,

의 추정은 수학식 15의

와

의 차로부터

Because of,

Can be obtained from a nonlinear observer designed as shown in Equation (21). From the text

Is converged,

Lt; RTI ID = 0.0 > (15)

Wow

From the car of

(16).

의 0이 아닌 성분을 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어진 경우에 얻을 수 있으므로,

은 수학식 16로부터

Can be obtained when one component of the relative velocity vector between the objects is given as not 0,

≪ RTI ID = 0.0 >

의 값을 이용할 때, 남은

의 미지 성분들과

는 수학식 16 및 수학식 15 를 이용하여 (17). From equation (17)

, The remaining

Unknown components of

Using Equation 16 and Equation 15

Can be obtained as follows.

에 대한 추정기는

를 정의함으로써 디자인될 수 있다. 이때,

는 측정 가능하고,

는 수학식 20와 같이 디자인된 관측기로부터 얻을 수 있다. 또한, 양의 유한한 상수 대각 행렬

로부터

정의된다. 따라서, 수학식 14에 대한 비선형 관측기는 Based on the SaM model in Equation 14,

The estimator for

Can be designed. At this time,

Lt; / RTI >

Can be obtained from an observer designed as shown in equation (20). Also, a positive finite constant diagonal matrix

from

Is defined. Thus, the nonlinear observer for (14)

는 Lt; / RTI >

The

이다. 따라서, 수학식 14과 수학식 20로부터 오차 동역학은 As shown in FIG.

to be. Thus, from equations (14) and (20), the error dynamics

및

가 제한되므로,

,

이다. 수학식 20 및 수학식 21에서의 비선형 관측기를 이용하여, 오브젝트들의 거리와

및

의 미지 성분을 후술하는 것과 같이 추정할 수 있다.Can be obtained.

And

As a result,

,

to be. Using the nonlinear observer in Equations (20) and (21), the distance of the objects and

And

Can be estimated as described below.

는 완전히 0은 아니며, 카메라 속도 벡터가 완전히 주어지고, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지는 가정이 만족된다고 가정한다. SaM 방정식수학식 14에 대하여, I는 1,2,3인 때,

행렬의 대각 성분은The two objects have the same distance from the camera, and the camera assumes a situation in which the camera looks at a vehicle or objects lying on the ground in the air. The camera speed and the speed of each object are not the same,

Is not completely zero, and it is assumed that the assumption that the camera velocity vector is completely given and that one component of the relative velocity vector between the objects is not zero is satisfied. SaM Equation For Equation 14, when I is 1, 2, 3,

The diagonal elements of the matrix are

는

에 대하여 수렴한다. 이와 같이

, 두 오브젝트들 간의 상대 속도

의 미지 성분 및

카메라와 오브젝트 간의 상대속도가 수학식 17,수학식 18 및 수학식 19로부터 구해질 수 있다.And using the nonlinear observer in equations (20) and (21)

The

Lt; / RTI > like this

, The relative speed between the two objects

Unknown component of

The relative speed between the camera and the object can be obtained from the equations (17), (18) and (19).

의 하나 또는 두 개의 성분이 주어지는 경우를 다음과 같이 고려한다. 카메라 속도 벡터

에 있어서, 두 개의 성분들이 주어지는 경우, 카메라 속도 벡터

에 있어서, 하나의 성분 만이 주어지는 경우,Camera speed vector

The following is taken into consideration. Camera speed vector

, When two components are given, the camera velocity vector

In the case where only one component is given,

의 주어진 성분수와 동일한 수의 오브젝트 속도

의 성분들을 다음과 같이 추정할 수 있다. then,

The same number of object speeds

Can be estimated as follows.

는 완전히 0은 아니며, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지고, 카메라 속도 벡터에 있어서 하나 또는 두 개의 성분이 주어진다고 가정한다. SaM 방정식을 위하여, 대각 상수 행렬

가 i=1,2,3일 때 수학식 24을 만족하는 경우,

는

에 대하여 수학식 20, 수학식 21의 비선형 관측기를 이용하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여,

,

및

의 미지 성분뿐 아니라 오브젝트 속도

의 미지 성분도 추정될 수 있다.The two objects have the same distance from the camera, and the camera assumes a situation in which the camera looks at a vehicle or objects lying on the ground in the air. The camera speed and the speed of each object are not the same,

Is not completely zero, one component of the relative velocity vector between the objects is given as not zero, and it is assumed that one or two components are given in the camera velocity vector. For the SaM equation, a diagonal constant matrix

When Equation 24 is satisfied when i = 1, 2, and 3,

The

Converges using the nonlinear observer of Equations (20) and (21). Using given components of camera speed,

,

And

Not only the unknown component of the object velocity

Can also be estimated.

와

의 조합으로부터

의 미지성분에 대한 추정을 얻을 수 있다.therefore,

Wow

From the combination of

Can be obtained.

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되는 경우 또는 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되는 경우에 해당하는 경우,

의 모든 0이 아닌 성분들을 추정할 수 있도록 할 수 있다.If two objects have their respective object velocity vectors

, If the two components are unknown and the remainder of the given component is zero, the objects of both are moved or held in a plane, or if each of the object velocity vectors

When only one component is unknown and the remaining components are zero, if the objects of both are moved or held along a line,

Lt; RTI ID = 0.0 > 0 < / RTI >

는 완전히 0은 아니며, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지고, 카메라 속도 벡터에 있어서 두 개의 성분이 주어지며, 각각의 오브젝트 속도 벡터의 두 개의 성분이 미지이고 하나가 0인 경우 즉 오브젝트들이 평면에서 이동하거나 존재하는 경우(또는 카메라 속도 벡터의 하나의 성분이 주어지는 경우)에 의할 때, 제너럴 카메라와 다이나믹 및 제너럴 오브젝트들의 기하학 구조는 도 11에 도시된다.The two objects have the same distance from the camera, and the camera assumes a situation in which the camera looks at a vehicle or objects lying on the ground in the air. The camera speed and the speed of each object are not the same,

Is not completely zero, one component of the relative velocity vector between the objects is given, not zero, given two components in the camera velocity vector, and two components of each object velocity vector are unknown and one is zero The geometry of the general camera and the dynamic and general objects is shown in FIG. 11 when the objects are moving in the plane or present (or when one component of the camera velocity vector is given).

및

의 미지 성분들을 다음 과같이 추정할 수 있다.In these cases, the distance of objects,

And

Can be estimated as follows.

는 완전히 0은 아니며, 오브젝트들 간의 상대 속도 벡터의 1 성분은 0이 아니고 주어지고, 카메라 속도 벡터에 있어서 두 개의 성분이 주어지며, 각각의 오브젝트 속도 벡터의 두 개의 성분이 미지이고 하나가 0인 경우 즉 오브젝트들이 평면에서 이동하거나 존재하는 경우,

이 성립한다고 가정한다. SaM 추정을 위하여, 상수 대각 행렬

가 i=1,2,3 에서 수학식 24을 만족하는 경우수학식 20, 수학식 21 에서의 비선형 관측기들을 이용하면

는

에 대하여 수렴한다. 카메라 속도 벡터의 주어진 성분들을 이용하면,

,

의 미지 성분 및

가 추정될 수 있다.The two objects have the same distance from the camera, and the camera assumes a situation in which the camera looks at a vehicle or objects lying on the ground in the air. The camera speed and the speed of each object are not the same,

Is not completely zero, one component of the relative velocity vector between the objects is given, not zero, given two components in the camera velocity vector, and two components of each object velocity vector are unknown and one is zero In other words, if the objects move or exist in a plane,

. For SaM estimation, a constant diagonal matrix

When the nonlinear observers in Equation (20) and Equation (21) are used when Equation (24) is satisfied in i = 1,

The

Lt; / RTI > Using the given components of the camera velocity vector,

,

Unknown component of

Can be estimated.

의 미지 성분들 및

를 다음 과 같이 추정할 수 있다.The camera has a static motion so that the two objects have the same motion from the camera and the camera looks down at the vehicle or objects located on the ground in the air and the camera speed and the speed of each object are not the same, The motion between the static camera motion and the dynamic and general objects according to the case where the velocity between the objects is not completely zero and one component of the relative velocity vector between the objects is given is not zero is shown in Fig. In this case, the distance of objects,

Unknown components of

Can be estimated as follows.

행렬의 대각 성분은 수학식 24을 만족하고, 수학식 20 및 수학식 21에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

,

의 미지 성분 및

가 수학식 17, 수학식 18 및 The camera has a static motion so that the two objects have the same motion from the camera and the camera looks down at the vehicle or objects located on the ground in the air and the camera speed and the speed of each object are not the same, Is not completely zero, one component of the relative velocity vector between the objects is given instead of zero, and it is assumed that the camera has static motion. SaM Equation For Equation (14), for SaM Equation (14), when I is 1, 2, and 3,

The diagonal elements of the matrix satisfy Equation (24), and when using the nonlinear observer in Equations (20) and (21)

The

Lt; / RTI > like this

,

Unknown component of

(17), (18) and

Lt; / RTI >

Equation (19) can be abbreviated as Equation (25) on the assumption that the camera has static motion.

In the case where the relative speed between the objects is unknown, the distance, the relative speed between the objects, and the speed of the objects can be estimated as follows.

는 일부 주어지는 경우, 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하고,

의 성분 중 하나는 0이 아니며 주어지는 것으로 가정할 수 있다.Assuming that the camera motion is dynamic rather than static, the motion of both objects is static or general,

The camera has dynamic motion, one of the objects is caused by static motion, the other is made by general motion,

It can be assumed that one of the components of < RTI ID = 0.0 >

Then the relative speed between the two objects is not zero or need to be given.

The camera has a static motion so that the two objects have the same motion from the camera and the camera looks down at the vehicle or objects located on the ground in the air and the camera speed and the speed of each object are not the same, The velocity between the cameras is not completely zero, the camera is assumed to have dynamic motion, one of the objects is given by static motion, the other is given by general motion, and one of the components of vc is not 0, And the geometric structure of the static and general objects are shown in Fig.

및 제너럴 오브젝트의 속도

를 다음과 같이 추정할 수 있다.In this case, the distance of the objects, the camera speed

And the speed of the general object

Can be estimated as follows.

행렬의 대각 성분은 수학식 24을 만족하고, 수학식 20 및 수학식 21에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

의 추정, 및

가 수학식 15로부터 The camera has a static motion so that the two objects have the same motion from the camera and the camera looks down at the vehicle or objects located on the ground in the air and the camera speed and the speed of each object are not the same, The velocity between them is not completely zero, the camera assumes that it has dynamic motion, one of the objects is given by static motion and the other is given by general motion, and one of the components of vc is not zero. SaM Equation For Equation 14, when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (24), and when using the nonlinear observer in Equations (20) and (21)

The

Lt; / RTI > like this

, And

From Equation (15)

,

의 추정은 Can be obtained as follows. Also,

The estimate of

. For a static object, Equation (15)

및

은 수학식 29 및 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지는 것으로부터, 수학식 26, 수학식 27과 같이 구해질 수 있다. 한편, 제너럴 오브젝트에 대하여, 수학식 28로부터 추정될 수 있는

로부터 수학식 15는 Can be written as:

And

(29) and the camera have dynamic motion, one of the objects is based on static motion and the other is based on general motion,

One of the components of " 0 " is not 0 and can be obtained from Equation (26) and Equation (27). On the other hand, with respect to the general object,

(15)

Can be written as

를 추정하는 데 이용될 수 있다.

및

의 조건들 역시

이 완전히 미지인 경우와 같이 완화될 수 있다.Thus, a pair of dynamic camera / static objects is an unknown value of distance and camera speed, and

As shown in FIG.

And

The conditions of

Can be mitigated as if it were completely unknown.

Here, it is assumed that two objects have different distances from the camera. When the objects are at the same distance from the camera, the camera speed is completely or partially unknown, and there are few restrictions on the motion of the camera and the two objects. A case where a relative distance between two objects is given and a case where a relative speed between two objects is given will be described later. The overall distance and the structure of the motion estimation are shown in Fig.

If two objects have different distances to the camera as described below, various methods of relative information and distance estimation of the two objects are presented.

는 완전히 0이 아니다.If the two objects have motion at different distances to the camera, and the camera speed and the speed of the objects are different,

Is not completely zero.

The SaM of objects can be estimated irrespective of whether a relative distance between two objects is given or not, or whether the relative speed between objects is partly given or not.

는 완전히 주어지며, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고, 카메라 속도와 오브젝트들의 속도는 다른 경우에, 다음 가정을 도입한다.When the relative distance between the objects is given and the relative speed between the objects is 0, the distance and the relative speed between the camera and the objects are estimated. For estimation, the camera velocity vector

Is given completely, and both objects have motion at different distances to the camera, and when the camera speed and the speed of the objects are different, the following assumptions are introduced.

가 수학식 11과 같이 표현될 수 있으므로,

과

이

로부터 유지될 수 있으므로, 이는

이 주어짐을 의미한다.The relative distance between the two objects is given, and the relative speed between the objects can be zero. At this time,

Can be expressed as Equation (11)

and

this

As shown in Fig.

Is given.

는 완전히 0이 아닌 경우의 조건이 각 오브젝트에 대하여 만족되지 않는 경우에, 카메라가 0의 상대 속도로 움직이는 오브젝트들을 관찰함으로써 추가적인 정보를 얻을 수 없다.Even if the relative distance between the two objects is given and the relative speed between the objects is zero, both objects with respect to the relative speed of the camera-object have motion at different distances to the camera, And therefore,

If the condition is not completely zero for each object, then the camera can not get any further information by observing the moving objects at a relative speed of zero.

Given the relative distance between objects and the relative distance between them instead of the relative velocity, the distance can be obtained by taking into account the geometric relationship between theorem 3 and objects. This is an extension of the discussion in Theorem 3.

는 완전히 주어지며, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고, 카메라 속도와 오브젝트들의 속도는 다르고, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0인 경우에, 제너럴 카메라와 다이나믹 오브젝트들의 모션의 기하 구조는 도 14와 같이 나타날 수 있다.Camera speed vector

The two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different, the relative distance between the two objects is given, and the relative speed between the objects is zero, The geometry of the motion of the camera and the dynamic objects may appear as shown in Fig.

는 완전히 주어지며, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고, 카메라 속도와 오브젝트들의 속도는 다르고, 두 오브젝트들 간의 상대적인 거리가 주어지고, 오브젝트들 간의 상대 속도는 0인 경우에, 수학식 12에서의 SaM 모델은 Camera speed vector

, The two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different, the relative distance between the two objects is given, and the relative speed between the objects is 0, The SaM model in Equation 12

Can be written as

는

와 같다.

와

이 모두 추정되어야 하므로, 수학식 31은 At this time, i = 1, 2, 3

The

.

Wow

Lt; RTI ID = 0.0 > (31) < / RTI &

1은 언제나 가역이다. 또한,

,

,

이 양의 상수

,

,

에 대하여 성립할 수 있다.Can be rewritten as. At this time,

1 is always reversible. Also,

,

,

This positive constant

,

,

. ≪ / RTI >

이므로,

의 추정은 후술하는 수학식 37과 같은 관측기로부터 얻어질 수 있으며,

에 대하여 수하고,

의 추정은 후술하는 것과 같이 구할 수 있다. 첫째로,

Because of,

Can be obtained from an observer such as Equation (37)

Lt; / RTI >

Can be obtained as described below. First,

및

를 얻을 수 있고, 결합하여Using

And

Lt; RTI ID = 0.0 >

는 완전히 0이 아닌 경우에, 수학식 34으로부터 얻은

의 추정을 이용하여,

의 추정을 수학식 33로부터 The two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different,

Lt; RTI ID = 0.0 > 34 < / RTI >

Lt; / RTI >

From the equation (33)

Can be obtained as follows.

의 추정은 수학식 33와 수학식 35를 이용하여 therefore,

Lt; RTI ID = 0.0 > (33) < / RTI &

As shown in Fig.

의 추정기는

를 정의함으로써 얻을 수 있다. 이때,

는 후술하는 수학식 37에서의 관측기로부터 얻어진다. 또한,

는 양의 유한 상수 대각 행렬

에 대해 정의된다. 그러면, 수학식 32에 대한 비선형 관측기는 Based on the SaM model in equation (32)

The estimator of

. ≪ / RTI > At this time,

Is obtained from the observer in the following Expression (37). Also,

Is a positive finite constant diagonal matrix

Lt; / RTI > Then, the nonlinear observer for (32)

는 ≪ / RTI >

The

Respectively. Thus, the error dynamics

이다.As shown in Fig.

to be.

와

가 제한되므로,

,

이다. 수학식 37 및 수학식 38로부터 정의되는 비선형 관측기를 이용하여, 오브젝트들의 거리 및

을 다음과 같이 추정할 수 있다.

Wow

As a result,

,

to be. Using the nonlinear observer defined by equations (37) and (38), the distance of objects and

Can be estimated as follows.

행렬의 대각 성분은Given that the camera velocity vector is completely given, the two objects have motions at different distances to the camera, the camera speed is different from the speed of the objects, and the relative distance between the two objects is given, . SaM model For equation 32, when I is 1, 2, 3,

The diagonal elements of the matrix are

는

에 대하여 수렴한다. 이와 같이

및

이 수학식 35, 수학식 36로부터 추정된다.And using the nonlinear observer in equations (37) and (38)

The

Lt; / RTI > like this

And

Is estimated from Equations (35) and (36).

는 수학식 34과같이 추정될 수 있다. 따라서,

는 수학식 35,

은 수학식 36와 같이 추정될 수 있다.At this time,

Can be estimated as shown in Equation (34). therefore,

(35)

Can be estimated as shown in Equation (36).

의 하나 또는 두 개의 성분들이 카메라 속도 벡터

에 있어서, 하나 또는 두 개의 성분들이 주어지는 경우를 가정한다. 이때, 전술한 바와 같이,

의 주어진 성분수와 동일한 수의

의 성분을 다음과 같이 추정할 수 있다.Camera speed vector

Lt; RTI ID = 0.0 >

, It is assumed that one or two components are given. At this time, as described above,

The same number of components as the given number of components of

Can be estimated as follows.

행렬의 대각 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분 정보를 이용하여,

,

,

뿐만 아니라 오브젝트 속도

의 두 개 혹은 한 개의 미지 성분도 추정할 수 있다. 따라서,

과

의 조합으로부터

의 미지성분을 추정할 수 있다.In the camera velocity vector, two components are given, 11, given the relative distance between the two objects, the two objects have motion at different distances to the camera, the camera speed and the speed of the objects are different, Assume that the assumption is that the relative speed between the two is zero. From the SaM estimate of equation (32), when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > Using the given component information of the camera speed,

,

,

In addition,

Can also estimate two or one unknown components of. therefore,

and

From the combination of

Can be estimated.

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되는 경우 또는 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되는 경우에 해당하는 경우, 다음을 구할 수 있다.For two objects, two objects are associated with each object velocity vector

, If the two components are unknown and the remainder of the given component is zero, the objects of both are moved or held in a plane, or if each of the object velocity vectors

, When only one component is unknown and the remaining components are zero, if the objects of both are moved or held along a line, the following can be obtained.

행렬의 대각 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여,

,

및

과

의 미지 성분을 추정할 수 있다.In the camera velocity vector, two components are given, 11, given the relative distance between the two objects, the two objects have motion at different distances to the camera, the camera speed and the speed of the objects are different, 12 is established. From the SaM equation of equation (32), when I is 1, 2, 3,

The diagonal elements of the matrix satisfy Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > Using given components of camera speed,

,

And

and

Can be estimated.

와

에 대하여 추정할 수 있다.It is assumed that the camera is static. Then, the static camera and two objects of dynamic and general objects have motions at different distances with respect to the camera, the camera speed and the speed of the objects are different, and the geometry under 12 can be shown as in FIG. In this case, the distance

Wow

. ≪ / RTI >

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

및

는 수학식 35와 수학식 42로부터 추정될 수 있다. The camera has static motion, the two objects have motion at different distances to the camera, the camera speed and the velocity of the objects are different, the relative distance between the two objects is given, and the relative velocity between objects is 0 . From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this,

And

Can be estimated from Equations (35) and (42).

Since the camera has static motion, we can simply write vc = 0 and Equation (36) as Equation (42).

Given a relative distance between the objects, given the relative speed between the objects, the distance, the relative speed between the camera and the object, and the unknown component of the relative speed between the objects can be estimated as follows.

For distance estimation, a camera complete vector is given completely, the two objects have motion at different distances to the camera, and when the camera speed and the speed of the objects are different, the following assumption is introduced.

The relative distance between the objects is given. In addition, some components of the relative speed between the objects are given.

Given that the camera velocity vector is fully given, the two objects have motion at different distances to the camera, the camera speed and the speed of the objects are different, and given the relative distance between the objects and the relative velocity between the objects, The geometry of the camera and the dynamic objects can be shown in FIG.

Given that the camera velocity vector is fully given, the two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different, and given the relative distance between the objects and the relative velocity between the objects, The SaM model of Equation 12 can be written as Equation (43).

와

가 모두 추정되어야 하므로, 수학식 43는 수학식 44과 같이 쓰여질 수 있다.

Wow

(43) can be written as shown in Equation (44).

는

을 만족하는 측정 가능한 벡터이고, 측정 불가능한 벡터는

으로 정의되며, 수학식 45와 같이 쓰여질 수 있다. At this time,

The

, And the non-measurable vector is a measurable vector satisfying

, And can be written as shown in Equation (45).

Equation (45)

Can be expressed as < RTI ID = 0.0 >

는 수학식 47으로부터

및

는 오브젝트들 간의 상대 거리가 주어진다. 또한, 오브젝트들간의 상대 속도의 일부 성분이 주어지는 가정으로부터 주어지므로, Respectively. therefore,

≪ RTI ID = 0.0 >

And

Is given a relative distance between the objects. Further, since it is given from the assumption that a part of the relative velocity between objects is given,

를 이용하면,

는 수학식 46로부터 Can be obtained. In Equation 48,

In this case,

From Equation (46)

로부터

는 Can be obtained. At this time,

from

The

As shown in Fig.

의 추정을 이용하여,

의 추정은 수학식 45와 수학식 49, 수학식 50을 이용하여 From equations (49) and (50)

Lt; / RTI >

Is calculated using Equation (45), Equation (49), and Equation (50)

의 미지 성분은 수학식 51로부터 Can be obtained,

The unknown component of equation

.

,

및

의 미지 성분들을 다음에 따라 구할 수 있다.Thus, using the nonlinear observer in equations (37) and (38), the distance of objects

,

And

Can be obtained as follows.

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

,

,

및

의 미지 성분을 수학식 49, 수학식 50, 수학식 51, 수학식 52로부터 추정할 수 있다. Given that the camera velocity vector is fully given, the two objects have motions at different distances to the camera, the camera speed and the speed of the objects are different, and given the relative distance between the objects and the relative velocity between the objects, From the SaM equation of Eq. (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this,

,

,

And

Can be estimated from Equation (49), Equation (50), Equation (51), and Equation (52).

는 완전히 주어지며, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고, 카메라 속도와 오브젝트들의 속도는 다른 경우에, 오브젝트들 간의 상대 거리와 오브젝트들간의 상대 속도의 일부 성분이 주어지는 경우에,

는 전술한 논의 수학식 49, 수학식 50에 따라 추정된다. 이에 따라,

및

의 미지 성분은 수학식 51 및 수학식 52에서와 같이 추정된다.Camera speed vector

Given two objects, two objects have motion at different distances to the camera, the camera speed and the speed of the objects are different, given a relative distance between the objects and some component of the relative speed between the objects,

Is estimated according to the above-described discussion formula (49), (50). Accordingly,

And

Is estimated as shown in Equation (51) and Equation (52).

의 하나 또는 두 개의 성분이 카메라 속도 벡터에 있어서, 하나 또는 두 개의 성분들이 주어진 경우,

의 주어진 성분 수와 동일한 수의

의 미지 성분을 다음과 같이 추정할 수 있다.Camera speed vector

One or both of the components are given in the camera velocity vector, given one or two components,

The same number of components as the given number of components of

Can be estimated as follows.

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용할 때,

및

의 미지 성분뿐만 아니라, 오브젝트 속도

의 미지성분 역시 동일한 성분 수만큼 추정할 수 있다.In the camera velocity vector, two components are given, 11, given the relative distance between the two objects, the two objects have motion at different distances to the camera, the camera speed and the speed of the objects are different, . From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > When using given components of camera speed,

And

Not only the unknown component of the object velocity

Can be estimated by the same number of components.

와

의 조합으로부터

의 추정을 얻을 수 있다. 두 오브젝트들이 각각의 오브젝트 속도 벡터

에 대하여, 두 개의 성분들이 미지이고 주어진 나머지 성분이 0인 경우, 양자의 오브젝트들은 평면 상에서 움직이거나 유지되는 경우 또는 각각의 오브젝트 속도 벡터

에 대하여, 하나의 성분만이 미지이고 나머지 성분들이 0인 경우에, 양자의 오브젝트 들은 선을 따라 움직이거나 유지되는 경우에 해당하는 경우,

가 추정될 수 있다. 제너럴 카메라와 다이나믹 및 제너럴 오브젝트의 모션의 기하 구조는 도 17와 같이 나타낼 수 있다.therefore,

Wow

From the combination of

Can be obtained. If two objects have their respective object velocity vectors

, If the two components are unknown and the remainder of the given component is zero, the objects of both are moved or held in a plane, or if each of the object velocity vectors

When only one component is unknown and the remaining components are zero, if the objects of both are moved or held along a line,

Can be estimated. The geometry of the motion of the general camera and the dynamic and general objects can be represented as shown in Fig.

및

와

의 미지 성분들을 다음과 같이 추정할 수 있다. 수학식 32의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 카메라 속도의 주어진 성분들을 이용하여

,

및

와

의 미지 성분을 추정할 수 있다In these cases, the distance of objects,

And

Wow

Can be estimated as follows. From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > Using given components of camera speed

,

And

Wow

It is possible to estimate the unknown component of

It is assumed that the camera has static motion. Then, in static camera motion and dynamic and general objects, the two objects have motions at different distances to the camera, the camera speeds and the speeds of the objects are different, and the geometry for motion when a relative distance between objects is given Can be represented as shown in Fig.

의 미지 성분들 및

를 다음과 같이 구할 수 있다.In this case, the distance of objects,

Unknown components of

Can be obtained as follows.

는 완전히 0이 아니라는 가정 및 오브젝트들 간의 상대 거리가 주어지며, 오브젝트들간의 상대 속도의 일부 성분이 주어진다는 가정이 성립한다고 가정한다. 수학식 44의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이,

,

,

및

의 미지 성분에 대하여 수학식 49, 수학식 50, 수학식 53 및 수학식 52을 이용하여 추정할 수 있다.Since the camera has static motion, the two objects have motion at different distances to the camera, and the camera speed and object speed must be different,

Is not completely zero, and assuming that the assumption is made that a relative distance between objects is given, and that some component of the relative velocity between objects is given. From the SaM equation of Equation (44), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this,

,

,

And

Can be estimated using Equation (49), Equation (50), Equation (53), and Equation (52) with respect to the unknown component of the input image.

If the camera has static motion, it can be simplified to equation (53) by combining vc = 0 and equation (51).

If the relative distance between the objects is unknown, and the relative speed between the objects is given in part, the distance, the camera speed and the speed of the general object can be estimated as follows.

가 일부 주어진 경우에, 오브젝트들 간의 상대 거리는 주어질 필요 없다. 대신에, 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지고, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고 카메라 속도와 오브젝트들의 속도는 달라야 하므로,

는 완전히 0이 아니라는 가정 및 다음 가정을 이용할 수 있다. 오브젝트들 간의 상대 거리가 주어지고, 두 오브젝트들 간의 상대 속도는 일부 주어지는 경우를 가정한다.If the camera has dynamic motion, both objects have static and general motion,

, The relative distance between the objects need not be given. Instead, the camera has dynamic motion, one of the objects is by static motion, the other by general motion,

One of the components of the image is given non-zero, the two objects must have motion at different distances to the camera, the camera speed and the speed of the objects must be different,

Is not completely zero and the following assumptions can be made. It is assumed that a relative distance between objects is given, and a relative speed between two objects is given in part.

의 성분 중 하나는 0이 아니며 주어지고, 11, 14 하에서의 다이나믹 카메라와 스태틱 및 제너럴 오브젝트들의 기하 구조는 도 19에서와 같이 나타낼 수 있다.The camera has dynamic motion, one of which is by static motion and the other by general motion,

One of the components of the dynamic camera is given as not 0, and the geometry of the dynamic camera and the static and general objects under 11 and 14 can be represented as shown in FIG.

는

과 같이 정의 될 수 있고, 오브젝트들의 거리,

의 미지 성분 및

를 후술하는 바와 같이 추정하고 이를 기초로 수학식 33 대신In this case, the SaM equation of the equation (32)

The

And the distance of objects,

Unknown component of

Is estimated as described later, and based on this,

Can be expressed as

의 성분 중 하나는 0이 아니며 주어지고, 두 오브젝트들은 카메라에 대하여 서로 다른 거리에서 모션을 가지고, 카메라 속도와 오브젝트들의 속도는 달라야 하므로,

는 완전히 0이 아니며, 오브젝트들 간의 상대 거리가 주어지는 한편, 두 오브젝트들 간의 상대 속도는 일부 주어진다는 가정이 성립한다고 가정한다. 수학식 32의 SaM 방정식으로부터, I는 1,2,3인 때, 상수 대각 행렬

의 성분은 수학식 41을 만족하고, 수학식 37 및 수학식 38에서의 비선형 관측기를 이용할 때

는

에 대하여 수렴한다. 이와 같이

및

의 추정을 다음과 같이 구할 수 있다.The camera has dynamic motion, one of which is by static motion and the other by general motion,

One of the components of is not 0 and given, the two objects have motion at different distances to the camera, the camera speed and the speed of the objects must be different,

Is not completely zero and the relative distance between the objects is given, while the assumption is made that the relative speed between the two objects is given in part. From the SaM equation of equation (32), when I is 1, 2, 3, the constant diagonal matrix

≪ / RTI > satisfies Equation (41), and when using the nonlinear observer in Equations (37) and (38)

The

Lt; / RTI > like this

And

Can be obtained as follows.

및

의 추정을 다음과 같이 구할 수 있다.Also,

And

Can be obtained as follows.

For a static object, Equation (54)

Can be expressed as:

는 0이 아님을 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지는 것으로부터 알 수 있으므로,

및

는 수학식 59로부터 수학식 55및 수학식 56와 같이 구할 수 있다. 한편, 제너럴 오브젝트에 대하여, 수학식 54는

Is not 0, the camera has dynamic motion, one of the objects is by static motion, the other is by general motion,

One of the components of < RTI ID = 0.0 > 1 < / RTI > is not 0 and is known from given,

And

Can be obtained from Equation (59) to Equation (55) and Equation (56). On the other hand, with respect to the general object,

Can be expressed as:

는

을 만족하며 카메라는 다이나믹 모션을 가지고, 둘 중 하나의 오브젝트는 스태틱 모션에 의하며 나머지 하나는 제너럴 모션에 의하며,

의 성분 중 하나는 0이 아니며 주어지고, 가정으로부터 0이 아니고 주어진다. 따라서,

는 수학식 60 로부터 수학식 57와 같이 구해지고,

는 수학식 58과 같이 추정될 수 있다.At this time,

The

, The camera has a dynamic motion, one of the objects is based on static motion, the other is based on general motion,

One of the components of is given non-zero and is given from the hypothesis, not zero. therefore,

Is obtained from Expression (60) as Expression (57)

Can be estimated as shown in Equation (58).

The method according to the present invention can be implemented as a computer-readable code on a computer-readable recording medium. A computer-readable recording medium includes all kinds of recording apparatuses in which data that can be read by a computer system is stored. Examples of the recording medium include a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like, and a carrier wave (for example, transmission via the Internet). The computer-readable recording medium may also be distributed over a networked computer system so that computer readable code can be stored and executed in a distributed manner.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, the true scope of the present invention should be determined by the technical idea of the appended claims

Claims (20)

A method for estimating spatial information of a plurality of objects,
An image input step of inputting images of the plurality of objects;
Obtaining coordinates of minutiae points of the plurality of objects to be traced from the input image;
Calculating normalized minutiae coordinates of at least three dimensions on the basis of the coordinates of the obtained minutiae points and the parameter information of the camera set in advance; And
And estimating spatial information of the plurality of objects using the three-dimensional normalized minutiae coordinates,
Wherein the estimating of the spatial information of the plurality of objects comprises:
Defining system models including a state vector as positional information for describing spatial information of the plurality of objects, the system models including a measurable vector and an unmeasurable vector; And
Defining nonlinear observers defined as derivatives of the system models, using the system models defined above,
The nonlinear observers,
Wherein the system model is defined by a measured or predefined variable, a velocity condition of the plurality of objects and a camera, a distance condition of the plurality of objects, and an estimated parameter that is a given or unmeasured variable. A method for estimating spatial information of an object. The method of claim 1, wherein estimating spatial information of the plurality of objects comprises:
Wherein the normalized minutiae point coordinates are computed to define state vectors and system models for describing spatial information of the object, the system models are divided into a measurable vector and a non-measurable vector, And estimating the spatial information of the plurality of objects from the estimation of the non-measurable vector. The method according to claim 1,
Wherein the estimating of the spatial information of the plurality of objects comprises:
Defining nonlinear estimators for estimating an unmeasurable vector of the nonlinear observers using an observer error defined as a difference between the system models and the estimate;
Estimating an unmeasurable vector of the nonlinear observer using the nonlinear estimators; And
Estimating unknown spatial information using the estimation of the unmeasurable vector. ≪ RTI ID = 0.0 > 8. < / RTI > The method of claim 3,
Wherein the measurable vector included in the system models includes first and second components of the state vectors and the unmeasurable vector included in the system models is a third component of the state vector, And a scale factor that is a constant that is a non-zero constant that controls the scale of the state vector associated with the distance of the object and the relative speed of the second object with respect to the camera. Way. 4. The method of claim 3, wherein the system models of the plurality of objects include:
And a scale for a state vector associated with the components of the state vectors, the velocity of the plurality of objects, the velocity of the camera, and the distance of the plurality of objects, And a scale factor, which is a constant. The method of claim 3,
Wherein the nonlinear estimators are defined to estimate the non-measurable vector using a non-measurable vector included in the non-linear observers and the observer errors, the non-linear estimators being derivatives of the non- Method for estimating spatial information of an object delete The method according to claim 1,
The spatial information is a factor that controls the scale of the state vector related to the speed of the objects, the relative speed of the objects with respect to the camera, the relative speed between the objects, the speed of the camera, Wherein the information includes at least one information selected from the group consisting of information on the at least one object. The method according to claim 1,
Wherein the step of calculating the coordinates of the three-
Calculating coordinates of the characteristic points of the camera defined by the parameter information of the camera and coordinates of the characteristic points, and calculating coordinates of the three-dimensional normalized characteristic points. The method according to claim 1,
Wherein the coordinates of the minutiae points are the coordinates of pixels associated with the minutiae vectors in the image space to which the objects belong, and the normalized three-dimensional space coordinates of the minutiae points are in a three-dimensional spatial coordinate system having the camera position as the origin, Wherein the coordinate information is set in a photographing direction of the camera and is a normalized coordinate based on the set photographing direction. The method of claim 3,
Wherein a distance condition of the plurality of objects is different from a distance of the plurality of objects from the camera, a difference of distances of the plurality of objects from the camera is given, and a relative speed between the plurality of objects is 0 Of the plurality of objects. 4. The method of claim 3, wherein in the unknown spatial information estimating step,
Wherein the unknown spatial information is a scale factor that is a non-zero constant that controls the scale of state vectors related to the relative speed of the objects with respect to the camera and the distance of the plurality of objects,
Estimating non-linear observers using the estimated non-measurable vectors;
Estimating a scale factor that is a non-zero constant as an element for adjusting a scale of a state vector associated with a distance of the plurality of objects using the estimated nonlinear observers; And
Estimating a relative speed of the plurality of objects with respect to the camera using the estimated scale factor; And estimating spatial information of the plurality of objects based on the spatial information. 5. The method of claim 4, wherein estimating the unmeasurable vector comprises:
Defining an observer error defined by a difference between the system model and the estimate;
Defining a filtered observer error that is defined as filtering the observer error through a low-pass filter; And
And defining an estimation equation of a non-measurable vector using the observer error and the filtered observer error. The method according to claim 1,
Wherein the image is photographed by a camera moving at a speed and an angular velocity according to a preset value or a function. A method for estimating spatial information of a plurality of objects,
The first and second minutiae coordinates of the first object and the second minutiae point of the second object are obtained from the input image, Obtaining normalized first and second minutiae coordinates of at least three dimensions on the basis of parameter information of a set camera;
Defining a first state vector and a second state vector for describing the spatial information of the first and second objects, respectively, as a constituent component of the obtained coordinates of the first and second feature points, Defining a first system model and a second system model including a first measurable vector and a second non-measurable vector;
Defining first and second nonlinear observers from the first and second system models using previously given spatial information;
Linear estimator for estimating an unmeasurable vector of the first and second nonlinear observers using first and second observer errors defined as a difference between the first and second system models and their respective estimates, Defining an estimator;
Estimating an unmeasurable vector of the first and second nonlinear observers using the first and second nonlinear estimators defined above; And
Estimating the first and second nonlinear observers using the estimated non-measurable vector; And estimating spatial information of the plurality of objects. 16. The method of claim 15,
Estimating scale factors that are non-zero constants as elements for adjusting the scale of state vectors associated with the distance of the plurality of objects using the estimated first and second nonlinear observers; And
And estimating a relative speed of the objects with respect to the camera from the estimated scale factors. 16. The method of claim 15,
Estimating the first and second system models using the estimated first and second nonlinear observers, and estimating the first and second system models using the estimated first and second nonlinear observers. A spatial information estimating apparatus for a plurality of objects,
An image input unit receiving images of the plurality of objects;
A minutiae point coordinate obtaining unit for obtaining coordinates of minutiae points of the plurality of objects to be tracked from the input image;
A normalized coordinate calculation unit for calculating coordinates of at least three-dimensionally normalized minutiae points using coordinates of the obtained minutiae points and parameter information of a camera set in advance; And
And a spatial information estimator for estimating spatial information of the plurality of objects using coordinates of the three-dimensional normalized minutiae points,
Wherein the spatial information estimating unit comprises:
Systematic models including state vectors as positional information for describing spatial information of the plurality of objects, which include a calculation of the components of the coordinates of the normalized minutiae points, and measurable vectors and non-measurable vectors, Define a system model definition part; And
And a nonlinear observer defining unit that defines nonlinear observers defined as derivatives of the system models using the defined system models,
The nonlinear observers,
Wherein the system model is defined by a measured or predefined variable, a velocity condition of the plurality of objects and a camera, a distance condition of the plurality of objects, and an estimated parameter that is a given or unmeasured variable. An apparatus for estimating spatial information of an object. 19. The method of claim 18,
Wherein the spatial information estimating unit comprises:
A nonlinear estimator defining a nonlinear estimator for estimating an unmeasurable vector of the nonlinear observers using observer errors defined as a difference between the system models and the estimates;
An unmeasurable vector defining unit that estimates non-measurable vectors of the nonlinear observers using the nonlinear estimators; And
An unknown spatial information estimator for estimating unknown spatial information using the estimation of the unmeasurable vectors; And a spatial information estimating unit for estimating spatial information of a plurality of objects. 20. The method of claim 19,
Wherein the unknown spatial information estimating unit comprises:
Wherein the unknown spatial information is a scale factor that is a non-zero constant that controls the scale of state vectors related to the relative speed of the plurality of objects to the camera and the distance of the objects,
Estimating the nonlinear estimators, the nonlinear observers and the system models using the estimated non-measurable vector, and adjusting the scale of the state vectors associated with the distance of the plurality of objects from the estimated system models And estimating distance information and state vectors using the estimated scale factor and estimating velocity vectors of the object based on the scale factors and the velocity factors of the plurality of objects, Of the spatial information.

Images