KR101456861B1 - Spatio-temporal calibration and tracking method and device with object dynamics in multiple camera systems - Google Patents

Abstract

The present invention relates to a space-time calibration tracking method and apparatus using dynamic information of an object in a multi-camera system, and more particularly, to a method and apparatus for extracting shape information and spatio- temporal information of an object from an image captured by a plurality of cameras, Tracks the trajectory of the object based on the extracted type information, time information, and spatial information, and performs calibration between the plurality of cameras based on the trajectory of the traced object.

Camera, Object, Trajectory Tracking, Calibration

Description

TECHNICAL FIELD [0001] The present invention relates to a space-time calibration tracking method and apparatus using dynamic information of an object in a multi-camera system,

The present invention relates to a method and apparatus for tracking space-time calibration using dynamic information of an object in a multiple camera system, and more particularly, The present invention relates to a space-time calibration tracking method and apparatus using dynamic information of an object in a multi-camera system capable of performing spatio-temporal calibration simultaneously.

In a multiple camera environment in which a plurality of cameras are installed, mutual correction of a plurality of cameras is required. Such multiple camera calibration is necessary to ensure consistency of object information photographed on the camera, and correction is made by using the correlation between cameras.

According to the above-described multi-camera calibration, accurate results can be obtained even for a complicated camera model.

In such a multi-camera calibration environment, a three-dimensional (3D) or two-dimensional (2D) correction grid is installed in order to convert the coordinates into reference coordinates using the relationship between the cameras, The overlapping occurred and the corrections and corrections were made accordingly.

SUMMARY OF THE INVENTION The present invention has been made in order to solve the above-described problems, and it is an object of the present invention to provide a method and apparatus for correcting a movement of an object by using a mathematical model of movement of an object, And to provide a camera calibration method capable of performing temporal correction between cameras by calculating rotation and translation between a plurality of cameras using the corresponding relationship.

Also, the present invention is intended to enable simultaneous spatio-temporal calibration using the same object motion model even when there is no overlap of views in a plurality of cameras.

Further, the present invention does not require analog cabling costs required for time correction to obtain a pattern image for setup correction, and does not require 3D cube or 2D cube, It is desired to perform correction of a plurality of cameras without installing a correction grid.

According to an aspect of the present invention, there is provided a method for time-space calibration and tracking using dynamic information of an object in a multi-camera system, the method comprising: Extracting spatio-temporal information, tracing the trajectory of the object based on the extracted type information, spatio-temporal information, and determining, based on the trajectory of the traced object, And performing calibration between the plurality of cameras.

According to another aspect of the present invention, the step of extracting shape information and spatio-temporal information of an object from the images photographed by the plurality of cameras may include generating a histogram that divides the object into parts, And extracting shape information of the object using a histogram.

According to another aspect of the present invention, the step of generating a histogram that divides the object into parts and extracting the type information of the object using the histogram may include a step of, when the object is a human, And extracting the morphological information of the person using the histogram.

According to another aspect of the present invention, the step of extracting shape information and spatio-temporal information of an object from an image photographed by a plurality of cameras comprises the steps of: Constructing a probability distribution of the movement of the object based on the generated time link information, and extracting space-time information based on the probability distribution.

According to another aspect of the present invention, the spatiotemporal information includes movement time information, which is information about a time at which the object is photographed by moving between the plurality of cameras, information about a speed at which the object is photographed by moving between the plurality of cameras Speed information as information, and path information that is information about a path taken by moving the object between the plurality of cameras.

According to another aspect of the present invention, the type information includes information related to the length of the object, and information related to the color of the object.

An apparatus for time-space calibration and tracking using dynamic information of an object in a multi-camera system according to an aspect of the present invention includes an information extracting unit for extracting shape information of an object and spatio-temporal information from an image photographed by a plurality of cameras A trajectory tracing unit for tracing a trajectory of the object based on the extracted shape information and the spatio-temporal information, and a controller for executing a correction between the plurality of cameras based on the trajectory of the traced object And an calibration section.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings and accompanying drawings, but the present invention is not limited to or limited by the embodiments. Like reference symbols in the drawings denote like elements.

1 is a conceptual diagram for explaining a space-time calibration and tracking method using dynamic information of an object in a multiple camera system according to an example of the present invention.

As shown in FIG. 1, the space-time calibration and tracking method according to an exemplary embodiment of the present invention includes a plurality of cameras 110, 120, and 130, and each camera includes areas 115 and 125 And 135, respectively.

When the object moves, each camera captures an image in which the object moves on a route in each photographing area.

The space time calibration and tracking method according to an exemplary embodiment of the present invention is not limited to the case where the areas 115, 125, and 135 that are photographed by the respective cameras overlap each other, It is possible to track an object and correct the camera accordingly, even when the two cameras 115, 125, and 135 do not overlap each other.

That is, although it is possible to track an object in a plurality of cameras, a synchronization between a plurality of cameras is not matched, and even in a situation where rotation and translation between the plurality of cameras are unknown, It is possible to perform calibration using spatio-temporal information between a plurality of cameras using the dynamic characteristics of the object.

Also, it is possible to track an object trajectory in an area that the camera can not cover, and to handover the ID associated with the object.

2 is a configuration diagram of a space-time calibration and tracking apparatus using dynamic information of an object in a multiple camera system according to an exemplary embodiment of the present invention.

2, the space time calibration and tracking apparatus 200 using dynamic information of an object in a multiple camera system according to an exemplary embodiment of the present invention includes an information extraction unit 210, a trajectory extraction unit 220, And an orthogonal unit 230.

The information extraction unit 210 extracts shape information and spatio-temporal information of an object from an image captured by a plurality of cameras.

In particular, the information extracting unit 210 generates a histogram that divides the object into parts, and extracts the shape information of the object using the histogram.

The information extracting unit 210 generates time link information based on the time taken to photograph the object among the plurality of cameras, and calculates a probability distribution about the movement of the object based on the generated time link information . The information extraction unit 210 extracts space-time information based on the probability distribution thus constructed.

In this case, the spatio-temporal information includes movement time information, which is information about a time when an object is photographed by moving between the plurality of cameras, speed information, which is information on the speed at which the object is photographed by moving among the plurality of cameras, And path information that is information on a path taken by the object moving between the plurality of cameras.

In particular, the shape information includes information related to the length of the object, and information related to the color of the object.

The locus extracting unit 220 tracks the locus of the object based on the type information, the time information, and the space information extracted by the information extracting unit 210.

Combining such appearance information with space-time information does not require additional camera calibration information, and even when the camera position changes, recognition through object ID is possible.

A kinematic model of an object can be used as a method for knowing the trajectory of the object, and the object kinematic model can be expressed by the following equation (1).

(i is the cubic curve and p is the polynomial coefficient).

i represents the cubic curve, and p is the polynomial coefficient.

Equation (2) shows a kinematic model when the order of p, the polynomial coefficient, i is 3.

P represents a coefficient according to the kind of model applied according to the kind of the object. As the polynomial coefficient P, a linear model, which is a constant-velocity model in the case of an automobile on an expressway, may be used as the object, and a quadratic kinematic model may be used when the object is a person .

The position and velocity of an object outside the camera shooting region can be calculated through curve fitting of the curve using the object kinematic model to enable ID handover of objects between a plurality of cameras, It is possible to know the rotation and translation between the cameras and calculate the temporal offset between the plurality of cameras.

The calibration unit 230 performs calibration between the plurality of cameras based on the trajectory of the tracked object. In this case, rotation and translation between the plurality of cameras and a time offset between the plurality of cameras and performs calibration between the plurality of cameras through temporal offset.

FIG. 3 is a diagram for explaining a method of extracting object shape information through a histogram that divides an object according to an example according to an embodiment of the present invention.

In order to extract the shape information of the object, a histogram that divides the object into parts is generated. The shape information of the object can be extracted using the histogram thus generated.

In particular, as shown in FIG. 3, when the object is a human, the human can be divided into the histogram by body, and if the histogram is used, the local color distribution information of the person is lost.

That is, when the object is a person, the person is divided into three parts of the head 310, the trunk 320, and the leg 330, and the whole histogram 340 of the person, the head histogram 350, the trunk histogram 360, The histogram 370 is obtained, and the histogram is used as the appearance descriptor of the person. Therefore, the information of the entire shape of the person and local information of each body part of the person are simultaneously obtained.

FIGS. 4, 5, and 6 are diagrams for explaining a method of extracting space-time information using time link information according to an example of the present invention.

When the object moves between a plurality of cameras, similar movement of the root movement is shown. Therefore, the probability distribution of the movement of the object can be obtained using a temporal link between the plurality of cameras.

As shown in Figure 4, when the objects 440 and 450 move on each route, the plurality of cameras 410, 420, and 430 may cause the objects 440 and 450 to display images 415, 425, and 435 ).

As shown in FIG. 5, a time link between each camera 510 and 520 can be generated using the images 510 and 520 and the moving time 540 outside the photographing region, .

6 illustrates a time link (T12, T13, T14, T23, T24, T34) between the four cameras 610, 620, 630, For example, 'T12' represents a temporal link between the first camera 610 and the second camera.

The probability distribution of the movement of the object can be obtained by using a temporal link between the plurality of cameras. The probability distribution of the movement time between the regions can be modeled as MOG (Mixture of Gaussian), and the distribution can be modeled by three distributions such as Original distribution, Mixed Gaussian, and Single Gaussian. By using the distribution, it is possible to model more accurately according to the moving speed of the object.

FIG. 7 is a flowchart illustrating a space-time calibration and tracking method using dynamic information of an object in a multiple camera system according to an exemplary embodiment of the present invention.

A space-time calibration and tracking method using dynamic information of an object in a multi-camera system according to an example of the present invention will be described with reference to FIG.

In step S710, shape information and spatio-temporal information of the object are extracted from the images photographed by the plurality of cameras.

More specifically, it generates time link information constituted based on a time taken by the plurality of cameras to photograph the object, constructs a probability distribution for the movement of the object based on the generated time link information, Space-time information can be extracted based on the probability distribution.

In particular, it is possible to generate a histogram that divides the object into parts and extract the morphological information of the object using the histogram. When the object is a human, a histogram is generated to classify the human by body, The type information of the person is extracted.

In this case, the spatio-temporal information includes movement time information, which is information about a time when an object is photographed by moving between a plurality of cameras, velocity information which is information about a speed at which the object is photographed by moving between the plurality of cameras, The object may include path information that is information about a path taken by moving between the plurality of cameras.

In addition, the appearance information includes information related to the length of the object, and information related to the color of the object.

The trajectory of the object is tracked based on the extracted shape information, time information, and spatial information (S720).

Based on the trajectory of the tracked object, calibration between the plurality of cameras may be performed (S730).

8 and 9 illustrate an ID handover of an object according to an example of the present invention. Fig. 8 shows an image taken by a first camera, Fig. 9 shows an image taken by a second camera And shows the photographed image.

8, an automobile as an object 810 and an ID 815 corresponding to the automobile are displayed on an image photographed by the first camera shown in Fig. 8, and as shown in Fig. 9, using the dynamics of the object The second camera can recognize the object 920 and the ID 915 corresponding thereto.

That is, even if there is no overlap between the images captured by the camera using the dynamic characteristics of the object, the trajectory can be estimated and the IDs in the shooting regions of the respective cameras can be obtained.

10 is a diagram for explaining a method of recognizing the arrangement of a plurality of cameras according to an example of the present invention.

As shown in FIG. 10, each of the plurality of cameras acquires the trajectory of the object (1010, 1020), and the trajectory of the object in the entire security area can be estimated using the dynamic characteristics of the object. Accordingly, it is possible to find the arrangement 1030 of the security area photographed by each camera through the match of such trajectories.

A method and apparatus for space-time correction and tracking using dynamic information of an object in a multiple camera system according to an example of the present invention is characterized in that when there is no overlapping of views in a plurality of cameras using a mathematical model of object motion Also, it is possible to calculate temporal corrections between the cameras by calculating the rotation and translation between a plurality of cameras using the correspondence relationship.

In addition, even when there is no overlap of views in a plurality of cameras, spatio-temporal correction can be performed simultaneously using the same object motion model.

Accordingly, correction of a plurality of cameras can be performed without the analog cabling cost required for time correction for obtaining a pattern image for an existing setup correction.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited to the described embodiments, but should be determined by the equivalents of the claims, as well as the claims.

1 is a conceptual diagram for explaining a space-time calibration and tracking method using dynamic information of an object in a multiple camera system according to an example of the present invention.

2 is a configuration diagram of a space-time calibration and tracking apparatus using dynamic information of an object in a multiple camera system according to an exemplary embodiment of the present invention.

FIG. 3 is a diagram for explaining a method of extracting object shape information through a histogram that divides an object according to an example according to an embodiment of the present invention.

FIGS. 4, 5, and 6 are diagrams for explaining a method of extracting space-time information using time link information according to an example of the present invention.

FIG. 7 is a flowchart illustrating a space-time calibration and tracking method using dynamic information of an object in a multiple camera system according to an exemplary embodiment of the present invention.

8 and 9 are diagrams for explaining ID handover of an object according to an example of the present invention.

10 is a diagram for explaining a method of recognizing the arrangement of a plurality of cameras according to an example of the present invention.

Description of the Related Art

210:

220: locus extracting unit

230:

Claims (10)

Extracting shape information and spatio-temporal information of an object from an image captured by a plurality of cameras; Tracking the trajectory of the object based on the extracted shape information and space-time information; And And performing calibration between the plurality of cameras based on the trajectory of the tracked object, The step of performing the calibration includes: Calculating a rotation and a translation between the plurality of cameras based on the movement of the object; And Performing time correction between the cameras based on the rotation and translation between the plurality of cameras And a time-space calibration and tracking method using dynamic information of an object in a multi-camera system. The method according to claim 1, The step of extracting shape information and spatio-temporal information of an object from the images photographed by the plurality of cameras includes the steps of: Generating a histogram that divides the object into parts, and extracting shape information of the object using the histogram Wherein the method further comprises the step of: 3. The method of claim 2, Wherein the step of generating a histogram that divides the object into parts and extracting the type information of the object using the histogram, If the object is a person, generating a histogram that distinguishes the person by body and extracting morphological information of the person using the histogram Wherein the method further comprises the step of: The method according to claim 1, The step of extracting shape information and spatio-temporal information of an object from an image captured by a plurality of cameras includes: Generating time link information based on a time when the object is photographed between the plurality of cameras; Constructing a probability distribution for the movement of the object based on the generated time link information; And Extracting space-time information based on the probability distribution And a time-space calibration and tracking method using dynamic information of an object in a multi-camera system. The method according to claim 1, The space- Speed information that is information on a speed at which the object is photographed when the object is moved between the plurality of cameras, and speed information indicating that the object is moving between the plurality of cameras And path information that is information on the photographed path. The method of claim 1, further comprising: The method according to claim 1, The type information includes: And information related to the length of the object and information related to the color of the object. An information extraction unit for extracting shape information and spatio-temporal information of an object from an image captured by a plurality of cameras; A trajectory tracing unit for tracing a trajectory of the object based on the extracted type information and space time information; And And a calibration unit that performs calibration between the plurality of cameras based on the trajectory of the tracked object, Wherein the calibration unit comprises: Calculating rotation and translation between the plurality of cameras based on the motion of the object and performing time correction between the cameras based on rotation and translation between the plurality of cameras, A Spatio - Temporal Calibration and Tracking System Using Dynamic Information of Objects in Multi - camera System. 8. The method of claim 7, The information extracting unit extracts, Wherein the histogram generating unit generates a histogram that divides the object into parts and extracts the shape information of the object using the histogram. 8. The method of claim 7, The information extracting unit extracts, Generating time link information based on a time at which the object is photographed between the plurality of cameras, constructing a probability distribution for the movement of the object based on the generated time link information, Space-time information is extracted based on the dynamic information of the object in the multi-camera system. 8. The method of claim 7, The space- Speed information that is information on a speed at which the object is moved between the plurality of cameras and speed information indicating that the object is moved between the plurality of cameras And path information which is information on the photographed moving path, The type information includes: Information relating to the length of the object, and information related to the color of the object.

Images