DE102008005993A1 - Object e.g. vehicle, tracking method, involves evaluating movement pattern of particular movement of camera, and integrating evaluated movement pattern of particular movement of camera in movement model of object - Google Patents

Abstract

The method involves providing a movement model for tracking an object. A movement pattern of a particular movement of a camera is evaluated based on a picture sequence recorded by the camera. The evaluated movement pattern of the particular movement of the camera is integrated in the movement model of the object, where a sinusoidal curve or a linear combination of sinusoidal curves is assumed as the particular movement of the camera. The frequency, amplitude, phase and direction are evaluated as parameters for the particular movement of the camera.

Description

Technical area

The The invention relates to a method for tracking objects using a camera, allowing for a tracking of an object a movement model is provided for this object.

State of the art

The Processing and evaluation of images for obtaining information from pictorial material as well as an interpretation of this information are Areas of current research. Also the tracking of objects by means of a camera, which is also called tracking or video tracking, based on the processing and evaluation of recorded with the camera Sequences of images, with the aid of these image sequences, for example, motion sequences, Movement speed or positions of objects measured or can be estimated.

At the Tracking is first an object based on taking pictures with the camera detected and determined a position of the object. Then follows a tracking system which is usually next to the camera for example a suspension device for the camera, a Analysis unit and control and control units includes the object. It must be a course of movement of the object or moving Camera, for example, fixed or movable on a vehicle can be mounted, an apparent movement of the recorded object be estimated on the basis of hypotheses. These hypotheses are based z. B. on physical or mathematical laws as well as empirical values. It is also possible that of one Camera several objects can be detected and tracked simultaneously, so these objects have similar or similar motion patterns exhibit.

The Tracking objects has, above all, the main task, the movement of the objects within the image area. It should essentially the positions of the objects in each frame of an image sequence estimated. Is it just an object and the camera mount is equipped with actuators, but this can also be used be used to control the camera so that the object as possible is held in the middle of the picture.

The For example, tracking can be divided into multiple processing steps become. In a processing step called prediction becomes a - mostly mathematical - prediction of a Position and movement information of the object or objects performed. In doing so, known physical or mathematical laws are used, about unlikely movements or positional positions excluded. For example, one on a street driving car tracked, so it is relatively unlikely that to move this car vertically without any external influence becomes.

a further processing step in tracking provides the so-called Association in which empirical values (eg information, Assumptions or movements from previous measurement cycles observed objects) one or more objects to be tracked be assigned. The association is used in particular when a measurement of the movement of the object to be tracked not can be done clearly. The object to be tracked is z. B. partially covered.

in the The processing step called innovation becomes the results (eg determining the current situation, movement-related information, etc.), which consists of prediction and current measurement or Association have been derived, weighted merged. In general, then for a movement of Objects models - so-called movement models - derived. Based Such a movement model then becomes the tracking of the object performed by the camera. The results as well as the Accuracy of tracking will be due to the quality of this Movement models or by the degree of approximation of Model determined to reality.

A well-known tracking algorithm is z. For example, the so-called Kalman filter, which is a stochastic state estimator for dynamic systems. Another example of a method for tracking objects by means of a camera, in which effectively integrates a relatively simple movement model, is the so-called Kanade-Lucas-Tomasi-Tracker (Literature: Birchfield, Stan: KLT: Implementation of the Kanade-Lucas-Tomasi Feature Tracker ; http://www.ces.clemson.edu/~stb/klt/ ; Shi, Jianbo and Tomasi, Carlo: Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994 ). For example, in the Kanade-Lucas-Tomasi tracker movement model used to track the object, it is assumed that there are no jumps from the movement of the object to be tracked.

During tracking, objects such as vehicles, traffic signs, persons, animals, etc. are tracked by means of a camera in order to estimate or measure, for example, their movement patterns, speed, position, etc. The camera used for tracking can either be stationary or moved. A stationary camera is fixed or movable in a certain place (ie the camera carries out defined movements (eg rotations, pans, etc.)). A moving camera is mounted on a movable object (eg a vehicle) fixed or movable and is moved with this. For example, stationary cameras are used to measure vehicle motion or to estimate speeds, typically assuming that the tracked vehicle is being moved smoothly (ie, without sudden changes in direction and / or speed). Moving cameras are used, for example, for tracking traffic signs. When tracking, the movement of the camera, for example, as an apparent movement of the object to be tracked, must be considered.

Indeed When tracking, take into account that the camera is mostly not really completely quiet or the assumed movement model follows exactly. Usually, the camera's own movements (eg vibrations, vibrations, etc.). So For example, a stationary attachment to which the camera is permanently mounted, swinging, or it can From the vehicle to which the camera is attached, vibration be transferred to the camera. Through these movements The camera shots are usually used for tracking negatively influenced. Tracking based measurements (eg Measuring the speed of an object, etc.) are usually used significantly less accurate.

Around to compensate for the effects of proper movements of the camera, are usually the recorded image sequences with mechanisms and processing algorithms for electronic image stabilization. In this case, by an image processing algorithm, for example tries to get interesting points in consecutive pictures (see above) called "points of interest") or edges the assumption that similar in successive pictures Scenarios are included, then z. For example, the selected ones Points or edges in the images matched and thus performed an adjustment of the image sequence. It tries to Distortions and blurring by the proper movement of the Minimize camera. Because the customization by these algorithms is common is done at the pixel level, this approach is usually associated with high computer work. There is also a search and find of appropriate points or edges in the images and a subsequent one Adjusting the image sequence is relatively time consuming. additionally can post-processing and matching the recorded image sequence to a reduction of the resolution and accuracy.

Presentation of the invention

Of the The invention is therefore based on the object of specifying a method through which in a simple and rapid way a higher Accuracy in tracking objects with a camera becomes.

The This problem is solved by a method of the beginning specified type, based on a recorded by the camera Image sequence estimated a movement pattern of a self-movement of the camera and then the estimated motion pattern of proper motion the camera is integrated into the movement model of the object.

Of the Main aspect of the invention proposed Method is that in a simple way when tracking the disturbing influence of the camera's own motion is minimized will be, without a time-consuming and with a lot of computer work connected post-processing for image stabilization is necessary. A movement pattern of the self-movement of the camera according to the invention already estimated during the tracking and in the tracking process integrated to disturbances due to proper movements of the camera to minimize in the recorded image sequence. That's how it works advantageously increases the accuracy of tracking.

It is favorable if for the movement pattern of Self-motion of the camera defined parameters from positions of the camera Object in the recorded image sequence are estimated. Because the camera's own motion usually has certain movement patterns (eg, sinusoidal oscillation, etc.) can be reduced to simple Way parameters are defined which are easily derived from positions of the Tracked object can be determined in the recorded image sequence can. These parameters then become the movement pattern the self-movement of the camera described.

In an advantageous embodiment of the invention Procedure is the proper motion of the camera as a sine wave or as a linear combination of sinusoids, given by the Camera mostly vibrations or vibrations executed as proper motion become. These movements can be very simple as sinusoidal or a combination of sinusoidal vibrations will be described.

An inherent movement of the camera, which is assumed to be a sinusoid, can be described completely in an advantageous manner by parameters such as frequency, amplitude, phase and direction and thus estimated. It is the frequency of the Speed, from the amplitude to the strength and from the phase the time of a zero crossing of the vibration of the camera specified. The direction tells you whether the camera swings from top to bottom, from left to right or diagonally, for example. The four parameters (frequency, amplitude, phase and direction) can be determined directly from the positions of the tracked object in the captured images, the estimation of these parameters being performed in a similar manner and in one step with the estimation of the motion model used for the tracking can.

It is advantageous if for estimating the movement pattern the self-motion of the camera a sequence of at least four Pictures is preset, since on the basis of at least four pictures the Self-movement of the camera can be well estimated. Especially, if it is assumed that a proper movement as a sine wave, can the necessary parameters (frequency, amplitude, phase and direction) be determined quickly and easily by at least four images.

at multiple objects to be tracked by the same camera may be more advantageous For all these objects, use the motion pattern of the self-motion the camera are assumed to be the same, since all of the camera recorded object in the same way affected by this movement pattern are. This means taking pictures of the objects in the sequence the movement pattern of the camera's own motion z. In the same way distorted or shifted.

It is also advantageous when estimating the motion pattern the proper movement of the camera is considered a fixing of the camera, because the mounting strongly influences the proper movement of the camera becomes.

typically, These fixings have certain resonance frequencies. The Resonance frequency is the so-called natural frequency of a vibratory System. It is therefore beneficial when estimating of the movement pattern of the camera's own motion one or more Dominant resonance frequencies of the attachment of the camera considered usually because of the frequency of self-motion the camera - especially with sinusoidal - either the value of the resonance frequency or a value very close to this Resonant frequency is assumed. This dependence of Self-movement of the camera from the attachment can be beneficial Be used in estimating the frequency. is For example, it is possible when installing the camera Resonance frequency, this can be estimated the parameter can be used. The frequency then becomes, for example constantly set to the value of the resonant frequency of the fixture.

Furthermore, Fixing the camera can cause the self-movement (eg sinusoidal oscillation) of the camera only in one through the attachment predetermined direction (eg from top to bottom, etc.) executed can be. This gives the advantage that the direction the self-motion of the camera no longer needs to be estimated, but can be assumed as known. When estimating the movement pattern of the camera's own motion can then turn on Simple way to be taken into account that by fixing the camera only allowed a swing in a certain direction becomes. In a known direction of the self-movement of the camera, which on simple manner can be determined by the attachment is for estimating the motion pattern of the self-motion the camera only a sequence of at least three images presuppose.

Brief description of the drawing

The invention will now be described by way of example with reference to the appended drawings 1 explained. These 1 shows by way of example a structogram for the sequence of the method according to the invention for tracking objects by means of a camera.

Embodiment of the invention

1 shows the schematic sequence of the method according to the invention for tracking objects by means of a camera. The method according to the invention starts with a starting step 1 began.

Before an object can be tracked by means of a camera, the object to be tracked (eg vehicle, animal, road signs, etc.) must be determined in an image recorded by the camera or in a recorded image sequence. Determining an object to be tracked typically involves two subtasks. A first subtask, which in a second step 2 is carried out, includes finding an object suitable for tracking in the image taken by a camera or in the image sequence recorded by the camera. This second process step 2 can therefore also be referred to as detecting the object to be tracked. In a second subtask, which in a third step 3 is carried out, then the position of the object based on the recorded image or on the basis of the recorded image sequence by estimation certainly.

In a fourth process step 4a On the basis of the image sequence recorded by means of the camera, a movement model for a tracking of the object is determined. For example, the so-called Kanade-Lucas-Tomasi-Tracker is used to derive an efficient movement model and to exclude improbable motion sequences of the object.

Since the detection and the position determination of the object based on an original image sequence - ie the image sequence used was not reworked by means of electronic image stabilization, this image sequence is adversely affected by proper movements of the camera (eg vibrations, vibrations). This means that the objects in the image sequence are distorted or displaced, for example due to vibrations or vibrations, which may occur, for example, due to movements of a camera mount or a vehicle on which the camera is mounted. Therefore, parallel to the fourth process step 4a a fifth process step 4b carried out. In this fifth process step 4b a motion pattern of the camera's own motion is estimated on the basis of the recorded image sequence, wherein parameters (eg frequency, direction, strength, etc.) defined for the estimation of the motion pattern of the camera's own motion are determined from the position of the object in the recorded image sequence become.

Frequently, this movement of the camera, from which a shot of the object is negatively affected, corresponds to a sine wave or a linear combination of sine waves. Will in the fifth step 4b For example, a sinusoidal vibration detected as a movement pattern of the camera's own motion, it can be described by four parameters such as frequency, amplitude, phase and direction. From the frequency while a speed of oscillation is specified. The amplitude defines how much the camera swings. The phase indicates a zero crossing of the oscillation, and the direction indicates where the oscillation will be carried out (eg from left to right, from top to bottom, diagonal, etc.). These four parameters can then be estimated from a sequence of at least four images.

If several objects are detected and tracked at the same time, then in the fifth method step 4b assume that these objects are equally negatively affected by the camera's own motion. Ie. For example, all of the objects are shifted or distorted in the same way because of the camera's own motion, and therefore, a motion pattern of the camera's own motion is estimated for all of these objects.

In the fifth process step 4b It is also possible to consider an attachment of the camera when estimating the movement pattern. On the one hand, the attachment of the camera can influence the direction of the camera's own movement, so that the camera, for example, can oscillate or vibrate only in one direction (eg from top to bottom). As a result, for example, the parameter of the direction for the movement pattern of the self-motion is predetermined and no longer needs to be estimated.

By On the other hand, the attachment can also be the parameter of the frequency to be influenced. So can the frequency of the camera's own motion for example, with the so-called resonance frequency of the attachment coincide or be located very close to the resonant frequency. In This case may help in the estimation of the movement pattern the self-motion of the camera, the frequency is assumed to be constant and set to the value of the resonant frequency, if that for example, during an installation of the camera measured accordingly has been.

After the estimation in the fifth process step 4b the estimated motion pattern of the camera's own motion thus becomes the sixth method step 5 in the process step 4a integrated movement model integrated. In a seventh process step 6 Then, the tracking of the object by means of the camera and on the basis of the movement model, in which the movement pattern of the self-movement of the camera has been integrated, performed.

The inventive method is the same for tracking applications with fix mounted at one location Cameras with cameras fixed to a moving object (eg Vehicle) are mounted and therefore moved with this, and also for tracking applications with movably mounted cameras, which can be stationary or moved again, used.

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited non-patent literature

• - Birchfield, Stan: KLT: Implementation of the Kanade-Lucas-Tomasi Feature Tracker [0008]
• - http://www.ces.clemson.edu/~stb/klt/ [0008]
• - Shi, Jianbo and Tomasi, Carlo: Good Features to Track. IEEE Conference on Computer Vision and Pattern Recognition, pages 593-600, 1994 [0008]

Claims (9)

Method for tracking objects by means of a camera, wherein a movement model for this object is provided for following an object ( 2 . 3 . 4a ), characterized in that on the basis of an image sequence recorded by the camera, a movement pattern of an own movement of the camera is estimated ( 4b ), and that this estimated movement pattern of the camera's own motion is integrated into the movement model of the object ( 5 ). Method according to Claim 1, characterized in that parameters defined for the movement pattern of the camera's own movement are estimated from positions of the object in the recorded image sequence ( 4b ). Method according to one of the preceding claims, characterized in that a sine curve or a linear combination of sinusoids is assumed as the proper motion of the camera ( 4b ). A method according to claim 3, characterized in that are estimated as a parameter for frequency, amplitude, phase and direction for the movement pattern of the self-motion of the camera as a parameter for the inherent motion of the camera assumed as sinusoid ( 4b ). Method according to one of the preceding claims, characterized in that for estimating ( 4b ) of the movement pattern of the camera's own motion is preceded by an image sequence of at least four images. Method according to one of the preceding claims, characterized in that, in the case of several objects to be tracked, an estimate of the motion pattern of the camera's own motion is assumed to be the same for all these objects ( 4b ). Method according to one of the preceding claims, characterized in that, when estimating the movement pattern of the camera's own movement, an attachment of the camera is taken into account ( 4b ). Method according to claim 7, characterized in that during estimation ( 4b ) of the movement pattern of the camera's own motion one or more dominant resonance frequencies of the attachment of the camera are taken into account. Method according to one of claims 7 or 8, characterized in that during estimation ( 4b ) of the movement pattern of the camera's own movement is taken into account, that by fixing the camera a swing in a certain direction is allowed.