JPH0696211A - Object tracking method - Google Patents

Abstract

PURPOSE:To track a target object for a long time with high accuracy even when plural moving objects are present in an image. CONSTITUTION:An image input device 100 continuously inputs moving images. An image 101 is an image '1' at time (t) and an image 102 is an image '2' newly inputted at time t+1 and contains one or more pieces of moving objects. An object position 102 is already fixed while being overlapped on the target object in the image '1'. Moving amount detection processing 110 inputs the images '1', '2' and object position 102, detects the moving amount 111 of the target object from (t) to t+1 and calculates a temporary object position 112 in the image '2'. In tracking energy image generation processing 120, a tracking energy image 121 is generated from the images '1' and '2' and the moving amount 111. In position correction processing 130, the temporary object position 112 is corrected according to the tracking energy image 121, and a new object position 131 is calculated.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to the field of image processing, and more particularly to a method for stably tracking a moving object existing in continuously input moving images.

[0002]

2. Description of the Related Art In order to track a moving object in a moving image, the moving amount of the object is mainly determined by associating the object to be tracked in the image one time before and the same object in the image one time after. The method of detection is taken. At this time, when a plurality of moving objects are included in the image, it is necessary to use the features relating to the objects to distinguish the individual objects. Conventionally, this includes a method of detecting the amount of movement by obtaining an area that is most similar in brightness to the object area, a method of detecting the amount of movement from the correspondence of the feature points of the object, and the amount of movement using temporal and temporal differentiation. There are detection methods, etc. (for example, Computer Vision, Masahiko Taniuchi, Maruzen, March 31, 1990)

[0003]

In the above-mentioned prior art, there is a problem in that the calculation accuracy of the movement amount deteriorates because the appearance changes greatly when the object to be tracked is occluded or the direction of the object changes. It was Also, when the motion of an object involves rotation or rotation, the amount of movement of the object surface and the amount of movement of the object itself differ, so if tracking is performed only according to the movement of the object surface, when the tracked part becomes invisible Tracking fails. Further, since the digital error is accumulated by repeating the tracking process, there is a problem that the tracking accuracy is lowered.

A method of predicting a past movement amount and compensating for the accuracy when the appearance of an object changes when repetitive processing is performed has been proposed, but the movement amount is gradually increased due to the accumulation of errors due to digital errors. However, there is a problem in that it is not possible to predict a sufficient amount of movement when it changes to.

An object of the present invention is to provide a method for preventing the deterioration of the object position estimation accuracy due to the occlusion and the change of the appearance and the tracking accuracy due to the repetitive processing, and enabling the object tracking always with high accuracy for a long time. Especially.

[0006]

In order to achieve the above object, in the object tracking method of the present invention, every time an image is input, an image immediately before the image (old image) and the newly input image are input. Image (new image) and the position on the object to be tracked (object position) in the old image that has already been obtained, the moving amount of the tracking object and the temporary object position in the new image corresponding to the object position in the old image are determined. A process for obtaining a tracking energy image having a small value in an area having the same amount of movement as the object to be tracked and a large value in an area having a different amount of movement based on the density value of each point of the old and new images and the movement amount And a process of moving the temporary object position to a position having a smaller value with the tracking energy image as a reference and setting the object position of a new image.

[0007]

[Function] First, at the time t using the continuous old and new images.
The object position at time t + 1 corresponding to the object position at 1 is solved by a matching problem, and the movement amount of the tracking object from time t to time t + 1 is calculated. Next, in order to track a moving object, it is advantageous that the position of the object is included in a region where the object moves as much as possible. Generate an energy image and move the object position to a point exhibiting less energy.

According to the present invention, the positional deviation due to the change in the shape of the object or the accumulated digital error that occurs during tracking is solved by moving the object position to an appropriate position on the object, so that the deterioration of accuracy due to repetition is prevented. You can

[0009]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings.

FIG. 1 shows a processing framework of the object tracking method according to the present invention. In the figure, the solid line shows the flow of control and the broken line shows the flow of data.

The image input processing 100 continuously reads and inputs images from an image file or the like. The image 2 of 102 is the newly input image (new image), and the image of 101 is the image input immediately before (old image). Here, image 1 is the image at time t, and image 2 is the time t +
The image in 1. That is, the images 1 and 2 are images captured with a slight time difference. In the image,
It is assumed that there is one or more moving objects. The object position 103 is a position on the object to be tracked in the image 1 and has already been determined when the image 2 is input.

When the image 2 is newly input, the movement amount detecting process 110 uses the images 1, 2 and the object position 103 to move the object to be tracked from the time t of the image 1 to the time t + 1 of the image 2. The amount 111 is detected and the object provisional position 112 in the image 2 is calculated. Movement amount 11
1 is obtained by a conventional tracking method such as a correlation method. The object provisional position 112 is calculated from the movement amount 111 and the object position 103 of the image 1 described above, and is a position that substantially overlaps the tracked object in the image 2.

In the tracking energy image generation process 120,
The density value of each point of the image 1 and the image 2 and the movement amount 1
From 11, a tracking energy image 121 having a small value at a point showing the same movement amount as the moving amount 111 and a large value at a point showing a different moving amount is generated.

In the position correction processing 130, the temporary object position 112 is corrected in accordance with the tracking energy image 121, and a new object position 131 is calculated.

Next, when an image following image 2 is input,
This is referred to as image 2, and image 2 up to now is referred to as image 1.
Then, the object position 131 obtained by the position complementing process 130 is newly set as the object position 103. Thereafter, newly input images are processed one after another and the tracking is repeated.

FIG. 2 is a diagram for explaining a concrete example of the processing according to the present invention. In the figure, 210 is an image (image 1) at time t, and 220 is an image (image 2) at time t + 1. Reference numeral 211 is a moving object to be tracked in the image 210 at time t, 212 is a moving object other than the object 211, and 213 is an object position to be tracked (position overlapping the object 211). 221 is the image 220 at time t + 1
Of the moving objects to be tracked, 222 is a moving object other than the object 221, 223 is the amount of movement of the objects 211, 221 to be tracked from time t to time t + 1, 224 is a temporary position of the object to be tracked at time t + 1, and 225 is time t + 1. The position of the object to be tracked at (position overlapping the object 221).

The image 210 and the image 220 are images obtained by a slight time difference, and are images similar to each other except for some moving object parts (eg, 211, 212, 221, 222). An object 211 in the image 210 is an object to be tracked, and an object 221 corresponds to this in the image 220.

Now, in the image 210, it is assumed that the object position 213 overlapping the object 211 is given as x ₀ . As a result of the tracking, the object position 213 in the image 210 is calculated as the object temporary position (x ₁ ) 224 in the image 220. For tracking, for example, the position having the density value closest to the object position 213 in the image 210 is searched for in the image 220.
Do by searching in. In this case, since the density information of each object is used, it is possible to correlate even a plurality of moving objects. As a result of this tracking, the movement amount 22 of the target object
3 is obtained, the moving amount 223 and the object position (x ₀ ) 213
The object provisional position (x ₁ ) 224 is obtained by. This is executed by the movement amount detection processing 110 of FIG.

In the image 220, the object temporary position 224 is on the moving object 221, but when the appearance of the object changes, the object temporary position 224 may be the end of the moving object 221. In this state, time t +
When the next tracking from 1 to time t + 2 is performed, there is a high possibility that the tracking will fail if a part of the vicinity of the provisional position of the object deviates from the object. This is likely to occur in a moving object in which the speed of the surface of the object and the speed of the object are different and the object rotates or rotates.
Therefore, the provisional object position (x ₁ ) 224 is corrected to obtain an object position that enables stable tracking. In FIG. 2, 225 is the new object position (x ₁ ′).

A method for correcting the provisional object position (x ₁ ) 224 and obtaining a new object position (x ₁ ′) 225 will be described below using mathematical expressions.

In the example of FIG. 2, the density value of the image 210 is f
Let (x, t). However, x ∈ ² is a point on the quadratic, t ∈ R
Is the time. Object position 213 to be tracked at time t
Is _denoted as x ₀ , and the point at the object provisional position 224 to be tracked at time t + 1 is _denoted as x ₁ . The movement amount 223 at the object position x ₀ is the 2 × 2 matrix A and the 2 × 1 vector V.
Can be expressed by a linear approximation. That is, the object provisional position x ₁ in the image 220 at the point x near x ₀ is A (x
It is expressed by −x ₀ ) + V + x ₁ .

At this time, V represents the movement vector at the point x ₀ when the time shifts from t to t + 1, and A represents the apparent rotation and expansion / contraction due to the change in the appearance of the object. For example, when the object position of interest moves in parallel,

[0023]

[Equation 1]

And V becomes a value x ₁ −x ₀ obtained by subtracting the object position 213 from the object temporary position 224.

When expressed in this way, at a point x near the point x ₀ , f (x, t) is f (Ax−x ₀ ) + V + x ₀ ,
It becomes a value close to t + 1). In other words, | f (x,
t) -f (A (x- x 0) + V + x 0, t + 1) | is a value close to zero. On the other hand, | f (x, t) -f (x, t +
1) | has a large value in a moving object portion. Here, g ₁ (x) = | f (A (x−x ₀ ) + V + x ₀ , t + 1) −f (x, t) | (Equation 1) g ₂ (x) = | f (x, t + 1) ) -F (x, t) | ... (Formula 2).

When the object is actually moving, the point x at which the expression 1 is small and the expression 2 is large has a high probability of being a point overlapping the moving object 211. In other cases, another moving amount is set. It can be said that there is a high probability that it is an object you have or a background that does not change. That is, the point where both Expression 1 and Expression 2 have a small value is a point having a uniform density value, and the point where movement cannot be specified, and the point where Expression 1 is large and Expression 2 has a small value are backgrounds that do not move.
It is considered that a point having a large value in both Expression 1 and Expression 2 is a moving object having a moving amount different from that of the moving object 211.

By integrating these, it is possible to represent by a numerical value whether or not a certain point is a point on the target moving object 211, 221. For example, G (x) = C ₁ g ₁ (x) ² + C ₂ {1-g ₂ (x)} ² G (x) = C ₁ g ₁ (x) + C ₂ / {C ₃ + g ₂ (x) } G (x) = g ₁ (x) / {C ₁ + g ₂ (x)} and the like have a smaller value, and the target moving object 21
There is a high probability that it is a point above 1, 221. Where G (x)
Will be called a tracking energy image. This G (x)
The tracking energy image generation process 120 of FIG.
Is.

Using this tracking energy image, the point x ₁ of the provisional position 224 of the object is corrected to obtain the object position 22 obtained.
Move to point x ₁ ′ of point 5. This is executed by the position correction processing 130 of FIG. For example,

[0029]

[Equation 2]

[0030] 'finds a, x _1' most point x ₀ in the vicinity of small to x ₀ the to _{= A (x 0 '-x 0} ) + V + x 0. However, r is a constant representing the allowable range of the search. The obtained object position (x ₀ ′) 225 is used as a predetermined position like the object position 213 in the next tracking process.

FIG. 3 shows a schematic block diagram of a hardware configuration example for realizing the object tracking method of the present invention. Under the control of the image processor 300, the images are sequentially read from the image file 330 and output to the display surface of the terminal device 320.
Looking at this display surface, the user points the target object. On the other hand, the image read from the image file 330 is also transferred to the image memory 310. The image memory 310 has two surfaces and alternately stores two consecutive images. The image processor 300 fetches two consecutive pieces of image information in the image memory 310, and executes the movement amount detection processing 110, the tracking energy image generation processing 120, and the position correction processing 130 with the object position designated by the user as an initial value. , Track the object of interest. The tracking result is output by an arrow or the like on the display surface of the terminal device 320.

[0032]

As described above, according to the present invention, since the area is corrected by the movement amount once obtained, only the target object can be cut out even if there are a plurality of moving objects, and the shape of the object changes. It is possible to prevent the position of the object to be tracked from the target object from protruding even if the object causing the noise is partially shielded. For this reason, it is possible to perform stable tracking for a long time as compared with the case where the conventional tracking without correction is repeatedly performed.

[Brief description of drawings]

FIG. 1 is a flow diagram showing a framework of processing of an object tracking method of the present invention.

FIG. 2 is a diagram illustrating a specific example of processing of the present invention.

FIG. 3 is a schematic block diagram of a hardware configuration example for realizing the object tracking method of the present invention.

[Explanation of symbols]

 100 image input processing 101 image 1 102 image 2 103 object position 110 movement amount detection processing 111 movement amount 112 object provisional position 120 tracking energy image generation processing 121 tracking energy image 130 position correction processing 131 new object position

Claims (1)

[Claims] 1. A method for sequentially tracking a moving object existing in continuously input moving images, comprising a newly input image (hereinafter referred to as a new image) and an image input immediately before (hereinafter referred to as an image). , The old image), and the position on the object to be tracked in the old image (hereinafter referred to as the object position), the amount of movement of the tracked object and the object in the new image corresponding to the object position in the old image. A tracking energy image having a small value in a region having the same movement amount as the tracking object and a large value in a region having a different movement amount by the process of obtaining the provisional position, and the density value of each point of the old and new images and the movement amount. And a process of moving the temporary object position to a position having a smaller value with the tracking energy image as a reference and setting the object position of a new image.