JPS60195682A - Tracking method of moving body - Google Patents

Abstract

PURPOSE:To surely catch the movement of a moving body even at a place where the optical environment is inferior and also to shorten the processing time, by providing plural picture memories, moving position calculating circuit, estimator, correlation operator, etc. CONSTITUTION:A TV camera 1 takes the picture of a moving body and outputs its video signal to an AD converter 2, etc. The data of the converter 2 are outputted to the 1st and 2nd picture memories 5 and 6 through a change-over switch 4 with a time lag. Data of partial picture extracted from the memories 5 and 6 through a moving position calculating circuit 8, estimator 9, etc., are supplied to a correlation operator 10 where the mutual correlation between both picture images is found. As a result, the discrepancy between both the partial pictures, at which the mutual correlation becomes the maximum, is added to an estimated position and the moving position of the moving body is detected. After a prescribed time has passed, the mutual correlation of partial pictures at another estimated position newly estimated from the detected moving position is again found by the operator 10 and another moving position is detected. By repeating these operations, movement of a moving body can be caught surely even at a place where the optical environment is inferior.

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention relates to a method for automatically tracking a moving object using an imaging device such as a television camera.

The main method used to capture the movement of moving objects without contact is using light. How to measure position, I
Although there is a method that uses an optical sensor called a motion sensor, the method that is widely used is a method that uses a television camera.

Conventionally, the method of tracking a moving object using a television camera is
The moving object can be tracked by attaching a light emitting body such as an LED to a moving object, detecting the brightest part corresponding to the light emitting object from the screen shot by a television camera, and sequentially performing this detection. I try to do it.

However, although this method has the advantage of being easy to implement, if a bright part other than the target (light emitter) appears on the screen, it may become impossible to track it. Therefore, Hiroko's method has problems in its implementation outdoors where there are many disturbances.

The present invention has been made in view of the above-mentioned circumstances, and provides a method for tracking a moving object that can reliably capture the movement of a moving object even in a place with a poor optical environment and that can shorten processing time. The purpose is to provide.

According to this invention, by utilizing the fact that in temporally continuous images, there is a high correlation between frames, unique images that are part of the image of the moving object can be extracted from one screen worth of images of the moving object. A partial image of a predetermined area including the turn image is extracted, and after a predetermined time has elapsed, a partial image of the predicted position predicted based on the movement trajectory of the moving body is extracted from the screen at that time), and both partial images are extracted. Based on the cross-correlation, the shift amount of both partial images with the maximum cross-correlation is added to the predicted position to detect the moving position of the moving body, the partial image at this moving position is extracted, and after a predetermined time elapses, the shift amount of both partial images is added to the predicted position. A partial image of the predicted position newly predicted from the detected moving position is extracted from the screen at that time,
The cross-correlation between the two partial images is again determined, and the moving position of the moving object is detected in the same manner as above, and this is repeated to track the moving object.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

First, one method of the present invention will be explained in principle. Figure 1(a)
shows an example of a screen taken by the TV right camera, and a moving object (a bulldozer) is shown on this screen. From this screen, an image (partial image) containing a unique pattern image that is a part of the image of the Schizordozer is extracted (FIG. 1(b)). Then, as the guru dope moves, the nog turn image is tracked.

Now, the coordinates of the center position (movement position) of the partial image surrounded by the solid line in FIG. 2 at an arbitrary time are (xl.

yI), and after a predetermined time Δ has passed, the coordinates move to (xt+++7t+1) as the bulldozer moves.

First, based on the movement trajectory of Gurudope, that is, n movement positions (xl, yl), ("I-1-3'1-
1),...(X+-n+1 + 3'l-1+1)
The movement position after the predetermined time Δ has elapsed is predicted based on .

Let this predicted position be ;). Subsequently, a partial image (the image surrounded by a chain line) related to the predicted position (x, y) is extracted from the screen after a predetermined period of time has elapsed.

Then, the deviation of the pattern images between the partial image surrounded by the solid line and the partial image surrounded by the dashed-dotted line is corrected.

When calculating this amount of deviation, the cross-correlation between both partial images is calculated, and the amount of deviation is calculated from the position where the peak value of the correlation appears. Now, this deviation amount is (ΔX.

Δy), the actual movement position (X
1+1+3'++1) is obtained by adding this amount of deviation and the predicted position.

Next, a partial image surrounded by a broken line based on this moving position (Xl+i + )'l+1) and a partial image gI( The moving position is determined in the same manner as above based on the correlation with [not shown]. By repeating this process, the Shibata turn image (bulldozer 1) is tracked.

Next, the method of the present invention will be explained in detail according to the embodiment shown in FIG. 3. In FIG. φ signal
Output to converter 2 and synchronization signal generator 3. The synchronization signal generator 3 separates the synchronization signal from the input video signal,
The A/D converter 2 converts the input video signal into image data of, for example, 16 gradations per pixel in synchronization with a synchronization signal from the synchronization signal generator 3, and outputs this to the changeover switch 4.

The changeover switch 4 has a 1 signal applied from the synchronization signal generator 3.
The movable contact piece 4m is connected to contact 4 for the signal every time the frame ends.
b and contact 4c alternately and output the input image data to the first image memory 5 and the second image memory 6. The first image memory 5 and the second image memory 6 each store image data for one screen, and each time the changeover switch 4 is switched, the stored contents of each memory are alternately changed to a new screen image f#f. -Updated in the evening. Therefore, a predetermined time Δ1 (=1730 seconds) is required to insert the image data between the first image memory 5 and the second image memory 6 (when one frame time is 1/3o seconds) (-number of screens). There will be a time difference.

The positions of the partial images extracted from the first image memory 5 and the second image memory 6 are controlled by a memory controller 7. That is, the memory controller 7 inputs a signal indicating the center position coordinates (z, y) of the main turn image from the movement position calculation circuit 8 as a signal indicating the movement position of the /4' turn image, and A signal indicating the predicted position (x, y) of the lean image after a predetermined time Δ has elapsed from the movement position is inputted from the device 9, and based on these position signals, the position (+c, y) and ( , :;
.

Now, at time t0, the lean image is at position (xo.

yo) Kashi, the screen at this point is stored in the first image memory 5, and after a predetermined time has elapsed, the screen at time 1. A case where the screen at (=1°+Δt) is stored in the second image memory 6 will be described. First, display the screen from the image ray' stored in the first image memory 5 on a monitor television (not shown), and specify the position of the pattern image (Xo + '16) using a light pen or the like.・This position (
The signal indicating x o + 3' 6 ) is input to the movement position calculation circuit 8 as an initial value, and is applied to the memory controller 7 and predictor 9 as is. The predictor 9 predicts the moving trajectory of the moving object, that is, a predetermined trajectory. The next movement position is predicted based on movement position signals from movement position calculation port 1f etc. added every time, and the predicted position (x
, y), but since the movement trajectory is undetermined in the initial state, in this case, the above initial position (
xor yo).

The memory controller 7 is connected to the movable contact piece 4 of the changeover switch 4.
1 is connected to contact 4c, and the latest screen image data is connected to the second
If stored in the image memory 6, a control signal is output to the second image memory 6 in order to extract a partial image centered at the predicted position based on the input signal indicating the predicted position (x, y). Based on the input signal indicating the movement position (x, y), a control signal is output to the image memory 5 for extracting a partial image centered at the movement position, and the movable contact of the changeover switch 4 is When the piece 4a is connected to the contact 4b and the latest screen image data is stored in the first image memory 5, control signals for extracting each partial image are generated based on signals opposite to those in the above case. Output. Note that at time t1, the control signal for the former case is output, and the position (
Since x, y) and (x, y) are the same, the partial images extracted from the first image memory 5 and the second image memory 6 are partial images at the same position on the screen, respectively.

The data of the partial images extracted from the first image memory 5 and the second image memory 6 are sent to the FFs of the correlation calculator 10, respectively.
The images are applied to T units 11 and 12, where they are subjected to a two-dimensional Fourier transform, the results from both images are multiplied by the corresponding frequency signal every minute in a multiplier 13, and this result is inversely Fourier transformed in an I FFT unit 14. Ru.

The inverse 7-lier transformed data is applied to the maximum value detector 15, which determines the amount of deviation (ΔX, Δy) between the two partial images that maximizes the cross-correlation between the two partial image data, and calculates the amount of deviation (ΔX, Δy) corresponding to this. A signal is sent to a movement position calculation circuit.

The movement position calculation circuit 8 adds the deviation amount (ΔX, Δy) to the initial value (xor yo), and uses a signal indicating this added value as the position (x, y) of the 4-turn image after a time period of Δ. Output as a signal indicating.

A signal indicating this position (x,y) is applied to a predictor 9 along with a memory controller 7. The predictor 9 predicts the movement position of the next Δ time after the input position and the previous position (initial value (x o ryo)), and outputs a signal indicating the predicted position (x, y). is output to the memory controller 7.

Similarly to the above, the memory controller 7 outputs control signals to extract partial images from the first image memory 5 and the second image memory 6, respectively, based on the two human input signals.

In this case, the movable contact piece 4a of the changeover switch 4 is switched to the contact point 4bK, and the first image memory 5 stores image data of a new screen. Also, the first
The partial image read out based on the position (x, y) from the image memory 5 and the partial image (2) read out based on the position (x, y) from the second image memory 6 are (x-x, y-y
) is a partial image shifted by an amount.

The correlation between these partial images is again determined, the amount of deviation (ΔX, Δy) at which the correlation is maximum is determined, and this is added to the current movement position (x, y) to determine the new movement position.

A moving object can be tracked by repeatedly executing the above process.

Note that the predictor 9 performs linear prediction based on the following equation, K, from n movement position data from the current time t to (ΔtXn) hours ago. The coefficients J and bl of the predictor 7 are determined in advance by analyzing the movement of the moving object.

If the predictor 7 does not predict the position after Δ, the partial images extracted from the first image memory 5 and the second image memory 6 will be at the same position, but in this case, the partial images will have MXN pixels. , and tracking is impossible unless the position change of the moving object is within (N/2, N/2) (
If sufficient correlation is not obtained), the speed vx+' of the moving body in the X direction/velocity t'y in the direction is limited to the following. However, Δt is the time to capture one screen.

However, when predicting the position after a time Δ by the predictor 9, it is sufficient that the amount of deviation between the predicted position and the actual movement position is within (N/2.N/2), and the movement of the moving body Does not depend on speed.

Note that the correlation calculation method between the two extracted partial images is not limited to every frame, but may be performed every several frames, for example. Furthermore, by tracking a moving object according to the present invention,
Its movement trajectory, speed, acceleration, etc. can be recorded.

As explained above, according to the present invention, it is possible to reliably capture the movement of a moving object even in a place with a poor optical environment, and the processing time is shortened due to correlation calculation between partial images extracted from one screen. Furthermore, since the moving position of the moving object is predicted and the partial images are extracted, it is possible to obtain a good correlation that is less affected by the moving speed of the moving object.

[Brief explanation of drawings]

FIG. 1(a) is a diagram showing an example of a screen photographed by a TV camera, FIG. 1(b) is an enlarged view of a partial image extracted from FIG. 1(a), and FIG. 2 is a diagram showing an example of a screen taken by a TV camera. FIG. 3 is a diagram showing a partial image used to explain the principle, and FIG. 3 is a diagram showing an embodiment of the method of the present invention. 1...TV left camera 2...A/D converter, 3...
Synchronous signal generator, 4... changeover switch, 5... first
Image memory, 6... Second image memory, 7... Memory controller, 8... Movement position calculation circuit, 9... Predictor, 10... Correlation calculator, 15... Maximum value detection vessel. Figure 1 (G) (b) Figure 2

Claims (4)

[Claims] (1) A moving object is photographed in a predetermined field of view, and a position of a first partial image of a predetermined area including a unique main turn clarity that is a part of the image of the moving object is determined from one screen of the photographed image. , and the first partial image at the specified time t0 is used as a reference, and the screen photographed at time t, (= t0+Jt) after a predetermined time Δ has elapsed. Calculate the cross-correlation with the second partial image extracted from the predicted position predicted based on the movement trajectory of the moving object, and calculate the amount of shift between the first and second partial images at which the cross-correlation is maximum. The moving position of the reference image is detected by adding it to the predicted position, and then the partial image is extracted from the screen photographed at time t1 based on the detected moving position, and the partial image and the time are t2 (=t, +Δt) and the third partial image extracted from the predicted position predicted based on the new movement trajectory of the moving object, and similarly, a reference image is obtained. 1. A method for tracking a moving object, characterized in that the moving object is tracked by detecting the moving position of the object and repeatedly performing this process. (2) The position of the first partial image is specified by displaying the one-screen image on a monitor television and specifying the position of the first partial image on the monitor television screen with a light pen. A method for tracking a moving object according to claim (1). (3) The position of the first partial image is determined by recording a unique 9-turn image of the moving object in advance, and maximizing the cross-correlation between the 3-turn image and the 1-screen image. A method for tracking a moving object according to claim (1), wherein the tracking method is specified by the position of the tracking image. (4) A patent claim in which the prediction based on the movement trajectory of the moving body predicts the next movement position using a linear prediction method based on n movement positions between the current time and a predetermined time (ΔtXn) before. A method for tracking a moving object according to scope (1).