JPH04115376A - Image tracking device - Google Patents

Abstract

PURPOSE:To attain highly accurate image correlation tracking even when a target moving distance from a target position is less than an image sampling interval by finding out the maximum value of a correlation value image with a target reference image and finding out an estimated maximum value extremely close to a real maximum value based upon respective correlation values of the position of the maximum value and its adjacent point position. CONSTITUTION:An image correlation arithmetic means 20 finds out the maximum value of a correlation value image by target extracting processing based upon correlation processing between an image obtained by an image pickup sensor and a target reference image. An estimation arithmetic means 21 estimates an estimated maximum value extremely close to a real maximum value based upon respective correlation values of the maximum value position and its adjacent point position and outputs the estimated maximum value position as a target position. Since correlation value distribution is approximated to a quadratic curve based upon the correlation maximum value position and its adjacent point position and the maximum value extremely approximate to a real value is estimated, the target can be traced with accuracy of the interval less than the sampling interval even if the sampling interval is larger than a set target moving distance.

Description

[Detailed Description of the Invention] [Summary] An image in which a target object is extracted by correlation processing between an image acquired by an image sensor and a reference image, and the visual axis of the image sensor is changed in accordance with the extracted coordinates to track the target object. Regarding the tracking device, the aim is to perform image correlation tracking with high precision even when the moving distance of the target is less than the image sampling interval, and to perform image correlation tracking with high precision even when the moving distance of the target is less than the image sampling interval. Based on the correlation value of , an estimated maximum value that is very close to the actual maximum value is estimated, and the estimated maximum value position is output as the target object position.

[Industrial application field]

The present invention relates to an image tracking device that extracts a target object through correlation processing between an image acquired by an image sensor and a reference image, and tracks the target object by changing the visual axis of the image sensor according to the extracted coordinates.

As such an image tracking device, an image tracking device that is mounted on an aircraft or the like to track a target on the ground is known. In this case, it is especially necessary to track a target with high accuracy even if the moving speed is fast.

[Conventional technology]

FIG. 6 shows a block diagram of a conventional example. In the figure, image data acquired by an image sensor 1 is sampled by an AD converter 2 and stored in a frame memory 4 according to a write address generated by an address generator 3. Next, the reference image of the shape corresponding to the target object stored in advance in the frame memory 5 and the sensor acquired image stored in the frame memory 4 are sent to the address generator 3.
The image is read out according to the read address generated by the image correlation calculator 6, where a correlation calculation is performed.
Correlation value is output. This correlation value forms a two-curved hilly distribution in the X direction and the X direction as shown in FIG.

The maximum correlation value of the thus obtained correlation value is detected by the maximum value detector 7, and is supplied to the target coordinate calculator 8, where the coordinates of the target object are determined. Next, the tracking command calculation unit 9 performs processing such as predicting the target object position based on the target object coordinates to obtain a tracking command, and this tracking command is sent to the visual axis deflection mechanism (gimbal) 10 and Activate.

The image sensor 1 continues to capture the target object through the operation of the visual axis deflection mechanism 10.

[Problem to be solved by the invention]

Now, to simplify the explanation, let's consider the correlation value only in the X direction. Let the pixel center point in the X direction be Xo Z
X+ −+ Pixel center point x0X+ +X*
. . . corresponds to sampling points in the AD converter 2, and between Xs- and X+-, between X, - and Xt', . . . are sampling intervals ΔX.

By the way, in the conventional example, the pixel position where the correlation value is maximum is determined as the target object position, so if the sampling interval ΔX is larger than the movement distance of the target object, the maximum value is the sampling point as shown in Fig. The target object may be located between the target object and the sampling point, and in such a case, there is a problem that the target object cannot be captured.

SUMMARY OF THE INVENTION An object of the present invention is to provide an image tracking device that can perform image correlation tracking with high precision even when the moving distance of a target is less than or equal to the image sampling interval.

[Means to solve the problem]

FIG. 1 shows a diagram of the principle of the present invention. In the figure, 2° is an image correlation calculation means, which calculates the maximum value of the correlation value image through target extraction processing based on correlation processing between the image captured by the imaging sensor and the target reference image. Reference numeral 21 denotes an estimation calculating means, which estimates an estimated maximum value that is very close to the actual maximum value based on the maximum value position of the correlation value image obtained above and the respective correlation values at adjacent point positions of the maximum value position. and outputs the estimated maximum value position as the target object position.

[Effect]

As shown in FIG. 5, the maximum correlation value obtained by the image correlation calculation means 20 is Cm (X-axis coordinate Xt-) + the correlation value at the adjacent point is C+ (x-axis coordinate X+''), Cm (
Assuming that the X-axis coordinate is X'), the sampling interval is ΔX, the difference between the coordinate of the maximum correlation value C8 and the estimated maximum value coordinate is It will be done.

x:)(,--X △X C −〇.

In this way, in the present invention, the correlation value distribution is approximated using quadratic surface approximation based on the correlation maximum value position and its adjacent point positions, and the maximum value position in the correlation image is estimated with an accuracy equal to or less than the sampling interval. .

〔Example〕

FIG. 2 shows a block diagram of an embodiment of the present invention. In the figure, the same components as those in FIG. 6 are given the same numbers, and the explanation of their operation will be omitted. In FIG. 2, the portion surrounded by a broken line is constituted by a microprocessor or digital signal processor.

In the second @, the correlation value obtained by the image correlation calculator 6 and the correlation maximum value obtained by the maximum value detector 7 are calculated by the difference calculator 1.
1, and here, a difference value is determined in each of the X and X directions of the image based on the correlation values of the maximum position and its adjacent point positions. That is, if the correlation value distribution obtained by the image correlation calculator 6 is as shown in FIG.
The maximum correlation value IE0 (as explained in FIG. 5, the sampling point of the maximum value E is located away from the actual maximum value) and the maximum value E. From the adjacent point correlation values El, E2°E, , E4 in the X direction and the X direction, Es-E4. E, -E. 9E4-E,,E,-E! ,
Correlation difference values E, -E, , E, -Eo were determined.

The correlation difference value obtained by the difference calculator 11 is sent to the peak estimator 1
2, where an estimated maximum value that is very close to the actual maximum correlation value is estimated. As shown in the correlation value schematic diagram in FIG. 4, the coordinates of the maximum correlation value E0 obtained by the maximum value detector 7 are (xo, yo).

The sampling interval in the X direction is Δx, the sampling interval in the y direction is Δy, and the coordinate of the estimated maximum value is (X.

9), the difference in the X direction and the X direction between the estimated maximum value coordinate and the coordinate of the maximum correlation value E0 obtained by the maximum value detector 7 is
, Y, the coordinates (x, y) of the estimated maximum value are (x, 9) = (XO+X, 3'o +Y)2
Es −2Eo +E4.

A tracking command is requested. This tracking command is supplied to the visual axis changing mechanism 10, which directs the visual axis of the image sensor l toward the target.
'Jru. , 2, in the present invention, instead of using the maximum correlation value obtained by the maximum value detector 7 as the target object position as in the conventional example shown in FIG. 6, the obtained correlation maximum value E is used. Since the estimated maximum value, which is very close to the actual maximum correlation value, is set as the target object position based on the correlation values E, ~E4, etc. of the adjacent points, even if the sampling interval is set larger than the target object movement distance, It is possible to track a target with an accuracy less than the sampling interval.

becomes. That is, the maximum correlation value E obtained by the maximum value detector 7
6 and the adjacent point correlation values E and -E4, the actual correlation value distribution is approximated using quadratic surface approximation to obtain an estimated maximum value that is very close to the actual maximum correlation value.

The estimated maximum value obtained in this way is supplied to the tracking command calculator 9 as the target object coordinates, and processing such as predicting the target object position is performed based on the target object coordinates. According to the present invention, the correlation value distribution is approximated to a quadratic surface based on the maximum correlation value position and its adjacent point positions to estimate the maximum value that is very close to the actual maximum value, so the sampling interval can be reduced. Even if the target movement distance is set larger than the target object movement distance, the target can be tracked with an accuracy less than the sampling interval, and even if an AD converter with the same sampling interval as the conventional example is used, image tracking is more accurate than the conventional example. It is possible to do this.

[Brief explanation of the drawing]

Fig. 1 is a diagram of the principle of the present invention, Fig. 2 is a block diagram of an embodiment of the present invention, Fig. 3 is a diagram explaining quadratic surface approximation in the present invention, and Fig. 4 is a correlation value model in the present invention. 5 is a diagram showing the correlation value distribution in one dimension, FIG. 6 is a block diagram of an example of the conventional technology, and FIG. 7 is a diagram showing the correlation value distribution in two dimensions. 7 is a maximum value detector, 9 is a tracking command calculator, 10 is a visual axis changing mechanism, 11.21a is a difference calculator, 12.21b is a peak estimator, 20 is an image correlation calculation means, 21 is an estimation calculation means show. Patent applicant: Fujitsu Limited In the figure, 1 is an image sensor, 2 is an AD converter, 4 is a frame memory (sensor acquired image), 5 is a frame memory (reference image), 6 is an image correlation calculator, this book Invention: Fig. 2 explaining quadratic surface approximation in this case Principle diagram of the invention Fig. 1 E3 (X3.Y3) Schematic diagram of correlation values in this invention I Fig. 4

Claims (2)

[Claims] (1) In the image correlation calculation means (20), the maximum value of the correlation value image is determined by the target object extraction process based on the correlation process between the image sensor acquired image and the target object reference image, and In an image tracking device that tracks a target with the position as the target position, the actual maximum value is determined based on the maximum value position of the correlation value image obtained above and the respective correlation values at the adjacent point positions of the maximum value position. Estimating calculation means (2) for calculating the closest estimated maximum value by estimation and outputting the estimated maximum value position as the target object position.
An image tracking device comprising: 1). (2) The estimation calculation means (21) includes a difference calculation unit (21a) that calculates a difference value between the maximum value position and the adjacent point position.
), a peak estimator (21
The image tracking device according to claim 1, characterized in that it comprises the following.