JPS5933858B2 - correlation tracking device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION The present invention is an apparatus for tracking a desired target or scene within the field of view of a television camera by performing digital correlation calculation between analog-to-digital converted images captured using a television camera. It is related to.

Conventionally, in this type of device, as shown in FIG.
Analog-to-digital converted luminance signals are extracted from n bits of pixels in cross-shaped horizontal and vertical correlation regions 1 and 2 with a width of m pixels, and a two-dimensional luminance signal is generated for each correlation region in the correlation width direction. The signal was compressed into a one-dimensional multivalued luminance signal.

Furthermore, for the purpose of real-time processing of correlation calculations and miniaturization of the device, the one-dimensional multivalued luminance signal obtained as described above is converted to a one-dimensional binary luminance signal by setting a fixed threshold. was. As shown in Fig. 1, the binarized luminance signals corresponding to the horizontal and vertical correlation windows 3 and 4 having lengths NH pixels and Nv pixels are used as reference signals for correlation calculation, and are applied to the screen of the next field or frame. Similarly, a 1-bit correlation operation with the compressed and binarized luminance signal is performed in each horizontal and vertical direction to calculate a correlation coefficient.

Next, detect the maximum value of this correlation coefficient, find the coordinates on the screen that give the maximum value, move the horizontal and vertical correlation windows to that position, and move the horizontal and vertical correlation windows to the desired target or I was tracking the scene.

With this device, indoors where the brightness can be kept constant,
Since there are few changes in the brightness of the desired target or scene within the field of view of the television camera, even if a fixed threshold is set and converted to a one-dimensional binary brightness signal, only a slight change in the brightness signal will be observed. When the correlation coefficient was calculated, there was little variation in its peak value.

However, when used outdoors, the brightness and the contrast between the target and the background change from moment to moment, so if a fixed threshold value is set and a one-dimensional binary luminance signal is obtained, the change in the luminance signal will be large. In many cases, the shape of the correlation coefficient changed and the peak value fluctuated so much that tracking became impossible. For this reason, the threshold value for obtaining a binary luminance signal was manually set depending on the desired target or scene within the field of view of the TV camera, but it was not stable because it could not follow changes in brightness and contrast. I couldn't expect a good tracking operation. In order to eliminate this drawback, the present invention increases the threshold value by 1 from 1 to (2n-2) for the n-bit reference and correlated luminance signals averaged in each horizontal and vertical direction. Convert it to a binary luminance signal, find (2n-12) correlation coefficients between both signals as a time series, detect the coordinate signal on the screen that gives the maximum value of each correlation coefficient, and calculate its generation. A histogram representing the frequency distribution is created to find the coordinate signal with the highest frequency of occurrence, and the horizontal and vertical correlation windows are moved to the position of this coordinate.

A detailed description will be given below with reference to the drawings. FIG. 2 is a diagram illustrating the operating principle of the correlation tracking device according to the present invention. In this invention, luminance information is extracted for each horizontal and vertical correlation area provided on the screen and correlation calculations are performed independently.
Here, explanation will be given taking the horizontal correlation region as an example. Figure 2A
shows the arrangement of horizontal correlation areas and horizontal correlation windows on the screen, where the screen size is NhXNv pixels, the correlation width of horizontal correlation window 3 in horizontal correlation area 1 is m pixels, and the correlation length is NH Let it be a pixel.

A one-dimensional n-bit luminance signal Σ is obtained by extracting an analog-to-digital converted luminance signal using n bits in the horizontal correlation region shown in FIG. 2A, adding m pixels in the correlation width direction, and obtaining an average value.

is shown in Figure 2B. It is now assumed that the luminance signal shown in FIG. 2B is obtained for the i-th screen. As a result of performing correlation processing on this screen, a correlation window 3 is set at the position shown in FIG. 2A, and correspondingly, the luminance signal between coordinate a and coordinate b in FIG. 2B becomes the reference signal. Let's say it's summer. Next, assume that a one-dimensional n-bit luminance signal obtained for the (1+1)th screen in the same manner as above is given as shown in FIG. 2c.

This will be called a correlated signal. Here, the threshold value is set from 1 to (2n-2
), and the binarized luminance signal q corresponding to each threshold value
Convert to

Now, the binarized reference and correlated luminance signals corresponding to the threshold value 1 {1≦i≦(2n-2)} are shown as D and E in FIG. 2. In these two binary luminance signals, the coordinate a of FIG. 2 D is
When a correlation calculation is performed by moving the signal from coordinate 1 to coordinate b one pixel at a time from coordinate 1 in FIG. 2E, a correlation coefficient Ci as shown in FIG. 2F is obtained.

Next, the coordinate Ki that gives the maximum value of this correlation coefficient is determined.

As described above, (2n - 2) coordinates Ki for threshold 1 are obtained from i = 1 to j - (2n - 2), and a histogram representing the distribution of the occurrence frequency ΣKj for that coordinate is created. Then, a graph as shown in FIG. 3 is obtained.

Here, the coordinate Knlax giving the maximum value of the histogram shown in FIG. 3 is determined and used as a tracking coordinate signal.

Correspondingly, the position of the reference window for the (1+1)th screen is moved to the section from coordinate a' to coordinate b' shown in Figure 2c, and the n-bit luminance signal within this interval is
) is used as a reference signal for correlation calculation with the (1+2)th screen, and at the same time, the n-bit luminance signal obtained for the i-th screen is erased from the memory, and the luminance signal of the (1+2)th screen is written. By the way, the above correlation calculation is
Each time the reference signal and correlated signal are shifted by one pixel, the sum of the products for each pixel is calculated and executed, or the time required to calculate one point of the correlation coefficient for each pixel shift is calculated. If t is the length of the one-dimensional reference signal, NH, and the length of the one-dimensional correlated signal is Nh, then shifting the reference signal and correlated signal while completely overlapping each other is (Nh - NH).
Since it is a pixel, the time for one correlation calculation is (Nh - NH) ×
It becomes t.

Since the device according to the invention changes the threshold value (2n-2) times, the total time T required for the correlation calculation. is (2n-2)x(
Nh-NH)×t. For example, n-4 bits, N
H-200 pixels Nh-400 pixels, t-100ns, T. -2.8×10 −4 s, which takes about 0.3 ms, and high-speed calculation can be easily realized. FIG. 4 is a block diagram of a practical example of the correlation tracking device according to the present invention.

In FIG. 4, a video signal supplied from a television camera 5 is passed through an analog-to-digital converter 6, and an n-bit digitized image is divided into regions to extract a luminance signal.

Each luminance signal is added by m pixels in orthogonal vertical and horizontal directions in an arithmetic averaging circuit 8, and then divided by m to obtain a one-dimensional n-bit luminance signal.

Next, one of the memories 10 is selected by the switch 9 and the luminance signal is stored therein.

It is assumed that the reference signal obtained for the previous screen is stored in advance in one memory 10. Here, the reference signal and the correlated signal are read out from each memory, and converted into a binary signal by changing the threshold value from 1 to 1 in the binarization circuit 11, and temporarily stored in the buffer memory 12. The reference signal and correlated signal read from each buffer memory are input to the digital correlator 13 to calculate the correlation coefficient, and the obtained correlation coefficient signal is passed through the maximum value detection circuit 14 to find the coordinates giving the maximum value. .

In this way, (2n-2) coordinate signals are input to the frequency counting circuit 15 to create a histogram. The coordinate signal that gives the maximum frequency for this histogram is determined using the maximum frequency detection circuit 16, and the address of the reference signal is specified in the memory 10, and the signal is returned to the region dividing circuit 7 to move the positions of the correlation region and the correlation window.

By performing the above operations independently in each horizontal and vertical direction, a desired target or scene within the field of view of the television camera can be tracked.

Although the above description has been made of a cross-shaped case in which correlation regions are set in the horizontal and vertical directions and correlation calculations are performed independently for real-time processing, the present invention is not limited to this. It can also be used when using

As described above, in the correlation tracking device according to the present invention, the threshold value is automatically set from 1 to (2n-2) for the one-dimensional n-bit reference and correlated luminance signals obtained in each horizontal and vertical direction. Correlation calculation is performed using the binarized luminance signal obtained by increasing the threshold value by 1, and (2n-2
), a histogram representing the frequency distribution of the coordinates is created, and the coordinate signal giving the maximum frequency of occurrence is obtained as the coordinate signal for tracking, so changes in brightness and contrast can be easily detected. Not only can the system easily track the target signal, but it also has the advantage of increasing tracking accuracy because the tracked target signal is obtained through total processing.

[Brief explanation of drawings]

FIG. 1 is a diagram showing the arrangement of correlation regions and correlation windows when a cross-shaped correlation region is used, FIG. 2 is a diagram of the operating principle of the correlation tracking device according to the present invention, and FIG. The histogram shown in FIG. 4 is a block diagram of an embodiment of the correlation tracking device according to the present invention.

Claims (1)

[Claims] 1 A television image supplied from a television camera is subdivided into pixels in the horizontal scanning direction and vertical scanning direction, and the luminance signal of each pixel is converted from analog to digital using n bits. A digital image in which a region including a desired target image or scene within the field of view of the television camera is stored in advance as a reference signal, and a new correlated signal is sequentially supplied from the television camera and digitized. Correlation is calculated with the digital image, and the coordinates on the orthogonal coordinate system set in the horizontal scanning direction and vertical scanning direction of the above image that maximize the correlation coefficient are calculated as tracking coordinates, and the tracking coordinates and the predetermined coordinates are calculated. By detecting an error signal with respect to the coordinates and controlling the attitude of the television camera so that the error signal becomes zero, tracking is performed so that a desired target image or scene is stopped at a predetermined position within the field of view of the television camera. In the correlation tracking device, the correlation calculating section and the tracking coordinate calculating section are connected to an area dividing circuit that sets rectangular areas in each of the horizontal scanning direction and the vertical scanning direction of the digital image, and a region dividing circuit that divides the luminance signals of pixels in each area horizontally. In the rectangular area in the scanning direction, in the vertical scanning direction,
In addition, in the rectangular area in the vertical scanning direction, after adding each in the horizontal scanning direction, an averaging circuit that averages the number of pixels added in each vertical scanning direction and horizontal scanning direction and converts it into a one-dimensional n-bit luminance signal is installed in advance. A memory that stores a one-dimensional n-bit luminance signal of a specific area including a desired target image or scene as a reference signal, which is determined for the image supplied from the television camera at a different time from the time stored in the memory. A buffer memory that stores a one-dimensional n-bit luminance signal as a correlated signal, and a threshold value is sequentially increased by 1 from 1 for the one-dimensional n-bit reference signal and correlated signal read from the memory and the buffer memory. (
A binarization circuit that repeatedly binarizes 2n-2) times and converts it into a one-dimensional 1-bit luminance signal, and a one-dimensional 1-bit luminance signal for each of the above thresholds.
A digital correlator that calculates a cross-correlation coefficient between a reference signal converted as a bit luminance signal and a correlated signal, and a digital correlator that detects the maximum value of the cross-correlation coefficient output from the digital correlator and calculates the correlation coefficient of the correlated signal. A maximum value detection circuit detects the coordinates in the horizontal scanning direction and the vertical scanning direction where an area that is the same as the reference signal or has a high similarity exists in the image at the time when the reference signal is stored, as described above (2n-2). A frequency counting circuit detects coordinates for each threshold value and calculates their occurrence frequency distribution, and a maximum frequency detection circuit calculates the coordinate giving the maximum occurrence frequency of the occurrence frequency distribution as a tracking coordinate. Correlation tracking device.