JPS6048952B2 - correlation tracking device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION This invention provides a method for detecting the line of sight of a scene or target within the field of view of an image captured using a television camera.
ofsight: Hereinafter referred to as L, O, and S.

This article relates to a device for spatially stabilizing the air. Conventionally, a device for maintaining a desired scene or object within the field of view of a television image in a predetermined position on the screen using a correlation method has the configuration shown in FIG.

In FIG. 1, a television camera 1 is mounted on a gimbal mount 4 consisting of pitch and yaw direction control servo motors 2 and 3 and their mechanical parts. A video signal obtained by this television camera 1 is passed through an area dividing circuit 5 and divided into two areas in the horizontal and vertical directions along the scanning line of the television image.

A video signal corresponding to each area is digitized using an A/D conversion circuit 6, and is written to a memory 8 in each horizontal and vertical direction using a switch 7, as a reference signal for performing a correlation calculation with the next screen. .

Next, the captured screen is digitized using the A/D conversion circuit 6 in the same manner as above, and this video signal and the reference signal are input to the correlation circuits 9 and 10 in each horizontal and vertical direction to calculate the correlation coefficient. and detecting the maximum value of the correlation coefficient.

In addition, the coordinate signals on the screen that give this maximum value are output as control signals in the pitch and yaw directions, respectively, and the servo motors 2 and 3 are driven to control the L and O of the desired scene due to the change in the field of view of the television camera 1. , S or by correcting the angular deviation of L, O, S of the target due to movement of the target, TV monitor 1
The scene or target is held at a predetermined position on the screen when displayed on the screen. At this time, the contents of the memory 8 are updated by using the video signal giving the maximum value of the correlation coefficient for each screen as a reference signal for performing correlation calculation with the next screen.
FIG. 2 shows a display screen on a television monitor for explaining the functions of the conventional device.

Here, to simplify the explanation, we will take a stationary rectangular target as an example. Figure 2a shows the screen when the correlation areas 13 and 14 in the horizontal and vertical directions are set with the target 12 as the center, and the correlation areas are made to stand still at these coordinates XO, YO, that is, the target 12 is made to stand still. do.

Now, if the field of view of the television camera is moved without operating the servo system, the correlation areas 13 and 14 in the horizontal and vertical directions will move to the coordinates Xn and Yn, as shown in Figure 2b. When the servo system is operated so that the angular deviations corresponding to -XOl and 1Yn -YOl become zero, the screen shown in FIG. 2a is obtained. However, when this field of view moves quickly, due to the time constant of the response of the servo system, a tracking delay of 1Xc-XOl and 1Yc-YOl occurs as shown in Figure 2c, and the target is stably held at a predetermined position on the screen. There was a drawback that made it difficult.

In order to eliminate such drawbacks, the present invention detects the angular deviation corresponding to the tracking delay caused by the servo system, and based on this angular deviation signal, determines an appropriate delay time for the horizontal and vertical synchronization signals of the television image. is applied and superimposed on the video signal to generate a composite video signal, thereby maintaining a desired scene or target at a predetermined position on the screen, which will be described in detail with reference to the drawings below.

FIG. 3 shows the operation of the correlation tracking device according to the present invention. A display screen on a television monitor is shown to explain the principle.

Here, to simplify the explanation, we will use a rectangular object as an example.

In FIG. 3a, the stationary coordinates on the screen of the vertical and horizontal correlation areas 13 and 14 surrounding the target 12 are XO, Y
0, and when the target 12 moves, the angular deviation is corrected using a servo system to keep the target stationary on the screen.

In this correlation tracking device, when the target 12 moves quickly, a tracking delay occurs due to the response time constant of the servo system l, and the vertical and horizontal correlation areas 13 and 14 are displayed on the screen at the coordinate
Suppose it remains at c, Yc. At this time, the tracking delay from the stationary coordinates Yc-YO and Xc in the vertical and horizontal directions
-XO is measured, and the delay time of the vertical synchronizing signal and horizontal synchronizing signal is controlled according to each measured value and superimposed on the video signal of the next screen to synthesize a composite video signal. When this is displayed on a TV monitor, as shown in Figure 3b, the vertical, horizontal and horizontal correlation areas 13 and 14 are displayed in the static coordinates XO on the screen.
, YO. In this case, since the synchronization signal is delayed, the vertical blanking area 15 and the horizontal blanking area 16 are displayed as shown in FIG. 3b. FIG. 4 shows a video signal waveform for explaining the above operating principle.

In FIG. 4, the waveform a is a composite video signal supplied from a television camera, the waveform b is a video signal after synchronization separation and separation, and the waveforms c and d are vertical and horizontal synchronization signals after synchronization separation. The waveforms of e and f are the vertical and horizontal synchronization signals when delayed according to the tracking delay, and the waveform of g is the waveform of B,
This is a composite video signal obtained by combining the waveforms of e and f.

Here, the vertical and horizontal delay times Tv and Th are set in proportion to the tracking delays Yc-YO and Xc-XO from the stationary coordinates, and Yc-YO<0 and Xc-XO<o
In this case, it is necessary to set Tv<0 and Th<0. Therefore, in the correlation tracking device according to the present invention, the above conditions are realized by using synchronization signals separated from the composite video signal of one frame before as the vertical and horizontal synchronization signals. FIG. 5 is a block diagram of an embodiment of a correlation tracking device according to the present invention.

In FIG. 5, a composite video signal supplied from a television camera 1 mounted on a gimbal mount 4 is passed through a region dividing circuit 5 and divided into vertical and horizontal correlated regions.

The video signal of each area is digitized using an A/D conversion circuit 6, and a correlation calculation with the next screen is performed using a switch 7.
The signal is recorded in the memory 8 as a reference signal for the operation. now,
As shown in FIG. 3, the area represented by the coordinates XO and YO on the screen is set as the initial value of the reference signal, lock-on (obtainment of the target) is performed, and the tracking operation is started. For the next screen, the A/D-converted video signal for each vertical and horizontal correlation area and the initial value of the reference signal read from the memory 8 are input to vertical and horizontal correlation circuits 9 and 10, respectively. Calculate the correlation coefficient and find the coordinate X that maximizes the correlation coefficient.
c and Yc are detected.

The video signal in the vertical and horizontal direction correlation area corresponding to the coordinates Xc and Yc at this time is written into the memory 8 using the switch 7 as a reference signal for the next screen, and the memory is sequentially updated. However, in each correlation circuit, vertical tracking delay Yc-YO and horizontal tracking delay Xc
-XO is measured and this value is used to drive the servo motors 2 and 3 for pitch direction and yaw direction control.

If the response of the above servo system is fast enough, XCJ-XO
= 0, Yc - YO = 0, but since the response time constant of the servo system is usually large, the device equipped with the correlation tracking device may move within the field of view of the television camera 1 due to high-speed movements such as vibrations generated by the device itself. The image of a certain scene or target vibrates, and it is inevitable that
c-XO half 0 and Yc-YO=O.

When a person views the screen displayed on the television monitor 11 under such conditions, there are problems in that the image becomes blurred and the display marks representing the correlation areas vibrate, making it difficult to see. Therefore, the composite video signal supplied from the television camera 1. The video signal is separated into a video signal and a synchronization signal through a synchronization separation circuit 17. The video signal is converted into a digital signal by an n-bit A/D conversion circuit 18 and stored in an image memory 19. The video signal of the next frame is similarly stored in the image memory B2O. At this time, the timing for reading the video signal stored in the image memory Al9 is set in proportion to the measured values of the tracking delays Yc-YO and Xc-XO, and the read video signal is converted into an n-bit D/A converter. The signal is input to a circuit 21 and converted into an analog signal. This video signal and the synchronization signal separated as described above are input to a composite video signal synthesis circuit 22 to generate a composite video signal. At this time, in order to store the video signal of the next frame, the video signal stored in the image memory Al9 is deleted. Such operations are alternately executed in the image memory Al9 and the image memory B2O. When this composite video signal is input to the television monitor 11 and displayed, the target and the vertical video signals set to surround the target are displayed. The horizontal correlation area can be frozen at a predetermined position on the screen. Although the above description has been made regarding the case where correlation regions are set in the vertical and horizontal directions and correlation calculations are performed in parallel to perform real-time processing, the present invention is not limited to this, and is applicable to the case where one rectangular correlation region is used. can also be used.

Furthermore, although the explanation has been given above using the simplest rectangular target as an example, the present invention may also be used for targets having complex shapes and targets in complex backgrounds.

Further, in the above description, the case where the target moves is taken as an example, but it goes without saying that the present invention can be similarly applied to the case where the field of view of the television camera changes with respect to a fixed target.

As described above, the correlation tracking device according to the present invention is configured to compensate for the tracking delay caused by the time constant of the response of the servo system by providing an electronic feedback loop for the synchronization signal that constitutes the composite video signal. Even when the target moves or the field of view changes rapidly, the target or scene can be stably kept still on the screen.

Furthermore, in the correlation tracking device according to the present invention, when the movement of the target or the variation in the field of view is small, the target or scene can be kept stationary on the screen using only the electronic feedback loop without activating the feedback loop of the servo system. It has the advantage of being able to

[Brief explanation of drawings]

FIG. 1 is a block diagram of a conventional correlation tracking device, FIG. 2 is a display screen on a television monitor for explaining the functions of the conventional device, and FIGS. 3 and 4 are diagrams of a correlation tracking device according to the present invention. A diagram showing a display screen on a television monitor and a video signal waveform diagram for explaining the principle of operation, and FIG. 5 are block diagrams of one embodiment of the correlation tracking device according to the present invention. In the figure, 1 is a television camera, 2 is a servo motor for pitch direction control, 3 is a servo motor for yaw direction control, 4 is a gimbal mount, 5 is an area division circuit, 6 is an A/D conversion circuit,
7 is a switch, 8 is a memo? 9 is a vertical correlation circuit;
10 is a horizontal correlation circuit, 11 is a telegraph monitor, 12 is a target, 13 is a vertical correlation area, 14 is a horizontal correlation area, 15 is a vertical blanking area, 16 is a horizontal blanking area, and 17 is a synchronization separation circuit. , 18 is an n-bit A/F/D conversion circuit, 19 is an image memory A) 20 is an image memory B) 21
2 is an n-bit D/A conversion circuit, and 22 is a composite video signal synthesis circuit.

Claims (1)

[Claims] 1 Find the cross-correlation coefficient between the screen obtained by analog-to-digital conversion of the video signal from the composite video signal supplied from the television camera mounted on the rotating table and the reference image stored in advance, and find the maximum correlation point. In a correlation tracking device that detects and tracks a desired scene or target within the field of view of the television camera, the movement position of the image of the scene or target on the screen due to the movement of the television camera or target is determined in advance. means for measuring the amount of deviation from the set position; means for separating the video signal and synchronization signal from the composite video signal supplied from the television camera; and two means for alternately storing the video signal for each frame. an image memory; means for adjusting the timing of reading out the video signal stored in the image memory based on the amount of deviation; and a composite video signal by superimposing the video signal read out from the image memory and the synchronization signal. A correlation tracking device comprising a means for generating a correlation.