JPH03216079A - Picture movement correction device - Google Patents

Abstract

PURPOSE:To prevent response of a moving correction means with respect to the movement by panning of a camera by using vector information of high frequency component in a changing frequency in time axis direction among sets of moving vector information as a moving correction signal. CONSTITUTION:A moving vector defection circuit 101 applies correlation calculation of inter-frame or inter-field picture elements to obtain moving vector information and a moving quantity arithmetic circuit 102 obtains the moving vector information representing the moving direction and quantity of the entire pattern in one frame period or one field period. Then a frequency separator circuit 103 extracts only a high frequency component with respect to a time axis direction among plural sets of moving vector information and uses it as a moving correction signal, which is fed to an address generating circuit 104. Thus, the moving vector by camera panning and the moving vector due to rocking are identified and only the information due to the movement of camera rocking is obtained.

Description

[Detailed Description of the Invention] [Purpose of the Invention (Industrial Application Field) This invention detects unnecessary movement of an image on a television screen due to parallel movement or vibration of an imaging camera, and corrects the movement. The present invention relates to an image motion correction device.

(Prior Art) Movement of an image on a television screen can be caused by movement of an object in the image or by parallel movement of a camera. The former is a local movement of the image, whereas the latter appears as a parallel movement phenomenon in which the entire image moves while maintaining its mutual relationship. Such parallel movement of the entire image can be further divided into fine movements caused by camera panning and camera vibration, and these fine movements result in an unsightly image. In order to improve such unnecessary motion, a correction device has been developed that detects a motion vector indicating the direction and magnitude of the motion and corrects the motion of the entire image.

FIG. 6 shows the principle of the correction device. If the image on the screen moves in parallel in the direction of the arrow as shown in (a) to (b) to (C) in the same figure, the motion vector (arrow) will move in the horizontal and vertical directions as shown in Figure 7. It is given by the deviation in direction (a, b). Therefore, by detecting this motion vector and using that information to shift the image of the current field in the opposite direction by the magnitude of the motion vector, it is possible to compensate for the motion caused by the parallel movement of the camera.

Incidentally, in order to perform the above compensation, image movement must be detected. In the conventional image motion detection method, a representative point is set in the video signal of one frame before (two fields before), and a correlation calculation is performed with the current video signal to obtain motion vector information.

That is, as shown in FIG. 8, for example, if field 3 is the video signal of the current field, the pixel at the representative point P in the area Al of the video signal two fields before this, and the area A3 of the third field. Perform calculations with multiple pixels of
A pixel having the same content as the pixel at the representative point P is detected, and the direction of the detected pixel is determined as the direction of movement. Then, a plurality of such regions are set, and the motion vector information obtained in each region is classified into vector information indicating the same direction.
The final motion vector method is obtained by using the direction and magnitude indicated by the most collected vector information as the direction and amount of parallel movement of the entire screen.

(Problems to be Solved by the Invention) The conventional motion correction device described above tends to correct all movements of the camera. In other words, it attempts to correct even the camera's banning. Since the motion vectors caused by banning are continuous in the same direction, if you correct the image movement using this, the correction limit will be reached and after that,
Correction becomes impossible. In other words, conventional devices always give the corrected image a vector in the direction of returning it to the center in order to maintain the shake correction operation, but if the panning speed increases and exceeds the amount of vector to return to the center, the correction The limit is reached and shaking correction becomes impossible.

SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an image motion correction device that can distinguish between motion vectors caused by panning of a camera and motion vectors caused by actual shaking, and can obtain information only about movements caused by camera shaking. .

[Structure of the Invention] (Means for Solving the Problems) The present invention provides a movement amount detection means for obtaining vertical and horizontal motion vector information between fields or frames of an image signal, and a movement amount detection means that obtains motion vector information in the vertical and horizontal directions between fields or frames of an image signal. Using a plurality of pieces of motion vector information in the time axis direction, separate the components with low and high change frequencies in the time axis direction from among the plurality of motion vector information, and calculate the motion vector information of the high frequency component. and vector variation identification means for outputting as a correction signal.

(Function) The above means prevents the motion correction means from reacting to movements caused by panning of the camera.

(Example) Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an embodiment of the present invention. For example, an interlaced digital video signal is supplied to the input terminal 10, and is supplied to a motion vector detection circuit 101 and a delay circuit 105. The motion vector detection circuit 101 obtains motion vector information by performing correlation calculations between pixels between frames or between fields. The detection principle of this motion vector method is the same as that explained in FIG.

Motion vector information in each area from the motion vector detection circuit 101 is supplied to the movement amount calculation circuit 102.

The movement amount calculation circuit 102 obtains motion vector information indicating the direction and amount of movement of the entire screen in one frame period or one field period. Further, the output of the movement amount calculation circuit 102 is input to a frequency separation circuit 103 which performs calculations to be described later and performs frequency separation using a plurality of pieces of motion vector information in the time axis direction (field direction). This frequency separation circuit 103 extracts only components with high frequencies in the time axis direction from among a plurality of pieces of motion vector information, and supplies this to the address generation circuit 104 as a motion correction signal.

The address generation circuit 104 is a circuit that controls the image memory 106 into which the output video signal of the delay circuit 105 is written and read. The delay circuit 105 is for obtaining time adjustment between the motion vector processing path and the video signal path. For example, one field is written into the image memory 106 in synchronization with the video signal on the input side. but,
Reading from the image memory 106 is performed by the address generation circuit 104.
The read address value is corrected by the motion correction signal and output, and as a result, the image position for one field is corrected and read out.

Furthermore, here, the operation of enlarging and reading out the image is also performed at the same time. That is, as a result of motion correction, if the entire image is shifted, a missing portion will occur at the edge of the screen, so the missing portion will be enlarged so that it is outside the effective screen area. Determine the correction limit in advance, that is, the maximum shift amount of the entire image,
If you set the image to be enlarged accordingly, there will be no missing parts of the screen. The video signal corrected in this way is input to the image interpolation circuit 107.

In the second example, an intermediate pixel is created from each pixel of the image read out from the image scale 106.

The address data output from the address generation circuit 104 has information that is narrower than the spacing between each pixel in order to control motion compensation and image enlargement, so this information is used to It is necessary to create new pixel data corresponding to any position in between, and the image interpolation circuit is responsible for this task.

The video signal obtained in this way is led out to the output terminal 108.

This embodiment is constructed as described above, and is particularly characterized in that a high frequency component is extracted from the motion vector information and is supplied to the address generation circuit 104 as a motion correction signal.

When the camera motion vector information is analyzed, it becomes as shown in FIG. In other words, when the actual movement has a change as shown in the figure (A), if this movement component is captured as a frequency change, the lower component (panning component) as shown in the figure (B) and the lower component (panning component) as shown in the figure (B). It can be classified into high components (sway components) as shown in C). Therefore, in this embodiment, only high-frequency components as shown in FIG. 2(C) are extracted by the frequency separation circuit 103 and used as a motion correction signal.

The frequency separation circuit 103 includes, for example, the following FI
R (fin1te impulses responses
e) Can be realized by a circuit that performs filter calculations. Here, IHz was selected as the threshold for separating banging and shaking. In other words, motion vector components below IHz are considered to be panning, and anything above that is judged to be shaking. Note that this value is not limited, and may be selected in various ways depending on the usage environment. The number of taps of the FIR filter is selected to be, for example, 57 taps based on the sampling period and separation accuracy.

Vv = (V-28 x K-28) +
(V-27 X K-27 )+ (V-28
XK-26 ) 10...10 (VO XKO) +
(VI XKI) + (V2 XK2
) +...10 (V27XK27) +
(V28XK28) vv: Motion vector due to shaking (
(The panning component is removed from the actual movement) n: Detected motion vector of the nth field when the corrected image is set to 0 field Kn: FIR filter coefficient By performing such calculations, the above 0th field image A motion compensation signal can be obtained.

According to the above formula, motion vector information in the time axis direction for 57 fields is required, and the motion correction signal is transmitted to the terminal 10.
This corresponds to an image signal 28 fields before the image signal inputted to the image signal. Therefore, the delay circuit 105 needs to have a delay time of 28 fields. Currently, there is a 1Mbit memory as a field memory in practical use, so the delay circuit 106 can be realized by using at least 56 of these.

In this embodiment, an FIR filter with 57 taps is used, but in order to reduce the number of memories in the delay circuit 105, an FIR filter with half the taps can be put to practical use.

When a motion compensation signal is obtained after such processing, only motion vector information with high frequency components is extracted, as shown in Figure 2, and no compensation is made for panning, but only for shaking. You will be able to get it.

FIG. 3 shows the motion vector detection circuit 101 shown in FIG.
A specific example of the movement amount calculation circuit 102 is shown.

The interlaced video signal is supplied to a vertical interpolation circuit 11 . The vertical interpolation circuit 11 performs interpolation using pixels within a field, creates interpolation pixels between vertical lines, and derives a supplementary video signal thereof. This compensated video signal is supplied to the latch circuit 12 and the correlator 17.

The latch circuit 12 latches the image information of the representative point of each area into which one field of the supplementary video signal is divided into a plurality of areas at the timing of the latch pulse Tl. The image information of this representative point is transferred to the representative point storage field memory 13 by the transfer pulse T2, and is stored at a predetermined address for the representative point.

The representative point storage field memory 13 is controlled by the write/read mode switching signal W/H, and its address input is provided by the address controller 14 in the write mode.
A write address generated from the address controller 14 and a read address generated from the address controller 14 in the read mode are respectively supplied via the address switching circuit 15. The image information of the representative point in the representative point storage field memory 13 is latched by a latch circuit 16 and supplied to a correlator 17.

The correlator 17 performs a correlation calculation between the compensated video signal from the vertical interpolation circuit 11 and the image information of the representative point latched in the latch circuit 16. Here, the latch circuit 16 stores the image information of each representative point from the representative point storage field memo 913 in correspondence with each representative point extraction area of the compensated video signal of the current field.
3.

The correlator 17 performs a correlation calculation between the image information of the representative point one field before stored in the latch circuit 16 and each pixel in the area of the current field corresponding to the area of this representative point image information. The cumulative adder 18 cumulatively adds the input correlation calculation results for each representative point. This cumulative addition result is input to one motion vector generation circuit. The motion vector generation circuit 19 generates, among the addition results of motion vectors indicating each direction,
The output unit 20 determines the addition result that is the largest, that is, the one with the largest number of motion vectors, and outputs final motion vector information that makes the direction of the motion vector the parallel movement direction of the entire screen.
Output to. This motion vector information is further input to the frequency separation circuit 103 described above.

Even if you simply try to perform a correlation calculation between the odd field line signal and the even field line signal of an interlaced video signal, the correlation calculation result will be inaccurate because there is no line that corresponds to the vertical position. Although motion vector information is shown, by creating an interpolation signal for each field and performing a correlation calculation as in this example, a motion vector for each field can be obtained.

Next, a method for obtaining final motion vector information for the entire screen based on the results of the correlation calculation will be explained in principle.

FIG. 4 is an example in which one field of video signal is divided into M in the horizontal direction and N in the vertical direction on the memory space, and divided into (M.XN) areas. Here, a pixel (x mark) serving as a reference point is selected for each region and is used as a representative point. By comparing the signal levels to detect in which direction and by how much the pixel at the representative point has moved for each field, a motion vector can be obtained for each region. Next, among the motion vectors detected for each region, the motion vector with the largest number is determined as the motion vector for the entire television screen.

? The principle of detecting a motion vector in one area will now be explained.

As shown in FIG. 5(a), it is assumed that one region has (mXn) pixels. Luminance signal level 1 of a pixel (p) at a representative point in a certain field. It is assumed that the following information is stored in the memory 13.

Then, the luminance signal level of all pixels in the next field is 1,
12, 1,...1■. Suppose it was. The correlation calculation is 1o-1 10 12, 1. -1. , 1 o -... 1 mfi. Here, as shown in FIG. 5(b), assume that the levels of both signals to be calculated are equal and the pixels for which the calculation result is zero are the three pixels (alsa2, a3) in the figure. Then, in this area, the representative point (p) and each pixel (
The vectors connecting the vectors (al, a2, a3) are determined as motion vectors bl, b2, b3, respectively.

In this way, a plurality of motion vectors are detected in one area, and a motion vector related to parallel movement of the entire screen cannot be uniquely determined. Therefore, in this embodiment, the number of motion vectors in the same direction and the same size is added up through each area, and the direction and size of the motion vector that forms the largest sum of the addition results is used to move the entire screen. It is determined as the direction and amount of movement. Note that it is not necessary to divide the entire screen into areas for detecting motion vectors, and the scale of calculation may be reduced by selecting an area to be used for motion detection in advance.

[Effects of the Invention] As described above, according to the present invention, it is possible to distinguish between a motion vector due to camera banning and a motion vector due to actual shaking, and to obtain information only about the movement due to camera shaking. This can contribute to obtaining reliable motion compensation.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention, FIG. 2 is an explanatory diagram showing an example of camera movement characteristics to explain the operation of the device of this invention, and FIG. 3 is a block diagram of the block diagram in FIG. 1. Figures 4 and 5 are circuit diagrams showing a part of the circuit in more detail. Figures 4 and 5 are explanatory diagrams for explaining the region where correlation calculations are performed and the process of determining motion vectors. Figure 6 is a diagram showing the parallel movement of the entire image captured by the camera. FIG. 7 is an explanatory diagram showing vectors of parallel movement of the entire image, and FIG. 8 is an explanatory diagram shown to explain a conventional motion detection method using correlation calculation. 101...Motion vector detection circuit, 102...Movement amount calculation circuit, 103...Frequency separation circuit, 104...
・Address generation circuit, 105...Delay circuit, 106・
...Image memory, 107...Image interpolation circuit.

Claims (1)

[Claims] A movement amount detection means for obtaining vertical and horizontal motion vector information between fields or frames of an image signal, and a plurality of movement vector information in the time axis direction obtained by the movement amount detection means. Among the plurality of motion vector information, a component having a low change frequency in the time axis direction,
1. An image motion correction device comprising: vector variation identification means for separating high-frequency components from high-frequency components and outputting vector information of the high-frequency components as a motion correction signal.