JP3225598B2 - Image shake detection device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a device for detecting a camera shake of an image, and more particularly, to a device for detecting and correcting an image movement caused by a so-called camera shake contained in video data such as an image pickup output of a handy type video camera. Applied to correction devices and the like.

[0002]

2. Description of the Related Art Generally, in a handy type video camera, camera shake during photographing, that is, camera vibration appears as image vibration. Therefore, as an image stabilization device for correcting an image vibration caused by such a camera shake, for example, as disclosed in Japanese Patent Application Laid-Open No. 63-166370, a motion vector of an image is detected and based on this motion vector. A technique for correcting video data stored in an image memory has been proposed.

For detecting a motion vector of an image, for example, a block matching method is employed. In the detection of a motion vector of an image by the block matching method, a screen is divided into a number of regions (referred to as blocks), and a representative point pixel of a previous field located at the center of each block and a pixel of each pixel in a block of the current field are determined. The absolute value of the field difference from the image data is calculated, the absolute value of the field difference of each block is integrated for each corresponding pixel to obtain a correlation integral value, and the correlation integration having coordinates corresponding to the pixel array of one block Form a value table. Then, the motion vector of the entire screen is determined using the coordinate value of the minimum value of the correlation integral value in the correlation integrated value table as the coordinate value of the motion vector of the image.

In general, in the above-described block matching method, a motion vector of an image due to camera shake and a motion vector of a motion of a subject occur simultaneously.
It is difficult to determine these. 2. Description of the Related Art Conventionally, as a method for improving the accuracy of detecting a motion vector of an image due to camera shake, one screen is divided into a plurality of regions (macroblocks), and motion vectors (macrovectors) for each macroblock are obtained. Techniques for determining vectors are known. For example, as shown in FIG.
The influence of the motion vector due to the moving object MV is removed by setting (2, 0) as the motion vector of the entire screen by majority decision on the macro vector obtained by dividing the macro vector into macro blocks.

[0005]

By the way, a method of detecting a motion vector by obtaining a minimum value of a correlation integrated value table in units of macroblocks and detecting a motion vector of the entire screen from the motion vector of each macroblock is described. ,
If there is a moving object, the background image including the information on the camera shake vector is masked by the moving object, so that as the ratio of the moving object in the image increases, the camera shake vector is detected as the motion vector in macroblock units. However, there is a problem that the detection ratio of the camera shake vector decreases and it becomes difficult to detect a correct camera shake vector.

Accordingly, the present applicant divides one screen into a plurality of macroblocks in such a manner that macroblocks that are not adjacent to each other are generated, and obtains a motion vector by obtaining a minimum value of a correlation integrated value table in macroblock units. As shown in FIG. 8, a method for shaking the same motion vector detected in at least two macroblocks located at spatially separated positions is described in, for example, Japanese Patent Application No. 3-100384. Has proposed.

However, even in this method, as shown in FIG. 9, when the proportion of a moving object in an image is extremely large, one of the macroblocks at spatially separated positions is not a background vector but a background vector. In some cases, a motion vector due to a moving object is detected, and a correct camera shake vector cannot be detected.

Accordingly, the present invention has been made in view of such circumstances, and has as its object to enable high-performance camera shake correction in a handy-type video camera or the like. An object of the present invention is to provide an image shake detection device capable of detecting a shake vector with high accuracy even in such a case.

[0009]

SUMMARY OF THE INVENTION In order to solve the above-mentioned problems, an image blur detection apparatus according to the present invention divides one screen into a plurality of macroblocks in such a manner that macroblocks not adjacent to each other are generated. The same motion is detected in the first detecting means for detecting a motion vector for each macroblock and at least two macroblocks at positions spatially separated in the output of the first detecting means. A second detection means for detecting whether or not there is a vector, and when there is the same one, outputting the motion vector as a motion vector of the entire screen. Outputting the minimum value coordinate of the integrated value table for each block as a camera shake vector candidate and a detected motion vector of the entire screen as a camera shake vector. A.

[0010]

In the image blur detecting apparatus according to the present invention, the first
The screen is divided into a plurality of macroblocks in such a manner that macroblocks that are not adjacent to each other are generated by the detecting means, and a motion vector is detected for each of the macroblocks. In at least two of the macroblocks at positions spatially separated from the output of the detecting means, it is detected whether or not any of the macroblocks has the same motion vector. Is output as a motion vector of the entire screen. Then, the second detection means outputs the minimum value coordinate of the integrated value table for each macroblock as a camera shake vector candidate, and outputs the detected motion vector of the entire screen as a camera shake vector.

[0011]

BRIEF DESCRIPTION OF THE DRAWINGS FIG. 1 is a block diagram showing an embodiment of an image blur detecting apparatus according to the present invention. An image shake detection apparatus according to the present invention is configured, for example, as shown in FIG.

The camera shake detecting device 10 shown in FIG.
The present invention is applied to a camera shake correction device that corrects image movement due to camera shake in a handy type video camera, and forms a camera shake correction device together with the correction signal generation unit 20 and the correction unit 30. In FIG. 1, a signal input terminal 1 is supplied with input video data obtained by digitizing a video signal obtained as an imaging output by an imaging unit (not shown) of the video camera.

The camera shake detecting device 10 includes a field difference detecting section 11 to which input video data is supplied via the signal input terminal 1, and a correlation integrated value table forming section to which an output of the field difference detecting section 11 is supplied. 12, a macro vector detecting unit 15 to which the output of the correlation integrated value table forming unit 12 is supplied, an effective vector determining unit 16 to which the output of the macro vector detecting unit 15 is supplied, Motion vector determination unit 1 to which the output is supplied
7 is provided.

The field difference detector 11 comprises a representative point memory 11A to which the input video data is supplied via the signal input terminal 1, and a subtraction circuit 11B.

The representative point memory 11A stores image data I _k (0,0) of representative point pixels for each block obtained by dividing an image of one field composed of input video data into a plurality of blocks.
Is stored. Specifically, for example, as shown in FIG.
The screen of the field is divided into blocks of m pixels × n lines, and as shown in FIG.
0) as a representative point, and the image data I of each representative point pixel
_k (0, 0) is stored in the representative point memory 11A for one field period. Note that the representative points are uniformly distributed on the screen. Then, the image data I _k-1 (0,0) of each representative point pixel one field before read from the representative point memory 11A is supplied to the subtraction circuit 11B.

Further, the subtraction circuit 11B converts m ×××××××××××××××××××××××××××××××××××××××××× s1 B1 B block distances of input video data supplied through the signal input terminal 1, that is, image data of the current field.
The difference between the image data I _k (x, y) of each of the n pixels and the image data I _k-1 (0, 0) of the representative point pixel of the corresponding block of the previous field read from the representative point memory 11A, that is, Absolute value of the difference between the fields | I _k-1 (0,
0) -I _k (x, y) |.

The field difference detector 11
Supplies the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the subtraction circuit 11B to the correlation integrated value table forming unit 12.

The correlation integrated value table forming unit 12 is supplied with the field difference absolute value | I _k-1 (0,0) −I _k (x, y) | obtained by the field difference detection unit 11. a macro blocking circuit 13, a macroblock of the field difference absolute value by the macro blocking circuit _{13 | I k-1 (0,0} ) -I k (x, y) | first through is supplied It comprises 13 absolute value integration circuits 14A to 14M.

The macroblocking circuit 13 converts the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the field difference detector 11 into macros that are not adjacent to each other. In the form that blocks occur,
One screen is divided into a plurality of macro blocks. For example, as shown in FIG. 4, one screen is divided into 12 4 × 3 macro blocks B1 to B12.

The field difference absolute value | I macro-blocked by the macro-blocking circuit 13
_k-1 (0,0) -I _k (x, y) | is supplied to the first to twelfth absolute value integration circuits 14A to 14L corresponding to the macroblocks B1 to B12 for each macroblock. At the same time, all macroblocks are supplied to the thirteenth absolute value integration circuit 14M.

The first absolute value integration circuit 14A includes a first absolute value integration circuit 14A.
Field difference absolute value | I _k-1 of macro block B1
For (0,0) −I _k (x, y) |, the field difference absolute values of the block composed of the m × n pixels are integrated for each corresponding pixel, and the first macro block B1 Is formed. Hereinafter, similarly, the second
The twelfth to twelfth absolute value integration circuits 14B to 14L respectively correspond to the second to twelfth macroblocks B2 to B1.
2 is formed. Further, the thirteenth absolute value integration circuit 14M forms a correlation integrated value table for the entire one screen including the first to twelfth macroblocks B1 to B12.

The correlation integrated value table forming section 12
The correlation integrated value of the correlation integrated value table of each of the macroblocks B1 to B12 formed by the first to twelfth absolute value integration circuits 14A to 14L and the thirteenth absolute value integration circuit 14
The correlation integrated value of the correlation integrated value table for one entire screen formed by M is supplied to the macro vector detecting unit 15.

The macro vector detection section 15 is supplied with a first correlation integrated value of a correlation integrated value table of each of the macro blocks B1 to B12 formed by the first to twelfth absolute value integration circuits 14A to 14L. To a twelfth macro vector detection circuit 15A to 15L, and a vector detection circuit 15M to which a correlation integrated value of a correlation integrated value table for one entire screen formed by the thirteenth absolute value integration circuit 14M is supplied.

The first to twelfth macro vector detection circuits 15A to 15L are provided for each of the macro blocks B1 to B12.
As the motion vector (macro vector) of each image, the coordinates of the minimum value of the integrated correlation value in the integrated correlation value table of each of the macro blocks B1 to B12 are detected. Further, the vector detection circuit 15M detects the coordinates of the minimum value of the correlation integrated value in the correlation integrated value table as a motion vector of one entire screen.

It should be noted that the correlation integrated value of each correlation integrated value table formed by the correlation integrated value table forming section 12 is based on the image data I _k-1 (0,0) of the representative point pixel of each block and other pixels. This indicates the inter-field correlation with the image data I _k (x, y) of the above. The coordinates corresponding to the pixels having a higher correlation have smaller values, and the integrated correlation value of the coordinates corresponding to the motion vector has the minimum value. The macro vector detecting section 15 can detect a motion vector by detecting the coordinates of the minimum value in the correlation integrated value table.

Here, the minimum value of the correlation integrated value table is one of the minimum values, and when the proportion of the moving object in the image is large as described above, for example, as shown in FIG. , The rate at which the motion vector due to the moving object is detected as the minimum value coordinate (3, 1) in the correlation integrated value table and the background vector, that is, the camera shake vector, is detected as the coordinate indicating the motion vector in the macroblock unit is reduced. However, the camera shake vector is detected as the coordinate (-1, 0) of the minimum value in the correlation integrated value table.

The macro vector detecting unit 15
Are the first to twelfth macro vector detection circuits 15
The macro vectors detected by A to 15L and the motion vector of one entire screen detected by the vector detection circuit 15M are supplied to the effective vector determination unit 16.

The effective vector determination unit 16 determines, for example, the minimum value of the integrated value, that is, the minimum value and the integrated value of the macro vector for each macro block detected by the macro vector detecting unit 15 and the motion vector of the entire screen. And the difference between the minimum value and the surrounding integrated value (degree of dent)
The validity determination is performed using the threshold as a threshold.

Based on the output of the effective vector determination unit 16, the vector determination unit 17 determines whether at least two macroblocks at the spatially separated positions have the same motion vector. Whether the motion vector is the same or not is determined as the motion vector of the entire screen, that is, the camera shake vector. For example, in the state shown in FIG. 5, the same motion vector (-1, 0) is present in the two macroblocks B1, 4 at spatially separated positions. Is the camera shake vector.

That is, the camera shake detecting device 1 of this embodiment
In the case of 0, the field difference detection unit 11, the correlation integrated value table forming unit 12, and the macro vector detection unit 15 generate 1
A motion vector is detected for each macroblock obtained by dividing the screen into a plurality. Then, the effective vector determination unit 16 and the vector determination unit 17 use the minimum value coordinate of the integrated value table for each macroblock as a camera shake vector candidate, and at least two macroblocks located at spatially separated positions have the same value. Whether or not there is a motion vector is detected, and when there is the same motion vector, the motion vector is output as a motion vector of the entire screen, that is, a camera shake vector.

As described above, not only the minimum value vector of the integrated value of each macroblock but also the minimum value vector is set as a candidate for the shake vector, so that even when the ratio of the moving object occupying the screen is very large, the correct value can be obtained. The probability of detecting a camera shake vector is increased, and the influence of a moving object is reduced, so that a camera shake vector with less error can be obtained.

Further, the correction signal generating section 20, 'as _{input, X t = X t-1} -V t' by the hand-shake is detected by the detection device 10 is hand shake vector V _t image stabilization becomes the correction amount X _t A signal is formed, and this camera shake correction signal is supplied to the correction unit 30.

As shown in FIG. 6, for example, the correction section 30 includes an address control circuit 31 and a select signal generation circuit 32 to which a camera shake correction signal is supplied from the correction signal generation section 20, and an address control circuit 31. Field memory 33 and peripheral memory 34 for writing / reading video data in accordance with an address signal supplied from
And a selector 35 for selectively outputting video data read out from the selector according to the select signal supplied from the select signal generating circuit 32.

Input video data supplied via the signal input terminal 1 is sequentially written into the field memory 33. The read address of the field memory 33 is controlled by the camera shake correction signal according to the camera shake vector. As a result, video data obtained by moving one field of input video data according to the camera shake vector is obtained from the field memory 33. Then, the video data read from the field memory 33 and the peripheral video data read from the peripheral memory 34 are combined by selection by the selector 35, and are output from the signal output terminal 2 as video data subjected to camera shake correction processing.

In the peripheral memory 34, the video data of the peripheral portion corresponding to the image correction range of the video data subjected to the camera shake correction output through the selector 35 is sequentially written as peripheral video data.

[0036]

As is apparent from the above description, in the image shake detecting apparatus according to the present invention, one screen is divided into a plurality of screens by the first detecting means in such a manner that macroblocks not adjacent to each other are generated. The motion vector is divided into macroblocks, a motion vector is detected for each macroblock, and the minimum value coordinate of the integrated value table for each macroblock is set as a camera shake vector candidate by the second detection means, and the output of the first detection means is output. In at least two of the macroblocks located at positions spatially separated within the macroblock, it is detected whether or not there is one having the same motion vector. Since it is a vector, that is, a camera shake vector, it is possible to detect the correct camera shake vector even when the proportion of the moving object occupying the screen is very large. Increased, to reduce the effect of the moving object can be obtained with less hand movement vector of errors.

[Brief description of the drawings]

FIG. 1 is a block diagram illustrating a configuration of a camera shake correction apparatus provided with an image camera shake detection apparatus according to the present invention.

FIG. 2 is a diagram showing a state in which a screen is divided into blocks in the camera shake detection device.

FIG. 3 is a diagram showing a structure of one block of a screen divided into blocks.

FIG. 4 is a diagram showing a state of a macro block obtained by dividing one screen into 12 parts.

FIG. 5 is a diagram illustrating a state of a motion vector of each macroblock.

FIG. 6 is a block diagram illustrating a configuration of a correction unit of the camera shake correction device.

FIG. 7 is a diagram for explaining a method of determining a camera shake vector by majority decision of macro vectors.

FIG. 8 is a diagram provided for describing a method of determining a camera shake vector using macro vectors at spatially separated positions.

FIG. 9 is a diagram for explaining a failure example of a method of determining a camera shake vector using macro vectors at positions spatially separated.

[Explanation of symbols]

 Reference numeral 10: Camera shake detection device 11: Field difference detection unit 12: Correlation integrated value table creation unit 15: Macro vector detection unit 16: Effective vector Judgment unit 17 ... Vector determination unit

──────────────────────────────────────────────────続 き Continuation of the front page (72) Inventor Tsukasa Hashino 6-35, Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation (56) References JP-A-4-309078 (JP, A) (58) Field surveyed (Int. Cl. ⁷ , DB name) H04N 5/232 G03B 5/00

Claims (1)

(57) [Claims] 1. A first detecting means for dividing one screen into a plurality of macro blocks and detecting a motion vector for each of the macro blocks in such a manner that macro blocks which are not adjacent to each other are generated; In at least two of the macroblocks at spatially separated positions in the output of the means, it is detected whether or not any of the macroblocks has the same motion vector. 2nd output as motion vector of entire screen
Wherein the second detecting means outputs the minimum value coordinate of the integrated value table for each macroblock as a camera shake vector candidate, and outputs the detected motion vector of the entire screen as a camera shake vector. Image shake detection device.