JPH05328194A - Picture hand jiggling detector - Google Patents

Abstract

PURPOSE:To detect a hand jiggle vector with high accuracy through interpola tion even when a moving object is in existence in a picture. CONSTITUTION:A field difference detection section 11, a correlation integration list generating section 12 and a macro vector detection section 15 are used to detect a motion vector for each of macro blocks being plural number of divisions of one pattern and a motion vector detection section 16 detects motion vectors of the entire pattern. The motion vector detection section 16, a peripheral integration value storage section 17, a effective integration value list generating section 18 and an interpolation processing section 19 are used to interpolate the motion vectors of the entire pattern with integration data around the motion vectors of each valid macro block in which the motion vectors in the vicinity of the motion vectors for the entire pattern are obtained and the result is outputted as a hand jiggle vector.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image shake detecting apparatus, and for example, an image shake detecting and correcting a movement amount of an image due to so-called shake included in video data such as an image pickup output of a handy type video camera. It is applied to correction devices.

[0002]

2. Description of the Related Art Generally, in a handy type video camera, camera shake during shooting, that is, camera vibration, appears as image vibration. Therefore, as an image shake correction apparatus for an image which corrects the image shake due to such a hand shake, for example, as disclosed in Japanese Patent Laid-Open No. 63-166370, a motion vector of the image is detected and based on this motion vector. Then, a method of correcting video data stored in the image memory has been proposed.

A block matching method, for example, is employed to detect the motion vector of the image. In the detection of the motion vector of the image by this block matching method, the screen is divided into a number of areas (referred to as blocks), and the representative point pixel of the previous field located at the center of each block and each pixel in the block of the current field are divided. The absolute value of the field difference from the image data is calculated, the absolute value of the field difference of each block is integrated for each corresponding pixel to obtain the correlation integral value, and the correlation integration having the coordinates corresponding to the pixel array for one block is calculated. Form a value table. Then, the motion vector of the entire screen is determined using the coordinate value of the minimum value of the correlation integrated value in the correlation integrated value table as the coordinate value of the motion vector of the image.

Generally, in such a block matching method, a motion vector is detected with a pixel interval as one unit, and a value below the decimal point cannot be obtained.

By the way, when the camera shake is corrected, a vector obtained by adding motion vectors detected for each field or frame is used as the camera shake vector.
The error for the correct vector can be large.
Therefore, the value below the decimal point is obtained by interpolating the motion vector using the determined motion vector and the integrated value data around it.

In the block matching method as described above, the motion vector of the image due to the camera shake and the motion vector due to the motion of the object are generated at the same time, so that it is difficult to distinguish between them. Conventionally, as a method of improving the detection accuracy of a motion vector of an image due to camera shake, there is known a method of dividing a screen into a plurality of regions (macroblocks) and detecting the motion vector of the entire screen from the motion vector of each macroblock. ing. The applicant of the present application has, for example, a patent application for a method of dividing one screen into a plurality of macroblocks in such a manner that macroblocks that are not highly adjacent to each other are generated and detecting the motion vector of the entire screen from the motion vector of each macroblock. It has been previously proposed in No. 3-100384.

[0007]

By the way, the screen is divided into a plurality of macroblocks of one screen, the motion vector of the entire screen is determined from the motion vector of each macroblock, and the integrated value data around this motion vector is determined. If a moving object is present in the image, the integrated value data around the shake vector of the macroblock in which the moving object exists will include the motion vector component of the moving object. The vector detection accuracy cannot be sufficiently increased, and the difference between the interpolated motion vector and the actual camera shake vector becomes large.

Therefore, the present invention has been made in view of such circumstances, and an object thereof is to enable high-performance image stabilization in a handy type video camera or the like, and a moving object is present in an image. Even in such a case, the present invention provides an image shake detection apparatus capable of detecting a shake vector with high accuracy by interpolation.

[0009]

In order to solve the above-mentioned problems, an image shake detecting apparatus according to the present invention divides one screen into a plurality of macroblocks and detects a motion vector for each macroblock. The first detection means and the first
Second detection means for detecting the motion vector of the entire screen based on the output of the detection means of (1), and a macroblock for obtaining a motion vector in the vicinity of the motion vector of the entire screen detected by the second detection means. As a block, an interpolation means for interpolating the motion vector of the entire screen by the integrated value data around the motion vector of each effective macroblock is provided, and the motion vector interpolated by the interpolation means is output as a camera shake vector. To do.

[0010]

In the image shake detecting apparatus according to the present invention, the first
The detection means detects the motion vector for each macroblock obtained by dividing one screen into a plurality of parts, and the second detection means detects the motion vector for the entire screen based on the motion vector of each macroblock. Then, using a macroblock for which a motion vector near the motion vector of the entire screen is obtained as an effective macroblock, the interpolation means interpolates the motion vector of the entire screen with integrated value data around the motion vector of each effective macroblock, Output as a shake vector.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image shake detecting apparatus according to the present invention will be described in detail below with reference to the drawings. The image shake detecting apparatus according to the present invention is configured as shown in FIG. 1, for example.

The camera shake detection device 10 shown in FIG.
The present invention is applied to a camera shake correction apparatus that corrects image movement due to camera shake in a handy type video camera, and constitutes a camera shake correction apparatus together with a correction signal generation unit 20 and a correction unit 30. In FIG. 1, a signal input terminal 1 is supplied with input video data obtained by digitizing a video signal obtained as an image pickup output by an image pickup unit (not shown) of the video camera.

The camera-shake detecting apparatus 10 includes a field difference detecting section 11 to which input video data is supplied through the signal input terminal 1 and a correlation integrated value table forming section to which an output of the field difference detecting section 11 is supplied. 12, the macro vector detection unit 15 to which the output of the correlation integrated value table forming unit 12 is supplied, the motion vector determination unit 16 and the peripheral integrated value storage unit 17 to which the output of the macro vector detection unit 15 is supplied, The effective integrated value table creation unit 1 to which the outputs of the motion vector detection unit 16 and the peripheral integrated value storage unit 17 are supplied.
8 and an interpolation processing unit 19 to which the output of the effective integrated value table creating unit 18 is supplied.

The field difference detector 11 comprises a representative point memory 11A to which the input video data is supplied via the signal input terminal 1 and a subtraction circuit 11B.

The representative point memory 11A stores image data I _k (0,0) of representative point pixels in each block obtained by dividing an image of one field composed of input video data into a plurality of blocks.
Memorize Specifically, for example, as shown in FIG.
The field screen is divided into blocks of m pixels × n lines, and as shown in FIG. 3, the pixel S (0,
0) as a representative point and image data I of each representative point pixel
_k (0, 0) is stored in the representative point memory 11A for one field period. The representative points are evenly distributed on the screen. Then, the image data I _k-1 (0,0) of each representative point pixel one field before read from the representative point memory 11A is supplied to the subtraction circuit 11B.

Further, the subtraction circuit 11B, for the input video data supplied through the signal input terminal 1, that is, the image data of the current field, m × m for each block.
The difference between the image data I _k (x, y) of each of the n pixels and the image data I _k-1 (0, 0) of the representative point pixel of the corresponding block of the previous field read from the representative point memory 11A, that is, Absolute value of field difference | I _k-1 (0,
0) -I _k (x, y) | is detected.

Then, the field difference detection section 11
Supplies the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the subtraction circuit 11B to the correlation integrated value table forming unit 12.

The correlation accumulated value table forming unit 12 is supplied with the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the field difference detecting unit 11. a macro blocking circuit 13, a macroblock of the field difference absolute value by the macro blocking circuit _{13 | I k-1 (0,0} ) -I k (x, y) | first through is supplied 13 absolute value integrating circuits 14A to 14M.

The macroblocking circuit 13 is a macroblock which is not adjacent to the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the field difference detecting unit 11. In the form of blocks,
One screen is divided into a plurality of macroblocks. For example, as shown in FIG. 4, one screen is divided into 12 4 × 3 macroblocks B1 to B12.

The field difference absolute value | I macroblocked by the macroblocking circuit 13
_k−1 (0,0) −I _k (x, y) | is supplied to each of the first to twelfth absolute value integrating circuits 14A to 14L corresponding to each of the macro blocks B1 to B12 for each macro block. At the same time, all the macro blocks are supplied to the thirteenth absolute value integration circuit 14M.

The first absolute value integrating circuit 14A has a first
Field difference absolute value of macroblock B1 of | I _k-1
For (0,0) -I _k (x, y) |, the field difference absolute values of the block composed of the m × n pixels are integrated for each corresponding pixel, and the first macroblock B1 is added. Form a correlation integrated value table of. Similarly, the second
To 12th absolute value integrator circuits 14B to 14L respectively correspond to the 2nd to 12th macroblocks B2 to B1.
2. Form a correlation integrated value table of 2. The thirteenth absolute value integration circuit 14M forms a correlation integrated value table for the entire one screen including the first to twelfth macro blocks B1 to B12.

The correlation integrated value table forming unit 12 is
The correlation integrated value of the correlation integrated value table of each macro block B1 to B12 formed by the first to twelfth absolute value integration circuits 14A to 14L and the thirteenth absolute value integration circuit 14 described above.
The correlation integrated value of the correlation integrated value table of the entire screen formed by M is supplied to the macro vector detection unit 15.

The macro vector detection unit 15 is supplied with the correlation integrated value of the correlation integrated value table of each macro block B1 to B12 formed by the first to twelfth absolute value integration circuits 14A to 14L. It is composed of a twelfth macro vector detection circuit 15A to 15L and a vector detection circuit 15M to which the correlation integrated value of the correlation integrated value table of the entire one screen formed by the thirteenth absolute value integration circuit 14M is supplied.

The first to twelfth macro vector detection circuits 15A to 15L are arranged in the respective macro blocks B1 to B12.
As the motion vector (macro vector) of each image, the coordinates of the minimum value of the correlation integrated value in the correlation integrated value table of each macro block B1 to B12 are detected. Further, the vector detection circuit 15M detects the coordinates of the minimum value of the correlation integrated values in the correlation integrated value table as the motion vector of the entire screen.

Here, the correlation integrated value of each correlation integrated value table formed by the correlation integrated value table forming unit 12 is the same as the image data I _k-1 (0, 0) of the representative point pixel of each block. This indicates the inter-field correlation with the image data I _k (x, y) of a pixel, and the coordinate corresponding to the pixel having a strong correlation has a smaller value, and the correlation integrated value of the coordinates corresponding to the motion vector has the minimum value. The macro vector detection unit 15 is
The motion vector can be detected by detecting the coordinates of the minimum value in the correlation integrated value table.

Then, the macro vector detector 15
Is the first to twelfth macro vector detection circuit 15
The macro vectors detected by A to 15L and the motion vector of the entire one screen detected by the vector detection circuit 15M are supplied to the motion vector determination unit 16.
Furthermore, the correlation integrated value table forming unit 12 supplies the correlation integrated value data around each macro vector detected by the first to twelfth macro vector detection circuits 15A to 15L to the peripheral integrated value storage unit 17. To do.

The motion vector determination unit 16 comprehensively determines the motion vector of the image from each macro vector supplied from the macro vector detection unit 15 and the motion vector of the entire screen. Then, the motion vector determination unit 16 supplies the determined motion vector of the image to the effective integrated value table creation unit 18.

In the peripheral integrated value storage unit 17, the first to twelfth macro vector detection circuits 15A to 1 are provided.
Correlation integrated value data around each macro vector detected by 5L is copied to each peripheral integrated value memory 17A to 17L.

Further, the effective integrated value table preparation section 18 is
Each peripheral integrated value memory 17A of the above peripheral integrated value storage unit 17
With respect to the correlation integrated value data around each macro vector copied to ˜17L, the macro block for which the macro vector near the motion vector of the image determined by the motion vector determination unit 16 is obtained is set as the valid macro block and each valid An effective integrated value table is created from integrated value data around the motion vector of the macro block.

Here, as shown in FIG. 5, for example, when a screen is divided into 4 × 3 12 macroblocks and the motion vector of the entire screen is determined from the motion vector of each macroblock, the shake vector is ( If it is assumed that the other vector is due to the movement of the object in -3, 1), the total integrated value table obtained by adding up the integrated value tables of all macroblocks (-
When the vector of (3, 1) is interpolated, the accumulated value data in the vicinity of (-3, 1) is contaminated by the motion of the object, except for the macroblock in which the vector of (-3, 1) appears. The interpolation accuracy of cannot be expected.

Therefore, as shown by hatching in FIG.
An effective integrated value that reduces the effect of a moving object by ignoring macroblocks in which a vector other than the shake vector (-3, 1) is detected and adding only the integrated value table of macroblocks in which the shake vector is detected. A table is created by the effective integrated value table creation unit 18. Although the shake vector is determined to be (-3, 1), it is actually (-2,
Since it may be in the middle of 1), the effective integrated value table creating unit 18 determines that the vector close to the motion vector determined by the motion vector determining unit 16 is, for example, as shown by hatching in FIG. Create an effective integrated value table by regarding it as an effective vector.

Then, the interpolation processing section 19 interpolates the motion vector of the image determined by the motion vector determination section 16 based on the effective integrated value table created by the effective integrated value table creating section 18. .. The motion vector interpolated by the interpolation processing unit 19 is supplied to the correction signal generation unit 20 as a camera shake vector indicating the camera shake of the image.

When the effective integrated value table creating unit 18 creates an effective integrated value table from the integrated value data around the motion vector of each effective macroblock, each correlation integrated value in the correlation integrated value table forming unit 12 is Although a part of the data has been updated with the data of the next field, the correct integrated value can be calculated correctly by copying the correlation integrated value data around each macro vector to each peripheral integrated value memory 17A to 17L. A value list can be created. Further, the peripheral integrated value memories 17A to 17L are
Although it is possible to copy all the correlation integrated value data, the storage capacity is reduced by copying only the peripheral integrated value data used for the interpolation processing of the motion vector.

That is, the camera shake detection device 1 of this embodiment
At 0, the field difference detection unit 11, the correlation integrated value table formation unit 12, and the macro vector detection unit 15 detect a motion vector for each macro block obtained by dividing one screen into a plurality of screens, and the motion vector determination unit 16 detects each motion vector. The motion vector of the entire screen is detected based on the motion vector of the macro block. Then, the peripheral integrated value storage unit 17 and the effective integrated value table creating unit 18 create an effective integrated value table for valid macroblocking a macroblock for which a motion vector near the motion vector of the entire screen is obtained, and the interpolation is performed. The processing unit 9 interpolates the motion vector of the entire screen by the integrated value data around the motion vector of each effective macroblock, and outputs it as a camera shake vector.

As described above, the effect of a moving object is reduced by creating an effective integrated value table for an effective macroblock of a macroblock for which a motion vector near the motion vector of the entire screen is obtained and performing interpolation processing. , It is possible to obtain a shake vector with less error.

Further, the above correction signal generating section 20, 'as _{input, X t = X t-1} -V t' by the hand-shake is detected by the detection device 10 is hand shake vector V _t image stabilization becomes the correction amount X _t A signal is formed and this image stabilization signal is supplied to the correction section 30.

As shown in FIG. 8, for example, the correction section 30 includes an address control circuit 31 and a select signal generation circuit 32 to which a shake correction signal is supplied from the correction signal generation section 20, and the address control circuit 31. A field memory 33 and peripheral memory 34 for writing / reading video data in accordance with an address signal supplied from the field memory 33 and the peripheral memory 34.
And a selector 35 for selectively outputting the video data read from the select signal generator circuit 32 according to the select signal supplied from the select signal generating circuit 32.

Input video data supplied through the signal input terminal 1 is sequentially written in the field memory 33. The read address of the field memory 33 is controlled by the camera shake correction signal according to the camera shake vector. As a result, from the field memory 33, video data in which one field of input video data is moved according to the camera shake vector is obtained. Then, the video data read out from the field memory 33 and the peripheral video data read out from the peripheral memory 34 are combined by the selection by the selector 35 and output from the signal output terminal 2 as the video data subjected to the image stabilization process.

In the peripheral memory 34, the video data of the peripheral portion corresponding to the image correction range of the image data subjected to the image stabilization processing output via the selector 35 is sequentially written as peripheral video data.

[0040]

As is apparent from the above description, in the image shake detecting apparatus according to the present invention, the first detecting means detects the motion vector for each macroblock obtained by dividing one screen into a plurality of screens. The second detection means detects the motion vector of the entire screen based on the motion vector of each macroblock, and the interpolation means uses each macroblock for which a motion vector near the motion vector of the entire screen is obtained as an effective macroblock. By interpolating the motion vector of the entire screen with the integrated value data around the motion vector of the macroblock and outputting it as a shake vector, the influence of the moving object can be reduced and a shake vector with less error can be obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a camera shake correction apparatus provided with an image camera shake detection apparatus according to the present invention.

FIG. 2 is a diagram showing a state of screen block division in the camera shake detection device.

FIG. 3 is a diagram showing a structure of one block of a screen divided into blocks.

FIG. 4 is a diagram showing a state of macroblocks obtained by dividing one screen into 12 parts.

FIG. 5 is a diagram showing a state of a motion vector of each macroblock.

6 is a diagram showing valid macroblocks in the state of the motion vector shown in FIG.

FIG. 7 is a diagram showing an example of an effective macroblock in the camera shake detection device.

FIG. 8 is a block diagram showing a configuration of a correction unit of the image stabilization apparatus.

[Explanation of symbols]

 10: Hand shake actor detection device 11: Field difference detection unit 12: Correlation integrated value table creation unit 15: Macro vector detection unit 16: Vector determination Part 17: peripheral integrated value storage unit 18: effective integrated value table creation unit 19: interpolation processing unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Tsukasa Hashino 6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation

Claims (1)

[Claims] 1. A first screen is divided into a plurality of macroblocks, and a motion vector for the entire screen is calculated based on an output of the first detection means for detecting a motion vector for each macroblock. Second detection means for detecting, and a macro block for which a motion vector near the motion vector of the entire screen detected by the second detection means is obtained as an effective macro block, and integration around the motion vector of each effective macro block An image shake detecting apparatus for an image, comprising: an interpolating unit that interpolates a motion vector of the entire screen based on value data, and outputs the motion vector interpolated by the interpolating unit as a shake vector.