JPH05328201A - Correction device for camera shake of picture - Google Patents

Abstract

PURPOSE:To provide the correction device for camera shake of a picture in which correction processing is ensured by discriminating whether a detected motion vector is a motion vector of a picture due to a camera shake or a motion vector due to an intended camera operation such as panning or tilt. CONSTITUTION:Motion vectors of a picture detected by a motion vector detection section 10 are stored in a vector memory 20 over lots of fields, and a discrimination section 30 discriminates whether or not a motion vector is due to a camera shake based on the history of the motion vector, a correction signal generating section 40 generates the camera shake correction signal of the correction quantity in response to the motion vector due to the camera shake and a correction section 50 applied camera shake correction processing to an input video signal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image shake correction apparatus for an image which detects and corrects a movement amount of an image caused by so-called camera shake included in video data such as an image pickup output of a handy type video camera.

[0002]

2. Description of the Related Art Generally, in a handy type video camera, camera shake during shooting, that is, camera vibration, appears as image vibration. Therefore, as an image shake correction apparatus for an image which corrects the image shake due to such a hand shake, for example, as disclosed in Japanese Patent Laid-Open No. 63-166370, a motion vector of the image is detected and based on this motion vector. Then, a method of correcting video data stored in the image memory has been proposed.

A block matching method, for example, is employed to detect the motion vector of the image. In the detection of the motion vector of the image by this block matching method, the screen is divided into a number of areas (referred to as blocks), and the representative point pixel of the previous field located at the center of each block and each pixel in the block of the current field are divided. The absolute value of the field difference from the image data is calculated, the absolute value of the field difference of each block is integrated for each corresponding pixel to obtain the correlation integral value, and the correlation integration having the coordinates corresponding to the pixel array for one block is calculated. Form a value table. Then, the motion vector of the entire screen is determined using the coordinate value of the minimum value of the correlation integrated value in the correlation integrated value table as the coordinate value of the motion vector of the image.

In the image shake correction apparatus for an image, the detected motion vector is converted into a correction signal, and the correction signal is used to move the current image. The correction accuracy of such an image shake correction apparatus depends on the detection accuracy of the motion vector of the image.

[0005]

By the way, as described above, the motion vector of the entire screen is determined by using the coordinate value of the minimum value of the correlation integrated value in the correlation integrated value table as the coordinate value of the motion vector of the image. In the detection of the motion vector of the image by the block matching method, the motion vector due to the camera shake and the motion vector due to the movement of the subject occur at the same time. Therefore, in the image shake correction apparatus for an image, it is necessary to discriminate between the motion vector caused by the hand shake and the motion vector caused by the movement of the subject, and form the hand shake correction signal only by the motion vector caused by the hand shake to perform the hand shake correction. When the motion vector due to the movement of the subject is erroneously determined as the motion vector due to the camera shake and the camera shake correction is performed, the background image or the like which should be still moves due to the camera shake correction, which is unnatural. Becomes an image. On the contrary, when the motion vector due to the camera shake is erroneously determined as the motion vector due to the movement of the subject, the camera shake correction becomes unsuccessful.

For example, when an intentional camera operation such as pan / tilt is performed, the camera shake correction process is performed up to the correction limit with the motion vector of the image by this camera operation, as shown in A and B of FIG. As you can see, the output image stops even though the camera is moving. Then, the camera shake correction is turned off after the correction limit, so that C in FIG.
As shown in, the image movement due to the above-mentioned intentional camera operation suddenly appears.

Therefore, for example, when a landscape is slowly panned and photographed, the output image is stopped due to camera shake correction even if the camera is moved, and the image tends to be different from the intended image. .. In addition, since the image does not move at first, the panning speed is increased, and when the correction range is exceeded, the output image suddenly starts to move and the image becomes difficult to see.

The present invention has been made in view of the above circumstances, and an object thereof is to enable high-performance image stabilization in a handy type video camera or the like,
It is determined whether the detected motion vector is the motion vector of the image due to camera shake or the motion vector of the image due to the intentional camera operation such as pan / tilt, and the motion vector of the image according to the above-mentioned intentional camera operation is used. It is an object of the present invention to provide an image shake correction apparatus for stopping an image shake correction process.

[0009]

In order to solve the above-mentioned problems, an image stabilization apparatus according to the present invention is representative of each block obtained by dividing an image of one field composed of an input video signal into a plurality of blocks. Image data of point pixels in memory 1
A motion vector that stores a field period and detects an image motion vector based on the absolute value of the difference between the image data of each pixel of the current field block and the image data of the representative pixel of the previous field block read from the memory. A detection unit, a vector memory that stores the motion vector detected by the motion vector detection unit over a large number of fields, and whether the motion vector of the image detected by the motion vector detection unit is caused by camera shake. A determination unit that determines whether or not the motion vector is stored based on the history of the motion vector stored in the vector memory, and a shake correction signal having a correction amount corresponding to the motion vector determined to be caused by the shake by the determination unit is generated. The input video by the correction signal generator and the image stabilization signal supplied from the correction signal generator. It is characterized in that comprising a correcting unit that performs camera shake correction process to issue.

[0010]

In the image shake correction apparatus according to the present invention, the motion vector of the image is block-matched by using the image data of the representative point pixel of each block obtained by dividing the image of one field formed by the input video signal into a plurality of blocks. Is detected by the motion vector detection unit, and the motion vector is stored in the vector memory over many fields. Then, the determination unit determines whether or not the motion vector detected by the motion vector detection unit is caused by camera shake, based on the history of the motion vector stored in the vector memory, and the motion vector caused by camera shake. A shake correction signal having a correction amount corresponding to the above is generated by the correction signal generation unit, and the correction unit performs the shake correction process on the input video signal.

[0011]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an image stabilization apparatus according to the present invention will be described in detail below with reference to the drawings. An image stabilization apparatus according to the present invention is configured as shown in FIG. 1, for example.

In the embodiment shown in FIG. 1, the present invention is applied to a camera shake correction device for correcting image movement due to camera shake in a handy type video camera. The motion vector detection unit 10, vector memory 20, and determination unit 30,
The correction signal generating section 40 and the correction section 50 are provided. In FIG. 1, a signal input terminal 1 is supplied with input video data obtained by digitizing a video signal obtained as an image pickup output by an image pickup unit (not shown) of the video camera.

In this camera shake correction device, the motion vector detection section 10 stores image data of representative point pixels for each block obtained by dividing an image of one field composed of an input video signal into a plurality of fields in a memory for one field period. Then
The motion vector of an image is detected based on the absolute value of the difference between the image data of each pixel of the block of the current field and the image data of the representative point pixel of the block of the previous field read from the memory. A field difference detecting unit 11 to which video data is supplied via the signal input terminal 1, a correlation integrated value table forming unit 14 to which a field difference output from the field difference detecting unit 11 is supplied, and a correlation integrated value table forming unit. And a motion vector estimation unit 18 to which the correlation integrated value of the correlation integrated value table formed by the unit 14 is supplied.

The field difference detection section 11 comprises a representative point memory 12 to which the input video data is supplied via the signal input terminal 1 and a subtraction circuit 13.

The representative point memory 12 stores image data I _k (0,0) of representative point pixels in each block obtained by dividing an image of one field composed of input video data into a plurality of blocks. Specifically, for example, as shown in FIG. 2, the screen of one field is divided into blocks of m pixels × n lines, and as shown in FIG. 3, the pixel S (0,
0) as a representative point and image data I of each representative point pixel
_k (0, 0) is stored in the representative point memory 12 for one field period. The representative points are evenly distributed on the screen. Then, the image data I of each representative point pixel one field before read from the representative point memory 12
_k-1 (0,0) is supplied to the subtraction circuit 13. Further, the subtraction circuit 13 performs image data I _k (x, y) of m × n pixels for each block for the input video data supplied through the signal input terminal 1, that is, the image data of the current field. And the image data I _k-1 (0,0) of the representative point pixel of the corresponding block of the previous field read from the representative point memory 12, that is, the absolute value of the inter-field difference | I _k-1 (0,0) ) -I _k (x,
y) | is detected.

Then, the field difference detector 11
Supplies the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the subtraction circuit 13 to the correlation integrated value table forming unit 14.

The correlation accumulated value table forming unit 14 is supplied with the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the field difference detecting unit 11. a macro blocking circuit 15, a macroblock of the field difference absolute value by the macro blocking circuit _{15 | I k-1 (0,0} ) -I k (x, y) | first through is supplied 10 absolute value integrating circuits 16A to 16J.

The macroblocking circuit 15 has macros which are not adjacent to each other with respect to the field difference absolute value | I _k-1 (0,0) -I _k (x, y) | obtained by the field difference detecting unit 11. In the form of blocks,
One screen is divided into a plurality of macroblocks. For example, as shown in FIG. 4, one screen is divided into 3 × 3 nine macroblocks B1 to B9.

The field difference absolute value | I macroblocked by the macroblocking circuit 15
_k−1 (0,0) −I _k (x, y) | is supplied to each of the first to ninth absolute value integrating circuits 16A to 16I corresponding to each of the macro blocks B1 to B9 for each macro block. At the same time, all macroblocks are supplied to the tenth absolute value integrating circuit 16J.

The first absolute value integrating circuit 16A has a first
Field difference absolute value of macroblock B1 of | I _k-1
For (0,0) -I _k (x, y) |, the field difference absolute values of the block composed of the m × n pixels are integrated for each corresponding pixel, and the first macroblock B1 is added. Form a correlation integrated value table of. Similarly, the second
The to ninth absolute value integrator circuits 16B to 16I form the correlation integrated value tables of the corresponding second to ninth macroblocks B2 to B9. Further, the tenth absolute value integrating circuit 16J includes the first to ninth macro blocks B1.
Form a correlation integrated value table for the entire screen consisting of B9 to B9.

The correlation integrated value table forming unit 14 is
The correlation integrated value of the correlation integrated value table of each macro block B1 to B9 formed by the first to ninth absolute value integrating circuits 16A to 16I and the entire one screen formed by the tenth absolute value integrating circuit 16J. The correlation integrated value of the correlation integrated value table of is supplied to the motion vector estimation unit 18.

The motion vector estimating section 18 estimates the motion vector by detecting the coordinates of the minimum value of the correlation integrated value in each correlation integrated value table formed by the correlation integrated value table forming section 14. Accumulated correlation value of each accumulated correlation value table is formed by the accumulated correlation value table forming circuit 14, image data I _k of image data I _k-1 (0,0) and the other pixels of the representative point pixels of each block This indicates the inter-field correlation with (x, y). The smaller the coordinate corresponding to a pixel having a stronger correlation is, the smaller the coordinate integrated value of the coordinates corresponding to the motion vector is. The motion vector can be estimated by detecting the motion vector.

The method of detecting the motion vector of the image by the macroblock adopted in the motion vector detecting section 10 was previously proposed by the applicant of the present application in Japanese Patent Application No. 3-100384.

The motion vector detecting section 10 is
The motion vector of the image estimated by the motion vector estimating unit 18 is stored in the vector memory 20 and the correction signal generating unit 4.
Supply to 0.

Further, the vector memory 20 stores the motion vector of the image detected by the motion vector detecting section 10 over many fields of several frames to several tens of frames.

Further, the judging unit 30 monitors the past correlation of the motion vectors stored in the vector memory 20, so that the motion vector of the image detected by the motion vector detecting unit 10 is caused by the camera shake. It is determined whether it is a thing. Here, with regard to the movement of an image due to a deliberate camera operation such as pan / tilt, the speed (the magnitude of the vector) is generally constant except when rising, when viewed with a motion vector.

At the time of rising, a uniform acceleration motion is often performed. The vector direction is almost constant. There are features such as. Therefore, in the determination unit 30, even if it is not possible to discriminate between intentional motion and camera shake only with the motion vector for one field, the magnitude and direction of the motion vector over many fields of the past several frames to several tens of frames can be determined. By monitoring the correlation, it is possible to distinguish between intentional movement and camera shake.

The determination unit 30 is configured to use the vector memory 2
On the basis of the motion vector stored in 0, by monitoring the correlation of the magnitude and direction of the motion vector over a large number of fields in the past several frames to several tens of frames, the intentional motion and the camera shake are discriminated. It is determined whether or not the motion vector of the image detected by the motion vector detection unit 10 is due to camera shake, and the determination output is supplied to the correction signal generation unit 40 as a stop control signal.

When the correction signal generating section 40 is supplied with a judgment output indicating that the motion vector detected by the motion vector detecting section 10 is caused by the image shake of the image, the correction signal generating section 40 receives the judgment output. _{'as, X t = X t-1} -V t' vector V _t shake motion vector to form a shake correction signal becomes the correction amount X _t, and supplies the image stabilization signal to the correcting unit 50. When the correction signal generator 40 is supplied with a determination output indicating that the motion vector detected by the motion vector detector 10 is not due to the image shake of the image, the shake vector V _{t'is a} zero vector V [0,
0] to form a camera shake correction signal, and the camera shake correction signal is supplied to the correction section 50.

Further, as shown in FIG. 5, the correction section 50 includes an address control circuit 51 and a select signal generation circuit 52 to which a shake correction signal is supplied from the correction signal generation section 40, and the address control circuit 51. Field memory 53 and peripheral memory 5 in which video data is written / read according to an address signal supplied from
4 and a selector 55 for selectively outputting the video data read from the field memory 53 and the peripheral memory 54 according to the select signal supplied from the select signal generating circuit 52.

Input video data supplied through the signal input terminal 1 is sequentially written in the field memory 53. The read address of the field memory 33 is controlled by the camera shake correction signal according to the camera shake vector. As a result, from the field memory 53, the video data obtained by moving the input video data of one field in accordance with the camera shake vector can be obtained. Then, the video data read from the field memory 53 and the peripheral video data read from the peripheral memory 54 are combined by the selection by the selector 55 and output from the signal output terminal 2 as the video data subjected to the image stabilization process.

In the peripheral memory 54, video data of the peripheral portion corresponding to the image correction range of the image data subjected to the image stabilization processing output via the selector 55 is sequentially written as peripheral video data.

In this camera-shake correction apparatus, it is determined whether or not the motion vector of the image detected by the motion-vector detecting section 10 is caused by camera-shake as described above, and the motion vector stored in the vector memory 20 is determined. Since the determination unit 30 can generate a camera shake correction signal having a correction amount according to the motion vector caused by the camera shake, the correction signal generation unit 40 can generate the correction signal based on the history of the input video. The image can be surely subjected to the image stabilization processing, and a natural image output can be obtained.

[0034]

As is apparent from the above description, in the image stabilization apparatus according to the present invention, the representative point pixel of each block obtained by dividing the image of one field composed of the input video signal into a plurality of blocks is selected. The motion vector of the image is detected by the motion vector detection unit by block matching using the image data, and the motion vector is stored in the vector memory over many fields. Therefore, the motion vector detected by the motion vector detection unit is detected. The determination unit can determine whether or not is caused by camera shake based on the history of motion vectors stored in the vector memory. Then, the camera shake correction signal having the correction amount corresponding to the motion vector caused by the camera shake can be generated by the correction signal generation unit, and the camera shake correction process can be reliably applied to the input video signal by the correction unit.

As described above, according to the present invention, it is possible to judge whether or not the motion vector of the image is caused by the camera shake, and surely perform the camera shake correction, and in a handy type video camera or the like. High-performance image stabilization can be enabled.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of an image shake correction apparatus according to the present invention.

FIG. 2 is a diagram showing a state of block division of a screen in a motion vector detection unit of the image stabilization apparatus.

FIG. 3 is a diagram showing a structure of one block of the screen divided into blocks.

FIG. 4 is a diagram showing a state of macroblocks obtained by dividing one screen into nine parts.

FIG. 5 is a block diagram showing a configuration of a correction unit of the image stabilization apparatus.

FIG. 6 is a diagram for explaining a problem of the conventional camera shake correction device.

[Explanation of symbols]

 10 --- Motion vector detection unit 20 --- Vector memory 30 --- Determination unit 40 --- Correction signal Generation unit 50 ・ ・ ・ ・ ・ ・ Correction unit

 ─────────────────────────────────────────────────── ─── Continuation of the front page (72) Inventor Tsukasa Hashino 6-735 Kita-Shinagawa, Shinagawa-ku, Tokyo Sony Corporation

Claims (1)

[Claims] 1. Image data of a representative point pixel for each block obtained by dividing an image of one field composed of an input video signal into a plurality of pieces is stored in a memory for one field period, and image data of each pixel of a block of the current field is stored. And a motion vector detection unit that detects a motion vector of the image based on the absolute value of the difference between the image data of the representative point pixel of the block of the previous field read from the memory, and the motion vector detected by the motion vector detection unit. Based on the history of the motion vector stored in the vector memory, and whether or not the motion vector of the image detected by the motion vector detection unit is caused by camera shake. Judgment unit to judge and motion vector judged to be caused by camera shake by the judgment unit A correction signal generator that generates a camera shake correction signal having a correction amount according to the image stabilization amount, and a camera shake correction signal that is supplied from the correction signal generator. Image stabilization device featuring images.