JPH04302591A - Blurring correction device for video data - Google Patents

Abstract

PURPOSE:To use a blurring vector as a blurring correction signal without integrating the blurring vector by detecting the blurring of the video camera from frame difference and maintaining an unused frame value for frame difference calculation when the blurring is corrected. CONSTITUTION:An absolute value integration circuit 8 integrates the frame difference between the representative point detected by a subtraction circuit 6 and each picture element of a block and detects the minimum value as motion vector. A motion amount decision circuit 10 decides a blurring vector from the motion vector. When the blurring vector is detected, switching circuits SW1 and SW2 are switched to states selecting input terminals (b) and (d) respectively. Thus, the update of the representative memories 3 and 5 is stopped and the stored representative point data is preserved.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction device for correcting camera shake contained in video data such as the output of a hand-held video camera.

0002

2. Description of the Related Art When taking pictures using a handheld video camera, there is a problem in that the playback screen shakes due to camera shake. To solve this problem, we detect the motion vector and
It has been proposed to correct video data stored in an image memory based on this motion vector (for example, Japanese Patent Application Laid-open No. 166370/1983). Detection of motion vectors is performed, for example, by block matching. In other words, the screen is divided into many areas (referred to as blocks),
The absolute value of the frame difference between the representative point of the previous frame located at the center of each block and the pixel data in the block of the current frame is calculated, the absolute value of this frame difference is integrated for one screen, and the integrated frame difference data is calculated. The motion vector is detected from the position of the minimum value. In addition, as described in this document, in order to prevent video data from being lost when motion compensation is performed, the image is enlarged, for example, by about 15% by controlling the readout of the image memory and using an interpolation circuit. ing.

0003

The camera shake vector is detected from the movement between frames, and is not the same as the camera shake correction amount. For example, in a period of 3 consecutive frames,
When camera shake occurs in the same direction during the second and third frames, the camera shake vector V1 between the first frame and the next frame is detected, and the camera shake vector V1 between the second and third frames is detected. A camera shake vector V2 between frames is detected. The correction amount for the second frame may be V1, but the correction amount for the third frame is (V1
+V2) is required. Therefore, conventional image stabilization devices require a circuit that generates a correction amount by integrating a motion vector. SUMMARY OF THE INVENTION Therefore, an object of the present invention is to provide an apparatus for correcting camera shake for video data, which does not require integrating camera shake vectors and converting them into correction amounts as in the prior art, and which can simplify the circuit.

0004

[Means for Solving the Problems] The present invention provides a circuit for detecting a camera shake vector indicating the direction and amount of camera shake by detecting a frame difference, and a system for correcting camera shake based on the output signal of this detection circuit. In the image stabilization device for video data, the detection circuit includes a memory (5) that stores representative points of each block when one frame of image data is divided into a plurality of blocks.
and a subtraction circuit (6) that subtracts the representative point data stored in the memory (5) and the pixel data in the block of the current frame.
), and when the vector detected by the detection circuit is determined to be a camera shake vector, the memory output is written into the memory (5) again.

[0005]

[Operation] Motion vectors are detected by block matching using representative points. A camera shake vector is detected from this motion vector. When a camera shake vector is detected, updating of the memory that stores representative points is stopped. Therefore, the camera shake vector detected after that corresponds to the movement of two frames, three frames, etc., rather than the movement between frames. Therefore, the camera shake vector corresponds to the camera shake correction amount, and there is no need to integrate the camera shake vector to form a camera shake correction signal, and the circuit configuration can be simplified.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. In one embodiment of the present invention, camera shake detection is performed because the camera shake is caused by the movement of the video camera, so the entire screen moves.On the other hand, the movement of the object is determined by the movement of a certain part of the screen, that is, the movement of the object. The distinction between the two is made by focusing on the fact that there is a certain amount of In FIG. 1, 1 is an input terminal for digital video data. This video data is captured, for example, by a CCD of a video camera, and is a series of interlaced scan sequences.

Input video data is supplied to the field memory 2, and is also supplied to the representative point memory 3 through the terminal a of the switching circuit SW1. The field memory 2 outputs data that has been corrected for camera shake as will be described later. When processing is performed frame by frame, a frame memory is used instead of the field memory 2. The output of the representative point memory 3 is supplied to the subtraction circuit 4, and is also returned to itself through the input terminal b of the switching circuit SW1. Therefore, when the input terminal b of the switching circuit SW1 is selected, updating of the representative point memory 3 is stopped and the stored representative point data is held.

Furthermore, the output of the representative point memory 3 is supplied to the representative point memory 5 through the input terminal c of the switching circuit SW2. The output of the representative point memory 5 is supplied to the subtraction circuit 6, and is also returned to itself through the input terminal d of the switching circuit SW2. Therefore, switching circuit SW2
When the input terminal d of is selected, updating of the representative point memory 5 is stopped and the stored data of the representative point is held. Subtraction circuits 4 and 6 are supplied with input video data. The representative point memory 3 stores the data of the representative point one field before, and the representative point memory 5 stores the data of the representative point two fields (that is, one frame) before.

As shown in FIG. 2, one field of screen is subdivided into blocks of m pixels×n lines, and the central pixel of each block is taken as a representative point. The representative point is on the screen,
It is evenly distributed. The subtraction circuit 4 detects the differences (ie, field differences) between each of the m×n pixel data of a certain block in the current field and the data at the representative point of the block at the same position in the previous field. This field difference is supplied to the absolute value integration circuit 7. Similarly, the subtraction circuit 6 detects the frame difference between the block of the current field and the representative point of two fields before. The frame difference is supplied to an absolute value integration circuit 8 and a frequency table creation circuit 9.

The motion vector is detected by the absolute value integration circuit 8.
is made using frame difference. The inter-field movement obtained from the inter-field difference by the absolute value integration circuit 7 is used as an auxiliary means. In other words, the difference between fields is characterized by good time precision but poor position precision.
The difference between frames is characterized by poor time precision but good position precision. Therefore, when detecting a motion vector from the frame difference, the detection accuracy can be improved by also using the inter-field difference.

As shown in FIG. 3, the motion search area is the same as the block size. For example (m=n=25
). For each block, the subtraction circuit 6 generates m×n differences between the representative point of two fields before and the pixel data in the block of the current field. In the absolute value integration circuit 8, the absolute value of the frame difference at each position within the block is integrated over one frame period to form an m×n distribution of integrated frame difference data, and the minimum value in this distribution is the motion vector. Detected as . Detection of this motion vector is the same as the conventional method.

The frame difference from the subtraction circuit 6 is supplied to the frequency distribution table creation circuit 9. The frequency distribution table creation circuit 9 includes a memory having a capacity corresponding to the above-mentioned motion search area. Then, when the following conditions are met, the frequency at that position is incremented. Ik-1 (0,0) ≒Ik (0,0)
And {Ik-1 (0,0) −Ik (x,y)
}Absolute value≧Th

[0013] Here, Ik (0,0)
is the value of the pixel data at the center of the block in the current field, Ik-1 (0,0) is the value of the pixel data one frame (two fields) before, and Th determines whether there is a level difference in space. This is the threshold for When the condition of the above expression is satisfied, the frequency f(x, y) in the frequency distribution table is incremented. This frequency distribution table creation process is accumulated over the entire one screen to form a frequency distribution table. This condition means that there is no frame difference and that this (x, y) position has an inclination, so if there is movement, a frame difference will definitely occur.

The motion vector detected by the absolute value integration circuit 8 and the output of the frequency distribution table creation circuit 9 are supplied to the motion amount determining circuit 10 . The motion amount determining circuit 10 verifies whether the motion vector can be used as a camera shake vector by referring to the frequency distribution table. That is, the frequency of the position of the motion vector is checked and this frequency is compared with a threshold value. If this frequency is less than or equal to the threshold value for hand shake vector determination, it is determined that the motion vector is derived from hand shake, and the motion vector is determined to be a hand shake vector. If this is not the case, it is determined that the number of still blocks in the screen is large, and it is determined that this motion vector is derived from the movement of the object in the screen. Note that instead of creating a frequency distribution table of the same size as the motion vector search area, the frequency distribution table creation circuit 9 creates a frequency distribution table of an area of about (3×3) pixels centered on the representative point. You may also do this.

The motion amount determination circuit 10 is supplied with inter-field motion vectors from the absolute value integration circuit 7 because, as described above, the stationarity of the detected motion vectors is checked and the accuracy is checked. This is to improve the results. Further, the motion amount determination circuit 10 generates a status signal indicating that a camera shake vector has been detected from the motion vector and the frequency distribution table. This state signal controls switching circuits SW1 and SW2.

[0016] If the status signal indicates that a camera shake vector has been detected, the switching circuit SW
1 selects the input terminal b, and the switching circuit S
W2 selects input terminal d. Otherwise, switching circuits SW1 and SW2 are connected to input terminals a and c.
Select each. In this way, when a camera shake vector is detected, updating of the representative point memories 3 and 5 is stopped, and the
The representative point before the frame is saved. As a result, after the camera shake vector is detected, motion vectors (camera shake vectors) are detected between 2 frames, 3 frames, etc.
This is done using the absolute value of the difference signal. This means that the detected camera shake vector no longer indicates the camera shake between one frame, but is the camera shake correction amount itself.

The output (hand shake vector) of the motion amount determination circuit 10 is supplied to a correction amount generation circuit 11 . As described above, since this camera shake vector corresponds to the camera shake correction amount, the correction amount generation circuit 11 does not need to integrate the camera shake vector, and the circuit is simplified. Therefore, it is not essential to provide the correction amount generation circuit 11. However, when image stabilization is performed on a field-by-field basis as in this embodiment, in order to convert the amount of movement between frames into the amount of movement between fields, the correction amount generation circuit 11 adjusts the amount of time difference between fields and frames. In order to do this, the motion vector is halved.

The image stabilization signal from the correction amount generation circuit 11 is transmitted to the address control circuit 12 and the selection signal generation circuit 1.
3. Address control circuit 12 generates address signals for field memory 2 and peripheral memory 16. One field of image is written in the field memory 2, and its readout address is controlled according to the amount of correction. Therefore, from the field memory 2, video data obtained by shifting one field of input video data according to the amount of correction is obtained. The output of the field memory 2 is supplied to the selector 14, and the output of the selector 14 is taken out to the output terminal 15 and is also supplied to the peripheral memory 16. Peripheral data that is the output of peripheral memory 16 is supplied to selector 14 . The selector 14 responds to a select signal from the select signal generating circuit 13 to select video data from the field memory 2 that has been corrected for camera shake and peripheral data stored in the peripheral memory 16.

As shown in FIG. 4A, field memory 2
A peripheral portion (area outside the dashed line) 22 of one field of the image (its image frame is represented by 21) captured in is stored in the peripheral memory 16. The width of the peripheral portion 22 is set in consideration of the range of camera shake correction, and is set to a width of about 10 to 20%, for example. In FIG. 4B, as shown by the image frame 21a, when the image that should be at the position shown in FIG. 4A moves toward the drawing due to camera shake, for example, toward the right, the image shake correction amount V moves the entire image to the position shown by the broken line. Corrected. In such a case, the image of the diagonally shaded portion 23 on the left side of the moved image is missing because it does not originally exist in the captured image. This missing portion 23 is replaced with an image at a corresponding position stored in the peripheral memory 16. This replacement is executed by address control by the address control circuit 12 and switching operation of the selector 14. Furthermore, peripheral image data other than the missing portion 23 in the captured video data is written to the peripheral memory 16, and the contents of the peripheral memory 16 are updated.

[0020] In the present invention, camera shake correction may be performed in real time for the image pickup output of a video camera, or may be performed on images recorded by a VTR. When correcting the imaging output of a video camera in real time, it is preferable to display a marker indicating that camera shake has occurred on the screen of the viewfinder.

As shown in FIG. 5A, an image frame 21 indicated by a broken line
It is assumed that this is an image that was actually captured, and that it has been corrected by the above-mentioned camera shake correction as shown in the image 21b indicated by the solid line. In this case, as shown in FIG. 5B, a vertical line marker 25 is displayed on the screen of the viewfinder 24 of the video camera at a position corresponding to the edge of the image frame 21 that is actually being imaged. The photographer looks at this marker 25, knows the direction and amount of camera shake, or the direction and amount of camera shake correction, and moves the video camera in the direction to correct the camera shake. In addition to the camera shake correction described above, feedback is applied based on the photographer's vision, so it is possible to significantly prevent the range of camera shake from exceeding the correction range. If camera shake is not displayed on the viewfinder, the screen may suddenly move at a certain point when camera shake cannot be corrected. As the marker 25, not only a vertical line but also an arrow or the like may be used.

[0022]

Effects of the Invention: This invention stops updating the previous pixel data when detecting a difference between frames when a motion vector is determined to be a camera shake vector, so the camera shake vector detected thereafter is corrected. It can be matched with the amount. Therefore, there is no need to integrate the camera shake vector and convert it into a camera shake correction amount, and the circuit configuration can be simplified.

[Brief explanation of the drawing]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram showing block division according to an embodiment of the present invention.

FIG. 3 is a schematic diagram showing a motion vector search range according to an embodiment of the present invention.

FIG. 4 is a schematic diagram used to explain camera shake correction according to an embodiment of the present invention.

FIG. 5 is a schematic diagram used to explain a camera shake display displayed on a viewfinder screen of a video camera.

[Explanation of symbols]

1 Video data input terminal 2 Field memory 3 Representative point memory 5 that stores representative point data from 1 field before Representative point memory 9 that stores representative point data from 2 fields before Frequency distribution table creation circuit 10 Motion amount determination circuit SW1, SW2 switching circuit

Claims (1)

[Claims] Claim 1: A camera shaker comprising: means for detecting a camera shake vector indicating the direction and amount of camera shake by detecting a frame difference; and means for correcting camera shake based on an output signal of the detection means. In the image stabilization device for video data, the detection means includes a memory that stores a representative point of each block when one frame of image data is divided into a plurality of blocks, and a memory that stores the representative point data stored in the memory. and a subtraction means for subtracting pixel data in a block of a frame, and when the vector detected by the detection means is determined to be a camera shake vector, the memory output is written into the memory again. Image stabilization device for video data.