JPH04302590A - Blurring detector for video data - Google Patents

Abstract

PURPOSE:To enable satisfactory blurring correction by discriminating the movement of the object in an image pickup signal and the blurring when the blurring of the video camera is corrected. CONSTITUTION:An absolute value integration circuit 8 integrates the frame difference between the representative points detected by a subtraction circuit 6 and each picture element of a block, and the minimum value is detected as motion vector. A frequency distribution table preparation circuit 9 is composed of a memory having an address corresponding to the search area of the motion vector, and the frame difference is approximately '0' in the area and the frequency of the position with tilted level is incremented. The result of this processing is integrated in one screen, and the integration frequency distribution table is prepared. The frequency of the integration frequency distribution table corresponding to the motion vector is checked and when the frequency is less than the threshold value, this motion vector is proved to be the blurring vector.

Description

[Detailed description of the invention]

0001

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake detection device for detecting the direction and amount of camera shake contained in video data such as the output of a hand-held video camera.

0002

2. Description of the Related Art When taking pictures using a handheld video camera, there is a problem in that the playback screen shakes due to camera shake. To solve this problem, we detect the motion vector and
It has been proposed to correct video data stored in an image memory based on this motion vector (for example, Japanese Patent Application Laid-open No. 166370/1983). Detection of motion vectors is performed, for example, by block matching. In other words, the screen is divided into many areas (referred to as blocks),
The absolute value of the frame difference between the representative point of the previous frame located at the center of each block and the pixel data in the block of the current frame is calculated, the absolute value of this frame difference is integrated for one screen, and the integrated frame difference data is calculated. The motion vector is detected from the position of the minimum value. In addition, as described in this document, in order to prevent video data from being lost when motion compensation is performed, the image is enlarged, for example, by about 15% by controlling the readout of the image memory and using an interpolation circuit. ing. In such a camera shake correction device, it is necessary that camera shake be detected correctly.

0003

In detecting motion vectors by block matching, it is difficult to determine whether the motion caused by camera shake is the motion of the video camera or the motion of an object on the screen. Therefore, there is a risk that movement of the object may be mistakenly determined to be camera shake. Therefore, as a conventional method for detecting camera shake, a method is known in which one frame of an image is divided into several regions (referred to as macroblocks). In this method, motion vectors are detected by block matching not for the entire frame but for each macroblock, and then the motion vectors for each macroblock are subjected to a majority decision and adopted as the majority motion vector. However, when an object that occupies a large area moves in one frame, there is a problem in that the movement of this object is interpreted as camera shake and is erroneously detected. SUMMARY OF THE INVENTION Accordingly, an object of the present invention is to provide a camera shake detection device for video data that can determine whether a detected motion vector is caused by the movement of an object or by camera shake.

0004

[Means for Solving the Problems] This invention divides one screen into a plurality of regions, detects a frame difference between a representative point of the previous frame and pixel data in each region of the current frame for each region, Means (5, 6, 8) for forming cumulative frame difference data by accumulating the absolute value of the frame difference for one screen and detecting a motion vector from the cumulative frame difference data; Therefore, the frame difference between the representative point and the pixel data corresponding to this representative point is approximately 0,
Further, when there is a slope of the level, the frequency of the pixel position in the direction of this slope is incremented, and the frequency is accumulated for the one screen to form an integrated frequency table (9
), a determining means (10) that receives the detected motion vector and the cumulative frequency table, and determines the motion vector as a camera shake vector if the frequency of the cumulative frequency table at the position corresponding to the motion vector is less than a predetermined value; This is a camera shake detection device for video data consisting of:

[0005]

[Operation] Motion vectors are detected by block matching using representative points. Further, a frequency distribution table creation circuit is provided that creates a frequency distribution table of the same size as the motion search area. If the frame difference is approximately 0 and there is movement, the position of the pixel where the frame difference occurs is incremented by +1. This frequency is accumulated over one screen,
A cumulative frequency distribution table is formed. From the detected motion vector and the integrated frequency distribution table, if the frequency corresponding to the motion vector is less than the threshold value, it is determined that the object is not moving.
It is determined to be camera shake.

[0006]

DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. This invention focuses on the fact that since camera shake is caused by the movement of a video camera, the entire screen moves.On the other hand, the movement of an object means that a certain part of the screen moves, that is, there are still parts of the screen. A distinction is made between the two. In FIG. 1, 1 is an input terminal for digital video data. This video data is captured, for example, by a CCD of a video camera, and is a series of interlaced scan sequences.

Input video data is supplied to a field memory 2 and a representative point memory 3. field memory 2
Data that has been corrected for camera shake using a correction signal, which will be described later, is read out from the output. When processing is performed frame by frame, a frame memory is used instead of the field memory 2. The output of representative point memory 3 is supplied to subtraction circuit 4 and representative point memory 5. The output of the representative point memory 5 is supplied to a subtraction circuit 6. Subtraction circuits 4 and 6 are supplied with input video data. The representative point memory 3 stores the data of the representative point one field before, and the representative point memory 5 stores the data of the representative point two fields (that is, one frame) before.

As shown in FIG. 2, one field of screen is subdivided into blocks of m pixels×n lines, and the central pixel of each block is taken as a representative point. The representative point is on the screen,
It is evenly distributed. The subtraction circuit 4 detects the differences between each of the m×n pixel data of a certain block in the current field and the data at the representative point of the block at the same position in the previous field (that is, the difference between fields). This inter-field difference is supplied to the absolute value integration circuit 7. Similarly, the subtraction circuit 6 detects the frame difference between the block of the current field and the representative point of two fields before. The frame difference is supplied to an absolute value integration circuit 8 and a frequency table creation circuit 9.

The motion vector is detected by the absolute value integration circuit 8.
is made using frame difference. The inter-field movement obtained from the inter-field difference by the absolute value integration circuit 7 is used as an auxiliary means. In other words, the difference between fields is characterized by good time precision but poor position precision.
The difference between frames is characterized by poor time precision but good position precision. Therefore, when detecting a motion vector from the frame difference, the detection accuracy can be improved by also using the inter-field difference.

As shown in FIG. 3, the motion search area is the same size as the block. For example (m=n=25
). For each block, the subtraction circuit 6 generates m×n differences between the representative point of two fields before and the pixel data in the block of the current field. In the absolute value integration circuit 8, the absolute value of the frame difference at each position within the block is integrated over one frame period to form an m×n distribution of integrated frame difference data, and the minimum value in this distribution is the motion vector. Detected as . Detection of this motion vector is the same as the conventional method.

The frame difference from the subtraction circuit 6 is supplied to the frequency distribution table creation circuit 9. The frequency distribution table creation circuit 9 includes a memory having a capacity corresponding to the above-mentioned motion search area. Then, when the following conditions are met, the frequency at that position is incremented. Ik-1 (0,0) ≒Ik (0,0)
And {Ik-1 (0,0) −Ik (x,y)
}Absolute value≧Th

[0012] Here, Ik (0,0)
is the value of the pixel data at the center of the block in the current field, Ik-1 (0,0) is the value of the pixel data one frame (two fields) before, and Th determines whether there is a level difference in space. This is the threshold for When the condition of the above expression is satisfied, the frequency f(x, y) in the frequency distribution table is incremented. This frequency distribution table creation process is accumulated over the entire one screen to form a frequency distribution table. This condition means that there is no frame difference and that this (x, y) position has an inclination, so if there is movement, a frame difference will definitely occur.

The motion vector detected by the absolute value integration circuit 8 and the output of the frequency distribution table creation circuit 9 are supplied to the motion amount determination circuit 10 . The motion amount determining circuit 10 verifies whether the motion vector can be used as a camera shake vector by referring to the frequency distribution table. That is, the frequency of the position of the motion vector is checked and this frequency is compared with a threshold value. If this frequency is less than or equal to the threshold value for hand shake vector determination, it is determined that the motion vector is derived from hand shake, and the motion vector is determined to be a hand shake vector. If this is not the case, it is determined that the number of still blocks in the screen is large, and it is determined that this motion vector is derived from the movement of the object in the screen.

The reason why the motion amount determination circuit 10 is supplied with inter-field motion vectors from the absolute value integration circuit 7 is to check the stationarity of the detected motion vectors and to check the accuracy. This is to improve the results. The output (hand shake vector) of the motion amount determination circuit 10 is supplied to the correction amount generation circuit 11 . The camera shake vector is detected from the movement between frames, and is not the same as the camera shake correction amount. For example, in a period of three consecutive frames, when camera shake occurs in the same direction in the second and third frames, the camera shake vector V1 between the first frame and the next frame is detected. , a camera shake vector V2 between the second and third frames is detected. Second
The correction amount for the third frame may be V1, but the correction amount for the third frame may be V1.
The amount of correction for the th frame needs to be (V1+V2). The correction amount generation circuit 11 generates a correction amount by integrating the camera shake vector. Also, as in this example,
When image stabilization is performed on a field-by-field basis, the motion vector is halved to accommodate the time difference between fields and frames in order to convert the amount of motion between frames into the amount of motion between fields.

The camera shake correction signal from the correction amount generation circuit 11 is transmitted to the address control circuit 12 and the selection signal generation circuit 1.
3. Address control circuit 12 generates address signals for field memory 2 and peripheral memory 16. One field of image is written in the field memory 2, and its read address is controlled according to the amount of correction. Therefore, from the field memory 2, video data obtained by shifting one field of input video data according to the amount of correction is obtained. The output of the field memory 2 is supplied to the selector 14, and the output of the selector 14 is taken out to the output terminal 15 and is also supplied to the peripheral memory 16. Peripheral data that is the output of peripheral memory 16 is supplied to selector 14 . The selector 14 responds to a select signal from the select signal generating circuit 13 to select video data from the field memory 2 that has undergone camera shake correction and peripheral data stored in the peripheral memory 16.

As shown in FIG. 4A, field memory 2
A peripheral portion (area outside the dashed line) 22 of one field of the image (its image frame is represented by 21) captured in is stored in the peripheral memory 16. The width of the peripheral portion 22 is set in consideration of the range of camera shake correction, and is set to a width of about 10 to 20%, for example. In FIG. 4B, as shown by the image frame 21a, when the image that should be at the position shown in FIG. 4A moves toward the drawing due to camera shake, for example, toward the right, the image shake correction amount V moves the entire image to the position shown by the broken line. Corrected. In such a case, the image of the diagonally shaded portion 23 on the left side of the moved image is missing because it does not originally exist in the captured image. This missing portion 23 is replaced with an image at a corresponding position stored in the peripheral memory 16. This replacement is executed by address control by the address control circuit 12 and switching operation of the selector 14. Furthermore, peripheral image data other than the missing portion 23 in the captured video data is written to the peripheral memory 16, and the contents of the peripheral memory 16 are updated.

[0017] In the present invention, camera shake correction may be performed in real time for the image pickup output of a video camera, or may be performed on images recorded by a VTR. When correcting the imaging output of a video camera in real time, it is preferable to display a marker indicating that camera shake has occurred on the screen of the viewfinder.

As shown in FIG. 5A, an image frame 21 indicated by a broken line
It is assumed that this is an image that was actually captured, and that it has been corrected by the above-mentioned camera shake correction as shown in the image 21b indicated by the solid line. In this case, as shown in FIG. 5B, a vertical line marker 25 is displayed on the screen of the viewfinder 24 of the video camera at a position corresponding to the edge of the image frame 21 that is actually being imaged. The photographer looks at this marker 25, knows the direction and amount of camera shake, or the direction and amount of camera shake correction, and moves the video camera in the direction to correct the camera shake. In addition to the camera shake correction described above, feedback is applied based on the photographer's vision, so it is possible to significantly prevent the range of camera shake from exceeding the correction range. If camera shake is not displayed on the viewfinder, the screen may suddenly move at a certain point when camera shake cannot be corrected. As the marker 25, not only a vertical line but also an arrow or the like may be used.

[0019]

According to the present invention, since it is possible to determine whether the detected motion vector is caused by camera shake or the movement of an object, camera shake correction can be performed satisfactorily.

[Brief explanation of drawings]

FIG. 1 is a block diagram of an embodiment of the present invention.

FIG. 2 is a schematic diagram showing block division according to an embodiment of the present invention.

FIG. 3 is a schematic diagram showing a motion vector search range according to an embodiment of the present invention.

FIG. 4 is a schematic diagram used to explain camera shake correction according to an embodiment of the present invention.

FIG. 5 is a schematic diagram used to explain a camera shake display displayed on a viewfinder screen of a video camera.

[Explanation of symbols]

1 Video data input terminal 2 Field memory 3 Representative point memory 5 that stores representative point data from 1 field before Representative point memory 9 that stores representative point data from 2 fields before Frequency distribution table creation circuit 10 Motion amount determination circuit

Claims (1)

[Claims] Claim 1: One screen is divided into a plurality of regions, and for each region, the frame difference between the representative point of the previous frame and the pixel data in each region of the current frame is detected, and the absolute value of the frame difference is determined. means for forming cumulative frame difference data by accumulating data for the one screen and detecting a motion vector from the cumulative frame difference data; When the frame difference between the pixel data corresponding to is approximately 0 and there is a level slope, the frequency of the pixel position in the direction of this slope is incremented, and the frequency is accumulated for the one screen. and means for receiving the detected motion vector and the cumulative frequency table, and when the frequency of the cumulative frequency table at a position corresponding to the motion vector is less than a predetermined value, the motion vector is A device for detecting camera shake for video data, comprising: determining means that uses a camera shake vector as a camera shake vector.