JP2007335917A - Motion vector detection apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector detection apparatus capable of detecting a motion vector with high accuracy by excluding the effect by a moving body and the effect by misdiscrimination of block reliability without increasing a processing load imposed on a sensor. <P>SOLUTION: An image is divided into a plurality of blocks (31), and a motion vector of each block is detected (32). The blocks are grouped by grouping the blocks whose motion vectors are similar to each other (34). An average motion vector of the blocks belonging to the same group is calculated (35), and the reliability of the groups is discriminated by comparing each average motion vector with a sensor motion vector (37). The whole motion vector is detected on the basis of the average motion vectors of the reliable groups (38). <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a motion vector detection device applied to a camera shake correction device used in a video camera or the like.

  Japanese Patent Application Laid-Open No. 2004-228688 discloses a motion vector detection device that detects a motion vector of a screen applied to camera shake correction in a video camera or the like. In this apparatus, a representative point matching method is used for motion vector detection. The representative point matching method is an absolute difference between the image signal level in the field immediately before the pixel defined as the representative point pixel and the image signal level in the current field of all the pixels in the region defined as the search range. The sum is calculated to obtain a correlation value, a highly correlated sampling point pixel having the smallest correlation value is obtained, and the difference in position between the sampling point pixel and the representative point pixel is used as a motion vector representing the motion of the subject. It is a method to specify.

  Specifically, one screen is divided into a plurality of regions (hereinafter referred to as “blocks”), and a motion vector for each block is detected by a representative point matching method. Further, the reliability of the motion vector for each block is determined using correlation value data indicating the tendency of the correlation value for each block, that is, the minimum value, average value, gradient, etc. of the correlation value. Then, based on the determined reliability, it is determined whether or not the detected motion vector for each block is due to camera shake, and the entire screen is determined from the average value or median value of the motion vectors of the blocks determined to be due to camera shake. The motion vector is detected.

JP-A-6-133298

  However, if there is a moving object in the screen, the detected motion vector for each block contains both motion vectors due to camera shake and motion vectors due to the motion of the moving object. If the average motion vector or median value of the motion vector of all blocks is used, the motion vector of the entire screen is affected by the motion vector due to the motion of the moving object, and the image stabilization cannot be performed sufficiently. was there.

  On the other hand, the present applicant pays attention to the points described above, and performs block grouping according to the degree of similarity of motion vectors for each detected block among the blocks determined to be reliable in the reliability determination. In addition, a motion vector detection device that determines reliability for each group and eliminates a motion vector of a block in which a moving object exists based on the determined reliability for each group is proposed. However, in this method, it is difficult to select parameters used for reliability determination for each group, and if an inappropriate parameter is selected, the accuracy of the overall camera shake correction may be reduced.

  On the other hand, there is also a method for obtaining a motion vector using an angular velocity sensor, an angular acceleration sensor, or the like that detects the motion of a video camera. However, with this method, in order to increase detection accuracy, it is necessary to increase the sampling frequency of the sensor output signal, and there is a problem that hardware and processing load increase.

  The present invention has been made paying attention to this point, and can detect a highly accurate motion vector by eliminating the influence of a moving object and the influence of erroneous determination of block reliability without increasing the processing load. An object of the present invention is to provide a possible motion vector detection device.

In order to achieve the above object, an invention according to claim 1 is used in an imaging device, and is a motion vector detection device that detects a motion vector of an image from a correlation between temporally continuous images, and a screen is divided into a plurality of regions. A correlation value calculating unit that calculates a correlation value between a pixel one field before the representative point included in each region and a pixel within the search range for performing motion detection by a dividing unit that divides into Correlation value data calculating means for calculating correlation value data indicating a tendency of the correlation value according to the correlation value;
Region motion vector detection means for detecting a motion vector for each region from the correlation value, reliability determination means for determining the reliability of each region based on the correlation value data, and reliability determined by the reliability determination means Grouping means for grouping areas similar to the motion vector based on the motion vectors of the areas determined to be effective by the average motion vector calculating means for calculating an average motion vector of blocks belonging to the same group; Speed parameter detection means for detecting a speed parameter indicating the angular acceleration or angular speed of the imaging device; and a sensor motion that calculates a speed parameter integrated value by integrating the speed parameter and calculates a sensor motion vector from the speed parameter integrated value Vector calculation means, average motion vector for each group and sensor motion The group reliability determination means for comparing the vectors and determining the reliability of the group, and the average motion vector of the group determined to be effective based on the reliability determined by the group reliability determination means, And an overall motion vector detecting means for detecting a motion vector.

  According to the first aspect of the present invention, one screen is divided into a plurality of areas, and motion vector detection for each area and reliability determination of the detected motion vector are performed. Based on the motion vector of the reliable region, regions having similar motion vectors are grouped, and an average motion vector of the regions belonging to the same group is calculated. A velocity parameter indicating angular acceleration or angular velocity of the imaging device is detected, and a sensor motion vector is calculated based on the velocity parameter. The average motion vector for each group and the sensor motion vector are compared, and the reliability of the group is determined. The motion vector of the entire screen is detected using the average motion vector of the reliable group. Thereby, without increasing the processing load, it is possible to exclude a moving object existing in the screen and erroneous detection of a motion vector due to various conditions, and a highly accurate motion vector can be detected.

Embodiments of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing a configuration of a camera shake correction apparatus to which a motion vector detection apparatus according to an embodiment of the present invention is applied. This apparatus includes a CCD 1 as an imaging device, a signal processor 2, a block motion vector detector 3, a CPU 6, a field memory 7, an angular acceleration sensor 21, and a filter 22. The video signal output from the CCD 1 is amplified by an amplifier (not shown), A / D converted by an A / D converter, and then input to the signal processor 2 for various signal processing such as color separation and γ correction. Done. The YCbCr signal of the luminance color difference signal of the image signal that has been subjected to signal processing is stored in the field memory 7. Video output from the field memory 7 is performed by read address control of the connected CPU 6.

  On the other hand, the Y signal of the luminance signal separated by the signal processor 2 is input to the block motion vector detector 3. The block motion vector detector 3 performs motion vector detection processing and correlation value data calculation processing for each block by the representative point matching method, and outputs the processing result to the CPU 6. Further, the angular acceleration sensor 21 acquires a detection signal at a cycle of 16 times in one field period, and outputs the detection signal SF to the CPU 6 via the filter 22. The CPU 6 integrates the 16 detection signals input from the angular acceleration sensor 21 in the horizontal direction and the vertical direction, respectively, and calculates a sensor motion vector corresponding to one field. The CPU 6 detects the motion vector of the entire screen based on the motion vector for each block input from the block motion vector detector 3, the correlation value data, and the calculated sensor motion vector. In this embodiment, an angular acceleration sensor is used, but an angular velocity sensor may be used instead of the angular acceleration sensor.

  The CPU 6 determines a read address according to the detected motion vector of the entire screen and outputs a control signal to the field memory 7. The field memory 7 outputs an image signal in which camera shake is corrected in accordance with a control signal input from the CPU 6.

  Next, the motion vector detection process for the entire screen according to the present embodiment will be described. FIG. 2 is a block diagram showing a configuration of a motion vector detection module including the block motion vector detector 3 and the CPU 6. This module includes a block dividing unit 31, a motion vector detecting unit 32, a reliability determining unit 33, a grouping unit 34, an average motion vector calculating unit 35, a sensor motion vector calculating unit 36, a group reliability determining unit 37, and an overall motion. It consists of a vector detection unit 38, and the function of this module is realized by processing in the block motion vector detector 3 and the CPU 6.

  The block dividing unit 31 divides the screen into blocks, and the motion vector detecting unit 32 detects the block motion vector for each divided block using the representative point matching method described above. FIG. 3 is a diagram for explaining the arrangement of blocks and representative point pixels used for detection at this time. FIG. 3 shows 16 blocks 9 divided into four in the horizontal direction and the vertical direction of the screen 8, and further representative points in which 8 in the horizontal direction and 4 in the vertical direction are arranged at equal intervals. Pixel 10 is shown. Note that the 16 blocks 9 are provided with labels B1 to B16 which will be described later.

  Next, the reliability determination unit 33 calculates correlation value data for each block, that is, the minimum value, average value, gradient, and the like of the correlation value, and determines the reliability of each block using the motion vector and correlation value data of each block. I do. That is, it is determined whether each block is valid or invalid. Blocks determined to be invalid are not used for subsequent processing.

The grouping unit 34 determines whether or not the motion vectors of the blocks are similar for the blocks determined to be valid by the reliability determination unit 33, and groups the blocks having similar motion vectors. To group the blocks. When the sum of the absolute differences in the horizontal direction and the absolute difference in the vertical direction (hereinafter referred to as “block correlation value”) BG of the motion vectors of the two blocks to be compared is obtained and the calculated block correlation value BG is smaller than the threshold value BGTH The two blocks to be compared are determined to belong to the same group (similar motion vectors). Specifically, when the motion vector of the block B1 determined to be valid is (X1, Y1) and the motion vector of the block B2 also determined to be valid is (X2, Y2), the block correlation is expressed by the following equation (1). The value BG (1, 2) is calculated.
BG (1,2) = | X1-X2 | + | Y1-Y2 | (1)

  When the calculated block correlation value BG (1,2) is smaller than the threshold value BGTH, it is determined that the blocks B1 and B2 belong to the same group. The block similarity value BG is used to determine block similarity for all blocks determined to be valid by the reliability determination unit 33.

  An example of grouping in this way is shown in FIG. FIG. 4 shows an image of a person located in front of the house and a truck traveling from right to left on the screen 8. The two blocks (blocks B3 and B4) with cross-hatching on the upper right side of the screen shown in FIG. 4 and the four blocks (B13, B14 with cross-hatching below the screen on which the ground portion is drawn are shown. , B15, B16) are determined to be invalid in the block reliability determination in the reliability determination unit 33, and are not used for the subsequent processing. Next, the six blocks (B1, B2, B5, B6, B9, and B10) with hatching on the left side of the screen on which the still person, the tree in the background, and the image of the house are drawn are expressed by the equation (1). ) Are grouped as group 1 based on the result of the determination.

  The average motion vector calculation unit 35 calculates an average motion vector for each divided group. The average motion vector is the average value of all block motion vectors belonging to the same group. An arrow Y1 drawn around the chest of a person in group 1 represents a motion vector of group 1 described later. Since the images in the group 1 are a stationary person and a background that does not move, it can be determined that the motion vector of the group 1 is a motion vector due to camera shake.

  Also, the four blocks (B7, B8, B11, B12) with double-line hatching on the right side of the screen including the image of the running track are also grouped as group 2 as a result of the determination according to the equation (1). ing. The motion vector of group 2 is indicated by an arrow Y2, which is a mixture of a motion vector caused by camera shake and a motion vector caused by track motion.

  On the other hand, the sensor motion vector calculation unit 36 integrates 16 detection values for each of the horizontal direction component and the vertical direction component of the detection signal SF output from the filter 22 to calculate a sensor motion vector corresponding to one field. .

  The group reliability determination unit 37 performs group reliability determination by vector comparison. The determination method will be described with reference to FIG. In FIG. 5, the average motion vector calculated by the average motion vector calculation unit 35 is indicated by a thick arrow for each group as in FIG. The average motion vector of group 1 is an arrow Y1, and the average motion vector of group 2 is an arrow Y2. A sensor motion vector calculated based on the detection signal SF of the angular acceleration sensor 21 is shown as an arrow Y3 on the right side of the screen 8. The vector values (vector components) of the sensor motion vector arrow Y3 and the average motion vector arrows Y1 and Y2 are respectively compared, and a group having a close vector value is determined as an effective group. The group determined to be invalid is not used for the subsequent processing. In the example shown in FIG. 5, since the direction and magnitude of the group 1 vector (arrow Y1) are substantially equal to those of the sensor motion vector (arrow Y3), that is, the vector values of the arrows Y1 and Y3 are substantially equal. It is determined to be an effective group. On the other hand, the vector of group 2 (arrow Y2) is different in direction and magnitude from that of the sensor motion vector (arrow Y3), that is, the vector values of arrow Y2 and arrow Y3 are completely different. Therefore, the group 2 is determined to be an invalid group.

  The overall motion vector detection unit 38 determines the motion vector of the entire screen (field) based on the average motion vector of the group determined to be valid by the group reliability determination unit 37. In this embodiment, the average motion vector of group 1 becomes the motion vector of the entire screen (field). If there are a plurality of effective groups, the average motion vectors of the groups determined to be effective are averaged to obtain a motion vector for the entire screen (field).

Specifically, the group trust determination is performed using the following equation (2). The average motion vector calculated by the average motion vector calculation unit 35 is (XGRP, YGRP), the sensor motion vector calculated by the sensor motion vector calculation unit 36 is (XCEN, YCEN), the determination threshold is FBTH, and the following formula (2 ) Is judged to be effective.
| XGRP-XCEN | + | YGRP-YCEN | <FBTH (2)

  In this example, the removal of a group in which a moving object is present has been described as an example. However, although there is no moving object in the block reliability determination of the reliability determination unit 33, it differs from the movement due to camera shake due to erroneous determination. For groups grouped by motion vectors, the effect can be eliminated by using this method. As a result, a highly accurate motion vector in field units can be detected.

It is a block diagram which shows the structure of the camera-shake correction apparatus of the video camera to which the motion vector detection apparatus concerning one Embodiment of this invention is applied. It is a block diagram which shows the structure of the motion vector detection module comprised by the block motion vector detector 3 and CPU6. It is a figure for demonstrating arrangement | positioning of the block on a screen of a video camera, and a representative point pixel. It is a figure for demonstrating grouping of a block. It is a figure for demonstrating group reliability determination by the comparison of an average motion vector and a sensor motion vector.

Explanation of symbols

1 CCD
2 Signal processor 3 Block motion vector detector 6 CPU
7 field memory 8 screen 9 block 10 representative pixel 21 angular acceleration sensor (speed parameter detection means)
22 Filter 31 Block division unit (division means)
32 motion vector detection unit (correlation value calculation means, region motion vector calculation means)
33 Reliability determination unit (correlation value data calculation means, reliability determination means)
34 Group division (grouping means)
35 Average motion vector calculation unit (average motion vector calculation means)
36 sensor motion vector calculation unit (sensor motion vector calculation means)
37 Group Reliability Judgment Unit (Group Reliability Judgment Unit)
38 Overall motion vector detection unit (overall motion vector detection means)

Claims (1)

In a motion vector detection device that is used in an imaging device and detects a motion vector of an image from a correlation between temporally continuous images,
A dividing means for dividing one screen into a plurality of areas;
Correlation value calculating means for calculating a correlation value between a pixel one field before the representative point included in each region and a pixel within a search range for performing motion detection by the representative point matching method;
Correlation value data calculating means for calculating correlation value data indicating a tendency of the correlation value according to the correlation value;
Area motion vector detection means for detecting a motion vector for each area from the correlation value;
Reliability determination means for determining the reliability of each region from the correlation value data;
Grouping means for grouping regions having similar motion vectors based on motion vectors of regions determined to be effective based on the reliability determined by the reliability determining means;
Average motion vector calculating means for calculating an average motion vector of regions belonging to the same group;
Speed parameter detection means for detecting a speed parameter indicating angular acceleration or angular velocity of the imaging device;
A sensor motion vector calculating means for calculating a speed parameter integrated value by integrating the speed parameter, and calculating a sensor motion vector from the speed parameter integrated value;
A group reliability determination means for comparing the average motion vector for each group and the sensor motion vector to determine the reliability of the group;
A motion comprising: an overall motion vector detecting means for detecting a motion vector of the entire screen using an average motion vector of the group determined to be effective based on the reliability determined by the group reliability determining means. Vector detection device.

Images