JP2009267716A - Motion vector detecting apparatus and method, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector detecting apparatus and method and an imaging apparatus for correctly detecting a motion vector between frames and capturing a motion image with reduced blur. <P>SOLUTION: A motion vector detection unit 16 which divides a frame of motion image data into a plurality of blocks and establishes matching between a previous frame and a current frame for each of the divided blocks, calculate an integrated value of difference absolute values in pixel data between the previous frame and the current frame of each block for each search position, sets a difference in position of the blocks between the previous frame and the current frame at the search position with the minimum integrated value as a motion vector of the blocks, and sets the degree of sharpness of the integrated value at the minimum search position as an evaluation value of the motion vector of the blocks. Further, the motion vector detection unit 16 sets the degree of sharpness of the minimum value as the smallest value other than zero when the integrated value of the difference absolute values for each search position calculated in each block is normalized within the range of 0 to 100. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a motion vector detection apparatus and method, and an imaging apparatus capable of detecting and correcting motion blur of a moving image, and more particularly, to a motion vector detected for each block by dividing moving image data into a plurality of blocks. This is related to the improvement of accuracy when evaluating the reliability of the.

  In recent years, camera miniaturization has progressed as an imaging apparatus, and camera shake correction technology has attracted attention.

  The reason for this is that the holdability of photography has been reduced due to the miniaturization of the camera, and the degree of freedom has increased compared to conventional photography forms such as one-handed photography as a method of photography such as a camera mounted on a mobile phone. Therefore, it is easy to cause camera shake.

  As described above, when shooting is performed in various methods or in an unstable state, the camera shakes when the shutter button is pressed, and camera shake occurs.

  In order to solve this problem, a device for physically correcting camera shake by providing a dedicated mechanism is also mounted on a single-lens reflex camera or recently a compact camera.

For example, an optical camera shake correction device that inclines and corrects a lens in accordance with a shake has been commercialized, but it is very difficult in terms of space in consideration of recent downsizing and mounting in a mobile phone camera.
For this reason, various imaging devices that can obtain a moving image without blurring by image processing have been proposed.

Image stabilization by image processing detects motion vectors between successive frames of a moving image, and shifts the region from which the image is read when outputting the image by the motion vector, thereby generating an image without blurring. .
Japanese Patent Laid-Open No. 6-169552 JP-A-7-177519 JP 2007-300592 A

  For example, a method such as block matching is generally used to detect a motion vector. An image is divided into a plurality of blocks, a motion vector is obtained by correlating the image for each block, and finally a motion vector of the entire screen is obtained. There is something to decide.

  However, it is easy to falsely detect blurry images, images of flat subjects, or images with many similar patterns such as vertical stripes, and if erroneous detection occurs in many blocks, it will hinder effective blur correction. May result.

  Patent Documents 1 and 2 disclose a technique related to a motion vector detection method, but do not mention its reliability. The motion vector may be erroneously detected depending on the subject and shooting conditions. If an incorrect motion vector is used, blurring may be increased when it is used for camera shake correction.

  Patent Document 3 discloses a method for selecting a block based on an integral value of a residual. Even in this method, it is difficult to select a motion vector that is truly reliable.

  An object of the present invention is to provide a motion vector detection apparatus and method, and an imaging apparatus capable of determining the reliability of a motion vector detected for each block and improving the detection accuracy.

  A first aspect of the present invention is a motion vector detection apparatus that divides a frame of moving image data into a plurality of blocks, and that matches the previous frame and the current frame for each divided block, The absolute value of the difference between the pixel data of the current frame and the current frame is obtained for each search position, and the difference between the position of the block of the previous frame and the current frame at the search position where the integrated value is the minimum is the motion vector of the block, A processing unit that uses the sharpness of the integrated value of the minimum search position as an evaluation value of the motion vector of the block;

  Preferably, when determining the motion vector of the entire screen from the motion vector obtained in each block, the processing unit validates only the motion vector for which the corresponding evaluation value is equal to or greater than a predetermined threshold. .

  Preferably, the processing unit employs, as the motion vector of the entire screen, a motion vector having the highest corresponding evaluation value among the motion vectors obtained in each block.

  Preferably, the processing unit sets the sharpness in each block as a difference between the minimum integrated value and the next smallest integrated value.

  Preferably, the processing unit sets the sharpness in each block to the smallest value other than 0 when normalized by the difference between the minimum value and the maximum value of the integrated values.

  A motion vector detection method according to a second aspect of the present invention includes a step of dividing each frame of a moving image into a plurality of blocks, and an integrated value of absolute values of differences between pixel data of the previous frame and the current frame of each block. The step of obtaining for each search position, and the difference between the block position of the previous frame and the current frame at the search position where the integrated value is the minimum as the motion vector of the block, and at the same time the sharpness of the integrated value of the minimum search position And setting the evaluation value of the motion vector of the block as the evaluation value.

  An imaging apparatus according to a third aspect of the present invention divides an optical system, an imaging element that captures a subject image that has passed through the optical system, and a frame of moving image data obtained from the imaging element into a plurality of blocks, A motion vector detecting device for matching the previous frame and the current frame for each divided block, wherein the motion vector detecting device is an integrated value of absolute values of differences between pixel data of the previous frame and the current frame of each block; For each search position, the difference between the position of the block of the previous frame and the current frame at the search position where the integrated value is the minimum as the motion vector of the block, and the sharpness of the integrated value of the minimum search position It has a processing part which makes an evaluation value of a motion vector of a block.

  According to the present invention, it is possible to accurately detect a motion vector between frames and obtain a moving image with reduced blur.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

  FIG. 1 is a block diagram showing an embodiment of an imaging apparatus according to the present invention.

The imaging apparatus 10 includes an optical system 11, an imaging control unit 12, an operation unit 13, an imaging unit 14, an image processing unit 15, a motion vector detection unit 16, a memory 17, and a recording unit 18.
Among these components, a processing unit is configured by the image processing unit, the motion vector detection unit 16, the memory 17, and the like.

In the motion vector detection method based on block matching, the imaging apparatus 10 realizes an evaluation value that is used to determine which of the motion vectors obtained for each block is to be used as the motion vector for the entire screen.
Specifically, the integrated value of the absolute value of the difference between the pixel data is obtained for each search position between frames, and if the difference (sharpness) between the minimum value and other values is large, the minimum value is obtained. It can be determined that the positional relationship between the previous frame and the current frame is likely to be a deviation amount.
The motion vector indicated by this positional relationship is highly reliable.

The imaging apparatus 10 according to the present embodiment includes an ON / OFF mode switching unit for camera shake correction photography.
When the camera shake correction shooting is ON, the imaging device 10 has a function of correcting a positional shift between the shot images and forming an image in which blur reduction is achieved.

  Upon receiving a shooting instruction from the operation unit 13 such as a shutter button, the shooting control unit 12 calculates optimum shutter speed, aperture, and gain values from the brightness of the subject, and sends a control signal to the imaging unit 14.

  The imaging control unit 12 performs exposure control and has operation inputs of the operation unit 13 and the like, determines the operation of the entire system according to those inputs, controls the optical system 11, the imaging unit 14, and the like, and It is responsible for overall mediation control.

  The imaging unit 14 is formed to include an imaging element 141 such as a CCD or a CMOS sensor, receives an optical image of a subject passing through the optical system 11 and converts it into an image signal, and converts YUV format image data from the converted image signal. Generate and output to the image processing unit 15.

  The image processing unit 15 stores the image data supplied from the imaging unit 14 in the memory 17 and performs signal processing on the image signal stored in the memory 17.

  The motion vector detection unit 16 detects a motion vector (position shift amount) and a correlation evaluation value between images of the current frame and the previous frame.

  For motion vector detection, an image is divided into small areas (blocks) of a certain size, where each block corresponds to in the previous frame, and the difference in the position of the corresponding block is used as a motion vector.

The cutout position of the image in the memory is changed according to the detected motion vector, and the moving image is recorded on the recording medium by the recording unit 18.

The motion vector detection unit 16 divides moving image data obtained from the image sensor into a plurality of blocks, and performs matching between the previous frame and the current frame for each of the divided blocks.
The motion vector detection unit 16 obtains an integrated value of absolute values of differences between pixel data of the previous frame and the current frame of each block for each search position.
Then, the motion vector detection unit 16 uses the difference between the position of the block of the previous frame and the current frame at the search position where the integrated value is the minimum as the motion vector of the block, and determines the sharpness of the integrated value of the minimum part of the block. Includes a function for evaluating motion vectors.

When determining the motion vector of the entire screen from the motion vector obtained for each block, the motion vector detection unit 16 validates only the motion vector whose corresponding evaluation value is equal to or greater than a predetermined threshold.
The motion vector detection unit 16 has a function of adopting the motion vector obtained by each block having the highest corresponding evaluation value as the motion vector of the entire screen.

  The motion vector detection unit 16 sets the sharpness in each block as the difference between the minimum integrated value and the next smallest integrated value.

The motion vector detection unit 16 sets the sharpness in each block to the smallest value other than 0 when normalized by the difference between the minimum value and the maximum value of the integrated values.
More specifically, when the motion vector detection unit 16 normalizes the minimum value sharpness to 0 for the integrated value of the absolute value of the difference for each search position obtained in each block, and 100 for the maximum value. The smallest value other than 0.

  Hereinafter, a method for obtaining a motion vector and a correlation evaluation value for each block will be described in more detail.

  FIG. 2 is a diagram schematically showing a standard block and a reference block of consecutive images.

  As shown in FIG. 2, in the previous frame 21 and the current frame 22 in which the moving image data is continuous, the reference image that is the area 23 obtained by dividing the previous frame 21 into a plurality of blocks is the reference image that is the search area 24 of the current frame 22. Matching is performed on a pixel-by-pixel basis while taking the correlation of the image (difference in pixel data) as to which position in the image corresponds to.

FIG. 3 is a diagram illustrating a difference integrated value (integral value of absolute values of pixel data differences) for each search position shifted in units of one pixel.
At the position where the correlation between the images is highest, the difference integrated value is minimal, and the difference between the positions of the blocks of the previous frame and the current frame at this point is used as a motion vector.

FIG. 4 is a graph showing the difference integrated value of one line including the minimum value of FIG. 3 for easy understanding.
In FIG. 4, the horizontal axis indicates the horizontal search position, and the vertical axis indicates the difference integrated value.
In FIG. 4, a local minimum point is shown at an acute angle at a certain point. In FIG. 4, the minimum point is shown when the horizontal search position is -3.

  However, if there is a subject with a similar pattern in the block or a flat image with a small change, there are a plurality of values close to the minimum point value as shown in FIGS. The correct correlation point cannot be detected as a minimum point due to the influence of noise and blurring.

In order to prevent this, an evaluation value of correlation is introduced in this embodiment.
Specifically, in the present embodiment, the magnitude between the minimum point value and the second smallest value is set as the evaluation value, and the greater the evaluation value, the higher the reliability of the detected motion vector.

  In addition, since the size of the difference integrated value itself changes depending on the brightness and the pattern of the subject depending on the block, the minimum value of the difference integrated value of the block is standardized to “0” and the maximum value is set to “100”. Obtain the evaluation value when

  Next, a motion vector detection operation with the above configuration will be described with reference to the flowchart of FIG.

<Step ST1>
In step ST1, the motion vector detection unit 16 generates a coarse image obtained by thinning out vertical and horizontal pixels from the images of the previous previous frame and current frame, and divides the image into a plurality of (n) blocks.

<Step ST2>
In step ST2, the motion vector detection unit 16 obtains an inter-frame difference integrated value for each search position of the search range with respect to the reference image for each block.

<Step ST3>
In step ST <b> 3, the motion vector detection unit 16 normalizes, for example, by setting the minimum value of the difference integrated value in the block as “0” and the maximum value as “100”.

<Step ST4>
In step ST4, the motion vector detection unit 16 obtains a minimum value other than “0” of the standardized difference integrated value, and uses this value as an evaluation value of the block.

<Step ST5>
In step ST5, the motion vector detection unit 16 obtains motion vectors and evaluation values for all blocks, and sets a plurality of (m [m <n]) blocks having large evaluation values as effective blocks.

<Step ST6>
In step ST6, the motion vector detection unit 16 performs detailed matching on the effective block of the image before thinning, and obtains a motion vector and an evaluation value.

<Step ST7>
In step ST7, the motion vector detection unit 16 sets the motion vector of the block having the largest evaluation value as the motion vector of the entire screen.
Then, motion compensation processing or the like is performed using the obtained motion vector.

As described above, according to the present embodiment, the motion vector detection unit 16 divides moving image data obtained from the image sensor into a plurality of blocks, and performs matching between the previous frame and the current frame for each of the divided blocks. Take.
The motion vector detection unit 16 obtains an integrated value of absolute values of differences between pixel data of the previous frame and the current frame of each block for each search position.
Then, the motion vector detection unit 16 uses the difference between the position of the block of the previous frame and the current frame at the search position where the integrated value is the minimum as the motion vector of the block, and determines the sharpness of the integrated value of the minimum part of the block. Includes a function for evaluating motion vectors.
Therefore, according to the present embodiment, the following effects can be obtained.

When matching images with clear edges, the minimum value of the difference integrated value is shown at an acute angle and the evaluation value is large, and in the case of an image with a small change, the vicinity of the minimum value of the difference integrated value is gentle and the evaluation value is also Get smaller.
Objects subject to errors can be omitted from the target block for determining the motion vector of the entire screen, and a block having a large evaluation value can be set as an effective block.
In the case of a two-step search for pattern matching, a motion vector and an evaluation value are obtained for each block in the coarse image at the first step, and only a block with a high evaluation value, that is, a highly reliable block is picked up. Can be matched.
Moreover, it contributes to greatly reducing false detections for continuous images having the same pattern such as vertical stripes.

1 is a block diagram showing an embodiment of an image processing apparatus according to the present invention. It is the figure which showed typically the reference | standard block and reference block of a continuous image. It is the figure which showed the difference integrated value of each search position in a block. It is the figure which showed the difference integrated value of each search position in a block. It is the figure which showed the difference integrated value of each search position in a block. It is the figure which showed the difference integrated value of each search position in a block. It is a flowchart for demonstrating the motion vector detection operation | movement which concerns on this embodiment.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 10 ... Imaging device, 11 ... Optical system, 12 ... Shooting control part, 13 ... Operation part, 14 ... Imaging part, 15 ... Image processing part, 16 ... Motion vector Detection unit, 17 ... memory, 18 ... recording unit.

Claims (7)

A motion vector detection apparatus that divides a frame of moving image data into a plurality of blocks and performs matching between the previous frame and the current frame for each divided block,
The absolute value of the difference between the pixel data of the previous frame and the current frame of each block is obtained for each search position, and the difference between the position of the block of the previous frame and the current frame at the search position where the integrated value is minimum A motion vector detection apparatus comprising: a processing unit that uses the minimum search position integrated value sharpness as an evaluation value of the motion vector of the block. The processor is
The motion vector detection according to claim 1, wherein when determining a motion vector of the entire screen from the motion vector obtained in each block, only the motion vector whose corresponding evaluation value is equal to or greater than a predetermined threshold is valid. apparatus. The processor is
The motion vector detection device according to claim 1, wherein among the motion vectors obtained in each block, a motion vector having the highest corresponding evaluation value is adopted as a motion vector for the entire screen. The processor is
The sharpness in each block is a difference between the minimum integrated value and the next smallest integrated value,
The motion vector detection device according to any one of claims 1 to 3. The processor is
The motion vector detection according to any one of claims 1 to 4, wherein the sharpness in each block is the smallest value other than 0 when normalized by the difference between the minimum value and the maximum value of the integrated values. apparatus. Dividing each frame of the moving image into a plurality of blocks;
Obtaining an integrated value of absolute values of differences between pixel data of the previous frame and the current frame of each block for each search position;
The difference between the block position of the previous frame and the current frame at the search position where the integrated value is minimum is used as the motion vector of the block, and at the same time, the sharpness of the integrated value of the minimum search position is evaluated for the motion vector of the block. A value step;
A motion vector detection method comprising: Optical system,
An image sensor that images a subject image that has passed through the optical system;
The motion vector detection according to any one of claims 1 to 5, wherein a frame of moving image data obtained from the image sensor is divided into a plurality of blocks, and the previous frame and the current frame are matched for each divided block. An imaging apparatus having the apparatus.