JP2007166545A - Motion vector detection device and method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a motion vector detection device which achieves motion vector detection with a few retrieval frequency and with good accuracy. <P>SOLUTION: From a motion vector detected for a plurality of blocks (contexts) in the vicinity of a present processing object block, a motion vector detecting initial vector for the present processing object block is predicted. At this time, a plurality of initial vector candidates are generated based on a context comprising blocks of different numbers, and one corresponding to a reference image block which is more similar to the present processing object block among individual candidate vectors is made to be an initial vector. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a motion vector detection apparatus and method for digital moving images.

  In general, when a moving image is shot with an imaging device such as a digital video camera or a digital camera, shooting frequency is high in a situation where vibration occurs, such as hand-held shooting or shooting from a moving body such as a car. When shooting in such a situation, the main body of the imaging apparatus moves and affects the movement of the entire image to be encoded.

  In motion-compensated predictive video coding systems, as represented by the MPEG (Moving Picture Experts Group) system, the inter-frame between the input signal block of the current frame and the locally decoded signal block of the prediction target frame obtained by motion compensation It is common to encode the difference signal. Then, a motion vector used for motion compensation is detected by a method called a normal block matching method.

  For example, the encoding target image frame is divided into blocks each having 16 pixels × 16 lines, and a motion vector is detected for each block. That is, the block being processed is compared with an area of the same size as the block in the past reference image, and the relative position of the block in the reference image that most closely matches the position of the block being processed is moved. Detect as a vector. The degree of coincidence between the image block being processed and the block in the reference image is often determined using the sum of absolute differences of pixels at corresponding positions as an evaluation value.

  When implementing such a predictive video coding method using motion compensation, how accurately motion vector detection is performed with a small number of comparisons is important for improving processing speed and compression rate. is there. The following method is known as one of methods for efficiently and accurately detecting a motion vector. That is, based on the motion vector information detected in the past, the motion vector of the image block being processed is predicted, and the motion vector is detected based on the predicted position of the motion vector.

  For example, one context is generated from a plurality of blocks near the current processing target block among a plurality of blocks in which motion vector detection has already been completed in the same frame. Then, based on the motion vector group already detected for each of the plurality of blocks constituting the context, the motion vector of the current processing target block is predicted and used as an initial vector for search.

FIG. 5 is a diagram illustrating a configuration example of a conventional motion vector detection device.
In FIG. 5, an input image is input in units of blocks for motion vector detection. The memory 501 stores detected motion vectors in the same frame. The stored motion vector is read to the initial vector generator 502.

  The initial vector generator 502 predicts a motion vector for the processing target block based on the already detected motion vector read from the memory 501, and outputs it as an initial vector to the motion vector search unit 503. The motion vector search unit 503 extracts a block at a desired position for performing block matching from the reference image, using the initial vector obtained from the initial vector generator 502 as an initial value for the current processing target block to be input. Motion vector detection is performed. The detected motion vector is used in a predictive coding circuit (not shown) or the like, and is also input to the memory 501 and used for initial vector prediction for the next and subsequent blocks.

Next, the operation of the motion vector detection device of FIG. 5 will be described more specifically with reference to FIG.
The input image divided into blocks for the motion vector detection unit is input to the motion vector search unit 503 and temporarily stored therein. The stored processing target block is read out as necessary for motion vector search. The memory 501 stores motion vectors detected for blocks in the same frame that have been processed before the current processing target block (current processing target block).

  In detecting a motion vector for the current processing target block, an initial vector used as an initial value for search is first generated. The initial vector generator 502 reads an already detected vector from the memory 501. At this time, a plurality of motion vectors are read according to a predetermined context. In the conventional example, the context in the initial vector generator is constituted by blocks at positions as shown in FIG.

  FIG. 4A is a diagram illustrating an example of the relationship between the context and the current processing target block, and each square represents a block that is a target of motion vector detection. In FIG. 4A, a portion surrounded by a thick line represents a context, and here, it is composed of three blocks. A block indicated by diagonal lines represents a current processing target block. In order to predict the motion vector of the current processing target block, a motion vector group corresponding to each of the three blocks constituting the context is used. For example, the elements of the three motion vectors of the context are arranged in descending order, and a vector composed of elements located in the middle is predicted as an initial vector.

  The motion vector search unit 503 takes out the reference image block corresponding to the input initial vector as an initial value and changes the vector, and repeats matching with the input current processing target block. Then, for example, a vector corresponding to a reference image block whose sum of pixel value differences is equal to or less than a predetermined threshold is output as a motion vector. The output motion vector is stored in the memory 501 in addition to being used in an encoding circuit (not shown), for example.

Japanese Patent Laid-Open No. 11-122618

  However, with this conventional method, there is a problem that the motion prediction accuracy of the block near the contour of the moving object tends to be lowered when there is movement of the entire screen caused by panning, device shake, etc. and movement of the subject itself. There is. FIG. 4C is a diagram showing a motion vector of each block extracted from a part of a block group constituting a certain frame. An arrow drawn in each block represents a motion vector of each block, and a region surrounded by a thick line represents a moving object.

  Now, it is assumed that a block having an upward motion vector is dominant as shown in the figure as a whole on the whole screen due to the shake of the apparatus. If there is a moving object in the right direction as shown in the figure, the block in the area occupied by the moving object is dominated by the block with the right (or upper right) direction motion vector as shown in the figure. It becomes the target. In this case, if the motion vector of each block is predicted using the context as shown in FIG. 4A, the accuracy of the prediction is reduced in the block indicated by hatching in FIG. In other words, the probability that the prediction is lost in the block located near the boundary of the same object is high.

  Also, in a region where there are many blocks with extremely low activity (motion), a vector that is completely different from the actual motion vector may be calculated. This is because there is almost no difference in the sum of absolute differences between the current processing target block and the reference image block in such an area. In this case, even if the detected vector is encoded as a motion vector as it is, it can be said that the code amount itself of the block does not have a bad influence. However, when the motion vector is used to predict the motion vector of another block, the motion vector prediction accuracy for the other block may be lowered.

As described above, the conventional motion vector detection method predicts the motion of the block near the contour of the moving subject when the motion of the entire screen and the motion of the subject itself, which are caused by panning or device shake, exist. There was a problem that was easy to come off.
In addition, there is a problem that the accuracy of motion prediction for motion vector detection decreases in an area where there are many blocks with extremely low activity.

  The present invention has been made in view of the above-described problems of the prior art, and provides a motion vector detection apparatus and method that can accurately detect a motion vector with a small number of searches.

  In order to solve such a problem, a motion vector detection apparatus according to the present invention is a motion vector detection apparatus that divides input moving image information into a plurality of blocks and detects a motion vector for each block, A first context is configured by configuring a first context from a plurality of blocks in the vicinity of the target block, and predicting a motion vector of the current processing target block based on a detected motion vector group corresponding to each block. A first predicted motion vector generating means for generating a vector; a plurality of blocks having a number smaller than the number of blocks constituting the first context as a block near the current processing target block; And the motion vector of the current processing target block based on the already detected motion vector group corresponding to each block. Second predicted motion vector generating means for predicting and generating a second predicted motion vector, and an initial motion vector used as an initial value for searching for a motion vector of the current processing target block, An initial motion vector generation unit that selects and generates one of predicted motion vectors, and searches for a motion vector of the current processing target block using the initial motion vector generated by the initial motion vector generation unit as an initial value. And motion vector search means for determining the motion vector.

  The motion vector detection method according to the present invention is a motion vector detection method that divides input moving image information into a plurality of blocks and detects a motion vector for each block. A first context is generated from the first block, and a first predicted motion vector is generated by predicting a motion vector of the current processing target block based on a detected motion vector group corresponding to each block. And a second context is formed by selecting a plurality of blocks in the vicinity of the current processing target block, the number of blocks smaller than the number of blocks constituting the first context. Based on the already detected motion vector group corresponding to the block, the motion vector of the current processing target block is predicted to generate the second predicted motion A second motion vector predictor generating step for generating a vector and an initial motion vector used as an initial value for searching for a motion vector of the current processing target block is one of the first and second motion vector predictors. An initial motion vector generation step of selecting and generating, and a motion vector search step of searching and determining a motion vector of the current processing target block using the initial motion vector generated in the initial motion vector generation step as an initial value; It is characterized by providing.

  Further features of the present invention will become apparent from the best mode for carrying out the present invention and the accompanying drawings.

  According to the motion vector detection apparatus and method of the present invention, motion vector detection can be realized with high accuracy with a small number of searches.

<First Embodiment>
FIG. 1 is a block diagram illustrating a configuration example of a motion vector detection device according to the first embodiment of the present invention.
In FIG. 1, an input image is input in units of blocks for motion vector detection. The memory 101 stores motion vectors that have already been detected in the same frame. The stored motion vector is read to the prediction vector generator # 1 (102) and the prediction vector generator # 2 (103). The prediction vector generator # 1 and the prediction vector generator # 2 generate prediction vectors # 1 and # 2, respectively, based on the already detected motion vectors read from the memory.

  Prediction vectors # 1 and # 2 generated by the prediction vector generator # 1 and the prediction vector generator # 2 are input to the initial vector generator 104. The initial vector generator 104 selects either one of the prediction vectors # 1 and # 2 generated by the prediction vector generator # 1 and the prediction vector generator # 2 and passes it to the motion vector search unit 105 as an initial vector. Output.

  The motion vector search unit 105 extracts a block (reference block) for performing block matching from the reference image, using the initial vector obtained from the initial vector generator 104 as an initial value for the current processing target block. Then, the motion vector search unit 105 detects the motion vector while repeating the comparison between the current processing target block and the reference block and the vector change, and outputs the motion vector. The output motion vector is used, for example, in an encoding circuit (not shown) in the subsequent stage, etc., or input to the memory 101 and used for motion vector detection of subsequent blocks.

  Next, the operation of the motion vector detection device of this embodiment will be described. The input image divided into blocks for the motion vector detection unit is input to the motion vector search unit 105 and temporarily stored therein. The stored processing target block is read out as necessary for motion vector search. The memory 101 stores motion vectors detected for blocks in the same frame that have been processed before the current processing target block. In detecting a motion vector, in order to generate a prediction vector that is a candidate for an initial vector used as an initial value for search, first, a detected vector is read from the memory 101.

  In prediction vector generators # 1 (102) and # 2 (103), a plurality of motions corresponding to a plurality of blocks constituting a predetermined context among blocks in which motion vector detection has already been completed in the same frame Each vector is read out. In the first embodiment, as the context in the prediction vector generator # 1 (102), a context larger than the conventional one as shown in the portion surrounded by the thick line in FIG. 4B is used. In FIG. 4B, the block indicated by diagonal lines represents the current processing target block, and the bold line indicates context. In order to predict the motion vector of the current processing target block, a motion vector group corresponding to each of all the blocks constituting the context is used.

  In the first embodiment, the prediction vector generator # 1 (102) outputs an average vector of all the motion vectors of the block in the context as the prediction vector # 1. As the context in the predictive vector generator # 2 (103), the conventional context indicated by the thick line in FIG. Here, the two contexts are in an inclusive relationship. The prediction vector generator # 2 (103) selects the motion vector having the largest difference from the prediction vector # 1 from the motion vectors of the blocks in the context, and outputs it as the prediction vector # 2. The magnitude of the difference can be determined by, for example, the sum of the magnitudes of the sum of absolute differences between the elements of the vector.

  The prediction vectors # 1 and # 2 generated in the prediction vector generators # 1 (102) and # 2 (103) are input to the initial vector generator 104. The initial vector generator 104 calculates the sum of absolute differences between the corresponding pixels of the reference image block and the current processing target block corresponding to the prediction vectors # 1 and # 2, respectively. Then, a prediction vector corresponding to a reference image block having a small difference absolute value sum is selected as an initial vector and output to the motion vector search unit 105.

  The motion vector search unit 105 changes the vector using the input initial vector as an initial value. Then, a reference image block corresponding to the vector is extracted, and a sum of absolute differences between corresponding pixels of the input current processing target block and the reference image block is calculated. The calculation of the sum of absolute differences is repeated while changing the vector, and a vector corresponding to the reference image block whose value is equal to or smaller than a predetermined threshold is output as a motion vector. The output motion vector is used in an encoding circuit (not shown) or the like and is stored in the memory 101.

  Thus, in this embodiment, an initial vector is predicted using two contexts having different sizes. Specifically, a first vector reflecting a wide range of motion vector trends is generated using a large context. On the other hand, in a small context, a motion vector different from the overall tendency in a large context is detected and detected as a second vector. Then, the reference image block acquired from the two vectors is compared with the current processing target block, and an appropriate vector is determined as an initial vector. As a result, for example, when the current processing target block corresponds to a moving subject, the second vector is selected, and a decrease in prediction accuracy in the boundary block of the moving object is prevented as in the conventional method. be able to. As a result, motion vector detection can be realized with high accuracy with a small number of searches.

<Second Embodiment>
Next, a second embodiment according to the present invention will be described. FIG. 2 is a block diagram illustrating a configuration example of a motion vector detection device according to the second embodiment.

  In FIG. 2, the prediction vector generator # 1 (202), the prediction vector generator # 2 (203), and the initial vector generator 204 are the same as those in the configuration of the first embodiment (FIG. 1), and thus description thereof is omitted. . That is, their configurations are the same as those of the prediction vector generator # 1 (102), the prediction vector generator # 2 (103), and the initial vector generator 104.

  In FIG. 2, the memory 201 stores the reference vector output from the motion vector search unit 205. The stored reference vector is read out to the prediction vector generator # 1 (202) and the prediction vector generator # 2 (203). The motion vector search unit 205 takes out a block at a desired position for block matching from the reference image, using the initial vector obtained from the initial vector generator 204 as an initial value for the current processing target block to be input. Then, a motion vector is detected and a motion vector is output. Further, a reference vector used for predicting an initial vector for motion vector detection of a block to be processed from the next time is generated and output to the memory 201.

Next, the operation of the motion vector detection device of the second embodiment will be described.
The input image divided into blocks for the motion vector detection unit is input to the motion vector search unit 205 and temporarily stored therein. The stored processing target block is read out as necessary for motion vector search. The memory 201 stores a reference vector processed before the current processing target block. The reference vector is a motion vector detected for a block in the same frame or a vector replaced with a motion vector. In detecting a motion vector, a reference vector is first read from the memory 201 in order to generate a prediction vector that is a candidate for an initial vector used as an initial value for a search.

  The operations in the prediction vector generators # 1 (202), # 2 (203), and the initial vector generator 204 are the same as those in the first embodiment, and thus the description thereof is omitted.

  The motion vector search unit 205 changes the vector using the input initial vector as an initial value, and extracts a reference image block corresponding to the vector. Then, matching with the input current processing target block is repeated, and a vector at the time when a value representing the difference (for example, the above-described sum of absolute differences) becomes equal to or less than a predetermined threshold is output as a motion vector. Alternatively, a vector at the time when the similarity is equal to or higher than a predetermined threshold may be output as a motion vector.

Next, a reference vector is generated. The motion vector search unit 205 checks whether the following conditions are satisfied.
(1) Whether the difference between the detected motion vector and the predicted vectors # 1 and # 2 for each of the x and y components exceeds the threshold (mvdx _th , mvdy _th ).
(2) and the sum of absolute differences S _ref of the reference image block and the current target block corresponding to the detected motion vector, the prediction vector # 1, # difference absolute value sum S ₁ obtained analogously using 2, Whether one of the differences from S ₂ is less than or equal to the threshold value d _th .

That is,
| S ₁ −S _ref | ≦ d _th or | S ₂ −S _ref | ≦ d _th
Is it?

When these conditions (1) and (2) are satisfied, a prediction vector that satisfies the expression of the condition (2) is output as a reference vector corresponding to the current processing target block.
If at least one of these conditions (1) and (2) is not satisfied, the detected motion vector is output as a reference vector corresponding to the current processing target block. The reference vector is output to the memory 201 and stored.

  According to the present embodiment, in addition to the effects of the first embodiment, a prediction vector in which the difference from the finally detected motion vector is equal to or greater than a certain value and the motion prediction accuracy difference is less than a threshold is actually detected. It saves as a reference vector instead of the motion vector. Therefore, it is possible to suppress a decrease in the accuracy of motion prediction, which is likely to occur in an area where there are many blocks with extremely low activity.

<Third Embodiment>
Next, a third embodiment according to the present invention will be described. FIG. 3 is a block diagram illustrating a configuration example of a motion vector detection device according to the third embodiment.

  In the present embodiment, the memory 301 is the same as the memory 101 in the first embodiment (FIG. 1). Further, the prediction vector generator # 1 (302), the prediction vector generator # 2 (303), and the motion vector search unit 305 are the prediction vector generator # 1 (102) and the prediction vector generator # 2 (103), respectively. , And the motion vector search unit 105. Therefore, description of these components is omitted.

  In FIG. 3, the initial vector generator 304 uses one of the prediction vectors # 1 and # 2 generated by the prediction vector generator # 1 (302) and the prediction vector generator # 2 (303) as blur information of the apparatus. Accordingly, the initial vector is selected. And it outputs to the motion vector search device 305.

Next, the operation of the motion vector detection device of the third embodiment will be described.
First, the operations in the memory 301, the prediction vector generator # 1 (302), the prediction vector generator # 2 (303), and the motion vector search unit 305 are the memory 101 in the configuration diagram (FIG. 1) of the first embodiment, Since the operations are the same as those in the prediction vector generator # 1 (102), the prediction vector generator # 2 (103), and the motion vector search unit 105, description thereof will be omitted.

The initial vector generator 304 acquires device shake information from an acceleration sensor (not shown), for example. Then, it is determined whether the difference for each component between the motion vector of the entire screen calculated from the blur information and the prediction vector # 1 exceeds a threshold value (mvadx _th , mvady _th ).

  When the difference does not exceed the threshold value, the prediction vector # 1 is considered to be dominated by the movement of the entire screen. Therefore, as in the first embodiment, the sum of absolute differences between the reference image block corresponding to each of the prediction vectors # 1 and # 2 and the current processing target block is calculated, and the prediction corresponding to the smaller value is calculated. Select a vector as the initial vector.

  On the other hand, when the difference exceeds the threshold, it is considered that the context of the prediction vector # 1 reflects the motion specific to the block in the context rather than the entire blur. Therefore, the prediction vector # 1 is immediately selected as an initial vector without considering the prediction vector # 2 based on a smaller context, and is output to the motion vector searcher.

  According to the present embodiment, the same effect as that of the first embodiment can be achieved, and the prediction vector can be determined more easily by using blur information.

Further, the above-described embodiment can be realized by software by a computer of a system or apparatus (or CPU, MPU, etc.).
Accordingly, the computer program itself supplied and installed in the computer in order to implement the functional processing of the present invention by the computer also realizes the present invention. That is, the computer program itself for realizing the functional processing of the present invention is also included in the present invention.

  In this case, the program may be in any form as long as it has a program function, such as an object code, a program executed by an interpreter, or script data supplied to the OS.

  In this case, a computer program for realizing the functional processing of the present invention by a computer is supplied to the computer by a storage medium or wired / wireless communication. Examples of the recording medium for supplying the program include a magnetic recording medium such as a flexible disk, a hard disk, and a magnetic tape, an optical / magneto-optical storage medium such as an MO, CD, and DVD, and a nonvolatile semiconductor memory.

  As a program supply method using wired / wireless communication, there is a method of using a server on a computer network. In this case, a data file (program data file) that can be a computer program forming the present invention is stored in the server. The program data file may be an executable format or a source code.

Then, the program data file is supplied by downloading to a client computer that has accessed the server. In this case, the program data file can be divided into a plurality of segment files, and the segment files can be distributed and arranged on different servers.
In other words, the present invention includes a server device that provides a client computer with a program data file for realizing the functional processing of the present invention on a computer.

  In addition, a storage medium in which the computer program of the present invention is encrypted and stored is distributed to users, key information for decryption is supplied to users who satisfy predetermined conditions, and installation on a user's computer is possible It is also possible to do. The key information can be supplied by downloading from a homepage via the Internet, for example.

In addition, a computer program for realizing the functions of the embodiments by a computer may realize the functions of the embodiments by using the functions of an OS already running on the computer.
Further, at least a part of the computer program constituting the present invention is provided as firmware such as an expansion board attached to the computer, and the functions of the above-described embodiments can be realized using a CPU provided in the expansion board. good.

It is a block diagram which shows the structural example of the motion vector detection apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structural example of the motion vector detection apparatus which concerns on the 2nd Embodiment of this invention. It is a block diagram which shows the structural example of the motion vector detection apparatus which concerns on the 3rd Embodiment of this invention. It is a figure explaining the example of a context, and a prediction motion vector generation method. It is a figure which shows the structural example of the motion vector detection apparatus of a prior art example.

Claims (8)

A motion vector detection device that divides input video information into a plurality of blocks and detects a motion vector for each block,
A first context is composed of a plurality of blocks in the vicinity of the current processing target block, and the motion vector of the current processing target block is predicted based on the detected motion vector group corresponding to each block, and the first First predicted motion vector generation means for generating a predicted motion vector;
For the blocks near the current processing target block, a plurality of blocks having a number smaller than the number of blocks constituting the first context are selected to constitute the second context, and the already detected motion corresponding to each block Second predicted motion vector generation means for generating a second predicted motion vector by predicting a motion vector of the current processing target block based on a vector group;
Initial motion vector generation means for generating an initial motion vector used as an initial value for searching for a motion vector of the current processing target block by selecting one of the first and second predicted motion vectors;
Motion vector search means for searching and determining a motion vector of the current processing target block using the initial motion vector generated by the initial motion vector generation means as an initial value;
A motion vector detection device comprising:   Further, based on the difference between the motion vector determined by the motion vector search means and the first and second predicted motion vectors, and the difference absolute value sum of the sample values of the current processing target block and the reference block, The motion vector according to claim 1, further comprising a vector replacement unit that replaces one of the motion vector corresponding to the first and second contexts and the first and second predicted motion vectors. Detection device.   The vector replacement means includes a difference between the motion vector determined by the motion vector search means and the first predicted motion vector, and the motion vector determined by the motion vector search means and the second predicted motion vector. Of the current processing target block and the sample value of the block in the reference frame at the displacement position corresponding to the motion vector determined by the motion vector search means. Sum of absolute differences, sum of absolute difference between sample value of current processing target block and sample value of block in reference frame at displacement position corresponding to first prediction motion vector, and sample value of current processing target block And a sample value of a block in a reference frame at a displacement position corresponding to the second predicted motion vector, Based on the comparison result between the sum of absolute differences, a motion vector detection device according to claim 2, characterized in that determining the vector substitution.   The initial motion vector generation means has a smaller sum of absolute differences between the sample value of the current processing target block and the sample value of the block in the reference frame at the displacement position corresponding to the first and second predicted motion vectors. The motion vector detection apparatus according to claim 1, wherein the predicted motion vector is selected.   5. The motion according to claim 1, wherein the first predicted motion vector generating unit generates an average vector of a motion vector group obtained from the first context as a predicted motion vector. Vector detection device.   The second predicted motion vector generation means selects a motion vector having the largest difference from the average vector of the motion vector group obtained from the first context among the motion vector group obtained from the second context. The motion vector detection device according to claim 1, wherein the motion vector detection device is a feature.   The motion vector detection according to any one of claims 1 to 6, wherein the initial motion vector generation means further receives motion blur information of the apparatus and selects a motion vector predictor according to the motion blur information. apparatus. A motion vector detection method for dividing input video information into a plurality of blocks and detecting a motion vector for each block,
A first context is composed of a plurality of blocks in the vicinity of the current processing target block, and the motion vector of the current processing target block is predicted based on the detected motion vector group corresponding to each block, and the first A first motion vector predictor generating step for generating a motion vector predictor;
For the blocks near the current processing target block, a plurality of blocks having a number smaller than the number of blocks constituting the first context are selected to constitute the second context, and the already detected motion corresponding to each block A second predicted motion vector generating step of generating a second predicted motion vector by predicting a motion vector of the current processing target block based on a vector group;
An initial motion vector generating step of generating an initial motion vector used as an initial value for searching for a motion vector of the current processing target block by selecting one of the first and second predicted motion vectors;
A motion vector search step of searching and determining a motion vector of the current processing target block using the initial motion vector generated in the initial motion vector generation step as an initial value;
A motion vector detection method comprising:

Images