JP4139342B2 - Motion vector detection method, motion vector detection device, motion vector detection program, and computer-readable recording medium recording the program - Google Patents

Description

  The present invention records a motion vector detection method and apparatus for detecting a motion vector of an image block used for encoding a digital moving image, a motion vector detection program used for realizing the motion vector detection method, and the program. The present invention relates to a computer-readable recording medium.

  In a conventional moving image compression encoding method (MPEG-1, MPEG-2, MPEG-4, etc.), motion vector detection between moving image frames is performed.

In this motion vector detection, a block (hereinafter referred to as block B _i ) in a certain image frame (hereinafter referred to as image frame 1) is included in another image frame (hereinafter referred to as image frame 2). A block having the highest similarity among the blocks (hereinafter referred to as a block B ₀ ) is obtained, and a process for obtaining a coordinate variation between the block B _i and the block B ₀ is performed.

In order to obtain the block B ₀ , a certain block (hereinafter referred to as the block B _j ) is extracted from the image frame 2, the similarity with the block B _i is calculated, and it is determined whether or not the calculated value is the highest. The block matching process is repeated.

Generally, SAD (Sum of Absolute Difference) is used for the similarity. SAD, when the block B _i and the block B _j are both containing N pixels, the absolute difference between the pixel value J _a in the pixel values I _a and the block B _j in the block B _i | I _a -J _a | Thus, Σ _a | I _a −J _a | is calculated. Here sigma _a means to integration by changing the suffixes a to refer to all the pixels in the block. To obtain SAD once, N absolute differences and N-1 additions are required. When SAD is used for the similarity, the block B ₀ having the highest similarity is a block having the minimum SAD.

In order to obtain the block B ₀ using SAD, the block B _j is extracted from the image frame 2, the SAD of the block B _i and the block B _j (hereinafter referred to as SAD _j ) is calculated, and the provisional minimum value during the search is calculated. Compared to SAD _cand ,
If SAD _j = Σ _a | I _a −J _a | <SAD _cand (1), SAD _cand is replaced with SAD _j and the coordinates of block B _j are recorded. SAD _cand when this process is completed for all the blocks B _j becomes SAD _{min of the} minimum SAD value.

Usually, the pixel value is represented as 8-bit data (value range is 0 to 255). Therefore, the absolute difference | I _a −J _a | is obtained by an 8-bit computing unit, and the integrated value SAD _j is represented by the number of bits determined by the number N of pixels in the block.

  An SEA search method is known as a method for efficiently performing this motion vector detection (see, for example, Non-Patent Document 1).

In the SEA search method, adjacent pixels in an image frame are grouped together to form a rough pixel (coarse image), SAD is obtained from the pixel value, and this is used as a lower limit threshold value, and SAD calculation using a true pixel value is omitted. is there. Since the number of pixels in the block is reduced by making the pixels coarse, the amount of calculation for the SAD calculation with coarse pixel values is smaller than that with the true pixel values. Therefore, it is expected that the amount of calculation required for the entire SAD calculation can be reduced.
"W.Li and E. Sarari, Successive Elimination Algorithm for Motion Estimation, IEEE Trans. Image Proc., Vol.4, pp.105-107, Jan.1995"

As can be seen from the above, in motion vector detection, the SAD must be calculated many times to obtain the block B ₀ even for one block B _i . Furthermore, this process must be performed for all blocks in the image frame 1, and requires a huge number of SAD calculations, that is, operations of absolute difference and addition. It takes a lot of processing time to process such processing with a realistic amount of computing resources.

One way to shorten the processing time is to reduce the number of bits that represent the pixel values I _a and J _a , and increase the number of pixels that can be computed at once with a constant bit width computing unit, thereby improving processing efficiency. A method to plan is conceivable.

However, the number of bits of each of the pixel values I _a and J _a is reduced to values Y _a and Z _a, and this is used simply as in the expression (1).
Σ _a | Y _a −Z _a | ≦ X _min
Even if a block having the minimum SAD value “X _min ” is found according to the update equation, it is not always the block B ₀ having the true minimum SAD value “SAD _min ”.

  Therefore, an attempt to increase the processing efficiency by performing SAD calculation with a pixel value with a reduced number of bits will not work as it is. This also applies to the SEA search method.

  The present invention has been made in view of such circumstances, and an object of the present invention is to provide a new motion vector detection technique that realizes a reduction in the processing time for detecting a motion vector of an image block.

[1] First Configuration To achieve the above object, the motion vector detection device of the present invention calculates a SAD value between image blocks and performs a process of detecting a motion vector of the image block based on the SAD value. Then, (b) detecting a motion vector by dividing the pixel value of an image to be detected by a motion vector by a number of 1 or more and generating an image with the integer part of the quotient as a new pixel value. A conversion means for converting a target image into a reduced bit image; and (b) an extraction means for calculating a SAD value between image blocks with the reduced bit image as a processing target and extracting a minimum value among them. (C) Whether or not the image block is a target for calculating a true SAD value by evaluating the SAD value at the pixel value of the reduced bit image on the basis of the minimum SAD value extracted by the extracting means. Judgment hand to judge (D) The SAD value at the true pixel value is calculated between the image blocks, and the image block that is the target of the true SAD value calculation is calculated, and the minimum value among them is specified. specifying means, form configured to include a (e) based on the specified image block became calculated original true SAD value of the specific means, detecting means for detecting a motion vector of an image block, further, the determination The means is configured to determine whether or not the image block is a true SAD value calculation target based on “X _min ≦ X _j ≦ X _min + 2 × N”. Here, X _j represents the SAD value at the pixel value of the reduced bit image of the image block to be determined, X _min represents the minimum value of the SAD value at the pixel value of the reduced bit image, and N represents the value in the image block Represents the number of pixels.

  The motion vector detection method of the present invention realized by the operation of each of the above processing means can be realized by a computer program, and this computer program is recorded on an appropriate recording medium such as a semiconductor memory and provided. Or provided via a network, installed when implementing the present invention, and operated on a control means such as a CPU, thereby realizing the present invention.

  In the motion vector detection device of the present invention configured as described above, first, the pixel value of the image to be detected as a motion vector is divided by a number of 1 or more, and the integer part of the quotient is determined as a new pixel value. By generating an image to be converted, an image that is a motion vector detection target is converted into a reduced bit image.

  Subsequently, using the reduced bit image obtained in this way as a processing target, SAD values are calculated between the image blocks, and a minimum value among them is extracted.

  Subsequently, by evaluating the SAD value at the pixel value of the reduced bit image on the basis of the extracted minimum SAD value (obtained from the pixel value of the reduced bit image), the true SAD value can be calculated. It is determined whether or not it is an image block.

Specifically, the SAD value at the pixel value of the reduced bit image of the image block to be determined is X _j , the minimum value of the SAD value at the pixel value of the reduced bit image is X _min , and the number of pixels in the image block is If represented by N,
X _min ≦ X _j ≦ X _min + 2 × N
Based on the evaluation formula, it is determined whether or not the image block is a target for calculating a true SAD value.

  Subsequently, the SAD value at the true pixel value is calculated between the image blocks, with the image block that is the target of calculation of the true SAD value obtained as described above being processed, and the minimum value among them is calculated. Is identified.

  Finally, the motion vector of the image block is detected based on the image block from which the specified true SAD value is calculated.

  As described above, according to the present invention, as a method of detecting a motion vector between two image frames, calculation is performed mainly by reducing the number of bits representing a pixel value. It becomes possible to improve the processing efficiency in the process.

[2] Second Configuration In order to achieve the above object, the motion vector detection device of the present invention performs processing for detecting a motion vector of an image block based on the SEA search method. A conversion means for converting an image to be a motion vector detection target into a coarse image by combining adjacent pixels with an image to be detected as a vector as a conversion target; and (b) a pixel value of the coarse image of 1 or more A conversion means for converting a coarse image into a reduced bit image by dividing the number by a number and generating an image having the integer part of the quotient as a new pixel value; Calculating means for calculating the SAD value between them, (d) the SAD value calculated by the calculating means, the numerical value used for the division to obtain the reduced bit image, the number of pixels in the block in the coarse image, and the true SAD value And provisional minimum And (e) an image block that cannot omit the calculation of the SAD value at the true pixel value. As a processing target, calculate a SAD value with a true pixel value, determine whether or not to update the provisional minimum value of the true SAD value accordingly, and update it if necessary. And (f) detecting means for detecting a motion vector of the image block based on the image block from which the provisional minimum value of the true SAD value is determined as the minimum value. form structure as further determining means, based on the _{"SAD cand <Q × XC j +} Q × NC ", whether or not an image block can be omitted to calculate the SAD value of a true pixel value determination To be configured. Here, XC _j represents the SAD value at the pixel value of the reduced bit image, Q represents the numerical value used for the division, NC represents the number of pixels in the block in the coarse image, and SAD _cand is a provisional value of the true SAD value. Represents the minimum value.

  The motion vector detection method of the present invention realized by the operation of each of the above processing means can be realized by a computer program, and this computer program is recorded on an appropriate recording medium such as a semiconductor memory and provided. Or provided via a network, installed when implementing the present invention, and operated on a control means such as a CPU, thereby realizing the present invention.

  In the motion vector detection device of the present invention configured as described above, first, an image that is a motion vector detection target is first set as a conversion target, and adjacent pixels are grouped together to roughly coarsen an image that is a motion vector detection target. Convert to image.

  Subsequently, by dividing the pixel value of the coarse image obtained in this way by a number of 1 or more and generating an image with the integer part of the quotient as a new pixel value, the coarse image is converted into a reduced bit image. To do.

  Subsequently, SAD values are calculated between image blocks using the reduced bit image obtained in this way as a processing target.

  Subsequently, the calculated SAD value (obtained from the pixel value of the reduced bit image), the numerical value used for division to obtain the reduced bit image, the number of pixels in the block in the coarse image, and the true SAD value Based on the provisional minimum value, it is determined whether or not the image block can omit the calculation of the SAD value with the true pixel value.

Specifically, the SAD value at the pixel value of the reduced bit image of the image block to be determined is XC _j , the numerical value used for division to obtain the reduced bit image is Q, and the number of pixels in the block in the coarse image is NC If the provisional minimum value of the true SAD value is represented by SAD _cand ,
SAD _cand <Q × XC _j + Q × NC
Based on the evaluation formula, it is determined whether or not the image block can calculate the SAD value with the true pixel value.

  Subsequently, the SAD value at the true pixel value is calculated for the image block in which the calculation of the SAD value at the true pixel value obtained in this way cannot be omitted, and the provisional of the true SAD value is correspondingly calculated. It is determined whether or not the minimum value is updated, and if it is necessary to update, it is updated.

  Finally, when the provisional minimum value of the true SAD value is determined as the minimum value, the motion vector of the image block is detected based on the image block that is the calculation source.

  Thus, according to the present invention, when performing a process of detecting a motion vector of an image block based on the SEA search method, a pixel value is used as a method of detecting a motion vector between two image frames. Since the calculation is performed mainly by reducing the number of bits to be expressed, it is possible to improve the processing efficiency with a calculation resource having a certain bit width.

[3] Theoretical basis of the present invention [3-1] Theoretical basis of the first configuration In the first configuration of the present invention, in order to solve the problem of the prior art, the pixel value obtained by reducing the number of bits is obtained. A method of deriving a relationship between the SAD value to be obtained and the SAD value obtained from the true pixel value and utilizing this is used to make the motion vector detection process more efficient.

First, pixel values I _a and J _a (both are non-negative integers) are respectively divided by a constant Q (a real number of 1 or more), and the integer parts are taken out to be U _a and V _a . Hereinafter, this method of reducing the number of bits will be referred to as decrementing by a constant Q.

With an appropriate constant Q, the values U _a and V _a can be expressed with fewer bits than the true pixel values I _a and J _a , respectively.

At this time, as will be described later,
_{| I a -J a | -Q ≦} Q × | U a -V a | ≦ | a -J a | + Q (2) expression holds.

When integrating the equation (2) in sigma _{a,
Σ a | I a -J a |} -Q × N ≦ Q × Σ a | U a -V a |
≦ Σ _a | I _a −J _a | + Q × N (3) is obtained. Here, N is the number of pixels in the block.

This expression (3) indicates that the SAD value “Σ _a | U _a −V _a |” obtained by subtracting a bit by a constant Q is Q times the true SAD value “Σ _a | I _a −J _a |”. Is used to give the upper and lower limits.

From this equation (3), a method of obtaining the block B ₀ using the SAD value after the reduced bit can be derived.

Assuming that the SAD value obtained by subtracting bits from the block B ₀ by a constant Q is X, as described later, X is in the range represented by the following equation (4). Here, X _min is the minimum value among the SAD values obtained by subtracting bits with a constant Q, and N is the number of pixels in the block.

X _min ≦ X ≦ X _min + 2 × N (4) In order to obtain the block B ₀ using this relationship, first, the block B _j is referred to from the image frame 2, and each of the block B _i and the block B _j is determined. The pixel value is decremented by a constant Q to obtain the SAD value “X _j ”, and X _j is recorded in the memory, and then compared with the provisional minimum value X _cand for the SAD value obtained by the decrement bit during the search
If X _j ≦ X _cand (5), X _cand is replaced with X _j .

This process, X _cand when finished for all the blocks B _j is the minimum SAD value obtained at reduced bit "X _min".

Next, the SAD value “X _j ” after bit reduction recorded in the memory is compared with this X _min for all the blocks B _j , and only the block B _j satisfying the expression (4) is true. A true SAD value is calculated using the pixel values I _a and J _a .

Then, a block B ₀ having a true minimum SAD value “SAD _min ” is obtained based on the equation (1).

  In this way, in the first configuration of the present invention, a relationship between the SAD value obtained by the pixel value obtained by subtracting the number of bits and the SAD value obtained by the true pixel value is derived and used. By doing so, a method of improving the efficiency of motion vector detection processing is adopted.

[3-2] Theoretical basis of the second configuration The second configuration of the present invention detects a motion vector using the SEA search method in order to solve the problems of the prior art.

  The SEA search method is a method in which adjacent pixels in an image frame are grouped together to form a coarse pixel (coarse image), SAD is obtained from the pixel value, and this is used as a lower limit threshold, and SAD calculation using a true pixel value is omitted. In the second configuration of the present invention, the basic configuration is to apply the SEA search method by reducing the bit of the coarse image.

The block BC ⁽¹⁾ _i from the coarse image frame 1 and the block BC ^{( 2)} _j from the coarse image frame 2 are referred to, and the SAD value is C _j, and the pixel value included in each block is reduced by a constant Q. If the bit and the SAD value calculated with the pixel value is XC _j , the SEA search method compares it with the provisional minimum value “SAD _cand ” of the true SAD value,
For blocks where SAD _cand <C _j (6) holds, SAD calculation with true pixel values can be omitted.

A similar conditional expression is considered for the SAD value “XC _j ” obtained by subtracting bits. Since the above equation (3) is an equation for determining an upper limit value between the SAD value obtained by subtracting bits and the original SAD value,
C _j ≦ Q × XC _j + Q × NC (7) is derived. Here, NC is the number of pixels in the block in the coarse image.

Then, from Equation (6) and Equation (7),
It can be seen that SAD _cand <Q × XC _j + Q × NC (8) is a conditional expression after the bit reduction.

  The second configuration of the present invention is to detect a motion vector by combining the SEA search method and the conversion to a reduced bit image. According to this equation (7), SAD calculation with a true pixel value is performed. The motion vector is detected using the SEA search method while determining whether the block can be omitted.

[3-3] Derivation of Expression (2) Next, Expression (2) is derived.

Divide the non-negative integers I _a and J _a by the constant Q, and let the integer parts be U _a and V _a , respectively.
U _a = I _a / Qd _i (A1-1)
V _a = J _a / Q−d _j (A1-2)
It expresses. Here, 0 ≦ d _i ≦ 1 and 0 ≦ d _j ≦ 1.

Then, from the minkowsky inequality theorem,
| I _a −J _a | = Q × | (U _a + d _i ) − (V _a + d _j ) |
≦ Q × | U _a −V _a | + Q × | d _i −d _j | (A1-3)
It becomes.

Since 0 ≦ d _i ≦ 1 and 0 ≦ d _j ≦ 1,
| D _i −d _j | ≦ 1 (A1-4)
Holds.

Therefore, from (A1-3) and (A1-4),
| I _a −J _a | ≦ Q × | U _a −V _a | + Q × | d _i −d _j |
≦ Q × | U _a −V _a | + Q (A1-5)
Holds.

From the triangle inequality theorem and (A1-4),
Q × | U _a −V _a | = | (I _a −J _a ) + Q × (d _j −d _i ) |
≦ | I _a −J _a | + Q × | d _j −d _i |
≦ | I _a −J _a | + Q (A1-6)
Also holds.

Therefore, from (A1-5) and (A1-6),
| I _a −J _a | −Q ≦ Q × | U _a −V _a | ≦ | I _a −J _a | + Q (A1-7)
The following equation (2) is derived (end of proof).

[3-4] Derivation of Expression (4) Next, when Expression (3) holds, the SAD value “X” obtained by subtracting the bit B ₀ having the true minimum SAD is expressed by Expression (4). Prove about meeting.

  First, formula (3) is re-displayed.

_{Σ a | I a -J a |} -Q × N ≦ Q × Σ a | U a -V a |
≦ Σ _a | I _a −J _a | + Q × N (3) Expression (3) is an expression for the SAD value “Σ _a | U _a −V _a |” obtained by subtracting bits from a certain block. The upper limit value and the lower limit value are given by using _a true SAD value “Σ _a | I _a −J _a |”. That is, the upper limit value is _{"Σ a | + Q × N} | I a -J a", the lower limit value is _{"Σ a | -Q × N |} I a -J a".

When the equation (3) is transformed,
Q × Σ _a | U _a −V _a | −Q × N ≦ Σ _a | I _a −J _a |
≦ Q × Σ _a | U _a −V _a | + Q × N (A2−1)
It becomes.

In contrast to equation (3), equation (A2-1) defines the true SAD value “Σ _a | I _a −J _a |”. The upper limit is “Q × Σ _a | U _a −V _a | + Q × N”, and the lower limit is “Q × Σ _a | U _a −V _a | −Q × N”.

Now, let X be the SAD value obtained by subtracting bits from the block B ₀ having the smallest true SAD value “SAD _min ”. Further, the minimum value among the SAD values obtained by subtracting bits is set to X _min .

First, it is assumed that SAD _min is larger than “Q × X _min + Q × N”. At this time, the true SAD value of the block to which X _min is given is “Q × X _min + Q × N” at the maximum, which is apparent from the equation (A2-2), and therefore has a value smaller than SAD _min . This is contrary to the assumption that SAD _min is minimal. Therefore, SAD _min must be equal to or less than “Q × X _min + Q × N”.

Furthermore, since X obtained for the block B ₀ can be seen from the equation (3), the upper limit value is “SAD _min + Q × N”.
Q × X ≦ SAD _min + Q × N ≦ Q × X _min + 2 × Q × N (A2-2)
Holds.

Also, for any block, since it is not possible SAD value obtained in reduced bits is less than X _min, obviously,
X _min ≦ X (A2-3)
It is.

Therefore, from (A2-2) and (A2-3),
X _min ≦ X ≦ X _min + 2 × N (A2-4)
It can be seen that the following equation (4) is satisfied (end of proof).

  As described above, the present invention mainly uses a calculation in which the number of bits representing a pixel value is reduced as a method for detecting a motion vector between two image frames. The processing efficiency can be improved.

  Hereinafter, the present invention will be described in detail according to embodiments.

  Here, in the embodiment described below, it is assumed that all blocks are ordered in each image frame and can be referred to by one reference variable. Further, it is assumed that the pixels in the block are also ordered, and can be referred to by one reference variable.

  First, an embodiment of the motion vector detection device 1 of the present invention according to the first configuration described above will be described.

  FIG. 1 shows a device configuration of a motion vector detection device 1 of the present invention according to the first configuration.

  As shown in this figure, when realizing the first configuration, the motion vector detection device 1 of the present invention stores two image frames, which are motion vector detection targets, an image frame 1 and an image frame 2. An image frame memory 10, a motion vector detection mechanism 11a that detects a motion vector between the image frame 1 and the image frame 2, and a reduced bit image memory 12 that stores a reduced bit image generated by the motion vector detection mechanism 11a. And a work memory 13 prepared for work, and a motion vector memory 14 for storing the motion vector of the detection result.

Here, memories M ⁽¹⁾ and M ⁽²⁾ described later correspond to the reduced bit image memory 12, a memory MX described later corresponds to the work memory 13, and a memory MV described later corresponds to the motion vector memory 14. Applicable.

  The motion vector detection mechanism 11a includes a reduced bit image generation unit that performs conversion into a reduced bit image, and a minimum reduced bit SAD that calculates an SAD value at the reduced bit pixel value and extracts a minimum one of them. Based on the value extraction unit, the true SAD value calculation necessity determination unit that determines whether the SAD value at the true pixel value needs to be calculated based on the extraction result, and the determination result A minimum true SAD value specifying unit for calculating a SAD value with a true pixel value and specifying a minimum one of them, and a motion vector detecting unit for detecting a motion vector based on the specifying result Prepare.

  2 and 3 show flowcharts executed by the motion vector detection mechanism 11a. Next, according to this flowchart, the motion vector detection process executed by the motion vector detection device 1 of the present invention configured as described above will be described in detail.

  First, the motion vector detection mechanism 11a initializes the variable i in step 10 and puts the number of pixels in the block into the variable N.

Subsequently, in step 11, the variable j is initialized. Further, the pixel values in the i-th block B ⁽¹⁾ _i of the image frame 1 are sequentially referred to, and the value obtained by decrementing by the constant Q is obtained and stored in the memory M ⁽¹⁾ sequentially. In addition, it puts a sufficiently large value to a variable X _min.

Subsequently, in step 12, the pixel values in the j-th block B ⁽²⁾ _j of the image frame 2 are sequentially referred to, and the value reduced by the constant Q is obtained and stored in the memory M ⁽²⁾ sequentially. deep.

Subsequently, in step 13, the pixel values after the reduced bit are sequentially read from the memory M ⁽¹⁾ and the memory M ⁽²⁾ , the SAD value “X _j ” after the reduced bit is calculated, and the jth of the memory MX is calculated. Remember it.

Subsequently, in step 14, when the provisional minimum value “X _min ” is compared with X _j and it is determined that “X _min > X _j ”, the process proceeds to step 15 where X _{min is set} to X _j . On the other hand, when it is determined that “X _min > X _j ” is not satisfied, this replacement process is omitted.

  Subsequently, in step 16, when the variable j is updated and it is determined that the updated value of j is smaller than the number of blocks of the image frame 2, since unprocessed blocks remain in the image frame 2, Return to 12.

In this way, by repeating the processing of step 12 to step 16, the minimum value “X _min ” in the SAD value “X _j ” after the reduced bit is obtained.

When it is determined in step 16 that the minimum value “X _min ” has been obtained by processing all the blocks of the image frame 2, the process proceeds to step 17 to initialize the variable j. Also, a sufficiently large value is set in the variable SAD _min . Further, the variable idx is initialized.

Subsequently, in step 18, the SAD value “X _j ” after the decrementing bit in the memory MX is referred to.

Subsequently, in step 19, the referred X _j is compared with the minimum value “X _min ”.
X _min ≦ X _j ≦ X _min + 2 × N
When it is determined whether or not this equation is satisfied, the processing of step 20 to step 22 to be described below is omitted and the processing proceeds to step 23.

On the other hand, when it is determined in step 19 that “X _min ≦ X _j ≦ X _min + 2 × N” is satisfied, it is determined that the block needs to calculate the SAD value at the true pixel value. Then, the process proceeds to step 20, and sequentially refers to the pixel values in the i-th block B ⁽¹⁾ _i of the image frame 1 and the pixel values in the j-th block B ⁽²⁾ _j of the image frame 2. Thus, the true SAD value “SAD _j ” is calculated.

Subsequently, in step 21, when the provisional minimum value “SAD _min ” is compared with SAD _j and it is determined that “SAD _min > SAD _j ”, the process proceeds to step 22 where SAD _{min is set} to SAD _j . At the same time, j is inserted into the variable idx. On the other hand, when it is determined that “SAD _min > SAD _j ” is not satisfied, this replacement process is omitted.

  Subsequently, in step 23, when the variable j is updated and it is determined that the updated value of j is smaller than the number of blocks of the image frame 2, since unprocessed blocks remain in the image frame 2, step 18 Return to.

In this way, by repeating the processing from step 18 to step 23, the minimum value “SAD _min ” in the true SAD value “SAD _j ” is obtained.

If it is determined in step 23 that the minimum value “SAD _min ” of the true SAD value has been obtained by processing all the blocks of the image frame 2, the process proceeds to step 24, and the block B in the image frame 1 is determined. ⁽¹⁾ A variation between the coordinates of _i and the idx-th block B ⁽²⁾ _idx in the image frame 2 is obtained and stored in the i-th address of the memory MV.

In step 24, a motion vector for block B ⁽¹⁾ _i is obtained.

  Subsequently, in step 25, when the variable i is updated and it is determined that the updated value of i is smaller than the number of blocks of the image frame 1, since unprocessed blocks remain in the image frame 1, Return to 11.

  In this way, the motion vector detection apparatus 1 according to the present invention detects a motion vector between two image frames based on a calculation in which the number of bits representing a pixel value is reduced according to the flowcharts of FIGS. Therefore, it is possible to improve the processing efficiency with an operation resource having a constant bit width.

  Next, an embodiment of the motion vector detection device 1 of the present invention according to the second configuration described above will be described.

  In order to follow the second configuration described above, the motion vector detection device 1 of the present invention performs a process of detecting a motion vector using the SEA search method.

  In the SEA search method, adjacent pixels in an image frame are grouped together to form a coarse pixel (coarse image), SAD is obtained from the pixel value, and this is used as a lower limit threshold value, and SAD calculation using a true pixel value is omitted. is there. Since the number of pixels in the block is reduced by making the pixels coarse, the amount of calculation for the SAD calculation with coarse pixel values is smaller than that with the true pixel values. Therefore, it is expected that the amount of calculation required for the entire SAD calculation can be reduced.

For example, two pixels (pixel values a ₁ and a ₂ ) of the image frame 1 are combined into one pixel (pixel value a ₁ + a ₂ ) to create a coarse image frame 1. It is assumed that the rough image frame 2 is created by combining two pixels (pixel values b ₁ and b ₂ ) into one pixel (pixel value b ₁ + b ₂ ). At this time, the number of pixels NC in the block in the coarse image frame is a half of the number N of pixels in the block in the original image frame.

Now, assuming that the SAD value is obtained between certain blocks for the coarse image frame 1 and the coarse image frame 2, the minkowsky inequality | (a ₁ + a ₂ ) − (b ₁ + b ₂ ) | ≦ | a ₁ −b ₁ | + | a ₂ −b ₂ |
Therefore, the lower limit is given to the true SAD value between the corresponding blocks of the original image frame 1 and the image frame 2.

Therefore, if the provisional minimum value of the true SAD value being searched is SAD _cand and the SAD value calculated between certain blocks of coarse image frame 1 and coarse image frame 2 is C,
SAD _cand <C
If so, it is not necessary to calculate a true SAD value for this block.

  This is the basic principle of the SEA search method. As can be seen from this, the SEA search method mainly performs SAD calculation using the pixel values of the coarse image frame. Since these pixel values are obtained by adding the pixel values of the original image frame, the number of bits used for the expression is larger than that of the original pixel values. Therefore, the SAD calculation in the SEA search method requires a calculation resource having a larger bit width than the SAD calculation with a true pixel value.

  Therefore, in the motion vector detection device 1 according to the second configuration according to the second configuration, the coarse image frame is subtracted to improve the efficiency of the SAD calculation required in the SEA search method.

The block BC ⁽¹⁾ _i from the coarse image frame 1 and the block BC ^{( 2)} _j from the coarse image frame 2 are referred to, and the SAD value is C _j, and the pixel value included in each block is reduced by a constant Q. If the bit and the SAD value calculated with the pixel value is XC _j , the SEA search method compares it with the provisional minimum value “SAD _cand ” of the true SAD value,
For blocks where SAD _cand <C _j (6) holds, SAD calculation with true pixel values can be omitted.

A similar conditional expression is considered for the SAD value “XC _j ” obtained by subtracting bits. Since the above equation (3) is an equation for determining an upper limit value between the SAD value obtained by subtracting bits and the original SAD value,
C _j ≦ Q × XC _j + Q × NC (7) is derived. Here, NC is the number of pixels in the block in the coarse image.

Then, from Equation (6) and Equation (7),
It can be seen that SAD _cand <Q × XC _j + Q × NC (8) is a conditional expression after the bit reduction.

  The motion vector detection apparatus 1 of the present invention according to the second configuration uses the SEA search method while determining whether or not the block can omit the SAD calculation with the true pixel value according to the equation (7). Thus, the motion vector is detected.

  FIG. 4 illustrates a device configuration of the motion vector detection device 1 of the present invention according to the second configuration.

  As shown in this figure, when realizing the second configuration, the motion vector detection device 1 of the present invention stores two image frames, which are motion vector detection targets, an image frame 1 and an image frame 2. An image frame memory 10, a motion vector detection mechanism 11b for detecting a motion vector between the image frame 1 and the image frame 2, a coarse image frame memory 15 for storing a coarse image generated by the motion vector detection mechanism 11b, and A reduced bit image memory 12 for storing a reduced bit image generated by the motion vector detection mechanism 11b, a working memory 13 prepared for work, and a motion vector memory 14 for storing a motion vector of a detection result. Prepare.

Here, memories MC ⁽¹⁾ and MC ⁽²⁾ to be described later correspond to the reduced bit image memory 12, and a memory MV to be described later corresponds to the motion vector memory 14.

  The motion vector detection mechanism 11b includes a coarse image generation unit that performs conversion to a coarse image, a reduced bit image generation unit that performs conversion to a reduced bit image, and a reduction that calculates an SAD value with a reduced bit pixel value. A true SAD value that determines whether or not the block needs to calculate the SAD value at the true pixel value based on the bit SAD value calculation unit, the calculation result, and the provisional minimum value of the true SAD value A calculation necessity determination unit, a true SAD provisional minimum value update unit that calculates a SAD value at a true pixel value based on the determination result, and updates a provisional minimum value of the true SAD value accordingly, And a motion vector detection unit that detects a motion vector based on the provisional minimum value of the true SAD value.

  5 and 6 show flowcharts executed by the motion vector detection mechanism 11b. Next, according to this flowchart, the motion vector detection process executed by the motion vector detection device 1 of the present invention configured as described above will be described in detail.

  First, in step 30, the motion vector detection mechanism 11b converts the image frame 1 and the image frame 2 into the coarse image frame 1 and the coarse image frame 2, respectively, and stores them in the coarse image frame memory 15. Further, the number of pixels in the block in the original image frame is entered in the variable N, and the number of pixels in the block in the coarse image frame is entered in the variable NC.

  Subsequently, in step 31, the variable i is initialized.

Subsequently, in step 32, the variable j is initialized. Then, the pixel values in the i-th block BC ⁽¹⁾ _i of the coarse image frame 1 are sequentially referred to, and the values are decremented by a constant Q and sequentially stored in the memory MC ⁽¹⁾ . Further, a sufficiently large value is _entered into the variable SAD _cand indicating the provisional minimum value of the true SAD value, and the variable idx is initialized.

Subsequently, in step 33, the pixel values in the j-th block BC ⁽²⁾ _j of the coarse image frame 2 are sequentially referred to, and the values are decremented by a constant Q and sequentially stored in the memory MC ^(2). .

Subsequently, in step 34, the pixel values after the reduced bit are sequentially read from the memory MC ⁽¹⁾ and the memory MC ^(2), and the SAD value “XC _j ” after the reduced bit is calculated.

Subsequently, in step 35, based on the calculated XC _j , SAD _cand , Q and NC,
SAD _cand <Q × XC _j + Q × NC
When it is determined whether or not the above equation is established, and it is determined that this equation is established, the processing of step 36 to step 38 described below is omitted, and the process proceeds to step 39.

On the other hand, when it is determined in step 35 that “SAD _cand <Q × XC _j + Q × NC” does not hold, it is determined that the block needs to calculate the SAD value at the true pixel value, Proceeding to step 36, the pixel values in the i-th block B ⁽¹⁾ _i of the image frame 1 and the pixel values in the j-th block B ⁽²⁾ _j of the image frame 2 are sequentially referred to. The true SAD value “SAD _j ” is calculated.

Subsequently, in step 37, when the provisional minimum value “SAD _cand ” is compared with SAD _j and it is determined that “SAD _cand > SAD _j ”, the _routine proceeds to step 38, where SAD _cand is _set to SAD _j . At the same time, j is entered in the variable idx, and when it is determined that “SAD _cand > SAD _j ” is not satisfied, this replacement process is omitted.

  Subsequently, in step 39, when the variable j is updated, and it is determined that the updated value of j is smaller than the number of blocks of the image frame 2, an unprocessed block remains in the image frame 2. Return to 33.

In this manner, the minimum value “SAD _min ” of the true SAD value is obtained by repeating the processing of step 33 to step 39.

If it is determined in step 39 that the minimum value “SAD _min ” of the true SAD value has been obtained by processing all the blocks of the image frame 2, the process proceeds to step 40 to block B in the image frame 1. ⁽¹⁾ A variation between the coordinates of _i and the idx-th block B ⁽²⁾ _idx in the image frame 2 is obtained and stored in the i-th address of the memory MV.

In step 40, a motion vector for the block B ⁽¹⁾ _i is obtained.

  Subsequently, in step 41, when the variable i is updated and it is determined that the updated value of i is smaller than the number of blocks of the image frame 1, an unprocessed block remains in the image frame 1, Return to 32.

  In this way, the motion vector detection device 1 according to the present invention performs a calculation by reducing the number of bits representing a pixel value when detecting a motion vector using the SEA search method according to the flowcharts of FIGS. 5 and 6. Since processing is performed so as to detect a motion vector amount between two image frames as a main subject, it is possible to improve processing efficiency with a calculation resource having a constant bit width.

  The motion vector detection device 1 according to the present invention is basically configured to calculate an SAD value for a reduced bit value. The calculation of the SAD value can be realized by hardware or can be calculated by software.

  FIG. 7 shows an example of a configuration for calculating the SAD value for the reduced pixel value using hardware.

As described above, a value obtained by subtracting each pixel value of the block B ⁽¹⁾ _i by the constant Q is stored in the memory MC ⁽¹⁾ , and a value obtained by subtracting each pixel value of the block B ⁽²⁾ _j is stored in the memory. Stored in MC ⁽²⁾ .

In the hardware shown in FIG. 7, the pixel value from the memory MC ⁽¹⁾ is given to the input terminal 1, the pixel value from the memory MC ⁽²⁾ is given to the input terminal 2, and the subtracter 100 And a difference signal between the input terminal 2 and the input terminal 2 are output to the terminal 3.

  In response to this, the sign determination unit 101 outputs the sign of the signal at the terminal 3 to the terminal 4, while the sign inverter 102 inverts the sign of the signal at the terminal 3 and outputs it to the terminal 5.

  In response to this, the selector 103 connects the terminal 3 and the output terminal 6 when the sign of the terminal 4 is positive, and the terminal 103 and the output terminal 6 when the sign of the terminal 4 shows negative. Connect between the two.

  In this subsequent stage, a cumulative adder having a hardware configuration is provided, and the SAD value for the reduced bit value is calculated according to this configuration.

  FIG. 8 shows an example of a configuration for calculating the SAD value for the reduced pixel value using software.

The case where the pixel values of the image frame 1 and the image frame 2 are both expressed by the number of bits greater than 8 is taken as an example. At this time, if the constant Q is appropriately set, the number of expression bits of each pixel value of the block B ⁽¹⁾ _i and the block B ⁽²⁾ _j can be reduced to 8. It is assumed that 8-bit data is stored in each of the memory M ⁽¹⁾ and the memory M ⁽²⁾ after being decremented by such a constant Q.

  Consider the use of parallel operation instructions (assembler instructions) possessed by many common CPUs installed in commercially available computers. For example, a certain CPU has an instruction called psadbw instruction, and can calculate SAD calculations for eight 8-bit pixel values at a time.

  If the present invention is applied, the general-purpose parallel operation instruction as described above can be used for multi-bit pixel values of 8 or more by subtracting bits.

FIG. 8 shows a configuration using this psadbw instruction.
(1) Load eight 8-bit pixel values from the Ath byte to the A + 7th byte counted from the start address M1 of the memory M ⁽¹⁾ into the register mm0.
(2) Eight 8-bit pixel values placed from the Ath byte to the A + 7th byte counted from the start address M2 of the memory M ⁽²⁾ are loaded into the register mm1.
(3) By the parallel operation instruction psadbw, the SAD value of eight pixels placed in the registers mm0 and mm1 is obtained in the register mm0.
According to the above procedure, the SAD value for the reduced pixel value is calculated.

  The present invention can be applied to an image encoding apparatus that encodes an image of a moving image. By applying the present invention, an encoding process that is faster than the conventional one can be realized.

It is an apparatus block diagram of the motion vector detection apparatus of this invention. It is a flowchart which a motion vector detection mechanism performs. It is a flowchart which a motion vector detection mechanism performs. It is an apparatus block diagram of the motion vector detection apparatus of this invention. It is a flowchart which a motion vector detection mechanism performs. It is a flowchart which a motion vector detection mechanism performs. It is a figure which shows an example of the hardware which implement | achieves calculation of a SAD value. It is a figure which shows an example of the software which implement | achieves calculation of a SAD value.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Memory for image frames 11 Motion vector detection mechanism 12 Memory for reduced bit images 13 Memory for work 14 Memory for motion vectors 15 Memory for coarse image frames

Claims (6)

A motion vector detection method for calculating a SAD value between image blocks and detecting a motion vector of the image block based on the SAD value,
A process of converting the image into a reduced bit image by dividing the pixel value of the image to be detected by the motion vector by a number of 1 or more and generating an image having the integer part of the quotient as a new pixel value; ,
A process of calculating SAD values between image blocks using the reduced bit image as a processing target, and extracting a minimum value among them;
Determining whether the image block is a true SAD value calculation target by evaluating the calculated SAD value with reference to the minimum SAD value;
The process of calculating the SAD value at the true pixel value between the image blocks with the image block that is the calculation target of the true SAD value as the processing target, and specifying the minimum value among them;
Based on the image block was the calculated original true SAD values specified above, Bei example a process of detecting a motion vector of an image block,
In the above determination process,
X _min ≦ X _j ≦ X _min + 2 × N
Where X _j : reduced bit image of the image block to be determined
SAD value at the pixel value of
X _min : Minimum SAD value at the pixel value of the reduced bit image
N: Number of pixels in the image block
Based on, and this determines whether or not an image block to be calculated subject to the true SAD value,
A feature motion vector detection method. A motion vector detection device for calculating a SAD value between image blocks and detecting a motion vector of the image block based on the SAD value,
Means for converting the image into a reduced bit image by dividing the pixel value of the image to be detected by the motion vector by one or more and generating an image having a new pixel value as an integer part of the quotient. ,
Means for calculating the SAD value between image blocks with the reduced bit image as a processing target, and extracting a minimum value among them;
Means for evaluating whether the image block is a true SAD value calculation target by evaluating the calculated SAD value with reference to the minimum SAD value;
Means for calculating the SAD value at the true pixel value between the image blocks, with the image block that is the calculation target of the true SAD value as a processing target, and specifying the minimum value among them;
Based on the image block was the calculated original true SAD values specified above, Bei example a means for detecting a motion vector of an image block,
The determination means is
X _min ≦ X _j ≦ X _min + 2 × N
Where X _j : reduced bit image of the image block to be determined
SAD value at the pixel value of
X _min : Minimum SAD value at the pixel value of the reduced bit image
N: Number of pixels in the image block
Based on, and this determines whether or not an image block to be calculated subject to the true SAD value,
A featured motion vector detection device. A motion vector detection method for detecting a motion vector of an image block based on a SEA search method,
A process for converting an image into a coarse image by combining adjacent pixels with an image to be detected as a motion vector as a conversion target;
Dividing the coarse image into a reduced bit image by dividing the pixel value of the coarse image by a number equal to or greater than 1 and generating an image with the integer part of the quotient as a new pixel value;
A process of calculating SAD values between image blocks using the reduced bit image as a processing target;
Calculation of the SAD value at the true pixel value is omitted based on the calculated SAD value, the numerical value used for the division, the number of pixels in the block in the coarse image, and the provisional minimum value of the true SAD value. A process of determining whether or not the image block can be performed,
If it is necessary to update the SAD value at the true pixel value for the image block that cannot be omitted, and determine whether or not to update the provisional minimum value accordingly, The process of updating
When the temporary minimum value is determined as the minimum value, based on the image block from which it calculates source Bei example a process of detecting a motion vector of an image block,
In the above determination process,
SAD _cand <Q × XC _j + Q × NC
XC _j : SAD value at the pixel value of the reduced bit image
Q: Numerical value used for division
NC: Number of pixels in block in coarse image
SAD _cand : provisional minimum value of true SAD value
Based on, and this determines whether or not an image block can be omitted to calculate the SAD value of the true pixel values,
A feature motion vector detection method. A motion vector detection device that detects a motion vector of an image block based on a SEA search method,
Means for converting an image into a coarse image by grouping adjacent pixels with an image to be detected as a motion vector as a conversion target;
Means for converting the coarse image into a reduced bit image by dividing the pixel value of the coarse image by a number of 1 or more and generating an image having the integer part of the quotient as a new pixel value;
Means for calculating the SAD value between image blocks with the reduced bit image as a processing target;
Calculation of the SAD value with the true pixel value is omitted based on the calculated SAD value, the numerical value used for the division, the number of pixels in the block in the coarse image, and the provisional minimum value of the true SAD value. Means for determining whether the image block is a possible image block;
If it is necessary to update the SAD value at the true pixel value for the image block that cannot be omitted, and determine whether or not to update the provisional minimum value accordingly, A means of updating
When the temporary minimum value is determined as the minimum value, based on the image block from which it calculates source Bei example a means for detecting a motion vector of an image block,
The determination means is
SAD _cand <Q × XC _j + Q × NC
XC _j : SAD value at the pixel value of the reduced bit image
Q: Numerical value used for division
NC: Number of pixels in block in coarse image
SAD _cand : provisional minimum value of true SAD value
Based on, and this determines whether or not an image block can be omitted to calculate the SAD value of the true pixel values,
A featured motion vector detection device. A motion vector detection program for causing a computer to execute processing used to realize the motion vector detection method according to claim 1 . A computer-readable recording medium on which a motion vector detection program for causing a computer to execute processing used to realize the motion vector detection method according to claim 1 or 3 is recorded.

Images