JP2006180014A - Apparatus and method of motion compensation prediction coding - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus and a method of motion compensation prediction coding which enable selection of the optimum block size while reducing a total arithmetic operation amount. <P>SOLUTION: In selecting either a first block or a second block obtained by dividing the first block into a plurality of blocks as a motion compensation prediction block, an estimated motion vector operating section 21 operates each of motion vectors of a plurality of second blocks at a first repetition of a searching algorithm or a less repetition not more than the maximum number of repetitions. If the motion vectors are equivalent, a first determining section 22 selects the first block. On the contrary, if the motion vectors are not equivalent, an SB searching section 25 operates each of motion vectors of the second block at the number of repetitions of a searching algorithm. A second determining section 26 determines the equivalence of the motion vector, selects the first block when it is equivalent, and selects the second block if it is not equivalent. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a motion compensation prediction encoding apparatus and a motion compensation prediction encoding method, and more particularly, to a motion compensation prediction encoding apparatus and a motion compensation prediction encoding method that can be used by selecting a size of a motion compensation prediction block.

  The moving image we see is just an afterimage of a number of frame images (still images) that are continuous on the time axis. For smooth motion without jerky feeling, the next frame image must be displayed before the afterimage effect is lost, which requires, for example, 30 or more frame images per second. Such a moving image composed of a large number of frame images has an enormous amount of information and presses the capacity of the storage medium and the bandwidth of the transmission path.

  Therefore, “motion compensated prediction coding” is used as a technique for reducing (compressing) the information amount of moving images.

  FIG. 16 is a simplified conceptual diagram of motion compensation predictive coding. In the left column, images (hereinafter referred to as original images) P1, P2, P3,..., P7 before encoding (before information amount reduction) are arranged. The order is from top to bottom along the time axis. In motion compensation predictive coding, at least one original image P1 (also referred to as an intra image) is sent as it is (compressed with uncompressed or still image coding technology) to the decoding side via a transmission path (or storage medium). However, with respect to the remaining original images P2, P3,..., P7, the “motion information” between the previous original image (reference image) that goes back in time to the past is extracted. .., P7 ′, which are the motion information, are sent to the decoding side via the transmission path. The amount of information of the encoded images P2 ′, P3 ′,..., P7 ′ is smaller than that of the original images P2, P3,. For this reason, the amount of information on the transmission path (or storage medium) is much reduced compared to the case of sending all original images.

  On the decoding side, the intra image (original image P1) is used as it is as the restored image P1 ″, but the other restored images P2 ′, P3 ′,. By repeating the procedure of reproducing an image with the previous restored image and “motion information”, the restored images P1 ″, P2 ″, P3 in the same arrangement as the original images P1, P2, P3,. “,..., P7” are obtained. In the description of the figure, the “previous” original image that goes back in time to the past is referred to as a “reference image”, but this is the case for transmission media. In the storage system, a future image on the time axis (one “after” original image) may be referred to as a “reference image”.

  FIG. 17 is a conceptual diagram of generation of a coded image in motion compensation predictive coding (see, for example, Patent Document 1). In this figure, in motion compensated prediction coding, an original image constituting a moving image is divided into small motion compensated prediction blocks (hereinafter simply referred to as “blocks”) such as 16 × 16 pixels, and temporally divided into blocks. An encoded image is created from a reference image located before (possibly later in the storage system).

  Specifically, for example, when it is assumed that the encoded image 1 is divided into 25 blocks from B1 to B25 (B is an abbreviation of blocks), first, a block (B1 having the same size as B1) is assumed. ′) Is extracted from the reference image 2, and a difference (residual signal) for each pixel between B1 ′ and B1 is obtained. Then, the residual signals are sequentially collected while shifting the position of B1 'on the reference image 2 in units of several pixels or 1 / m pixels, and B1' when the residual signal is minimized (see FIG. Then, the magnitude of the relative deviation between B1 ″) and B1 and the direction of deviation are detected with the vector quantity 3 (motion vector). By repeating this for the other blocks (B2 to B25) in the same manner, Finally, the motion vectors and residual signals of all the blocks (B1 to B25) can be obtained, and these motion vectors and residual signals are stored in the respective blocks (B1 to B25) on the decoding side. The encoded image 4 is sent out.

  On the decoding side, using the received motion vectors B1 to B25 of the encoded image 4, the block of the restored image corresponding to the target reference image is selected, and the remaining blocks and the remaining blocks B1 to B25 are selected. The difference signal is added for each pixel, B1 to B25 are reproduced, and the same restored image as the original image is reconstructed.

  Here, if the block size is reduced, more detailed motion prediction can be performed, so that there was only one type of “16 × 16” in the initial moving picture coding system (MPEG-2: Moving Picture Experts Group). There are two types of block sizes, “16 × 16” and “8 × 8” in the higher version (MPEG-4), and “16 × in the latest version (MPEG-4 / AVC or H.264 / AVC)”. 16 types, “16 × 8”, “8 × 16”, “8 × 8”, “8 × 4”, “4 × 8”, and “4 × 4”.

  As a result, a small block can be selected to improve the accuracy of motion prediction. On the other hand, a small block increases the amount of encoding processing, and increases the amount of transmission information, processing overhead, and power consumption. Therefore, it is inappropriate for small-motion images, and corresponds to the amount of motion. There is a need for a technique that can select an appropriately sized block.

  As a conventional technique capable of selecting a block having an appropriate size corresponding to the amount of motion, for example, a 16 × 16 motion vector detection unit, an 8 × 8 motion vector detection unit, and these two motion detection units can be used. In addition, a device including a vector selection unit that selects one of the motion compensation prediction results is known (see, for example, Patent Document 2).

  According to this, it is possible to select an appropriate one of the motion compensation prediction results obtained for each of the 16 × 16 block and the 8 × 8 block obtained by dividing the block into four. For this reason, when there is little movement, a large size “16 × 16 block” is selected to reduce the amount of transmission information, reduce power consumption, and reduce processing overhead. By selecting the “8 × 8 block” of the size, it is possible to perform more detailed motion prediction.

Special table 2003-533142 gazette JP 2000-102016 A

  However, since the motion compensation prediction is performed in each of the “16 × 16 block” and the “8 × 8 block” obtained by dividing the block into four in the above-described conventional technology, for example, the “full search method” When the prediction is performed, the total calculation amount reaches almost twice the calculation amount for one 16 × 16 block. This is because the amount of calculation increases in proportion to the area of the block, and the amount of calculation for four 8 × 8 blocks is almost equal to the amount of calculation for one 16 × 16 block.

  Note that this (the calculation amount increases twice) is the same even when another search method (such as the nearest neighbor search method) having a smaller calculation amount than the “full search method” is used. For example, when the calculation amount for one 16 × 16 block when performing the nearest neighbor search method is A (although this A is smaller than the full search method), the nearest neighbor search is similarly performed 8 × 8 This is because the calculation amount for the four blocks is almost equal to A, so the total calculation amount is also doubled (≈2A).

  Accordingly, an object of the present invention is to provide a motion compensated prediction coding apparatus and a motion compensated prediction coding method capable of selecting an optimum block size while reducing the total amount of calculation.

The invention according to claim 1 is a motion compensated prediction encoding apparatus comprising block selection means for selecting one of a first block and a second block obtained by dividing the first block as a plurality of motion compensated prediction blocks. The block selection means includes first motion vector calculation means for calculating a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined number of iterations, and the first motion vector calculation And a first determination unit that determines whether or not each motion vector of the second block calculated by the unit is completely equivalent or approximate equivalent, and the first determination unit determines if the determination result of the first determination unit is affirmative While one block is selected as a motion compensated prediction block, if the determination result of the first determination means is negative, the second block is selected as a motion compensated prediction block. A motion compensated predictive coding apparatus characterized by selecting as.
In the present invention, when a motion vector of each of a plurality of second blocks is calculated using a predetermined search algorithm with a small number of iterations equal to or less than a predetermined number of iterations, and the motion vectors are completely equivalent or approximate equivalent. The first block is selected as the motion compensated prediction block, while the second block is selected as the motion compensated prediction block when their motion vectors are not completely equivalent or approximate equivalent. Here, since the second block is obtained by dividing the first block into a plurality of blocks, the first block is a “large motion compensation prediction block” and the second block is a “small motion compensation prediction block”.
The fact that the motion vectors of the plurality of second blocks (small motion compensation prediction blocks) are aligned (completely equivalent or approximately equivalent) means that the images are moving in the same direction and in the same amount. This means that this represents the motion of the image of the first block (large motion compensation prediction block). In this case, a large motion compensation prediction block, that is, the first block is selected for the purpose of reducing the amount of calculation. .
On the other hand, when the motion vectors of the plurality of second blocks (small motion compensation prediction blocks) are not aligned (not completely equivalent or approximate equivalent), the images are moving apart. In this case, in order to detect a fine motion vector, a small motion compensation prediction block, that is, a second block is selected.
The time required for “block selection” according to the present invention is given solely by the calculation time of each motion vector of the plurality of second blocks, and this calculation is performed with a small number of iterations less than the “predetermined number of iterations” of the search algorithm. Therefore, the time required for the search algorithm is significantly shorter than when the search algorithm is performed with a predetermined number of iterations.
Therefore, it is possible to provide a motion compensated prediction encoding apparatus that can select an optimum block size while reducing the total amount of calculation.
According to a second aspect of the present invention, there is provided a motion compensated prediction encoding apparatus including block selection means for selecting any one of a first block and a second block obtained by dividing the first block as a plurality of motion compensated prediction blocks. The block selecting means includes first motion vector computing means for computing a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined first number of iterations; First determination means for determining whether or not each motion vector of the second block calculated by the motion vector calculation means is completely equivalent or approximate equivalent, and when the determination result of the first determination means is affirmative A first selection unit that selects the first block as a motion compensated prediction block; and a motion vector of the first block when the determination result of the first determination unit is affirmative. Second motion vector computing means for computing a predetermined number of second iterations with a predetermined search algorithm, and when the judgment result of the first judging means is negative, Third motion vector calculation means for calculating each motion vector by a predetermined search algorithm up to a predetermined third number of iterations, and each of the second blocks calculated by the third motion vector calculation means Second determination means for determining whether or not the motion vector is completely equivalent or approximate equivalent; and when the determination result of the second determination means is affirmative, the first block is selected as a motion compensated prediction block, Motion compensated prediction, comprising: a second selection unit that selects the second block as a motion compensated prediction block when the determination result of the second determination unit is negative It is a Goka apparatus.
In the present invention, in addition to the invention of claim 1, when the motion vectors of each of the plurality of second blocks are not aligned (completely equivalent or not approximately equivalent), the motion vectors of each of the second blocks are Search again. This re-search is performed by an accurate calculation of a predetermined search algorithm with a predetermined third number of iterations as an upper limit, and thus has higher accuracy than the calculation result of the first motion vector calculation means.
For this reason, if the reverse determination result (equivalent) is obtained by the second determination means, it is determined that the determination result of the first determination means is wrong, and the large motion compensation prediction block, that is, the first When one block is selected and the same determination result (not equivalent) is obtained by the second determination unit, it is determined that the determination result of the first determination unit is correct, and a small motion compensated prediction block, that is, Select the second block.
Therefore, even if there is an error in the determination result of the first determination means, correct block selection can be performed, which is practically preferable.
According to a third aspect of the present invention, the predetermined search algorithm is a nearest neighbor search method, an N-step search method, or a logarithmic search method. It is.
According to a fourth aspect of the present invention, the second motion vector computing means and the third motion vector computing means start repeating from n ′ + 1 times when the first motion vector computing means is repeated n ′ times. The motion compensated predictive coding apparatus according to claim 2, wherein an intermediate value obtained by n 'iteration of the first motion vector computing means is used as an initial value of the iteration.
In this invention, since the intermediate value obtained by the calculation of the first motion vector calculation means is diverted by the second motion vector calculation means and the third motion vector calculation means, the second motion vector calculation means, The amount of processing can be reduced by reducing the number of repetitions of the third motion vector calculation means.
According to a fifth aspect of the present invention, when one of the first block and the second block obtained by dividing the first block is selected as a motion compensated prediction block, each motion vector of the plurality of second blocks is selected. Or a first motion vector calculation step that calculates a predetermined number of iterations with a predetermined number of iterations or less, and each motion vector of the second block calculated in the first motion vector calculation step is completely equivalent or A first determination step for determining whether or not they are approximately equivalent; and when the determination result of the first determination step is affirmative, the first block is selected as a motion compensated prediction block, while the determination of the first determination step When the result is negative, a motion compensation prediction encoding method is provided, wherein a selection step of selecting the second block as a motion compensation prediction block is executed.
According to a sixth aspect of the present invention, when any one of the first block and the second block obtained by dividing the first block is selected as a motion compensated prediction block, each motion vector of the plurality of second blocks is selected. Are calculated by a predetermined search algorithm with a small number of iterations less than or equal to a predetermined first number of iterations, and each motion vector of the second block calculated in the first motion vector calculation step is A first determination step of determining whether or not they are completely equivalent or approximate equivalent, and a first selection step of selecting the first block as a motion compensated prediction block when the determination result of the first determination step is affirmative; When the determination result of the first determination step is affirmative, the motion vector of the first block is expressed by a predetermined search algorithm with an upper limit of a predetermined second iteration number. A predetermined search with a predetermined third iteration number as an upper limit for each motion vector of the plurality of second blocks when the determination result of the second motion vector calculation step and the first determination step is negative A third motion vector calculation step calculated by an algorithm, and a second determination step of determining whether or not each motion vector of the second block calculated in the third motion vector calculation step is completely equivalent or approximate equivalent When the determination result of the second determination step is affirmative, the first block is selected as a motion compensated prediction block, while when the determination result of the second determination step is negative, the second block is selected as a motion compensated prediction block. And a second selection step of selecting as a motion compensated predictive coding method.
The invention according to claim 7 is characterized in that the predetermined search algorithm is a nearest neighbor search method, an N-step search method or a logarithmic search method. It is.
In the invention according to claim 8, the second motion vector calculation step and the third motion vector calculation step start repeating from n ′ + 1 times when the first motion vector calculation step is repeated n ′ times. The motion compensated predictive coding method according to claim 6, wherein an intermediate value obtained by n 'iteration of the first motion vector calculation step is used as an initial value of the iteration.

According to the present invention, the time required for “block selection” is given solely by the motion vector calculation time for each of the plurality of second blocks. Since this calculation is performed with “a smaller number of iterations less than or equal to a predetermined number of iterations”, the time required for the search algorithm is clearly shorter than when the search algorithm is performed with a predetermined number of iterations.
Therefore, it is possible to provide a motion compensated prediction encoding apparatus that can select an optimum block size while reducing the total amount of calculation.
In addition, according to another invention, it is possible to back up the erroneous determination of the first determination means and perform correct block selection, which is practically preferable.
According to another invention, the intermediate value obtained by the calculation of the first motion vector calculation means is diverted by the second motion vector calculation means and the third motion vector calculation means. It is possible to reduce the repetition of the motion vector calculation means and the third motion vector calculation means, and to reduce the processing amount.

  Embodiments of the present invention will be described below with reference to the drawings. It should be noted that the specific details or examples in the following description and the illustrations of numerical values, character strings, and other symbols are only for reference in order to clarify the idea of the present invention, and the present invention may be used in whole or in part. Obviously, the idea of the invention is not limited. In addition, a well-known technique, a well-known procedure, a well-known architecture, a well-known circuit configuration, and the like (hereinafter, “well-known matter”) are not described in detail, but this is also to simplify the description. Not all or part of the matter is intentionally excluded. Such well-known matters are known to those skilled in the art at the time of filing of the present invention, and are naturally included in the following description.

<Motion compensation prediction block>
First, the motion compensation prediction block will be described.
FIG. 1 is a diagram illustrating types of motion compensation prediction blocks. (A) to (h) are seven types of motion-compensated prediction blocks each approved for use in MPEG-4 / AVC (or H.264 / AVC). Specifically, (a) is a motion compensated prediction block having a size of “16 × 16”, and (b) is a motion having a size of “16 × 8” obtained by dividing a motion compensated prediction block of “16 × 16” into two vertically. Compensated prediction block, (c) is a motion compensated prediction block having a size of “8 × 16” obtained by dividing a motion compensated prediction block of “16 × 16” into left and right parts, and (d) is also “16 × 16”. The motion compensation prediction block is a motion compensation prediction block having a size of “8 × 8”, which is obtained by dividing the motion compensation prediction block into four parts in the vertical and horizontal directions.

  (E) is a motion compensated prediction block of “8 × 8”, and (f) is a motion compensated prediction block of the size of “8 × 4” obtained by dividing the motion compensated prediction block of “8 × 8” into two vertically. , (G) is a motion compensated prediction block having a size of “4 × 8” obtained by dividing a motion compensated prediction block of “8 × 8” into left and right parts, and (h) is a motion compensated prediction block of “8 × 8”. Each block is a motion-compensated prediction block having a size of “4 × 4” obtained by dividing the block into four parts vertically and horizontally.

  The selection of the motion compensation prediction block is performed in the groups (a) to (d). That is, the maximum size “16 × 16”, the maximum size divided into two up and down “16 × 8”, the maximum size divided into two left and right “8 × 16”, and the maximum size divided into four up and down and left and right It is selected from “8 × 8”. Further, when the selected motion compensation prediction block is (d) (“8 × 8” obtained by dividing “16 × 16” into four parts vertically and horizontally), for each “8 × 8” block, ( It is hierarchized so that it can be selected from the groups e) to (h).

  In general, the motion compensated prediction block (“16 × 16”) in (a) is called “macroblock”, and the divided motion compensated prediction blocks in (b) to (h) are called “subblocks”.

  As described above, since there are many types of motion compensated prediction blocks and there are various combinations of block selections, in this embodiment, for simplicity of explanation, a “16 × 16” motion compensated prediction block (a), It is assumed that a motion compensated prediction block (d) of “8 × 8” having a size obtained by dividing into four is selected.

FIG. 2 is a diagram illustrating two motion compensated prediction blocks that are objects of convenient selection according to the present embodiment. In this figure, a “16 × 16” motion compensation prediction block and a “8 × 8” motion compensation prediction block having a size obtained by dividing the motion compensation prediction block into four are illustrated. Hereinafter, following the above general designation, a motion compensated prediction block of “16 × 16” is referred to as a “macroblock (abbreviation: MB)”, and a motion compensated prediction of “8 × 8” having a size obtained by dividing the block into four. The block is referred to as “sub-block (abbreviation: SB)”. However, this is merely an explanatory convenience for distinguishing both blocks, and the outer edge of the invention should not be grasped from these names (macro..., Sub...). Further, each of the four SBs is identified by adding subscripts 0, 1, 2, 3 in order from the upper left. That is, the upper left SB is identified as SB ₀ , the upper right SB as SB ₁ , the lower left SB as SB ₂ , and the lower right SB as SB ₃ .

<Configuration of Motion Compensated Predictive Encoding Device>
Next, the configuration of the motion compensated prediction encoding apparatus will be described.

  FIG. 3 is a block diagram of the motion compensated prediction encoding apparatus. The motion-compensated predictive coding apparatus 10 includes a subtractor 11 that takes a difference between a current frame (original image) in an input bitstream and a predicted image, and a finite series conversion equation (for example, a certain two-dimensional function) Two-dimensional discrete cosine transform (DCT) is displayed as a transform unit 12 for compression, a quantization unit 13 for quantizing the coefficients of the transform equation, and the quantized data and motion information and division mode to be described later are encoded. (For example, entropy encoding) and outputting the encoded bit stream to a transmission medium or a storage medium; an inverse quantization unit 15 for inversely quantizing the quantized data; An inverse transformation unit 16 that inversely transforms data to return to the original difference, an addition unit 17 that adds the difference and the predicted image to reproduce the restored image, a restored image memory 18 that holds the restored image, A motion estimation unit 19 (block selection) that estimates motion information (such as a motion vector) and a block division mode based on the current original image and the previous restored image (reference image) held in the restored image memory 18 Means) and a motion compensation unit 20 that generates a predicted image based on the motion information.

<Configuration and operation of motion estimation unit>
FIG. 4 is a conceptual block diagram of the motion estimator 19. The motion estimation unit 19 includes an estimated motion vector calculation unit 21 (first motion vector calculation unit), a first determination unit 22 (first determination unit), an MB (macroblock) search unit 23 (second motion vector calculation unit), First MB motion vector output unit 24 (first selection unit), SB (sub-block) search unit 25 (third motion vector calculation unit), second determination unit 26 (second determination unit), second MB motion vector output A unit 27 (second selection unit) and an SB motion vector output unit 28 (second selection unit).

The estimated motion vector calculation unit 21 performs, for each of SB _{0 to} SB ₃ (see FIG. 2), a first iteration of a search algorithm (for example, a fast search algorithm such as nearest neighbor search method). Only a predetermined number of iterations or a smaller number of iterations equal to or less than a first predetermined number of iterations) is performed, and rough estimated motion vectors fv _{0 to} fv ₃ of SB ₀ to SB ₃ are calculated and output. Details regarding this calculation will be described later in an estimated motion vector calculation process (see FIGS. 10 and 11).

First determination unit 22 determines the equivalent estimated motion vector _{fv 0 ~fv 3 (fv 0 =} fv 1 = fv 2 = fv 3), if equivalent, start MB search unit 23, be equivalent Then, the SB search unit 25 is activated.

When the estimated motion vectors fv _{0 to} fv ₃ are equivalent (fv ₀ = fv ₁ = fv ₂ = fv ₃ ), the MB search unit 23 outputs a 16 × 16 division mode and, for example, a nearest neighbor search method or the like Thus, the motion vector mv of the MB is obtained, and the motion vector mv is output via the first MB motion vector output unit 24.

When the estimated motion vectors fv _{0 to} fv ₃ are not equivalent (fv ₀ = fv ₁ = fv ₂ = fv ₃ ), the SB search unit 25 performs SB ₀ , SB ₁ , SB ₂ and The respective motion vectors mv ₀ , mv ₁ , mv ₂ , mv ₃ of SB ₃ are obtained.

The second determination unit 26 calculates the equivalent (mv ₀ = mv ₁ = mv ₂ = mv ₃ ) of the motion vectors mv ₀ , mv ₁ , mv ₂ , and mv ₃ of SB ₀ , SB ₁ , SB _2, and SB _3. If it is determined that it is equivalent, the 16 × 16 division mode and one motion vector (mv ₀ ) are output via the second MB motion vector output unit 27, while if not equivalent, the SB motion vector output unit 28 is Via the 8 × 8 division mode and four motion vectors (mv ₀ , mv ₁ , mv ₂ , mv ₃ ).

5 and 6 are conceptual diagrams for explaining the operation of the motion estimation unit 19. First, as shown in FIG. 5, the estimated motion vectors (fv ₀ , fv ₁ , fv ₂ ) of the four sub-blocks (SB ₀ , SB ₁ , SB ₂ , SB ₃ ) calculated by the estimated motion vector calculator 21 are When fv ₃ ) is equivalent, the motion vectors (mv) of the macroblocks (MB) corresponding to these four sub-blocks are almost the same with high probability. That is, the estimated motion vectors (fv ₀ , fv ₁ , fv ₂ , fv ₃ ) of the four sub-blocks are equivalent (equal), so that the images in the four sub-blocks are in the same direction. In addition, the movement is the same amount, that is, it represents the movement of the image of the macroblock (MB) corresponding to those four sub-blocks. Therefore, in this case, a large division mode (for example, “16 × 16”) is selected (first MB motion vector output unit 24) from the viewpoint of reducing the amount of calculation, suppressing power consumption, and reducing processing overhead.

On the other hand, as shown in FIG. 6, the estimated motion vectors (fv ₀ , fv ₁ , fv ₂ ) of the four sub-blocks (SB ₀ , SB ₁ , SB ₂ , SB ₃ ) calculated by the estimated motion vector computing unit 21 If fv ₃ ) is not equivalent (uneven), it indicates that the images in those four sub-blocks are moving apart. However, this suggestion may not be correct. The estimated motion vector (fv ₀ , fv ₁ , fv ₂ , fv ₃ ) is only an “estimated value” of only one iteration (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined number of first iterations). Because it is not perfect. In the final determination, the SB search unit 25 obtains accurate motion vectors (mv ₀ , mv ₁ , mv ₂ , mv ₃ ) of the four sub-blocks (SB ₀ , SB ₁ , SB ₂ , SB ₃ ) and performs the second determination. The unit 26 determines the equivalence (mv ₀ = mv ₁ = mv ₂ = mv ₃ ). And even in the final judgment, if it is still not equivalent, the above “suggestion” was correct, so in order to detect the accurate motion for each sub-block, a small division mode (for example, four “8 × 8”) is used. Select (SB motion vector output unit 28). On the other hand, when the determination result of the second determination unit 26 is “not equivalent”, it is determined that the above “suggestion” is an error, and a large division mode (for example, “16 × 16” is intended to reduce the amount of calculation). ") Is selected (second MB motion vector output unit 27).

<Detailed operation of motion estimation unit>
Next, the operation of the motion estimation unit 19 will be described in more detail. In addition, although demonstrated in the form of a flowchart here, this has two implications. The first meaning is to help understanding by explaining the operations in order along the flow, and the second meaning is only an embodiment in which each function of the motion estimation unit 19 is configured by hardware logic. In addition, an embodiment realized by an organic combination of software and a computer designed along the flow of such a flowchart is not excluded.

First, terms used in the flowchart will be described.
FIG. 7 is a diagram for explaining the block position (coordinates) and block size of the original image. In this figure, each cell 30 of the original image 29 is a block (MB or SB), and the block position thereof is expressed by pixel coordinates (x, y) of the upper left corner of the cell 30, for example. To do. The block size is expressed as sx in the horizontal direction (row direction) and sy in the vertical direction (column direction).

  FIG. 8 is a diagram illustrating an offset given to the reference image. In this figure, it is assumed that the offset of the block 32 on the reference image 31 is represented by (ox, oy).

<Operation flow of motion estimation unit>
FIG. 9 is a diagram showing an operation flowchart of the motion estimator 19. When this flowchart is started, first, for each of SB _{0 to} SB ₃ (see FIG. 2), a first iteration (for example, a predetermined number of iterations or a predetermined number of iterations) Estimated motion vector arithmetic processing (step S10: first motion) in which only rough estimated motion vectors fv _{0 to} fv ₃ of SB ₀ to SB ₃ are calculated and output by performing only a small number of iterations equal to or less than the predetermined first number of iterations. Vector calculation step) is executed. Next, the expression “fv ₀ = fv ₁ = fv ₂ = fv ₃ ” is evaluated (step S11: first determination step), and it is determined whether or not the evaluation result is “True” (step S12). If the determination result is “True”, then the MB search process (step S13: second motion vector calculation step) is executed, which is larger than the MB motion vector (mv) obtained by the MB search process. After outputting the division mode (for example, “16 × 16”) (step S14: selection step, first selection step), the flowchart is ended.

On the other hand, if the determination result in step S12 is not “True”, an SB search process (step S15: third motion vector calculation step) is executed, and the expression “mv ₀ = mv ₁ = mv ₂ = mv ₃ ” is evaluated ( Step S16: second determination step), it is determined whether or not the evaluation result is “True” (Step S17). If the determination result is “True”, then an MB motion vector (mv ₀ ) and a large division mode (for example, “16 × 16”) are output (step S18: selection step, second selection step). ) To end the flowchart, and if the determination result is not “True”, an SB motion vector (mv _{0 to} mv ₃ ) and a small division mode (for example, “8 × 8”) are output (step S19: selection) Step, second selection step) and the flowchart is terminated.

<Estimated motion vector calculation processing>
10 and 11 are flowcharts showing the operation of the estimated motion vector calculation process. When this flowchart is started, first, initial values are respectively set in the block size variables sx and sy and the loop variable i (step S20). The initial value of i is “0”, and the initial values of sx and sy are “8” corresponding to the horizontal size and vertical size of SB.

First time (i = 0):
First, i = 0 is determined (step S21), and x = X and y = Y are set (step S22). X and Y represent the position coordinates of the SB _0. Therefore, at this time (i = 0), the processing for the upper left sub-block (SB ₀ ) in FIG. 2 is performed.

Next, using the SAD function, residuals evaluation of residual evaluation value A of the search center _{(x 0, y 0),} 1 pixel right search center _{(x 0, y 0) (} x 0 + 1, y 0) value B, the search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ + 1) residual evaluation value D and the residual evaluation value E one pixel below the search center (x ₀ , y ₀ ) (x ₀ , y ₀ −1) are calculated (steps S23 to S27). . The SAD function is a function for calculating a sum of absolute difference.

Note that the search center offset (x ₀ , y ₀ ) may be a zero vector or a prediction vector given by some method (for example, median prediction defined in H.264).

The SAD function is calculated for each of the residual evaluation values A to E. That is,
A = SAD (x, y, x ₀ , y ₀ , sx, sy)
B = SAD (x, y, x ₀ +1, y ₀ , sx, sy)
C = SAD (x, y, x ₀ -1, y ₀ , sx, sy)
D = SAD (x, y, x ₀ , y ₀ +1, sx, sy)
E = SAD (x, y, x ₀ , y ₀ −1, sx, sy)
Is calculated five times.

Since x = X, y = Y, sx = 8, and sy = 8,
A = SAD (X, Y, x 0, y 0, 8,8)
B = SAD (X, Y, x ₀ +1, y ₀ , 8, 8)
C = SAD (X, Y, x ₀ -1, y ₀ , 8, 8)
D = SAD (X, Y, x ₀ , y ₀ +1, _8, 8)
E = SAD (X, Y, x 0, y 0 -1,8,8)
It becomes. These expressions correspond to, for example, one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined first number of iterations).

  Next, using the Min function, the minimum values of A, B, C, D, and E are extracted (step S28), and the minimum values are stored in the variable SADmin.

  Next, it is determined which of A to E is the minimum value stored in SADmin (steps S29 to S32). If SADmin = A, (0,0) is set to fv (i) (step S33), and if SADmin = B, (1,0) is set to fv (i) (step S34). ), If SADmin = C, set (−1, 0) to fv (i) (step S35), and if SADmin = D, set (0, 1) to fv (i) (step S35). (S36) If SADmin = E, (0, −0) is set in fv (i) (step S37).

Since i = 0 now, this fv (i) is fv ₀ . Accordingly, at this stage, the estimated motion vector fv ₀ of the upper left sub-block (SB ₀ ) in FIG. 2 is obtained. The estimated motion vector fv ₀ is subjected to an operation corresponding to one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined first number of iterations). Although the accuracy is inferior to the value obtained by performing the algorithm the maximum number of iterations (predetermined number of iterations or predetermined number of first iterations), the approximate motion vector It can be used as a value for grasping (estimating) the tendency.

First loop (i = 1):
Next, the loop variable i is incremented by 1 (step S38), and it is determined whether or not i> 3 (step S29). Since i = 1, a loop is made, i = 1 is determined (step S40), and x = X + 8 and y = Y are set (step S41). X + 8 and Y represent the position coordinates of SB ₁ . Therefore, in this time (i = 1), the process for the upper right sub-block (SB ₁ ) in FIG. 2 is performed.

Next, using the SAD function, residuals evaluation of residual evaluation value A of the search center _{(x 0, y 0),} 1 pixel right search center _{(x 0, y 0) (} x 0 + 1, y 0) value B, the search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ + 1) residual evaluation value D and the residual evaluation value E one pixel below the search center (x ₀ , y ₀ ) (x ₀ , y ₀ −1) are calculated (steps S23 to S27). .

Similar to the first time, the SAD function is calculated for each of the residual evaluation values A to E. That is,
A = SAD (x, y, x ₀ , y ₀ , sx, sy)
B = SAD (x, y, x ₀ +1, y ₀ , sx, sy)
C = SAD (x, y, x ₀ -1, y ₀ , sx, sy)
D = SAD (x, y, x ₀ , y ₀ +1, sx, sy)
E = SAD (x, y, x ₀ , y ₀ −1, sx, sy)
Is calculated five times.

Since x = X + 8, y = Y, sx = 8, and sy = 8,
A = SAD (X + 8, Y, x ₀ , y ₀ , 8, 8)
B = SAD (X + 8, Y, x ₀ +1, y ₀ , 8, 8)
C = SAD (X + 8, Y, x 0 -1, y 0, 8,8)
D = SAD (X + 8, Y, x ₀ , y ₀ +1, 8, 8)
E = SAD (X + 8, Y, x ₀ , y ₀ −1, 8, 8)
It becomes. These expressions correspond to, for example, one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined first number of iterations).

  Next, using the Min function, the minimum values of A, B, C, D, and E are extracted (step S28), and the minimum values are stored in the variable SADmin.

  Next, it is determined which of A to E is the minimum value stored in SADmin (steps S29 to S32). If SADmin = A, (0,0) is set to fv (i) (step S33), and if SADmin = B, (1,0) is set to fv (i) (step S34). ), If SADmin = C, set (−1, 0) to fv (i) (step S35), and if SADmin = D, set (0, 1) to fv (i) (step S35). (S36) If SADmin = E, (0, −0) is set in fv (i) (step S37).

Since i = 1 at this time, fv (i) is fv ₁ . Therefore, at this stage, the estimated motion vector fv ₁ of the upper right sub-block (SB ₁ ) in FIG. 2 is obtained. Similarly to the estimated motion vector fv ₀ , the estimated motion vector fv _{1 is} also one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined iteration number or a predetermined first iteration number or less). It is obtained by performing an operation corresponding to a small number of iterations), and is compared with a value obtained by performing the same algorithm the maximum number of iterations (a predetermined number of iterations or a predetermined first number of iterations). Although it is inferior in accuracy, it can be used as a value for grasping (estimating) the tendency of an approximate motion vector.

Second loop (i = 2):
Next, the loop variable i is incremented by 1 (step S38), and it is determined whether or not i> 3 (step S29). Since i = 2, a loop is made, i = 2 is determined (step S42), and x = X and y = Y + 8 are set (step S43). X and Y + 8 represents the position coordinates of SB _2. Therefore, in this time (i = 2), the processing for the lower left sub-block (SB ₂ ) in FIG. 2 is performed.

Next, using the SAD function, residuals evaluation of residual evaluation value A of the search center _{(x 0, y 0),} 1 pixel right search center _{(x 0, y 0) (} x 0 + 1, y 0) value B, the search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ + 1) residual evaluation value D and the residual evaluation value E one pixel below the search center (x ₀ , y ₀ ) (x ₀ , y ₀ −1) are calculated (steps S23 to S27). .

Similar to the first and first loops, the SAD function is calculated for each of the residual evaluation values A to E. That is,
A = SAD (x, y, x ₀ , y ₀ , sx, sy)
B = SAD (x, y, x ₀ +1, y ₀ , sx, sy)
C = SAD (x, y, x ₀ -1, y ₀ , sx, sy)
D = SAD (x, y, x ₀ , y ₀ +1, sx, sy)
E = SAD (x, y, x ₀ , y ₀ −1, sx, sy)
Is calculated five times.

Since x = X, y = Y + 8, sx = 8, and sy = 8,
A = SAD (X, Y + 8, x ₀ , y ₀ , 8, 8)
B = SAD (X, Y + 8, x ₀ +1, y ₀ , 8, 8)
C = SAD (X, Y + 8, x ₀ -1, y ₀ , 8, 8)
D = SAD (X, Y + 8, x ₀ , y ₀ +1, 8, 8)
E = SAD (X, Y + 8, x 0, y 0 -1,8,8)
It becomes. These expressions correspond to, for example, one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined first number of iterations).

  Next, using the Min function, the minimum values of A, B, C, D, and E are extracted (step S28), and the minimum values are stored in the variable SADmin.

  Next, it is determined which of A to E is the minimum value stored in SADmin (steps S29 to S32). If SADmin = A, (0,0) is set to fv (i) (step S33), and if SADmin = B, (1,0) is set to fv (i) (step S34). ), If SADmin = C, set (−1, 0) to fv (i) (step S35), and if SADmin = D, set (0, 1) to fv (i) (step S35). (S36) If SADmin = E, (0, −0) is set in fv (i) (step S37).

Now, since it is i = 2, the fv (i) is a fv _2. Therefore, at this stage, the estimated motion vector fv ₂ of the lower left sub-block (SB ₂ ) in FIG. ₂ is obtained. Similarly to the estimated motion vectors fv ₀ and fv ₁ , this estimated motion vector fv ₂ is one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a predetermined first iteration). Obtained by performing an operation corresponding to the number of iterations (less than a few iterations), and obtained by performing the same maximum number of iterations (a predetermined number of iterations or a predetermined number of first iterations). Although the accuracy is inferior to the value, it can be used as a value for grasping (estimating) the tendency of a general motion vector.

Third loop (i = 3):
Next, the loop variable i is incremented by 1 (step S38), and it is determined whether or not i> 3 (step S29). Since i = 3, a loop is performed, i = 3 is determined (“NO” in step S42), and x = X + 8 and y = Y + 8 are set (step S44). X + 8 and Y + 8 represent the coordinates of SB ₃ . Therefore, in this time (i = 3), the process for the lower left sub-block (SB ₃ ) in FIG. 2 is performed.

Next, using the SAD function, residuals evaluation of residual evaluation value A of the search center _{(x 0, y 0),} 1 pixel right search center _{(x 0, y 0) (} x 0 + 1, y 0) value B, the search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ + 1) residual evaluation value D and the residual evaluation value E one pixel below the search center (x ₀ , y ₀ ) (x ₀ , y ₀ −1) are calculated (steps S23 to S27). .

Similar to the first to second loops, the SAD function is calculated for each of the residual evaluation values A to E. That is,
A = SAD (x, y, x ₀ , y ₀ , sx, sy)
B = SAD (x, y, x ₀ +1, y ₀ , sx, sy)
C = SAD (x, y, x ₀ -1, y ₀ , sx, sy)
D = SAD (x, y, x ₀ , y ₀ +1, sx, sy)
E = SAD (x, y, x ₀ , y ₀ −1, sx, sy)
Is calculated five times.

Since x = X + 8, y = Y + 8, sx = 8, and sy = 8,
A = SAD (X + 8, Y + 8, x ₀ , y ₀ , 8, 8)
B = SAD (X + 8, Y + 8, x ₀ +1, y ₀ , 8, 8)
C = SAD (X + 8, Y + 8, x ₀ -1, y ₀ , 8, 8)
D = SAD (X + 8, Y + 8, x ₀ , y ₀ +1, 8, 8)
E = SAD (X + 8, Y + 8, x ₀ , y ₀ −1, 8, 8)
It becomes. These expressions correspond to, for example, one iteration of a fast search algorithm such as the nearest neighbor search method (a predetermined number of iterations or a smaller number of iterations equal to or less than a predetermined first number of iterations).

  Next, using the Min function, the minimum values of A, B, C, D, and E are extracted (step S28), and the minimum values are stored in the variable SADmin.

  Next, it is determined which of A to E is the minimum value stored in SADmin (steps S29 to S32). If SADmin = A, (0,0) is set to fv (i) (step S33), and if SADmin = B, (1,0) is set to fv (i) (step S34). ), If SADmin = C, set (−1, 0) to fv (i) (step S35), and if SADmin = D, set (0, 1) to fv (i) (step S35). (S36) If SADmin = E, (0, −0) is set in fv (i) (step S37).

Now, since i = a 3, the fv (i) is a fv _3. Therefore, at this stage, the estimated motion vector fv _{3 of the} lower right sub-block (SB ₃ ) in FIG. 2 is obtained. Similarly to the estimated motion vectors fv ₀ , fv ₁ , and fv ₂ , this estimated motion vector fv _{3 is} also one iteration of a fast search algorithm such as nearest neighbor search (a predetermined number of iterations or a predetermined number of iterations). Obtained by performing an operation corresponding to the number of iterations equal to or less than one iteration), and performing the maximum number of iterations (predetermined number of iterations or predetermined number of first iterations). Although the accuracy is inferior to the obtained value, it can be used as a value for grasping (estimating) the tendency of an approximate motion vector.

  Next, the loop variable i is incremented by 1 (step S38), and it is determined whether i> 3 (step S29). Since i = 4, the process exits the loop and ends the flowchart.

According to the estimated motion vector calculation processing, the respective rough estimate motion vectors fv ₀ ~fv ₃ of the four sub-blocks in FIG. 2 (SB ₀ ~SB ₃₎ is obtained.

That is, the residual evaluation value A of the searching center of the SB ₀ (x _0, y _0), the search center (x _0, y ₀₎ residual evaluation value of one pixel right _{(x 0 + 1, y 0} ) B, search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ +1) Among the residual evaluation value D and the residual evaluation value E one pixel below (x ₀ , y ₀ −1) of the search center (x ₀ , y ₀ ), the smallest one is defined as “fv ₀ ”.
Also, the search center of SB ₁ (x _0, y ₀₎ residual evaluation value A, the search center (x _0, y ₀₎ residual evaluation value of one pixel right _{(x 0 + 1, y 0} ) B, search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ +1) The smallest one of the residual evaluation value D and the residual evaluation value E one pixel below (x ₀ , y ₀ −1) of the search center (x ₀ , y ₀ ) is defined as “fv ₁ ”.
Also, the search center of SB ₂ (x _0, y ₀₎ residual evaluation value A, the search center (x _0, y ₀₎ residual evaluation value of one pixel right _{(x 0 + 1, y 0} ) B, search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ +1) The smallest one of the residual evaluation value D and the residual evaluation value E one pixel below (x ₀ , y ₀ −1) of the search center (x ₀ , y ₀ ) is defined as “fv ₂ ”.
Also, the search center of SB ₃ (x _0, y ₀₎ residual evaluation value A, the search center (x _0, y ₀₎ residual evaluation value of one pixel right _{(x 0 + 1, y 0} ) B, search center (x _0, y ₀₎ 1 pixel left (x ₀ -1, y ₀₎ of the residual evaluation value C for the search center (x _0, y ₀₎ of 1 on the pixel (x _0, y ₀ +1) Of the residual evaluation value D and the residual evaluation value E one pixel below (x ₀ , y ₀ −1) of the search center (x ₀ , y ₀ ), the smallest one is obtained as “fv ₃ ”. It is done.

These estimated motion vectors fv _{0 to} fv ₃ are obtained by performing five SAD operations on each of SB _{0 to} SB ₃ , and this calculation is performed by a fast search algorithm such as the nearest neighbor search method. Since this corresponds to one iteration (a predetermined number of iterations or a smaller number of iterations less than or equal to a predetermined number of first iterations), all the iterations of the fast search algorithm (a predetermined number of iterations or a predetermined number of iterations) The number of iterations of 1) results can be obtained in a shorter time than the calculation, and the processing amount of the calculation is also small.

<MB search process>
FIG. 12 is a diagram showing an operation flowchart of the MB search process. When this flowchart is started, first, it is determined whether or not fv ₀ is an initial value (x ₀ , y ₀ ) (step S50). If it is an initial value, (x ₀ , y ₀ ) is set to mv. After (step S51), the flowchart ends.

On the other hand, if fv ₀ is not the initial value (x ₀ , y ₀ ), first, x = X, y = Y, ox = fv ₀ , oy = fv ₀ , sx = 16, sy = 16, n = 14 is set (step S52). Next, NNS function processing (FIG. 14 or FIG. 15) is executed using them as arguments, the return value of the function is set to mv (step S53), and the flowchart ends. Here, since x = X, y = Y, ox = fv ₀ , oy = fv ₀ , sx = 16, sy = 16, and n = 14, NNS (X, Y, fv ₀ , fv ₀ , 16, 16, 14). The NNS function is a function that performs a Nearest Neighbors Search on condition of a given argument and returns a result (motion vector). The processing contents of the NNS function will be described later (FIGS. 13 and 14).

  Thus, in this MB search process, a fast search algorithm, for example, the NNS function of the nearest neighbor search method is used, and the maximum number of iterations (n) is, for example, 14 times (this number is described in the gist of the invention). (Corresponding to “predetermined second number of iterations”), an accurate motion vector (mv) of one macroblock (MB) is calculated.

<SB search process>
FIG. 13 is a diagram illustrating an operation flowchart of the SB search process. When this flowchart is started, first, sx = 8, sy = 8, and n = 14 are set (step S60). Next, x = X, y = Y, ox = fv ₀ , oy = fv ₀ are set (step S 61), NNS function processing (FIG. 14 or FIG. 15) is executed using them as arguments, and the function returns. The value is set to mv ₀ (step S62). Here, since x = X, y = Y, ox = fv ₀ , oy = fv ₀ , sx = 8, sy = 8, and n = 14, NNS (X, Y, fv ₀ , fv ₀ , 8, 8, 14).

Next, x = X + 8, y = Y, ox = fv ₁ , oy = fv ₁ are set (step S 63), NNS function processing (FIG. 14 or FIG. 15) is executed using them as arguments, and the function returns The value is set to mv ₁ (step S64). Here, since x = X + 8, y = Y, ox = fv ₁ , oy = fv ₁ , sx = 8, sy = 8, and n = 14, NNS (X + 8, Y, fv ₁ , fv ₁ , 8, 8, 14).

Next, x = X, y = Y + 8, ox = fv ₂ , oy = fv ₂ are set (step S65), NNS function processing (FIG. 14 or FIG. 15) is executed with these as arguments, and the function returns The value is set to mv ₂ (step S66). Here, since x = X, y = Y + 8, ox = fv ₂ , oy = fv ₂ , sx = 8, sy = 8, and n = 14, NNS (X, Y + 8, fv ₂ , fv ₂ , 8, 8, 14).

Next, x = X + 8, y = Y + 8, ox = fv ₃ , oy = fv ₃ are set (step S 67), and NNS function processing (FIG. 14 or FIG. 15) is executed using them as arguments. The return value is set to mv ₃ (step S68). Here, since x = X + 8, y = Y + 8, ox = fv ₃ , oy = fv ₃ , sx = 8, sy = 8, and n = 14, NNS (X + 8, Y + 8, fv ₃ , fv ₃ , 8, 8, 14).

Thus, in this SB search process, a fast search algorithm, for example, the NNS function of the nearest neighbor search method is used, and the maximum number of iterations (n) is, for example, 14 times (this number is described in the gist of the invention). (Corresponding to a “predetermined third number of iterations”), the accurate motion vectors (mv ₀ , mv ₁ , mv ₂ ) for each sub-block (SB ₀ , SB ₁ , SB ₂ , SB ₃ ) mv ₃ ) is calculated.

  In the above example, the maximum number n of MB search processes (predetermined second iteration number) and the maximum number of SB search processes n (predetermined third iteration number) are the same. Although the value (“14”) is set, the present invention is not limited to this. It may be a different value.

<NNS function processing>
FIG. 14 is a flowchart of the NNS function process. This function is given arguments (x: block X coordinate, y: block Y coordinate, ox: offset X coordinate, oy: offset Y coordinate, sx: block size X, sy: block size Y, n: maximum number of iterations) The nearest neighbor search method which is one of the fast search algorithms is executed according to, and the result is returned as a return value.

  The NNS function first sets an initial value 0 to a loop variable i for iteration (step S70), then a residual evaluation value A at the search center, a residual evaluation value B right one pixel from the search center, a search A residual evaluation value C left one pixel at the center, a residual evaluation value D one pixel above the search center, and a residual evaluation value E one pixel below the search center are calculated (step S71). These residual evaluation values A to E can be obtained using the SAD function in the same manner as the estimated motion vector calculation process (see FIGS. 10 and 11).

  Next, the minimum value among A to E is obtained (step S72). This minimum value can also be obtained using the Min function in the same manner as the estimated motion vector calculation process (see FIGS. 10 and 11).

  Next, it is determined whether or not the minimum value is obtained at the search center (step S73). If the minimum value is obtained at the search center, the coordinates of the search center are set as return values (step S74), and the flow is ended (iteration is ended).

On the other hand, if the minimum value is not obtained at the search center, the pixel from which the minimum value is obtained is set as a new search center (step S75). That is, if the minimum value = B, one pixel right of the search center is set as a new search center, and if the minimum value = C, one pixel left of the search center is set as a new search center, and the minimum value = D If so, one pixel above the search center is set as a new search center, or if the minimum value = E, one pixel below the search center is set as a new search center.
Next, the loop variable i is incremented by 1 (step S76), and it is determined whether or not i exceeds the maximum number of iterations n (step S77). If not, the process returns to step S71 and the above processing is repeated. If it has exceeded (repeated), the coordinates of the search center at that time are set as return values (step S74), and the flow is terminated (repetition is terminated).

  As described above, in the NNS function processing, it is possible to perform a search sequentially while repeating iterations up to the maximum number of iterations n of a given argument, and return an accurate motion vector.

  By the way, in this NNS function processing, the processing is started from the first iteration (i = 0), but this is also equivalent to this first iteration in the estimated motion vector calculation processing (see FIGS. 10 and 11). Since the processing (step S23 to step S27) is performed, it is a double operation and is useless, so it is desirable to improve as follows.

  FIG. 15 is a diagram showing a main part flowchart of the improved NNS function process. The difference from before the improvement is that a new process (step S78) is performed before calculating a total of five residual evaluation values A to E of the search center and four points in the vicinity of the search center (step S71). .

  In this process (step S78), it is determined whether i = 0 (that is, whether it is the first iteration) (step S78a), and if it is not the first iteration (second iteration, third iteration). ,...), A total of five residual evaluation values A to E of the search center and four points in the vicinity of the search center are calculated in the same way as before the improvement (step S71). In this case, “the intermediate value of the estimated motion vector calculation process (A to E obtained in step S23 to step S27 in FIG. 10) is taken, and these are left as a total of five remaining points including the search center and four points in the vicinity of the search center. It is used as the difference evaluation values A to E "(step S78b).

  Therefore, according to the improved NNS function process, the intermediate value of the estimated motion vector calculation process is reused as the initial value of the iteration of the MB search process (FIG. 12) and the SB search process (FIG. 13). The processing time can be shortened by reducing the number of iterations of the MB search process and SB search process. The effect of shortening the processing time is enormous even if the number of iterations of the estimated motion vector calculation process is minimum (only the first iteration).

  This is because MB search processing and SB search processing need to be executed by the number of blocks, and a huge amount of search processing of “number of blocks × number of frames” is performed per video stream. Even if the number of iterations of the estimated motion vector calculation process is the smallest (only the first iteration), a large amount of iterations corresponding to “number of blocks × number of frames” can be omitted when viewed in total. .

<Summary of Embodiment>
As described above, according to the present embodiment, only the operation corresponding to the first iteration of the nearest neighbor search method (predetermined number of iterations or a smaller number of iterations equal to or less than the predetermined number of first iterations) is obtained. Based on the estimated motion vectors (fv _{0 to} fv ₃ ) of the sub-blocks (SB _{0 to} SB ₃ ), it is determined at an early stage whether or not the motion in the macroblock (MB) is nearly uniform. Since the search block division method (large division mode or small division mode) is determined according to the determination result, the time required for the determination can be shortened compared to the prior art, For the following reasons, the amount of calculation can be almost halved.

  The reason why the amount of calculation can be almost halved is as follows. In the prior art, the search block division method is determined after performing motion compensation prediction for each macro block (16 × 16 block) and four sub-blocks (8 × 8 block) obtained by dividing the macro block. When the calculation amount of one macroblock is A, the calculation amount of the four sub-blocks is substantially equivalent to A, and the total calculation amount is almost doubled (≈2A).

  In contrast, in the present embodiment, MB search (MB search unit 23 in FIG. 4 / MB search process in step S13 in FIG. 9) and SB search (SB search unit 25 in FIG. 4 / SB in step S15 in FIG. 9). 4) Since the calculation amount is A, the total calculation amount is calculated by adding the estimated motion vector calculation unit 21 in FIG. 4 (or the estimated motion in step S10 in FIG. 9). It is only necessary to add (A + α) to the amount of calculation α in the vector calculation process. Moreover, the calculation amount α of the estimated motion vector calculation unit 21 (or the estimated motion vector calculation process in step S10 of FIG. 9) is one iteration (a predetermined number of iterations or a predetermined number) of a fast search algorithm such as the nearest neighbor search method. Therefore, the total calculation amount (A + α) of the present embodiment is changed to that of the prior art (≈2A). In comparison, it can be almost halved.

  Further, when the above-described improved NNS function processing (FIG. 15) is used, intermediate values (A to E obtained in steps S23 to S27) obtained in the middle of the estimated motion vector calculation processing (FIGS. 10 and 11). That is, the motion vector of each of the plurality of second blocks calculated by a predetermined number of iterations of the search algorithm or a small number of iterations (n ′) equal to or less than the predetermined number of first iterations is subjected to MB search processing ( It can be reused as an initial value for iteration of FIG. 12) or SB search processing (FIG. 13). As a result, the number of iterations of MB search processing and SB search processing can be reduced (-n '), and the processing time can be shortened.

The present invention is not limited to the above embodiment. Of course, various modifications are included within the scope of the technical idea, and for example, the following may be adopted.
(1) The size, number and shape of the sub-block are arbitrary. Also, if the division method has a hierarchical relationship, it can be similarly applied. For example, H.M. In H.264, “8 × 8” obtained by dividing “16 × 16” can be further divided into “4 × 8”, “8 × 4”, and “4 × 4”, and the hierarchical structure is doubled. Can be done between any of these hierarchies or between both hierarchies.
(2) The residual evaluation value may be SSD (sum of squared difference) or the like instead of SAD. If block evaluation values can be calculated from a collection of sub-block evaluation values, the same applies. Also, the code allocation cost of the motion vector may be added to the evaluation value.
(3) Although the number of iterations for obtaining the estimated motion vector (fv _{0 to} fv ₃ ) (a predetermined number of iterations or a smaller number of iterations equal to or less than the predetermined number of first iterations) is set to one time, two times One or more iterations may be performed. It may be at least equal to or less than the maximum number of repetitions (a predetermined number of repetitions or a predetermined first number of repetitions). The greater the number of iterations, the greater the certainty of the estimation.
(4) The equivalence determination of the estimated motion vectors (fv _{0 to} fv ₃ ) may be performed not only by complete equivalence but also by approximate equivalence such that the difference absolute value of each component is within a predetermined value.
(5) As a search algorithm for motion estimation, not only the nearest neighbor search method but also other algorithms such as an N-step search method and a logarithmic search method may be used.

It is a figure which shows the kind of motion compensation prediction block. It is a figure which shows the two motion compensation prediction blocks used as the selection object for convenience of this embodiment. It is a block diagram of a motion compensation predictive coding apparatus. 3 is a conceptual block diagram of a motion estimation unit 19. FIG. It is a conceptual diagram (1/2) for demonstrating operation | movement of the motion estimation part 19. FIG. 6 is a conceptual diagram (2/2) for explaining the operation of the motion estimation unit 19; FIG. This is a description of the block position (coordinates) and block size of the original image. It is a figure which shows the offset given to a reference image. It is a figure which shows the operation | movement flowchart of the motion estimation part 19. FIG. It is a figure (1/2) which shows the operation | movement flowchart of an estimated motion vector calculation process. It is a figure (2/2) which shows the operation | movement flowchart of an estimated motion vector calculation process. It is a figure which shows the operation | movement flowchart of MB search process. It is a figure which shows the operation | movement flowchart of SB search processing. It is a figure which shows the flowchart of a NNS function process. It is a figure which shows the flowchart of the improved NNS function process. It is the conceptual diagram which simplified the motion compensation prediction encoding. It is a conceptual diagram of the encoding image production | generation in motion compensation prediction encoding.

Explanation of symbols

S10 Estimated motion vector calculation process (first motion vector calculation step)
S11 First determination process (first determination step)
S13 MB search process (second motion vector calculation step)
S14 MB motion vector output (selection process, first selection process)
S15 SB search process (third motion vector calculation step)
S16 Second determination process (second determination step)
S18 MB motion vector output (selection process, second selection process)
S19 SB motion vector output (selection step, second selection step)
DESCRIPTION OF SYMBOLS 10 Motion compensation prediction encoding apparatus 19 Motion estimation part (block selection means)
21 Estimated motion vector calculation unit (first motion vector calculation means)
22 1st determination part (1st determination means)
23 MB search unit (second motion vector calculation means)
24 First MB motion vector output unit (first selection means)
25 SB search unit (third motion vector calculation means)
26 Second determination unit (second determination means)
27 Second MB motion vector output unit (second selection means)
28 SB motion vector output unit (second selection means)

Claims (8)

In the motion compensated prediction encoding apparatus including block selection means for selecting either the first block or the second block obtained by dividing the first block into a plurality of motion compensation prediction blocks,
The block selection means includes
First motion vector computing means for computing a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined number of iterations;
First determination means for determining whether or not each motion vector of the second block calculated by the first motion vector calculation means is completely equivalent or approximate equivalent,
When the determination result of the first determination means is affirmative, the first block is selected as a motion compensated prediction block, while when the determination result of the first determination means is negative, the second block is selected as a motion compensated prediction block. A motion-compensated predictive coding apparatus, characterized by: In the motion compensated prediction encoding apparatus including block selection means for selecting either the first block or the second block obtained by dividing the first block into a plurality of motion compensation prediction blocks,
The block selection means includes
First motion vector computing means for computing a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined first number of iterations;
First determination means for determining whether or not each motion vector of the second block calculated by the first motion vector calculation means is completely equivalent or approximate equivalent;
First selection means for selecting the first block as a motion compensated prediction block when the determination result of the first determination means is affirmative;
Second motion vector computing means for computing a motion vector of the first block by a predetermined search algorithm with an upper limit of a predetermined second iteration number when the judgment result of the first judging means is affirmative;
A third motion vector calculation for calculating a motion vector of each of the plurality of second blocks with a predetermined third iteration number as an upper limit when the determination result of the first determination means is negative Means,
Second determination means for determining whether or not each motion vector of the second block calculated by the third motion vector calculation means is completely equivalent or approximate equivalent;
When the determination result of the second determination means is affirmative, the first block is selected as a motion compensated prediction block, while when the determination result of the second determination means is negative, the second block is selected as a motion compensated prediction block. A motion-compensated predictive coding apparatus comprising: a second selecting means for selecting. 3. The motion compensated predictive coding apparatus according to claim 1, wherein the predetermined search algorithm is a nearest neighbor search method, an N-step search method, or a logarithmic search method. The second motion vector computing means and the third motion vector computing means start the iteration from n ′ + 1 times when the first motion vector computing means is repeated n ′ times, and as initial values of the iterations The motion compensated predictive coding apparatus according to claim 2, wherein an intermediate value obtained by n 'iteration of the first motion vector computing means is used. When selecting one of the first block and the second block obtained by dividing the first block into a plurality of motion compensation prediction blocks,
A first motion vector calculation step of calculating a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined number of iterations;
A first determination step of determining whether or not each motion vector of the second block calculated in the first motion vector calculation step is completely equivalent or approximate equivalent;
When the determination result of the first determination step is positive, the first block is selected as a motion compensated prediction block, while when the determination result of the first determination step is negative, the second block is selected as a motion compensated prediction block. And a selection step of selecting. A motion compensated predictive encoding method, comprising: When selecting one of the first block and the second block obtained by dividing the first block into a plurality of motion compensation prediction blocks,
A first motion vector calculation step of calculating a motion vector of each of the plurality of second blocks by a predetermined search algorithm with a small number of iterations equal to or less than a predetermined first iteration number;
A first determination step of determining whether or not each motion vector of the second block calculated in the first motion vector calculation step is completely equivalent or approximate equivalent;
A first selection step of selecting the first block as a motion compensated prediction block when the determination result of the first determination step is positive;
A second motion vector calculation step of calculating a motion vector of the first block by a predetermined search algorithm with an upper limit of a predetermined second iteration number when the determination result of the first determination step is affirmative;
A third motion vector calculation for calculating a motion vector of each of the plurality of second blocks by a predetermined search algorithm up to a predetermined third number of iterations when the determination result of the first determination step is negative Process,
A second determination step of determining whether or not each motion vector of the second block calculated in the third motion vector calculation step is completely equivalent or approximate equivalent;
When the determination result of the second determination step is positive, the first block is selected as a motion compensated prediction block, while when the determination result of the second determination step is negative, the second block is selected as a motion compensated prediction block. A motion-compensated predictive encoding method comprising: performing a second selection step. 7. The motion compensated predictive coding method according to claim 5, wherein the predetermined search algorithm is a nearest neighbor search method, an N-step search method or a logarithmic search method. In the second motion vector calculation step and the third motion vector calculation step, when the first motion vector calculation step is repeated n 'times, the iteration starts from n' + 1 times and is used as an initial value of the iteration. The motion compensated predictive encoding method according to claim 6, wherein an intermediate value obtained by n 'iteration of the first motion vector calculation step is used.

Images