JP2012070278A - Video decoding device, video decoding method and video decoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To solve the problem in which, since motion compensated prediction is performed with prepared shape patterns in conventional video encoding/decoding, an optimum shape pattern cannot be selected, so that improvement of encoding efficiency is limited.SOLUTION: A variable length decoding unit 201 decodes plural motion vectors for a decoding target block from an encoded stream. An actual region division/motion compensation unit 203 generates plural prediction blocks corresponding to the decoding target block from a reference image by using the plural motion vectors, determines a boundary based on at least one of activities of the plural prediction blocks and an adjacent region evaluation value obtained by evaluating an encoded region adjacent to an encoding target block, and when combining, between the respective prediction blocks, regions obtained by dividing the respective prediction blocks with the boundary, by making the respective prediction blocks overlap each other in a peripheral region of an actual boundary and performing weighting addition of the respective prediction blocks with a predetermined ratio, generates a composite prediction block.

Description

  The present invention relates to a moving picture decoding technique, and more particularly to a moving picture decoding technique using motion compensated prediction.

  In compression coding of moving images represented by MPEG (Moving Picture Experts Group), motion compensation prediction that compresses a code amount by using correlation between screens is often used. In motion compensated prediction used in MPEG or the like, a predicted image is generated for each block of a predetermined size from a reference image that has already been decoded using a motion vector that indicates the relative positional relationship between the encoding target image and the reference image. Thereafter, the encoding side calculates a prediction error that is a difference between the original image to be encoded and a prediction image generated by motion compensated prediction, and transmits only the prediction error to the decoding side. Thereby, compared with the case where motion compensation prediction is not used, the code amount transmitted can be reduced significantly.

  In general, encoding / decoding processing is performed in units of macroblocks (pixel group having a predetermined block size, for example, 16 × 16). Motion compensation prediction is often performed in units of macroblocks, but in this case, it is difficult to capture the motion of an object or the like that is smaller than the macroblock, resulting in a decrease in coding efficiency. Therefore, multi-block pattern motion compensated prediction is used as a method for making motion compensated prediction function more efficiently.

  In multi-block pattern motion compensation prediction, a macro block can be further divided into sub blocks, and motion compensation prediction can be performed using different motion vectors for each sub block. The division pattern of the motion compensation prediction block to be used is defined in advance as the same rule on the encoding side and the decoding side. On the encoding side, a block pattern for motion compensation prediction is selected from the defined block patterns, and block pattern selection information is transmitted to the decoding side. On the decoding side, motion compensation prediction is performed based on the received block pattern selection information. In multi-block pattern motion compensated prediction, when the optimum block pattern for motion compensated prediction is selected on the encoding side, the prediction error after motion compensated prediction is reduced, and the encoding efficiency is improved.

  As an example of a specific block pattern, in MPEG-4 AVC (Advanced Video Coding) internationally standardized by a joint video team (JVT) of ISO / IEC and ITU-T, there are further 16 macroblocks (16 × 16 blocks). It is possible to perform motion compensation prediction by dividing into block patterns of × 8/8 × 16/8 × 8/8 × 4/4 × 8/4 × 4. On the encoding side, a block pattern is selected, and block pattern selection information is encoded in the bitstream. On the decoding side, the macroblock is divided into regions according to the block pattern encoded in the bitstream, and motion compensation prediction is performed for each divided region.

  Furthermore, Patent Literature 1 and Patent Literature 2 disclose techniques for defining motion compensation prediction shape patterns and performing motion compensation prediction with a more flexible shape.

Japanese Patent No. 4025570 Republished Patent No. WO2003-026315

  However, in the methods disclosed in Patent Document 1 and Patent Document 2, when the number of motion compensated prediction shape patterns to be defined is increased, the amount of code consumed for selection information of the shape pattern to be transmitted increases as the number of shape patterns increases. End up. In other words, a decrease in prediction error due to an increase in shape patterns and an increase in code amount related to shape pattern selection are in a trade-off relationship, so it is difficult to improve overall coding efficiency simply by increasing the number of shape patterns. .

  As described above, in the conventional moving image encoding / decoding, only motion compensation prediction of a predefined shape pattern can be performed. Therefore, motion compensation prediction cannot be performed with an optimal shape, and encoding efficiency cannot be improved. . In addition, when the number of motion compensation predicted shape patterns to be defined is increased, the amount of additional information related to the selection of motion compensated predicted shape patterns increases, and thus the overall coding efficiency is not always improved.

  The present invention has been made in view of such circumstances, and an object of the present invention is to enable motion compensation prediction in various block patterns without increasing the amount of additional information regarding the block pattern of motion compensation prediction. Therefore, an object of the present invention is to provide a technique for improving the coding efficiency by reducing the prediction error.

  In order to solve the above problems, a video decoding device according to an aspect of the present invention uses a decoding unit (201) that decodes a plurality of motion vectors for a decoding target block from an encoded stream, and a plurality of motion vectors. Generating a plurality of prediction blocks corresponding to the decoding target block from a reference image, determining a boundary based on activities of the plurality of prediction blocks, and dividing each prediction block by the boundary When combining between the prediction blocks, the motion compensation unit (203) for generating a composite prediction block by overlapping each prediction block and weighting and adding at a predetermined ratio for the peripheral region of the boundary, and the composite prediction An adding unit that generates a decoded image by adding a block and a prediction difference block decoded from the decoding target block 209) and a.

  Another aspect of the present invention is also a video decoding device. The apparatus includes: a decoding unit (201) that decodes a plurality of motion vectors for a decoding target block from an encoded stream; a plurality of prediction blocks corresponding to the decoding target block from a reference image using the plurality of motion vectors; Generating a plurality of extended prediction images corresponding to decoded regions adjacent to the block to be decoded, evaluating adjacent prediction values between each extended prediction image and the decoded image of the decoded region, and predicting each prediction Movement to generate a composite prediction block by determining a boundary based on at least one of block activities and combining regions obtained by dividing the prediction blocks by the boundary between the prediction blocks Compensating unit (203), adding the combined prediction block and the prediction difference block decoded from the decoding target block Ri, and an adder for generating a decoded image (209).

  Yet another embodiment of the present invention is also a moving picture decoding apparatus. The apparatus includes: a decoding unit (201) that decodes a plurality of motion vectors for a decoding target block from an encoded stream; a plurality of prediction blocks corresponding to the decoding target block from a reference image using the plurality of motion vectors; Generating a plurality of extended prediction images corresponding to decoded regions adjacent to the block to be decoded, evaluating adjacent prediction values between each extended prediction image and the decoded image of the decoded region, and predicting each prediction A boundary is determined based on at least one of the block activities, and when a region obtained by dividing each prediction block by the boundary is combined between the prediction blocks, A motion compensation unit (203) for generating a composite prediction block by overlapping each prediction block and performing weighted addition at a predetermined ratio; Wherein it comprises a combined predicted block by adding the prediction difference block decoded from the decoding target block, an adder for generating a decoded image and a (209).

  Yet another aspect of the present invention is a video decoding method. In this method, a plurality of motion vectors for a decoding target block are decoded from an encoded stream, and a plurality of prediction blocks corresponding to the decoding target block are generated from a reference image by using the plurality of motion vectors. The boundary is determined based on the activity of the prediction block, and the regions obtained by dividing the prediction blocks by the boundary are combined between the prediction blocks. A decoded image is generated by adding a step of generating a combined prediction block by overlapping and weighting and adding at a predetermined ratio, adding the combined prediction block, and a prediction difference block decoded from the decoding target block Steps.

  Yet another embodiment of the present invention is also a moving image decoding method. The method includes a step of decoding a plurality of motion vectors for a decoding target block from an encoded stream, a plurality of prediction blocks corresponding to the decoding target block from a reference image using the plurality of motion vectors, and the decoding target. A plurality of extended prediction images corresponding to a decoded region adjacent to the block, an adjacent evaluation value that evaluates a prediction error between each extended prediction image and the decoded image of the decoded region, and an activity of each prediction block; Generating a composite prediction block by determining a boundary based on at least one of the two and combining regions obtained by dividing the prediction blocks by the boundary between the prediction blocks; and A decoded image is generated by adding a prediction block and a prediction difference block decoded from the decoding target block. That and a step.

  Yet another embodiment of the present invention is also a moving image decoding method. The method includes a step of decoding a plurality of motion vectors for a decoding target block from an encoded stream, a plurality of prediction blocks corresponding to the decoding target block from a reference image using the plurality of motion vectors, and the decoding target. A plurality of extended prediction images corresponding to a decoded region adjacent to the block, an adjacent evaluation value that evaluates a prediction error between each extended prediction image and the decoded image of the decoded region, and an activity of each prediction block; A boundary is determined based on at least one of the prediction blocks, and when the regions obtained by dividing each prediction block by the boundary are combined between the prediction blocks, each prediction block is assigned to a peripheral region of the boundary. Generating a composite prediction block by overlapping and weighting and adding at a predetermined ratio; and If, by adding the prediction difference block decoded from the decoding target block, and generating a decoded image.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to reduce the prediction error and increase the coding efficiency by enabling the motion compensation prediction in various block patterns without increasing the code amount of the additional information regarding the block pattern of the motion compensation prediction. Can be improved.

1 is a block diagram illustrating a configuration of a moving image encoding apparatus according to Embodiment 1. FIG. 1 is a block diagram illustrating a configuration of a moving image decoding apparatus according to Embodiment 1. FIG. It is a figure explaining the pattern which divides a macroblock into 2 in a horizontal direction. It is a figure explaining the pattern which divides a macroblock into 2 in the perpendicular direction. It is a figure which shows the motion vector detected for every temporary area | region of a macroblock. It is a figure which shows the motion compensation estimated image produced | generated from the motion vector detected for every temporary area | region. It is a figure which shows the synthetic | combination motion compensation prediction image obtained by synthesize | combining the motion compensation prediction image produced | generated from the motion vector of each temporary area | region. It is a figure explaining the method of synthesize | combining the motion compensation estimated image produced | generated from the motion vector of each temporary area | region, and producing | generating a synthetic | combination motion compensation prediction image. 6 is a flowchart for explaining a main boundary determination procedure by the main region dividing / motion compensating unit in FIG. 1. It is a figure explaining the activity regarding this boundary determination about a motion compensation prediction image. It is a figure explaining the method of calculating an adjacent area evaluation value about the encoded area | region adjacent to an encoding object block. It is a figure explaining the method to synthesize | combine, combining a 1st motion compensation prediction image and a 2nd motion compensation prediction image in the peripheral region of this boundary, and produce | generate a synthetic | combination motion compensation prediction image. It is a figure explaining the superposition coefficient at the time of superimposing the 1st motion compensation prediction picture and the 2nd motion compensation prediction picture in the peripheral field of this boundary. 10 is a flowchart for explaining a motion vector adjustment procedure by the moving picture encoding apparatus according to the second embodiment. 10 is a flowchart for explaining a procedure for adjusting a temporary boundary by the moving image encoding apparatus according to the third embodiment. It is a figure which shows the 1st syntax pattern of the bit stream of the moving image encoded by the moving image encoder of Embodiment 1-3. It is a figure which shows the semantics of macroblock type mb_type in case the 1st flag which shows whether the shape of a motion compensation prediction is determined automatically by a decoding side per slice is ON / OFF. It is a figure which shows the 2nd syntax pattern which transmits the 2nd flag which shows whether the shape of motion compensation prediction is determined automatically by a decoding side per macroblock. It is a figure which shows the 3rd syntax pattern which switches the algorithm which determines automatically the shape of a motion compensation prediction at the slice level by the decoding side. It is a figure which shows the 4th syntax pattern which does not distinguish the division | segmentation direction of a macroblock on a syntax. It is a figure which shows the macroblock type semantics of a 4th syntax pattern. It is a figure which shows the 5th syntax pattern which determines area | region division | segmentation without interlocking | linking with a macroblock type. It is a figure explaining the case where embodiment of this invention is applied with respect to bidirectional | two-way prediction. It is a figure which shows the method of dividing | segmenting a macroblock into two dimensions. It is a flowchart explaining the procedure which divides | segments a macroblock into two dimensions and performs motion compensation. It is a figure which shows the method of dividing | segmenting a macroblock into three area | regions. It is a flowchart explaining the procedure which divides a macroblock into 3 and performs motion compensation.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of the moving picture encoding apparatus according to the first embodiment. The moving image encoding apparatus according to Embodiment 1 includes a provisional region dividing unit 101, a motion vector detecting unit 102, a main region dividing / motion compensating unit 103, an orthogonal transform / quantizing unit 104, a variable length encoding unit 105, and an inverse quantum. And an inverse orthogonal transform unit 106, a reference image memory 107, a subtraction unit 108, and an addition unit 109.

  The temporary area dividing unit 101 divides an area at an arbitrary boundary for a pixel group to be encoded, such as a 16 × 16 macroblock. This region division is a unique method on the coding side for motion vector detection, and the region may be divided at any boundary, but it is desirable to perform region division at a boundary that improves the efficiency of motion compensation prediction. Here, as the simplest provisional area division, a case where a macroblock is divided into two only in the horizontal direction or the vertical direction will be described as an example.

  FIGS. 3A to 3C are diagrams for explaining a pattern in which a macroblock is divided into two in the horizontal direction. FIG. 3A shows a pattern in which a macroblock is divided into a 16 × 4 upper region and a 16 × 12 lower region at the horizontal boundary of the fourth pixel from the top, and FIG. This is a pattern in which the macroblock is divided into a 16 × 8 upper area and a 16 × 8 lower area at the horizontal boundary of the eighth pixel, and FIG. 3C shows the macroblock at the horizontal boundary of the twelfth pixel from the top. Is divided into a 16 × 12 upper region and a 16 × 4 lower region.

  FIGS. 4A to 4C are diagrams for explaining a pattern in which a macroblock is divided into two in the vertical direction. FIG. 4A shows a pattern in which a macroblock is divided into a 4 × 16 left region and a 12 × 16 right region at the vertical boundary of the fourth pixel from the left, and FIG. The macroblock is divided into two 8 × 16 left and 8 × 16 right regions at the vertical boundary of the pixel, and FIG. 4C shows 12 macroblocks at the vertical boundary of the 12th pixel from the left. The pattern is divided into two parts, a left side area of x16 and a right side area of 4x16.

  The boundary of the macroblock determined by the provisional region dividing unit 101 is called a “provisional boundary”, and each region in the macroblock divided by the provisional boundary is called a “provisional region”.

  Here, three patterns that are divided into two in the horizontal direction or the vertical direction are shown, but four or more division patterns may be provided by increasing the candidate positions of the horizontal boundary or the vertical boundary. Further, it may be divided at an oblique boundary or may be divided at a bent boundary.

  The temporary area dividing unit 101 calculates a horizontal activity related to each horizontal boundary or a vertical activity related to each vertical boundary for the original image signal to be encoded. As described later, when a macroblock is divided at a bent boundary, the activity is calculated along the bent boundary.

The activity of the image signal is a value obtained by performing some operation on the pixel. For example, an absolute difference sum (SAD) between two pixels across the boundary can be used as an activity related to the boundary for dividing the macroblock. For example, assuming that the X coordinate in the macroblock is i, the Y coordinate is j, and the pixel value at the point (i, j) is _{Ai, j} , the horizontal activity at the horizontal boundary j of the jth pixel from the top is as follows: Defined in
Horizontal activity = Σ _{i = 0} ¹⁵ | A _{i, j} −A _{i, j−1} |
Here, Σ _{i = 0} ¹⁵ is the total sum when the subscript i is moved from 0 to 15.

However, the activity may not be an absolute difference sum (SAD). An absolute sum of squares (SSD) as shown below may be used.
Horizontal activity = Σ _{i = 0} ¹⁵ (A _{i, j} −A _{i, j−1} ) ²

Similarly, the vertical activity at the vertical boundary i of the i-th pixel from the left of the macroblock is defined as follows.
Vertical activity = Σ _{j = 0} ¹⁵ | A _{i, j} −A _{i−1, j} |

  The activity becomes a large value at the edge of the object. The temporary area dividing unit 101 tentatively divides a macroblock area at a boundary having the largest activity value.

  The motion vector detection unit 102 detects a motion vector for each temporary region divided by the temporary region division unit 101.

  FIG. 5 is a diagram showing motion vectors detected for each temporary area of the macroblock. For example, when the temporary area dividing unit 101 tentatively divides the macroblock at the horizontal boundary of the fourth pixel from the top as shown in FIG. 3A, the motion vector detecting unit 102 is as shown in FIG. , Motion vectors are detected for each of the 16 × 4 upper region and the 16 × 12 lower region.

  Here, the motion vector is detected using a block matching method. As a motion vector detection algorithm, there are various methods such as a full search that evaluates all candidate vectors in a designated search region and a high-speed search that narrows down and searches candidate vectors based on various motion characteristics. Any motion vector detection algorithm can be used as long as a motion vector can be detected for each block divided by the temporary region dividing unit 101.

  The region segmentation / motion compensation unit 103 performs motion compensation prediction from the reference image stored in the reference image memory 107 using the motion vector of each temporary region detected by the motion vector detection unit 102. The region segmentation / motion compensation unit 103 synthesizes a plurality of motion compensated prediction images generated from the motion vectors of the temporary regions in a procedure described later to generate a combined motion compensated prediction image.

  6A and 6B show motion compensated prediction images generated from motion vectors detected for each temporary region. FIGS. 7A to 7C show synthesized motion compensated predicted images obtained by synthesizing motion compensated predicted images generated from the motion vectors of the temporary regions.

  An example will be described in which the temporary region dividing unit 101 tentatively divides a macroblock at the horizontal boundary of the fourth pixel from the top as shown in FIG. As illustrated in FIG. 5, the motion vector detection unit 102 detects motion vectors for each of the 16 × 4 upper region and the 16 × 12 lower region, and as a result, the motion target macroblock includes two motion vectors ( There are first and second motion vectors). As shown in FIGS. 6A and 6B, the region segmentation / motion compensation unit 103 uses a reference image for each motion vector detected by the motion vector detection unit 102 and temporarily determines the size of the macroblock. Then, an image in the case of motion compensation prediction is generated at (here, 16 × 16). FIG. 6A is a first motion compensated prediction image generated using the first motion vector of the 16 × 4 upper region, and FIG. 6B is a second image of the lower region of 16 × 12. It is the 2nd motion compensation prediction picture generated using a motion vector.

  The region segmentation / motion compensation unit 103 synthesizes the first and second motion compensated prediction images shown in FIGS. 6A and 6B into any one of patterns (a) to (c) shown in FIG. Thus, a combined motion compensated prediction image is generated. The first and second motion compensated prediction images are the same size as the encoding target macroblock, and are divided at a certain boundary in order to generate a combined motion compensation prediction image. The boundary of the macroblock determined by the region dividing / motion compensating unit 103 is referred to as “actual boundary”, the region of the macroblock divided by the boundary is referred to as “actual region”. It is distinguished from “temporary boundary” and “temporary region” determined by the region dividing unit 101.

  FIGS. 7A to 7C show candidates for the main boundary determined by the region division / motion compensation unit 103. FIG. 7A, the horizontal boundary of the fourth pixel from the top is the main boundary, FIG. 7B is the horizontal boundary of the eighth pixel from the top, and FIG. 7C is the twelfth pixel from the top. With the horizontal boundary as the main boundary, the main region above the main boundary is represented by the region corresponding to the first motion compensated prediction image, and the main region below the main boundary is represented by the region corresponding to the second motion compensated prediction image. Is.

  The main region dividing / motion compensating unit 103 determines the main boundary based on the evaluation value indicating the edge strength and the like, and divides and synthesizes the first and second motion compensated prediction images at the determined main boundary.

  FIGS. 8A to 8C are diagrams illustrating a method of generating a combined motion compensated prediction image by combining motion compensated prediction images generated from the motion vectors of the temporary regions. After the boundary determination, the main segmentation / motion compensation unit 103 determines the first motion compensated prediction image corresponding to the first motion vector in FIG. 8A and the second motion vector corresponding to the second motion vector in FIG. Using the motion compensated prediction image, motion compensation prediction is performed using the region above the main boundary as the first motion compensation prediction image, and the region below the main boundary is used as the second motion compensation prediction image. In this example, the corresponding region of the first motion compensated prediction image is duplicated in the 16 × 4 region above the main boundary, and the second motion compensated prediction image is reproduced in the 16 × 12 region below the main boundary. The corresponding region of is duplicated.

  Here, the horizontal region division has been described as an example, but the vertical region division can also be performed in the same manner. Moreover, although the case where the main boundary candidates are three kinds of FIGS. 7A to 7C has been described this time, the main boundary candidates may be further increased. However, as described later, it is necessary to sufficiently consider the relationship with the orthogonal transform size of the orthogonal transform / quantization unit 104.

  Returning to FIG. 1, the subtraction unit 108 calculates a prediction residual component based on the difference between the original image to be encoded and the motion compensated prediction image calculated by the main region division / motion compensation unit 103, and generates an orthogonal transform / quantization unit. 104. The orthogonal transform / quantization unit 104 performs orthogonal transform / quantization of the prediction residual component.

  Here, the orthogonal transform / quantization unit 104 performs orthogonal transform using an orthogonal transform size corresponding to the size of motion compensation prediction. That is, when 16 × 4/16 × 8/16 × 12 (multiple of 4 in the vertical direction) is allowed as the size of motion compensation prediction, at least 16 × 4, 8 × 4, or 4 × 4 (vertical direction) An orthogonal transform size of a multiple of 4). As another example, when 16 × 2/16 × 4/16 × 6/16 × 8/16 × 10/16 × 12 (a multiple of 2 in the vertical direction) is allowed as the size of motion compensation prediction, at least An orthogonal transform size of 16 × 2 or 8 × 2 or 4 × 2 (a multiple of 2 in the vertical direction) can be used. As a result, when the prediction error of motion compensation prediction is orthogonally transformed, the motion compensation prediction boundary is not included in the prediction error set to be orthogonally transformed. As a result, it is possible to prevent a decrease in orthogonal transform efficiency caused by performing orthogonal transform together as a prediction error on pixels that straddle the boundary of motion compensation prediction, and the effect of further improving the coding efficiency is achieved.

  The variable length coding unit 105 performs variable length coding on the prediction residual component orthogonally transformed and quantized by the orthogonal transformation / quantization unit 104 and variable length coding the motion vector detected by the motion vector detection unit 102. To do. When a macroblock is divided at a fixed boundary as in the prior art, motion vectors are transmitted in raster order (ie, from the upper left block to the lower right block). When the shape of motion compensated prediction is automatically determined on the decoding side as in the present embodiment, the position of the upper left pixel in each region of motion compensated prediction is the motion vector in the raster order from the previous one. To transmit. As a result, as in the prior art, the target region for motion compensation prediction can be uniquely expressed by the order in which a plurality of motion vectors are transmitted.

  The inverse quantization / inverse orthogonal transform unit 106 performs inverse orthogonal transform and inverse quantization on the prediction residual component orthogonally transformed / quantized by the orthogonal transform / quantization unit 104. Similar to the orthogonal transform / quantization unit 104, inverse orthogonal transform can be performed with a size corresponding to the size of motion compensation prediction.

  The adding unit 109 generates a reference image by adding the prediction residual component decoded by the inverse quantization / inverse orthogonal transform unit 106 and the motion compensated prediction image calculated by the region division / motion compensation unit 103. And stored in the reference image memory 107.

  FIG. 2 is a block diagram illustrating a configuration of the moving picture decoding apparatus according to the first embodiment. The moving picture decoding apparatus according to Embodiment 1 includes a variable length decoding unit 201, a main region division / motion compensation unit 203, an inverse quantization / inverse orthogonal transform unit 206, an addition unit 209, and a reference image memory 207.

  The variable length decoding unit 201 performs variable length decoding on the prediction residual component signal and the motion vector that have been orthogonally transformed and quantized. In the bit stream encoded by the moving image encoding apparatus in FIG. 1, since a motion vector is encoded for each area into which a macroblock is divided, the variable length decoding unit 201 decodes the motion vector for each divided area. Is done. Here, the target region for motion compensation prediction can be uniquely determined by the order in which the motion vectors in the macroblock are decoded.

  The region segmentation / motion compensation unit 203 has the same function as the region segmentation / motion compensation unit 103 of the moving image coding apparatus in FIG. 1, and uses the motion vector decoded by the variable length decoding unit 201 as a reference image. Motion compensation prediction is performed from the reference image stored in the memory 207. Here, the motion vector is acquired for each divided area of the macroblock. The region segmentation / motion compensation unit 203 performs a plurality of motion compensated prediction images generated from the motion vectors of each segment region in the same procedure as the region segmentation / motion compensation unit 103 of the video encoding device in FIG. A combined motion compensated prediction image is generated by combining.

  The inverse quantization / inverse orthogonal transform unit 206 has the same function as the inverse quantization / inverse orthogonal transform unit 106 of the moving image coding apparatus in FIG. 1, and uses the prediction residual component decoded by the variable length decoding unit 201. On the other hand, inverse orthogonal transform and inverse quantization are performed.

  The addition unit 209 adds the prediction residual component decoded by the inverse quantization / inverse orthogonal transformation unit 206 and the motion compensated prediction image calculated by the region division / motion compensation unit 203, thereby obtaining an image signal. Decrypt. The reference image memory 207 is the same as the reference image memory 107 of the moving image encoding apparatus in FIG. 1, and stores the decoded reference image.

  The operation of the moving picture encoding apparatus having the above configuration, particularly the operation of the region dividing / motion compensating unit 103 will be described.

  FIG. 9 is a flowchart for explaining the procedure for determining the main region by the main region dividing / motion compensating unit 103.

  First, the motion vector detecting unit 102 detects a motion vector for each of N (N ≧ 2) temporary regions divided by the temporary region dividing unit 101. Here, N = 2. Using the first motion vector detected by the motion vector detection unit 102, motion compensation prediction is performed with the same size as the macroblock, and a first motion compensation predicted image is generated (S01). Similarly, using the second motion vector detected by the motion vector detection unit 102, motion compensation prediction is performed with the same size as the macroblock, and a second motion compensation predicted image is generated (S02).

  Next, the first to fourth activities shown in FIGS. 10A to 10D are calculated for each candidate of this boundary (S03 to S06). However, steps S03 to S06 can be performed in any order. Further, it is of course possible to calculate only the activity that is desired to be used for boundary evaluation without performing all of steps S03 to S06.

  First, as shown in FIG. 10A, a boundary activity (first activity) related to the boundary candidate is calculated for the first motion compensated prediction image (S03). Here, the absolute difference sum (SAD) between two pixels straddling the boundary candidate is used for the activity. Since the value of the first activity increases at the edge of the object, the prediction efficiency of the motion compensation prediction is improved by dividing the region at the corresponding boundary as the value increases. Similarly, as shown in FIG. 10B, a boundary activity (second activity) regarding the boundary candidate is calculated for the second motion compensated prediction image (S04). Similar to the first activity, the second activity improves the prediction efficiency of motion compensation prediction by dividing the region at the corresponding boundary as the value increases.

  Subsequently, as shown in FIG. 10C, the region above the main boundary candidate in the macroblock is motion-compensated and predicted as the first motion compensated prediction image, and the region below the main boundary candidate is the second motion compensated prediction image. As for the synthesized motion compensated prediction image synthesized as follows, a boundary activity (third activity) related to this boundary candidate is calculated (S05). Since the third activity is the sum of absolute differences between two pixels straddling the boundary candidate, the absolute difference between the values of the pixels of the first motion compensated prediction image and the pixels of the second motion compensated prediction image located above and below the boundary candidate. The sum of Therefore, the smaller the value of the third activity, the smoother the boundary of the synthesized motion compensated prediction image, and the high frequency component is less likely to be generated in the prediction error signal, so that the prediction efficiency of motion compensation prediction is improved.

  Finally, as shown in FIG. 10D, a boundary activity (fourth activity) regarding the boundary candidate is calculated for the difference image between the first motion compensated predicted image and the second motion compensated predicted image (S06). Since the value of the fourth activity increases at the boundary of the object, the prediction efficiency of the motion compensation prediction is improved by dividing the region at the boundary as the value increases.

  Further, in the present embodiment, an extended predicted image obtained by extending the first motion compensated predicted image and the second motion compensated predicted image to the encoded region adjacent to the encoding target block, and encoding of the adjacent region, respectively. Each prediction error when compared with a local decoded image is evaluated, and an adjacent region evaluation value is calculated (S07).

  FIGS. 11A to 11C are diagrams illustrating a method for calculating an adjacent region evaluation value. Here, the calculation of the adjacent region evaluation value when the region is divided into two in the horizontal direction will be described with reference to FIG. When the encoding order is raster order, that is, from left to right and from top to bottom, the left and upper adjacent areas of the encoding target block are already encoded. When the region is divided into two in the horizontal direction, as shown in FIG. 11A, the local decoded image 302 of the encoded adjacent region on the left side of the encoding target region 300 is used for evaluation.

  First, as shown in FIG. 11B, a first extended predicted image 312 is generated for an adjacent region extended by a predetermined number of pixels (for example, two pixels) in the left direction of the first motion compensation predicted image 310. Similarly, FIG. As in 11 (c), the second extended predicted image 322 is generated for the adjacent region extended by a predetermined number of pixels in the left direction of the second motion compensation predicted image 320. The first and second extended prediction images are generated by performing motion compensation prediction on the left adjacent region of the encoding target block using the first and second motion vectors, respectively.

Next, as shown in FIG. 11D, an error evaluation value between each of the extended predicted images 312 and 322 and the local decoded image 302 of the encoded adjacent region is calculated by SAD or the like, and the region above the boundary candidate is calculated. Is calculated using the error evaluation value with the first extended prediction image and for the region below the boundary candidate, the error evaluation value with the second extended prediction image is used to calculate the sum of these error evaluation values. To calculate the adjacent region evaluation value in this boundary candidate.
Adjacent region evaluation value NEI = U1 + D2
Here, U1 is an error evaluation value of the upper region of the first extended prediction image, and D2 is an error evaluation value of the lower region of the second extended prediction image. That is, the adjacent region evaluation value NEI is a numerical value of the poor prediction efficiency when the peripheral region of the encoding target block is divided by the boundary candidates.

  Since the encoding target block and the peripheral block are continuous, the encoding target image is predicted by dividing the encoding target image at the boundary where the prediction efficiency of the peripheral region is high (the adjacent region evaluation value is small). Efficiency is also improved.

  In order to generate the extended prediction image, when generating the first motion compensated prediction image and the second motion compensated prediction image for the encoding target block, the size of the prediction image is increased by a predetermined number of pixels in the extension direction. There is no need to newly access the reference image.

  In FIGS. 11A to 11D, the case where the region is divided in the horizontal direction has been described. However, in the case where the region is divided in the vertical direction, the encoded adjacent region on the upper side of the encoding target block is used for evaluation. Since the other points are the same, the description is omitted.

After calculating all the activities and adjacent area evaluation values used for the boundary evaluation, the boundary candidates are evaluated using the evaluation values defined in advance (S08). For example, the evaluation value is defined as follows.
Evaluation value = -A * ACT1-B * ACT2 + C * ACT3-D * ACT4 + E * NEI
Here, ACT1 represents a first activity value, ACT2 represents a second activity value, ACT3 represents a third activity value, ACT4 represents a fourth activity value, and NEI represents an adjacent region evaluation value. A, B, C, D, and E are constants of 0 or more, respectively, and usually A = B.
In addition, since the adjacent area evaluation value cannot be calculated when the adjacent area is uncoded such as the edge of the image or the boundary of the slice, the adjacent area evaluation value is excluded from the evaluation value.

  However, if the magnitudes of the first motion vector and the second motion vector are used to make A and B variables instead of constants, the accuracy of the boundary evaluation is further improved. For example, when the size of the first motion vector is small and the size of the second motion vector is large, in the first activity and the second activity, since the second activity is a more reliable index, A < A and B are set to be B. This is because the edge of the moving region is more likely to be the boundary of the actual motion than the edge of the non-moving region, and the coding efficiency is improved when the boundary of the motion compensation prediction is set near the boundary of the actual motion. This is because of the increase.

  As another method using the magnitude of the motion vector, when the difference between the first motion vector and the second motion vector is less than a predetermined threshold T1, the first motion compensation predicted image and the second motion compensation predicted image are similar. Since it is an image, the effect of determining the boundary other than the center may be small. In this case, it is possible to fix the boundary to the center of the block without performing the boundary determination process. Conversely, if the difference between the first motion vector and the second motion vector is greater than or equal to the predetermined threshold T2, it may be better to have less influence of motion compensation from the other region across this boundary. You may make it not carry out, you may make it the range which uses OBMC narrow, or make the intensity | strength which uses OBMC weak.

  The evaluation values are calculated for all main boundary candidates, and the main boundary candidate having the minimum value is determined as the final main boundary (S09).

  After the boundary determination, as described in FIGS. 8A to 8C, the first motion compensation predicted image and the second motion compensation predicted image are synthesized to generate a synthesized motion compensated prediction image. In the embodiment, it is also possible to perform motion compensation prediction in which the first motion compensation prediction image and the second motion compensation prediction image are superimposed on the peripheral region of the boundary. A technique for superimposing a plurality of motion compensated prediction images across block boundaries is called OBMC (Overlapped Block Motion Compensation). OBMC superimposes a plurality of motion-compensated prediction images, and can suppress the energy of prediction errors by averaging errors, thereby improving coding efficiency and making it difficult to visually recognize block boundaries. The image quality is also improved. In particular, when the orthogonal transform corresponding to all the motion compensation sizes cannot be used and the motion compensation prediction boundary is included in the prediction error set to be orthogonal transform, the motion compensation boundary is straddled. By superimposing a plurality of motion compensated prediction images, the energy of the transform coefficient after the orthogonal transform can be significantly suppressed, so that the effect of further improving the coding efficiency is achieved.

  FIGS. 12A to 12C are diagrams illustrating a method of generating a combined motion compensated prediction image by combining the first motion compensated prediction image and the second motion compensation prediction image using OBMC. When OBMC is not used, as described in FIGS. 8A to 8C, in the example in which the region is divided into two in the horizontal direction, the first motion compensated prediction image is displayed in the region above the main boundary after the main boundary is determined. And the second motion compensated prediction image is employed in the region below the boundary. On the other hand, when OBMC is used, the first motion compensated prediction image is extended by a predetermined number of pixels below the main boundary as shown in FIG. 12A, and the second motion compensated prediction image is obtained as shown in FIG. 12B. Is extended by a predetermined number of pixels above the main boundary, and the first motion compensation predicted image and the second motion compensated predicted image are extended for the peripheral region of the predetermined number of pixels across the main boundary as shown in FIG. The superimposed image (referred to as “OBMC image”) is used as a final motion compensated prediction image.

  FIG. 13 is a diagram for explaining a superposition coefficient when an OBMC image is generated by superimposing the first motion compensated predicted image and the second motion compensated predicted image in the peripheral region of the boundary. Here, an example of the superposition coefficient when the range to which OBMC is applied is set to four pixels above and below this boundary is shown. The superposition coefficients of the first motion compensation predicted image and the second motion compensation predicted image are symmetric with respect to the main boundary, and the effect of motion compensation from the other region becomes smaller as the pixel moves away from the main boundary. A superposition coefficient is set.

  For the pixel row immediately above this boundary, the overlapping ratio of the first motion compensated predicted image and the second motion compensated predicted image is 2: 1, and the pixels of the two predicted images are weighted and added at this ratio. A pixel of the OBMC image is generated. For the pixel column immediately below this boundary, the overlapping ratio of the first motion compensated predicted image and the second motion compensated predicted image is 1: 2.

  In the pixel row two, three, four above the boundary, the overlapping ratio of the first motion compensated predicted image and the second motion compensated predicted image is 4: 1, 6: 1, or 8: 1. Yes, the influence of the second motion compensated prediction image decreases as the distance from the boundary increases. In the pixel rows two, three, four, and four pixels below the boundary, the overlapping ratio of the first motion compensated predicted image and the second motion compensated predicted image is 1: 4, 1: 6, and 1: 8. Yes, the influence of the first motion compensated prediction image decreases as the distance from the boundary decreases downward.

  The application range of OBMC is four pixels above and below the main boundary, and outside the application range of OBMC, the first motion compensated prediction image is adopted in the upper region as in the case where OBMC is not used, and in the lower region from the main boundary. 2. Adopt a motion compensated prediction image. In this example, 5 pixels or more from the main boundary, the upper pixel row uses the first motion compensated prediction image, and the second motion compensated prediction image is not used. The first motion compensated predicted image is used without using the second motion compensated predicted image.

  Here, in the present embodiment, when determining the main boundary of the motion compensation prediction, the first and second motion prediction images are generated in advance by performing motion compensation with the size of the encoding target block larger than the divided region. Therefore, it is not necessary to newly access the reference image in order to obtain a motion prediction image for the overlapping region necessary for OBMC. In other words, in order to perform the process for automatically determining the boundary of motion compensation prediction on the decoding side and the OBMC process, the common process for motion compensation prediction from other regions needs to be performed only once, and duplication of the process can be omitted.

  Furthermore, it is possible to generate an OBMC image with high prediction efficiency by adjusting the application range of OBMC and the intensity of OBMC using the boundary activity information used for the boundary determination. Specifically, if the difference between the first motion compensated predicted image and the second motion compensated predicted image is evaluated and it is determined that these predicted images are similar images, is the application range of OBMC widened? Increase the strength of OBMC. That is, the range of the peripheral region of the main boundary where the weighted addition of the first motion compensated predicted image and the second motion compensated predicted image is expanded, the weight of the other region is increased when performing weighted addition, The overlapping ratio should not be reduced so much even when they are separated. When the first motion compensation prediction image and the second motion compensation prediction image are similar images, prediction with less prediction error is performed by averaging errors as represented by bidirectional prediction of B pictures by performing OBMC. An image is obtained. Conversely, if the difference between the first motion compensated predicted image and the second motion compensated predicted image is large and these predicted images are not similar images, it is better that the influence of motion compensation from the other region is less. Therefore, the application range of OBMC is narrowed or the strength of OBMC is weakened.

  Here, it is desirable that the main boundary and the temporary boundary are the same, but the determined main boundary is not necessarily the same as the temporary boundary. The temporary boundary for motion vector detection is a boundary for obtaining an optimal motion vector on the encoding side, and can be calculated using an original image to be encoded that can be used only on the encoding side. On the other hand, this boundary must be able to be uniquely calculated on both the encoding side and the decoding side, and a plurality of calculated motion vectors (transmitted on the decoding side) and their motion compensated prediction images (that is, prediction residuals). Image before the component is added). For this reason, there is no mismatch between the encoding side and the decoding side even if the main boundary and the temporary boundary are not the same.

  However, the difference between the main boundary and the temporary boundary means that an optimal motion vector has not been detected for the combined motion compensated prediction image after the determination of the main boundary, and the prediction efficiency may not necessarily be improved. Therefore, by adjusting the temporary region set by the temporary region dividing unit 101 or adjusting the motion vector detected by the motion vector detecting unit 102, the optimal temporary boundary or the optimal motion vector and the optimal main boundary are adjusted. If both are realized at the same time, the encoding efficiency can be further improved.

  Hereinafter, a configuration for optimizing the prediction efficiency of the combined motion compensated prediction image generated by the region division / motion compensation unit 103 by adjusting the motion vector detected by the motion vector detection unit 102 as the second embodiment will be described. explain. Further, a configuration for optimizing the prediction efficiency of the combined motion compensated prediction image generated by the main region division / motion compensation unit 103 by adjusting the temporary boundary set by the temporary region division unit 101 is described as a third embodiment. explain.

(Embodiment 2)
The moving picture coding apparatus according to the second embodiment has the same configuration as that of the moving picture coding apparatus shown in FIG. 1, but in the second embodiment, the motion dividing unit 103 moves from the region dividing / motion compensating unit 103 to the motion vector detecting unit 102. A route for sending a signal for instructing the vector adjustment is further added. Thereby, the processing of the motion vector detection unit 102 and the main region dividing / motion compensating unit 103 forms a loop, and the main boundary determined by the main region dividing / motion compensating unit 103 is the temporary boundary by the temporary region dividing unit 101. The motion vector detection unit 102 adjusts the motion vector until they match or are close enough.

  FIG. 14 is a flowchart for explaining a motion vector adjustment procedure by the moving picture coding apparatus according to the second embodiment. The motion vector detection unit 102 detects a motion vector based on the temporary boundary (S11), and then the region segmentation / motion compensation unit 103 performs a main motion detection based on the motion vector detected by the motion vector detection unit 102. A boundary determination process is performed (S12). As described with reference to FIG. 9, this boundary determination process is performed by selecting the boundary candidate having the best evaluation value of the boundary activity from among the boundary candidates.

  When the main boundary determined by the main area division / motion compensation unit 103 is equal to the temporary boundary, the motion vector detection by the motion vector detection unit 102 is terminated. When the determined main boundary is different from the temporary boundary, the motion vector The motion vector detection by the detection unit 102 is continued (S13). When the main boundary and the temporary boundary are different, the motion vector detection unit 102 redoes the motion vector detection in one of the regions temporarily divided into two regions, for example. Any method may be used for redoing, for example, the second smallest value (second minimum value) of the error evaluation values at the time of motion vector search is left, and the motion vector is detected from the position of the second minimum value. It is possible to continue the process.

  Using the motion vector redetected by the motion vector detection unit 102, the region segmentation / motion compensation unit 103 performs the boundary determination process again. The motion vector detection processing by the motion vector detection unit 102 and the main boundary determination processing by the region division / motion compensation unit 103 are repeated until the main boundary coincides with the temporary boundary or becomes sufficiently close.

(Embodiment 3)
The moving picture coding apparatus according to the third embodiment has the same configuration as that of the moving picture coding apparatus shown in FIG. 1, but in the third embodiment, the temporary area dividing unit 101 sets a plurality of temporary boundary candidates and moves The vector detection unit 102 detects a motion vector for each temporary boundary candidate, and the region segmentation / motion compensation unit 103 performs main boundary determination processing for each temporary boundary. Then, the region segmentation / motion compensation unit 103 selects the one having the best prediction efficiency of the combined motion compensated prediction image among the plurality of temporary boundaries.

  FIG. 15 is a flowchart for explaining a procedure for adjusting a temporary boundary by the moving picture coding apparatus according to the third embodiment. The temporary region dividing unit 101 sets a plurality of temporary boundary candidates, and the motion vector detecting unit 102 detects a motion vector for each temporary region divided by each temporary boundary candidate (S21). The region segmentation / motion compensation unit 103 performs the boundary determination process based on the motion vector detected for each temporary region divided by each temporary boundary candidate (S22), and evaluates the prediction efficiency of the combined motion compensated prediction image. (S23). The prediction efficiency is evaluated by SAD or the like for the difference between the original image and the synthesized motion compensated prediction image. This evaluation is performed for a plurality of temporary boundary candidates (S24), and the temporary boundary with the highest prediction efficiency of the motion compensated prediction image is selected from the temporary boundary candidates. The region segmentation / motion compensation unit 103 outputs, as a final result, a combined motion compensated prediction image generated under the main boundary determined for the motion vector for the selected temporary boundary.

  As another method, as in the second embodiment, a path for sending a signal for instructing the adjustment of the temporary boundary from the main region dividing / motion compensating unit 103 to the temporary region dividing unit 101 is added. The processes of the motion vector detection unit 102 and the region segmentation / motion compensation unit 103 may be configured to form a loop. The temporary region dividing unit 101 adjusts the temporary boundary until the main boundary determined by the main region dividing / motion compensating unit 103 matches the temporary boundary by the temporary region dividing unit 101 or becomes sufficiently close. When the main boundary determined by the main region dividing / motion compensating unit 103 is equal to the temporary boundary, the setting of the temporary boundary by the temporary region dividing unit 101 ends. However, when the determined main boundary is different from the temporary boundary, The region dividing unit 101 sets another temporary boundary candidate, and the motion vector detecting unit 102 redetects a motion vector for each temporary region divided by the reset temporary boundary, and this region dividing / motion compensating unit 103 performs the boundary determination process again. The temporary boundary setting process by the temporary area dividing unit 101 and the main boundary determination process by the main area dividing / motion compensating unit 103 are repeated until the main boundary matches or becomes sufficiently close to the temporary boundary.

  Regardless of which method is used, as a result, the main boundary finally determined by the main region dividing / motion compensating unit 103 is coincident with or sufficiently close to the temporary boundary set by the temporary region dividing unit 101. Efficiency is improved.

  Next, the syntax of a moving image bit stream encoded by the moving image encoding apparatus according to the first to third embodiments will be described.

  FIG. 16 shows a first syntax pattern based on the syntax of MPEG-4 AVC. As shown in FIG. 16A, first, a first flag (use_auto_mc_size) indicating whether or not the shape of motion compensated prediction is automatically determined on the decoding side using the feature amount of the predicted image in units of slices is transmitted. When the first flag use_auto_mc_size is OFF, the motion compensation prediction is performed by dividing the macroblock fixedly based on the macroblock type mb_type shown in FIG. When the first flag use_auto_mc_size is ON, motion compensated prediction is performed by automatically determining the shape of motion compensated prediction on the decoding side using the feature amount of the predicted image. In the macro block unit, as in MPEG-4 AVC, mb_type is transmitted to determine the shape of motion compensation prediction.

  FIG. 17 shows the semantics of the macroblock type mb_type when the first flag use_auto_mc_size is ON / OFF. When the macroblock type mb_type = 0, motion compensation prediction is performed without dividing a 16 × 16 macroblock into regions. When the macro block type mb_type = 1, motion compensation prediction is performed with 16 × 8 when the first flag use_auto_mc_size is OFF, but when the first flag use_auto_mc_size is ON, the macro blocks are 16 × A and 16 × (16− A motion compensation prediction is performed by automatically dividing into two regions A). Similarly, when the macro block type mb_type = 2, when the first flag use_auto_mc_size is OFF, motion compensation prediction is performed with 8 × 16 blocks. However, when the first flag use_auto_mc_size is ON, the macro block is A × 16 ( Motion-compensated prediction is automatically performed by dividing into two areas of 16-A) × 16. When the macro block type mb_type = 3, motion compensation prediction is performed with 8 × 8 blocks.

  FIG. 18 shows a second syntax pattern for transmitting a second flag (auto_mc_size_enable) indicating whether or not to automatically determine the shape of motion compensated prediction not only at the slice level but also in macroblock units on the decoding side. When the second flag auto_mc_size_enable is OFF, as in the case where the first flag use_auto_mc_size is OFF at the slice level, the macroblock is fixedly divided based on the macroblock type mb_type as before, and motion compensation prediction is performed. When the second flag auto_mc_size_enable is ON, motion compensated prediction is performed by automatically determining the shape of motion compensated prediction on the decoding side using the feature amount of the predicted image. When the second syntax is used, it is possible to eliminate the case where the prediction efficiency is lowered by automatically determining the boundary.

  FIG. 19 shows a third syntax pattern for switching an algorithm for automatically determining the shape of motion compensation prediction at the slice level on the decoding side. When the first flag use_auto_mc_size is ON for each slice, an algorithm type auto_mc_algorithm indicating an algorithm type for automatically determining the shape of motion compensated prediction on the decoding side is transmitted. For example, when the algorithm type auto_mc_algorithm = 0, the shape of the motion compensation prediction is determined based on the evaluation values using all of the first to fourth activities ACT1 to ACT4. When the algorithm type auto_mc_algorithm = 1, the shape of the motion compensation prediction is determined based on the evaluation value using only the first to third activities ACT1 to ACT3 excluding the fourth activity ACT4. In this way, by associating the type of activity used with the value of the algorithm type auto_mc_algorithm, the algorithm for determining the shape of the motion compensation prediction can be switched. When the third syntax is used, it is possible to determine an optimal algorithm type on the encoding side and to perform automatic segmentation on the decoding side, thereby further improving the encoding efficiency.

  FIG. 20 shows a fourth syntax pattern in which it is not distinguished on the syntax whether the macro block is divided into the horizontal direction or the vertical direction (division direction). It is the same as the first syntax pattern to transmit the macroblock type mb_type in units of macroblocks and determine the shape of the motion compensation prediction, but the macroblock type mb_type has different semantics.

  FIG. 21 shows the semantics of the macro block type mb_type of the fourth syntax pattern. When the first flag use_auto_mc_size is OFF, the macroblock is fixedly divided based on the macroblock type mb_type as before, and motion compensation prediction is performed. When the first flag use_auto_mc_size is ON, the feature amount of the predicted image is used. The motion compensation prediction is performed by automatically determining the shape of the motion compensation prediction on the decoding side, as in the first syntax pattern. However, dividing into two areas of 16 × A and 16 × (16−A) in the horizontal direction and dividing into two areas of A × 16 and (16−A) × 16 in the vertical direction are possible. Unlike the first syntax pattern, the macroblock type mb_type = 1 is treated without distinction. When the macroblock type mb_type = 1, all boundary evaluation values of the horizontal main boundary candidate and the vertical main boundary candidate are calculated, and the boundary candidate having the minimum evaluation value is determined as the main boundary. That is, the shape of motion compensation prediction including the division direction (horizontal direction / vertical direction) is automatically determined on the decoding side. When the fourth syntax pattern is used, since it is not necessary to transmit division direction information for distinguishing between the horizontal direction and the vertical direction, the code amount of the macroblock type mb_type is reduced, and the encoding efficiency is further improved.

  FIG. 22 shows a fifth syntax pattern for determining region division without being linked with the macroblock type mb_type. The motion vector number motion_vector_num_minus1 is transmitted instead of the macroblock type mb_type in units of macroblocks. The number of motion vectors motion_vector_num_minus1 represents a value of (number of motion vectors in the macroblock-1). In the embodiment of the present invention, since the motion compensation prediction area is divided by the number of motion vectors to be transmitted, the shape of motion compensation prediction can be automatically determined on the decoding side if at least the number of motion vectors is transmitted. there is a possibility. In the first to third embodiments, an example in which there are two motion vectors has been described. However, an example in which there are more than two motion vectors, for example, three, will be described in a fifth embodiment.

  Up to this point, the embodiments of the present invention have been described for unidirectional prediction used in MPEG P-pictures. With reference to FIG. 23, a case will be described in which the embodiment of the present invention is applied to bi-directional prediction (usually forward prediction and backward prediction) used in a B picture or the like. Here, a case where a macroblock is divided into two in the horizontal direction will be described as an example. Bidirectional prediction is a technique for obtaining a predicted image by averaging or weighted averaging two images obtained by performing motion compensation prediction from two reference images.

  First, a motion vector in each prediction direction (forward direction, backward direction) is detected for each temporary region. FIGS. 23A and 23B show the first motion vector in each prediction direction (forward direction, backward direction) with respect to the first temporary area. FIGS. 23A and 23B show the second motion vector in each prediction direction (forward direction, backward direction) with respect to the second temporary area.

  FIG. 23 (e) shows a first motion compensated prediction image that has been bi-directionally predicted using forward and backward first motion vectors, and FIG. 23 (f) shows forward and backward second motion vectors. The 2nd motion compensation prediction image bi-predicted using is shown.

  FIG. 23 (g) shows that the main region above the main boundary of the macroblock is copied from the corresponding region of the first motion compensated prediction image after the main boundary is determined, and the main region below the main boundary is copied to the second motion compensation. The synthetic | combination motion compensation prediction image produced | generated by duplicating from the area | region corresponding to a prediction image is shown.

  As described above, it is easy to apply the embodiment of the present invention to bidirectional prediction, and prediction errors can be further reduced by appropriate region division while reducing prediction errors by bidirectional prediction.

  The embodiment of the present invention does not directly transmit a motion vector, but automatically uses a motion vector automatically calculated based on a motion vector of a peripheral block or a motion vector of a reference image to automatically change the shape of motion compensation prediction. Of course, it is also possible to decide.

  Although a single image component (luminance) has been described so far, it can be applied to a plurality of components (luminance and color difference) such as YUV4: 4: 4 / YUV4: 2: 2 / YUV4: 2: 0. Is possible. However, since motion compensation prediction performs luminance and color difference at the same time, in a format with different luminance and color difference samples such as YUV4: 2: 2 and YUV4: 2: 0, the region is divided based on the luminance with a large number of samples. In this case, the area division position of the color difference may be ambiguous. For example, when the luminance is divided into 16 × 5 and 16 × 11 in the YUV 4: 2: 0 format, it is unknown whether the color difference area division is 8 × 2 or 8 × 3. As measures to prevent this ambiguity, the area is divided in advance based on the color difference with a small number of samples, and the division rule for the color difference at the ambiguous position is determined in advance (the block center is discarded). There are a method and a method using an average value (filtering) of both motion compensated prediction images for pixels on the boundary.

(Embodiment 4)
The moving picture encoding apparatus and moving picture decoding apparatus according to the fourth embodiment have the same configurations as those of the first to third embodiments. However, the moving picture encoding apparatus according to the fourth embodiment divides a macroblock into two dimensions. Then, motion compensation is performed and two motion vectors are transmitted, and the video decoding device divides the motion compensated prediction shape into two dimensions on the decoding side, and performs motion compensation using the transmitted two motion vectors. .

  24A to 24F show a method of dividing a macroblock into two dimensions. As shown in FIG. 24A, the horizontal boundary and the vertical boundary are defined for the macroblock by executing the procedure described in the first to third embodiments. The two two-dimensional division patterns shown in FIGS. 24B to 24E can be defined by combinations above or below the horizontal boundary and left or right from the vertical boundary.

  In the two-dimensional division pattern of FIG. 24B, the area above the horizontal boundary and to the left of the vertical boundary is the first area, and the remaining area (below the horizontal boundary or left of the vertical boundary) Area) is the second area.

  In the two-dimensional division pattern of FIG. 24C, the area below the horizontal boundary or the left of the vertical boundary is the first area, and the remaining area (above the horizontal boundary and to the right of the vertical boundary). Area) is the second area.

  In the two-dimensional division pattern of FIG. 24D, the area above the horizontal boundary or right of the vertical boundary is the first area, and the remaining area (below the horizontal boundary and left of the vertical boundary). Area) is the second area.

  In the two-dimensional division pattern of FIG. 24 (e), the area above the horizontal boundary or the left of the vertical boundary is the first area, and the remaining area (below the horizontal boundary and to the right of the vertical boundary). Area) is the second area.

  In FIG. 24F, one of these four two-dimensional division patterns is selected, the corresponding region of the motion compensated prediction image based on the first motion vector is used for the first region, and the second motion vector is used for the second region. The composite motion compensation prediction image obtained by duplicating the corresponding region of the motion compensation prediction image is shown.

FIG. 25 is a flowchart illustrating a procedure for performing motion compensation by dividing a macroblock into two dimensions. First, both the horizontal boundary and the vertical boundary are determined by the same method as in the first embodiment (S31). Next, the evaluation value of the boundary activity is calculated for the two-dimensional divided area of the macroblock divided into two by the combination of the horizontal boundary and the vertical boundary (S32). For example, if the X coordinate in the macroblock is i, and the pixel value at the Y coordinate is j is A _{i, j} , the horizontal activity at the horizontal boundary j is changed from i = a to i = The two-dimensional average activity when applying up to b and applying the vertical activity at the vertical boundary i from j = c to j = d can be defined as follows.
Two-dimensional average activity = Σ _{i = a} ^b | A _{i, j} −A _{i, j−1} | / (b−a) + Σ _{j = c} ^d | A _{i, j} −A _{i−1, j} | / (d -C)

  By using the above two-dimensional average activity as the evaluation value, it is possible to evaluate the two-dimensional region division without depending on the number of samples used for the activity calculation. Step S32 is repeated for all two-dimensional division patterns (here, four patterns) (S33), a two-dimensional division candidate having the minimum evaluation value is selected, and a combined motion compensated prediction image is generated.

  As described above, in the fourth embodiment, since motion compensation prediction can be performed with a more flexible shape than in the first embodiment, the coding efficiency is further improved.

(Embodiment 5)
The moving picture encoding apparatus and moving picture decoding apparatus of the fifth embodiment have the same configurations as those of the first to third embodiments, but the moving picture encoding apparatus of the fifth embodiment uses three motion vectors. The macroblock is divided into three to perform motion compensation, and three motion vectors are transmitted. The moving picture decoding apparatus divides the shape of motion compensated prediction into three on the decoding side using the transmitted three motion vectors. Perform motion compensation.

  FIGS. 26A and 26B show a method of dividing a macroblock into three regions. As shown in FIG. 26A, first, the procedure described in the first to third embodiments is executed to divide the macroblock into two at the horizontal boundary or the vertical boundary. Next, as shown in FIG. 26B, the larger region is further divided into two by setting a horizontal boundary or a vertical boundary for the larger region among the two divided regions. Thereby, the macroblock is divided into three, and a motion vector is detected in each region.

  FIG. 27 is a flowchart illustrating a procedure for performing motion compensation by dividing a macroblock into three. In order to perform motion compensation by dividing a macroblock into three regions, it is necessary to detect and transmit three motion vectors. First, using the first motion vector and the second motion vector in the same manner as in the first embodiment, the region is divided in the horizontal direction or the vertical direction, and motion compensation is performed (S41). Next, the size of the region divided by the first motion vector and the second motion vector is compared, and the larger region is determined (S42). This is because the larger region is more affected by the region division, and the prediction efficiency can be expected to improve more greatly. Also, when the size of the area is the same, previously you decide in advance which of the regions are preferentially. Finally, for the larger region, using the motion vector of the region (first motion vector or second motion vector) and the third motion vector, the region is divided in the horizontal direction or the vertical direction, and motion compensation is performed. Perform (S43).

  As described above, in the fifth embodiment, the motion compensation prediction region can be divided into three using three motion vectors. As a result, it is possible to cope with a plurality of small movements and the like, so that the encoding efficiency is further improved. Similarly, by further dividing the region, the region of motion compensation prediction can be divided into four or more and the number of motion vectors can be four or more.

  As described above, according to the embodiment of the present invention, by using the feature amounts of a plurality of predicted images obtained from a plurality of motion vectors, the shape of motion compensated prediction is automatically determined on the decoding side. The shape of the motion compensation prediction can be made variable without transmitting the information of the motion compensation prediction shape pattern. Therefore, flexible motion compensation prediction in various shapes is possible. As a result, the prediction error of motion compensation prediction can be reduced without increasing the code amount of the additional information, and the coding efficiency is improved.

  Also, in the motion vector detection process on the encoding side, both the optimal motion vector and the optimal main boundary were realized simultaneously by evaluating the motion vector while calculating the main boundary determined on the decoding side. Thereby, the prediction efficiency of motion compensation prediction improves. On the decoding side, a moving image can be decoded only by performing motion compensation prediction using this boundary calculated from the transmitted motion vector.

  The above processing relating to encoding and decoding can be realized as a transmission, storage, and reception device using hardware, and is stored in a ROM (Read Only Memory), a flash memory, or the like. It can also be realized by firmware or software such as a computer. The firmware program and software program can be recorded on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting Is also possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  101 Temporary region segmentation unit, 102 Motion vector detection unit, 103 Region segmentation / motion compensation unit, 104 Orthogonal transformation / quantization unit, 105 Variable length coding unit, 106 Inverse quantization / inverse orthogonal transformation unit, 107 Reference image memory , 108 subtraction unit, 109 addition unit, 201 variable length decoding unit, 203 area division / motion compensation unit, 206 inverse quantization / inverse orthogonal transform unit, 207 reference image memory, 209 addition unit.

Claims (10)

A decoding unit that decodes a plurality of motion vectors for the decoding target block from the encoded stream;
Generate a plurality of prediction blocks corresponding to the decoding target block from a reference image using a plurality of motion vectors, determine a boundary based on activities of the plurality of prediction blocks, and divide each prediction block by the boundary A motion compensation unit that generates a combined prediction block by overlapping each prediction block with respect to a peripheral region of the boundary and weighting and adding them at a predetermined ratio when the regions obtained in the above are combined between the prediction blocks; ,
A moving picture decoding apparatus comprising: an adding unit that generates a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. A decoding unit that decodes a plurality of motion vectors for the decoding target block from the encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. A motion compensation unit that generates a combined prediction block by combining the regions obtained by combining the prediction blocks;
A moving picture decoding apparatus comprising: an adding unit that generates a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. A decoding unit that decodes a plurality of motion vectors for the decoding target block from the encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. A motion compensation unit that generates a combined prediction block by overlapping each prediction block with respect to a peripheral region of the boundary and weighting and adding them at a predetermined ratio when the regions obtained in the above are combined between the prediction blocks; ,
A moving picture decoding apparatus comprising: an adding unit that generates a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block.   The motion compensation unit adjusts at least one of a range of the surrounding area of the boundary and a ratio of weighted addition that overlap each prediction block based on activities of the plurality of prediction blocks. Item 4. The moving picture decoding device according to Item 1 or 3. Decoding a plurality of motion vectors for the decoding target block from the encoded stream;
Generate a plurality of prediction blocks corresponding to the decoding target block from a reference image using a plurality of motion vectors, determine a boundary based on activities of the plurality of prediction blocks, and divide each prediction block by the boundary When combining the regions obtained in the above, between each of the prediction blocks, for the peripheral region of the boundary, by overlapping each prediction block and weighted and added at a predetermined ratio, to generate a combined prediction block;
A moving picture decoding method comprising: a step of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. Decoding a plurality of motion vectors for the decoding target block from the encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. Generating a combined prediction block by combining the regions obtained by combining the prediction blocks;
A moving picture decoding method comprising: a step of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. Decoding a plurality of motion vectors for the decoding target block from the encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. When combining the regions obtained in the above, between each of the prediction blocks, for the peripheral region of the boundary, by overlapping each prediction block and weighted and added at a predetermined ratio, to generate a combined prediction block;
A moving picture decoding method comprising: a step of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. A function of decoding a plurality of motion vectors for a decoding target block from an encoded stream;
Generate a plurality of prediction blocks corresponding to the decoding target block from a reference image using a plurality of motion vectors, determine a boundary based on activities of the plurality of prediction blocks, and divide each prediction block by the boundary When combining the regions obtained in the above, between each of the prediction blocks, for the peripheral region of the boundary, by overlapping each prediction block and weighted and added at a predetermined ratio, to generate a combined prediction block,
A moving picture decoding program that causes a computer to realize a function of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. A function of decoding a plurality of motion vectors for a decoding target block from an encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. A function of generating a synthesized prediction block by combining the regions obtained by combining the prediction blocks,
A moving picture decoding program that causes a computer to realize a function of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block. A function of decoding a plurality of motion vectors for a decoding target block from an encoded stream;
A plurality of prediction blocks corresponding to the decoding target block and a plurality of extended prediction images corresponding to decoded regions adjacent to the decoding target block are generated from a reference image using a plurality of motion vectors, and each extended prediction is generated. A boundary is determined based on at least one of an adjacent evaluation value obtained by evaluating a prediction error between an image and a decoded image of the decoded area and an activity of each prediction block, and each prediction block is divided by the boundary. When combining the regions obtained in the above, between each of the prediction blocks, for the peripheral region of the boundary, by overlapping each prediction block and weighted and added at a predetermined ratio, to generate a combined prediction block,
A moving picture decoding program that causes a computer to realize a function of generating a decoded image by adding the synthesized prediction block and a prediction difference block decoded from the decoding target block.