JP2011254395A - Moving image encoding apparatus, moving image encoding method and moving image encoding program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve an encoding efficiency of the conventional moving image encoding/decoding which is motion compensation predicted in a shape pattern prepared in advance.SOLUTION: A first motion vector detection section 108 detects a first motion vector in a local decoded template area adjacent vertically or horizontally to an encoding target block. A virtual area division section 101 divides the encoding target block into two virtual areas. A second motion vector detection section 102 detects a second motion vector by using an original image of the encoding target block to the virtual area having a smaller part adjacent to the template area. A main area division and motion compensation section 103 generates two prediction blocks corresponding to the encoding target block from a reference image by using the first and the second motion vectors, determines a main boundary based on activities of the two prediction blocks and combines areas which are obtained by dividing the each prediction block by the main boundary to generate a synthesis prediction block.

Description

  The present invention relates to a moving picture coding technique, and more particularly to a moving picture coding technique using motion compensated prediction.

  In compression coding of moving images represented by MPEG (Moving Picture Experts Group), motion compensation prediction that compresses a code amount by using correlation between screens is often used. In motion compensated prediction used in MPEG or the like, a predicted image is generated for each block of a predetermined size from a reference image that has already been decoded using a motion vector that indicates the relative positional relationship between the encoding target image and the reference image. Thereafter, the encoding side calculates a prediction error that is a difference between the original image to be encoded and a prediction image generated by motion compensated prediction, and transmits only the prediction error to the decoding side. Thereby, compared with the case where motion compensation prediction is not used, the code amount transmitted can be reduced significantly.

  In general, encoding / decoding processing is performed in units of macroblocks (pixel group having a predetermined block size, for example, 16 × 16). Motion compensation prediction is often performed in units of macroblocks, but in this case, it is difficult to capture the motion of an object or the like that is smaller than the macroblock, resulting in a decrease in coding efficiency. Therefore, multi-block pattern motion compensated prediction is used as a method for making motion compensated prediction function more efficiently.

  In multi-block pattern motion compensation prediction, a macro block can be further divided into sub blocks, and motion compensation prediction can be performed using different motion vectors for each sub block. The division pattern of the motion compensation prediction block to be used is defined in advance as the same rule on the encoding side and the decoding side. On the encoding side, a block pattern for motion compensation prediction is selected from the defined block patterns, and block pattern selection information is transmitted to the decoding side. On the decoding side, motion compensation prediction is performed based on the received block pattern selection information. In multi-block pattern motion compensated prediction, when the optimum block pattern for motion compensated prediction is selected on the encoding side, the prediction error after motion compensated prediction is reduced, and the encoding efficiency is improved.

  As an example of a specific block pattern, in MPEG-4 AVC (Advanced Video Coding) internationally standardized by a joint video team (JVT) of ISO / IEC and ITU-T, there are further 16 macroblocks (16 × 16 blocks). It is possible to perform motion compensation prediction by dividing into block patterns of × 8/8 × 16/8 × 8/8 × 4/4 × 8/4 × 4. On the encoding side, a block pattern is selected, and block pattern selection information is encoded in the bitstream. On the decoding side, the macroblock is divided into regions according to the block pattern encoded in the bitstream, and motion compensation prediction is performed for each divided region.

  Furthermore, Patent Literature 1 and Patent Literature 2 disclose techniques for defining motion compensation prediction shape patterns and performing motion compensation prediction with a more flexible shape.

Japanese Patent No. 4025570 Republished Patent No. WO2003-026315

  However, in the methods disclosed in Patent Document 1 and Patent Document 2, when the number of motion compensated prediction shape patterns to be defined is increased, the amount of code consumed for selection information of the shape pattern to be transmitted increases as the number of shape patterns increases. End up. In other words, a decrease in prediction error due to an increase in shape patterns and an increase in code amount related to shape pattern selection are in a trade-off relationship, so it is difficult to improve overall coding efficiency simply by increasing the number of shape patterns. .

  As described above, in the conventional moving image encoding / decoding, only motion compensation prediction of a predefined shape pattern can be performed. Therefore, motion compensation prediction cannot be performed with an optimal shape, and encoding efficiency cannot be improved. . In addition, when the number of motion compensation predicted shape patterns to be defined is increased, the amount of additional information related to the selection of motion compensated predicted shape patterns increases, and thus the overall coding efficiency is not always improved.

  The present invention has been made in view of such circumstances, and an object of the present invention is to enable motion compensation prediction in various block patterns without increasing the amount of additional information regarding the block pattern of motion compensation prediction. Therefore, an object of the present invention is to provide a technique for improving the coding efficiency by reducing the prediction error.

  In order to solve the above-described problem, a moving image encoding apparatus according to an aspect of the present invention includes a first motion vector determination unit (108) that determines a first motion vector of a predetermined region that has been locally decoded and is adjacent to an encoding target block. ), A temporary area dividing unit (101) that divides the encoding target block into two temporary areas at a predetermined temporary boundary, and a temporary area that has a smaller portion adjacent to the predetermined area of the two temporary areas On the other hand, a second motion vector detection unit (102) that detects a second motion vector using the original image of the encoding target block, and a reference image using the first motion vector and the second motion vector Obtained two prediction blocks corresponding to the block to be encoded, determined the main boundary based on the activities of the two prediction blocks, and obtained by dividing each prediction block at the main boundary A motion compensation unit (103) that generates a composite prediction block by combining a region between the prediction blocks, a prediction difference block obtained by subtracting the composite prediction block from the encoding target block, and a motion of each temporary region And an encoding unit (105) for encoding the vector.

  Another aspect of the present invention is also a moving image encoding apparatus. The apparatus includes: a first motion vector of a first predetermined region that has been locally decoded adjacent to a block to be encoded; and a second motion vector of a second predetermined region that has been decoded and that is adjacent to the block to be encoded. A motion vector determination unit (108) that determines two motion vectors, and using the first motion vector and the second motion vector, generate two prediction blocks corresponding to the coding target block from a reference image; Motion compensation that generates a combined prediction block by determining a main boundary based on activities of two prediction blocks and combining regions obtained by dividing the prediction blocks at the main boundary between the prediction blocks Part (103), a prediction difference block obtained by subtracting the combined prediction block from the encoding target block, and a motion vector of each temporary area Comprising encoding unit for the (105).

  Yet another embodiment of the present invention is also a moving image encoding apparatus. The apparatus includes: a first motion vector for a first predetermined region that has been locally decoded adjacent to a current block to be encoded; and a second motion vector for a second predetermined region that has been locally decoded and adjacent to a current block to be encoded. A first and second motion vector determination unit (108) that determines a motion vector and adopts one of the first motion vector and the second motion vector; and A temporary area dividing unit that divides into two temporary areas at the boundary, and a temporary area that is not adjacent to the predetermined area in which the motion vector employed by the first and second motion vector determining units is determined, of the two temporary areas On the other hand, the third motion vector detection unit (102) that detects the third motion vector using the original image of the block to be encoded and the first and second motion vector determination units adopt it. Using the generated motion vector and the third motion vector, generate two prediction blocks corresponding to the block to be encoded from a reference image, determine a boundary based on activities of the two prediction blocks, A motion compensation unit (103) that generates a combined prediction block by combining regions obtained by dividing a prediction block at the main boundary between the prediction blocks, and the combined prediction block from the encoding target block An encoding unit (105) that encodes the subtracted prediction difference block and the motion vector of each temporary area is provided.

  Yet another embodiment of the present invention is also a moving image encoding apparatus. The apparatus includes a first motion vector of a first predetermined region that has been locally decoded adjacent to a block to be encoded in a horizontal direction, and a second motion vector of a second predetermined region that has been decoded and that is adjacent to the block to be encoded in a vertical direction. First and second motion vector determination units that determine a motion vector and select one of the first motion vector and the second motion vector as a main motion vector and the other motion vector as a sub motion vector ( 108) and the first motion vector and the second motion vector to generate first and second prediction blocks corresponding to the coding target block from a reference image, and activities of the first and second prediction blocks A motion compensator (103) for determining a first main boundary based on the first main boundary, and the first and second main regions divided by the first main boundary. The main region having a larger portion adjacent to the predetermined region where the motion vector selected as the sub motion vector by the two motion vector determination unit is selected is selected, and the selected main region is further divided into two temporary regions at a predetermined temporary boundary. A temporary area dividing unit (101) that divides the area into two areas, and a temporary area that is not adjacent to the predetermined area in which the motion vector selected as the sub-motion vector among the two temporary areas is determined. And a third motion vector detection unit (102) that detects a third motion vector using the original image. The motion compensation unit generates a third and a fourth prediction block corresponding to the coding target block from a reference image using the motion vector selected as the sub motion vector and the third motion vector, And a region obtained by determining a second main boundary based on the activity of the fourth prediction block and dividing the first or second prediction block at the first main boundary; and third and fourth prediction blocks Are combined with the regions obtained by dividing the second main boundary by the second main boundary to generate a combined prediction block. The apparatus further includes an encoding unit (105) that encodes a prediction difference block obtained by subtracting the combined prediction block from the encoding target block and a motion vector of each temporary region.

  Yet another aspect of the present invention is a video encoding method. This method includes a first motion vector for a first predetermined region that has been locally decoded adjacent to an encoding target block and a second motion vector for a second predetermined region that has been adjacent to the encoding target block and that is adjacent in the vertical direction. Determining a motion vector, adopting one of the first motion vector and the second motion vector, and dividing the encoding target block into two temporary regions at a predetermined temporary boundary; Among the two temporary areas, for the temporary area that is not adjacent to the predetermined area for which the motion vector adopted by the first and second motion vector determination units is used, using the original image of the encoding target block, A step of detecting a third motion vector, and using the motion vector employed by the first and second motion vector determination units and the third motion vector Two prediction blocks corresponding to the encoding target block are generated from an image, main boundaries are determined based on activities of the two prediction blocks, and regions obtained by dividing the prediction blocks at the main boundaries are obtained as described above. Generating a combined prediction block by combining the prediction blocks, encoding a prediction difference block obtained by subtracting the combined prediction block from the encoding target block, and a motion vector of each temporary region; Is provided.

  It should be noted that any combination of the above-described constituent elements and a conversion of the expression of the present invention between a method, an apparatus, a system, a recording medium, a computer program, etc. are also effective as an aspect of the present invention.

  According to the present invention, it is possible to reduce the prediction error and increase the coding efficiency by enabling the motion compensation prediction in various block patterns without increasing the code amount of the additional information regarding the block pattern of the motion compensation prediction. Can be improved.

1 is a block diagram illustrating a configuration of a moving image encoding apparatus according to Embodiment 1. FIG. 1 is a block diagram illustrating a configuration of a moving image decoding apparatus according to Embodiment 1. FIG. It is a figure which shows the motion vector detected for every temporary area | region of the macroblock divided | segmented by the horizontal boundary. It is a figure which shows the motion vector detected for every temporary area | region of the macroblock divided | segmented by the vertical boundary. It is a figure explaining the motion vector detection method using a template. It is a figure which shows the motion compensation estimated image produced | generated from the motion vector detected for every temporary area | region. It is a figure which shows the synthetic | combination motion compensation prediction image obtained by synthesize | combining the motion compensation prediction image produced | generated from the motion vector of each temporary area | region. It is a figure explaining the method of synthesize | combining the motion compensation estimated image produced | generated from the motion vector of each temporary area | region, and producing | generating a synthetic | combination motion compensation prediction image. It is a flowchart which shows the procedure of temporary area | region division | segmentation and this boundary determination. It is a figure explaining the suitable method of the temporary division | segmentation of an encoding object block. 6 is a flowchart for explaining a main boundary determination procedure by the main region dividing / motion compensating unit in FIG. 1. It is a figure explaining the activity regarding this boundary determination about a motion compensation prediction image. It is a figure which shows the 1st syntax pattern of the bit stream of the moving image encoded by the moving image encoding device of embodiment. It is a figure which shows the semantics of macroblock type mb_type in case the flag which shows whether the shape of a motion compensation prediction is determined automatically on a decoding side by slice unit is ON / OFF, respectively. FIG. 10 is a block diagram illustrating a configuration of a moving image encoding apparatus according to a second embodiment. 6 is a block diagram illustrating a configuration of a moving image decoding apparatus according to Embodiment 2. FIG. 10 is a flowchart showing a main boundary determination procedure in the second embodiment. It is a figure which shows a mode that this boundary is determined using the motion vector calculated from each template area | region. It is a figure which shows the semantics of macroblock type mb_type in case the flag which shows whether the shape of a motion compensation prediction is determined automatically on a decoding side by slice unit is ON / OFF, respectively. 10 is a flowchart showing a main boundary determination procedure in the third embodiment. It is a figure which shows a mode that this boundary is determined using the motion vector calculated from each template area | region. FIG. 10 is a block diagram illustrating a configuration of a moving image encoding device according to a fourth embodiment. FIG. 10 is a block diagram illustrating a configuration of a moving picture decoding apparatus according to a fourth embodiment. 10 is a flowchart showing a main boundary determination procedure in the fourth embodiment. It is a figure which shows a mode that this boundary is determined using the motion vector calculated from each template area | region.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing a configuration of the moving picture encoding apparatus according to the first embodiment. The moving image encoding apparatus according to Embodiment 1 includes a temporary region dividing unit 101, a second motion vector detecting unit 102, a main region dividing / motion compensating unit 103, an orthogonal transform / quantizing unit 104, a variable length encoding unit 105, An inverse quantization / inverse orthogonal transformation unit 106, a reference image memory 107, a first motion vector detection unit 108, an addition unit 109, and a subtraction unit 110 are provided.

  The first motion vector detection unit 108 detects a motion vector of the template region using a pixel group of a locally decoded region adjacent to the encoding target block (hereinafter referred to as “template”). The first motion vector detection unit 108 is characterized in that a motion vector can be automatically calculated on the decoding side without transmitting a motion vector. Therefore, the amount of motion vector codes can be reduced, and the overall coding efficiency can be improved.

  Motion vector detection using a template (hereinafter referred to as “template ME”) transmits a motion vector using the characteristics of a moving image that the motion of an encoding / decoding target region is very close to the motion of an adjacent region. This is a technique for obtaining a motion vector of an encoding / decoding target region without performing the processing. Therefore, when a region that is adjacent to the encoding / decoding target region as much as possible is selected as a region to be used as a template, a highly accurate motion vector can be obtained.

  The temporary area dividing unit 101 divides an area at an arbitrary boundary for a pixel group to be encoded, such as a 16 × 16 macroblock. This region division is a unique method on the coding side for motion vector detection, and the region may be divided at any boundary, but it is desirable to perform region division at a boundary that improves the efficiency of motion compensation prediction.

  Preferably, the temporary area dividing unit 101 evaluates a prediction error and a boundary activity when the macroblock is motion-compensated and predicted by the first motion vector detected by the first motion vector detecting unit 108, and determines the macroblock as a temporary block. To divide.

  The boundary of the macroblock determined by the provisional region dividing unit 101 is called a “provisional boundary”, and each region in the macroblock divided by the provisional boundary is called a “provisional region”.

  Due to the temporary boundary determined by the temporary region dividing unit 101, the macroblock has a temporary region adjacent to the template region used for detection of the first motion vector and a portion adjacent to the template region (or a template region). It is divided into temporary areas that are not adjacent.

  The second motion vector detecting unit 102 selects the original image to be encoded for one of the temporary regions divided by the temporary region dividing unit 101, preferably for the temporary region having a smaller part adjacent to the template region. Used to detect motion vectors by normal motion detection. The motion vector of the temporary area detected by the second motion vector detection unit 102 needs to be transmitted to the decoding side.

  FIG. 3 is a diagram showing motion vectors detected for each temporary area of the macroblock. The case where the temporary area dividing unit 101 temporarily divides a macroblock at a horizontal boundary is taken as an example. The first motion vector detection unit 108 detects a motion vector 310 that can be calculated on the decoding side and does not need to be transmitted using the template region 304 for the region above the horizontal temporary boundary 302 of the encoding target block 300. . On the other hand, the second motion vector detection unit 102 detects a motion vector 320 that needs to be transmitted using a normal motion vector search algorithm for a region below the temporary boundary 302 of the encoding target block 300.

  FIG. 4 is a diagram showing motion vectors detected for each temporary area of the macroblock when the macroblock is temporarily divided at the vertical boundary. The first motion vector detection unit 108 uses the template region 306 to detect a motion vector 330 that can be calculated on the decoding side and does not require transmission for the region on the left side of the temporary boundary 302 in the vertical direction of the block 300 to be encoded. . On the other hand, the second motion vector detection unit 102 detects a motion vector 320 that needs to be transmitted using a normal motion vector search algorithm for a region on the right side of the temporary boundary 302 of the encoding target block 300.

  FIGS. 5A to 5C are diagrams illustrating a motion vector detection method using a template. The same detection method is used on both the encoding side and the decoding side. As shown in FIG. 5A, a region that has already been encoded / decoded adjacent to the upper side and / or the left side of the encoding / decoding target block 300 is set as a template region 304, and a predetermined method is used. Detect motion vectors. Here, description will be made based on a full search for searching all candidates within a predetermined search range.

  As shown in FIGS. 5B and 5C, in the full search, the template region 304 in the encoding / decoding target image 500 and the reference image 510 are the same for all candidate motion vectors in the search range. Block matching is performed with the template-shaped region 314, and an optimal region where an evaluation value such as SAD is minimized is set as a motion vector 310. The motion vector 310 determined in this way is an optimal vector for the template region 304. If the motion of the encoding / decoding target block 300 is the same as that of the template region 304, the encoding / decoding target block 300 also has the motion vector 310 determined in this way. It becomes the optimal motion vector. However, when the template region 304 is too far away from the encoding / decoding target block 300 or when a different motion occurs in the encoding / decoding target block 300, the optimal motion vector is not necessarily obtained. Therefore, it is important to select a template appropriately.

  When the template ME is used, since the motion vector can be calculated on the decoding side without transmitting the motion vector, the code amount of the motion vector can be reduced. In the present invention, the shape of motion compensation prediction is automatically determined on the decoding side using the motion vector calculated using the template ME.

  The region segmentation / motion compensation unit 103 uses the motion vector of each temporary region detected by the first motion vector detection unit 108 and the second motion vector detection unit 102 to store the reference image stored in the reference image memory 107. Motion compensation prediction. The region segmentation / motion compensation unit 103 synthesizes a plurality of motion compensated prediction images generated from the motion vectors of the temporary regions in a procedure described later to generate a combined motion compensated prediction image.

  6A and 6B show motion compensated prediction images generated from motion vectors detected for each temporary region. FIGS. 7A to 7C show synthesized motion compensated predicted images obtained by synthesizing motion compensated predicted images generated from the motion vectors of the temporary regions.

  An example will be described in which the temporary area dividing unit 101 temporarily divides a macroblock at a horizontal boundary. As shown in FIG. 3, the first motion vector is detected for the region above the temporary boundary, and the second motion vector is detected for the region below the temporary boundary. As shown in FIGS. 6A and 6B, the region segmentation / motion compensation unit 103 temporarily uses the reference image for each detected motion vector to temporarily determine the size of the macroblock (here, 16 × An image in the case of motion compensation prediction in 16) is generated. FIG. 6A is a first motion compensated prediction image generated using the first motion vector in the upper region, and FIG. 6B is generated using the second motion vector in the lower region. It is a 2nd motion compensation prediction image.

  The region segmentation / motion compensation unit 103 synthesizes the first and second motion compensated prediction images shown in FIGS. 6A and 6B into any one of patterns (a) to (c) shown in FIG. Thus, a combined motion compensated prediction image is generated. The first and second motion compensated prediction images are the same size as the encoding target macroblock, and are divided at a certain boundary in order to generate a combined motion compensation prediction image. The boundary of the macroblock determined by the region dividing / motion compensating unit 103 is referred to as “actual boundary”, the region of the macroblock divided by the boundary is referred to as “actual region”. It is distinguished from “temporary boundary” and “temporary region” determined by the region dividing unit 101.

  FIGS. 7A to 7C show candidates for the main boundary determined by the region division / motion compensation unit 103. FIG. 7A, the horizontal boundary of the fourth pixel from the top is the main boundary, FIG. 7B is the horizontal boundary of the eighth pixel from the top, and FIG. 7C is the twelfth pixel from the top. With the horizontal boundary as the main boundary, the main region above the main boundary is represented by the region corresponding to the first motion compensated prediction image, and the main region below the main boundary is represented by the region corresponding to the second motion compensated prediction image. Is.

  The main region dividing / motion compensating unit 103 determines the main boundary based on the evaluation value indicating the edge strength and the like, and divides and synthesizes the first and second motion compensated prediction images at the determined main boundary.

  FIGS. 8A to 8C are diagrams illustrating a method of generating a combined motion compensated prediction image by combining motion compensated prediction images generated from the motion vectors of the temporary regions. After the boundary determination, the main segmentation / motion compensation unit 103 determines the first motion compensated prediction image corresponding to the first motion vector in FIG. 8A and the second motion vector corresponding to the second motion vector in FIG. Using the motion compensated prediction image, motion compensation prediction is performed using the region above the main boundary as the first motion compensation prediction image, and the region below the main boundary is used as the second motion compensation prediction image. In this example, the corresponding region of the first motion compensated prediction image is duplicated in the 16 × 4 region above the main boundary, and the second motion compensated prediction image is reproduced in the 16 × 12 region below the main boundary. The corresponding region of is duplicated.

  Here, the horizontal region division has been described as an example, but the vertical region division can also be performed in the same manner. Moreover, although the case where the main boundary candidates are three kinds of FIGS. 7A to 7C has been described this time, the main boundary candidates may be further increased. However, as described later, it is necessary to sufficiently consider the relationship with the orthogonal transform size of the orthogonal transform / quantization unit 104.

  Returning to FIG. 1, the subtraction unit 110 calculates a prediction residual component based on the difference between the original image to be encoded and the motion compensated prediction image calculated by the main region division / motion compensation unit 103, and generates an orthogonal transform / quantization unit. 104. The orthogonal transform / quantization unit 104 performs orthogonal transform / quantization of the prediction residual component.

  Here, the orthogonal transform / quantization unit 104 performs orthogonal transform using an orthogonal transform size corresponding to the size of motion compensation prediction. That is, when 16 × 4/16 × 8/16 × 12 (multiple of 4 in the vertical direction) is allowed as the size of motion compensation prediction, at least 16 × 4, 8 × 4, or 4 × 4 (vertical direction) An orthogonal transform size of a multiple of 4). As another example, when 16 × 2/16 × 4/16 × 6/16 × 8/16 × 10/16 × 12 (a multiple of 2 in the vertical direction) is allowed as the size of motion compensation prediction, at least An orthogonal transform size of 16 × 2 or 8 × 2 or 4 × 2 (a multiple of 2 in the vertical direction) can be used. As a result, when the prediction error of motion compensation prediction is orthogonally transformed, the motion compensation prediction boundary is not included in the prediction error set to be orthogonally transformed. As a result, it is possible to prevent a decrease in orthogonal transform efficiency caused by performing orthogonal transform together as a prediction error on pixels that straddle the boundary of motion compensation prediction, and the effect of further improving the coding efficiency is achieved.

  The variable length coding unit 105 performs variable length coding on the prediction residual component orthogonally transformed and quantized by the orthogonal transformation / quantization unit 104 and variable length coding the motion vector detected by the motion vector detection unit 102. To do. When a macroblock is divided at a fixed boundary as in the prior art, motion vectors are transmitted in raster order (ie, from the upper left block to the lower right block). When the shape of motion compensated prediction is automatically determined on the decoding side as in the present embodiment, the position of the upper left pixel in each region of motion compensated prediction is the motion vector in the raster order from the previous one. To transmit. As a result, as in the prior art, the target region for motion compensation prediction can be uniquely expressed by the order in which a plurality of motion vectors are transmitted.

  The inverse quantization / inverse orthogonal transform unit 106 performs inverse quantization and inverse orthogonal transform on the prediction residual component orthogonally transformed / quantized by the orthogonal transform / quantization unit 104. Similar to the orthogonal transform / quantization unit 104, inverse orthogonal transform can be performed with a size corresponding to the size of motion compensation prediction.

  The adding unit 109 generates a reference image by adding the prediction residual component decoded by the inverse quantization / inverse orthogonal transform unit 106 and the motion compensated prediction image calculated by the region division / motion compensation unit 103. And stored in the reference image memory 107.

  FIG. 2 is a block diagram illustrating a configuration of the moving picture decoding apparatus according to the first embodiment. The moving picture decoding apparatus according to Embodiment 1 includes a variable length decoding unit 201, a main segmentation / motion compensation unit 203, an inverse quantization / inverse orthogonal transform unit 206, a first motion vector detection unit 208, an addition unit 209, and a reference. An image memory 207 is provided.

  The variable length decoding unit 201 performs variable length decoding on the prediction residual component signal and the motion vector that have been orthogonally transformed and quantized. In the bit stream encoded by the moving image encoding apparatus in FIG. 1, since a motion vector is encoded for each area into which a macroblock is divided, the variable length decoding unit 201 decodes the motion vector for each divided area. Is done. Here, the target region for motion compensation prediction can be uniquely determined by the order in which the motion vectors in the macroblock are decoded.

  The first motion vector detection unit 208 has the same function as that of the first motion vector detection unit 108 of the moving picture encoding device in FIG. 1 and selects a pixel group (“template”) in a decoded area adjacent to the decoding target block. Use to detect the motion vector of the template region.

  The region segmentation / motion compensation unit 203 has the same function as the region segmentation / motion compensation unit 103 of the moving image encoding device in FIG. 1, and the motion vector decoded by the variable length decoding unit 201 and the first motion Motion compensation prediction is performed from the reference image stored in the reference image memory 207 using the motion vector detected by the vector detection unit 208. Here, the motion vector is acquired or detected for each divided area of the macroblock. The region segmentation / motion compensation unit 203 performs a plurality of motion compensated prediction images generated from the motion vectors of each segment region in the same procedure as the region segmentation / motion compensation unit 103 of the video encoding device in FIG. A combined motion compensated prediction image is generated by combining.

  The inverse quantization / inverse orthogonal transform unit 206 has the same function as the inverse quantization / inverse orthogonal transform unit 106 of the moving image coding apparatus in FIG. 1, and uses the prediction residual component decoded by the variable length decoding unit 201. On the other hand, inverse quantization and inverse orthogonal transform are performed.

  The addition unit 209 adds the prediction residual component decoded by the inverse quantization / inverse orthogonal transformation unit 206 and the motion compensated prediction image calculated by the region division / motion compensation unit 203, thereby obtaining an image signal. Decrypt. The reference image memory 207 is the same as the reference image memory 107 of the moving image encoding apparatus in FIG. 1, and stores the decoded reference image.

  The operation of the moving picture coding apparatus having the above configuration will be described. First, with reference to FIG. 9 and FIG. 10, a process of dividing a temporary area based on a motion vector detected using a template area and determining a main boundary will be described. Next, the boundary determination process will be described in detail with reference to FIGS. 11 and 12.

  FIG. 9 is a flowchart showing a procedure for temporary area division and main boundary determination. Here, a case where the region is divided in the horizontal direction will be described.

  First, the first motion vector detection unit 108 detects a first motion vector using a predetermined template region (S101). As a motion vector detection algorithm, there are various methods such as a full search that evaluates all candidate vectors in a designated search region and a high-speed search that narrows down and searches candidate vectors based on various motion characteristics. Any motion vector detection algorithm can be used as long as a motion vector can be detected using a predetermined template region and used as a motion vector of an encoding / decoding target region. Further, the definition of the template area is not limited to that illustrated in FIGS. 3 and 4 and can be defined in various shapes as long as it is an area adjacent to the encoding target block.

  Next, the temporary area dividing unit 101 performs motion compensation prediction of the encoding target area using the first motion vector detected by the first motion vector detecting unit 108, and uses the prediction error and the boundary activity to encode the encoding target. The block is temporarily divided (S102).

Here, the activity of the image signal is a value obtained by performing some operation on the pixel. For example, an absolute difference sum (SAD) between two pixels across the boundary can be used as an activity related to the boundary for dividing the macroblock. For example, assuming that the X coordinate in the macroblock is i, the Y coordinate is j, and the pixel value at the point (i, j) is _{Ai, j} , the horizontal activity at the horizontal boundary j of the jth pixel from the top is as follows: Defined in
Horizontal activity = Σ _{i = 0} ¹⁵ | A _{i, j} −A _{i, j−1} |
Here, Σ _{i = 0} ¹⁵ is the total sum when the subscript i is moved from 0 to 15.

However, the activity may not be an absolute difference sum (SAD). An absolute sum of squares (SSD) as shown below may be used.
Horizontal activity = Σ _{i = 0} ¹⁵ (A _{i, j} −A _{i, j−1} ) ²

Similarly, the vertical activity at the vertical boundary i of the i-th pixel from the left of the macroblock is defined as follows.
Vertical activity = Σ _{j = 0} ¹⁵ | A _{i, j} −A _{i−1, j} |

  The activity becomes a large value at the edge of the object. As one method of provisional division of an encoding target block, the provisional area division unit 101 provisionally divides a macroblock area at a boundary having the largest activity value.

  Since the method of provisional division of the encoding target block is a method only on the encoding side, any algorithm can be applied, but it is better to perform provisional division that is as close to the main region division as possible. High motion vector can be detected.

  FIG. 10 is a diagram for explaining a preferred method of provisional division of the encoding target block. The error between the motion compensated prediction image 360 and the encoding target original image 300 when performing motion compensation prediction on the entire encoding target region using the first motion vector was evaluated by an absolute difference sum (SAD) or the like. A temporary boundary 302 is determined based on the result. In other words, for regions where this error is large, prediction efficiency improves when motion compensation prediction is performed using other motion vectors, so the block to be encoded is temporarily divided into regions where the error is greater than or equal to a predetermined threshold and regions where the error is less than the threshold. For the region with low prediction efficiency (here, the lower region not adjacent to the template region), the motion vector is detected and transmitted on the encoding side.

  Returning to the flowchart of FIG. For the temporarily divided lower region that is not adjacent to the template region, the second motion vector detection unit 102 detects the second motion vector by normal motion detection (S103). Finally, the main region division / motion compensation unit 103 performs main region division of the encoding target region based on the boundary activity of the image subjected to motion compensation prediction using the first motion vector and the second motion vector (S104). ). Details of this area division processing will be described next.

  FIG. 11 is a flowchart for explaining the determination procedure of the main region by the main region division / motion compensation unit 103.

  The region segmentation / motion compensation unit 103 uses the first motion vector detected by the first motion vector detection unit 108 to perform motion compensation prediction with the same size as the macroblock, and generates a first motion compensation predicted image. (S01). Similarly, using the second motion vector detected by the second motion vector detection unit 102, motion compensation prediction is performed with the same size as the macroblock, and a second motion compensation predicted image is generated (S02).

  Next, the first to fourth activities shown in FIGS. 12A to 12D are calculated for each candidate of this boundary (S03 to S06). However, steps S03 to S06 can be performed in any order. Further, it is of course possible to calculate only the activity that is desired to be used for boundary evaluation without performing all of steps S03 to S06.

  First, as shown in FIG. 12A, a boundary activity (first activity) related to this boundary candidate is calculated for the first motion compensated prediction image (S03). Here, the absolute difference sum (SAD) between two pixels straddling the boundary candidate is used for the activity. Since the value of the first activity increases at the edge of the object, the prediction efficiency of the motion compensation prediction is improved by dividing the region at the corresponding boundary as the value increases. Similarly, as shown in FIG. 12B, a boundary activity (second activity) regarding the boundary candidate is calculated for the second motion compensated prediction image (S04). Similar to the first activity, the second activity improves the prediction efficiency of motion compensation prediction by dividing the region at the corresponding boundary as the value increases.

  Subsequently, as shown in FIG. 12C, the region above the main boundary candidate in the macroblock is subjected to motion compensation prediction as the first motion compensated prediction image, and the region below the main boundary candidate is set to the second motion compensated prediction image. As for the synthesized motion compensated prediction image synthesized as follows, a boundary activity (third activity) related to this boundary candidate is calculated (S05). Since the third activity is the sum of absolute differences between two pixels straddling the boundary candidate, the absolute difference between the values of the pixels of the first motion compensated prediction image and the pixels of the second motion compensated prediction image located above and below the boundary candidate. The sum of Therefore, the smaller the value of the third activity, the smoother the boundary of the synthesized motion compensated prediction image, and the high frequency component is less likely to be generated in the prediction error signal, so that the prediction efficiency of motion compensation prediction is improved.

  Finally, as shown in FIG. 12D, a boundary activity (fourth activity) regarding the boundary candidate is calculated for the difference image between the first motion compensated predicted image and the second motion compensated predicted image (S06). Since the value of the fourth activity increases at the boundary of the object, the prediction efficiency of the motion compensation prediction is improved by dividing the region at the boundary as the value increases.

After all the activities used for the boundary evaluation are calculated, this boundary candidate is evaluated using a predefined evaluation value (S07). For example, the evaluation value is defined as follows.
Evaluation value = −A * ACT1-B * ACT2 + C * ACT3-D * ACT4
Here, ACT1 represents a first activity value, ACT2 represents a second activity value, ACT3 represents a third activity value, and ACT4 represents a fourth activity value. A, B, C, and D are constants of 0 or more.

  The evaluation value is calculated for all main boundary candidates, and the main boundary candidate having the minimum value is determined as the final main boundary (S08).

  Here, it is desirable that the main boundary and the temporary boundary are the same, but the determined main boundary is not necessarily the same as the temporary boundary. The temporary boundary for motion vector detection is a boundary for obtaining an optimal motion vector on the encoding side, and can be calculated using an original image to be encoded that can be used only on the encoding side. On the other hand, this boundary must be able to be uniquely calculated on both the encoding side and the decoding side, and a plurality of calculated motion vectors (transmitted on the decoding side) and their motion compensated prediction images (that is, prediction residuals). Image before the component is added). For this reason, there is no mismatch between the encoding side and the decoding side even if the main boundary and the temporary boundary are not the same.

  However, the difference between the main boundary and the temporary boundary means that an optimal motion vector has not been detected for the combined motion compensated prediction image after the determination of the main boundary, and the prediction efficiency may not necessarily be improved. Therefore, by adjusting the temporary area set by the temporary area dividing unit 101 or adjusting the motion vector detected by the first motion vector detecting unit 108 and the second motion vector detecting unit 102, an optimal temporary boundary or If both the optimal motion vector and the optimal main boundary are realized at the same time, the encoding efficiency can be further improved.

  Specifically, when the main boundary determined by the main region division / motion compensation unit 103 is different from the temporary boundary, the first motion vector detection unit 108 and / or the second motion vector detection unit 102 may detect the motion vector. Redo detection. Any method may be used for redoing, for example, the second smallest value (second minimum value) of the error evaluation values at the time of motion vector search is left, and the motion vector is detected from the position of the second minimum value. It is possible to continue the process. Using the motion vector redetected by the motion vector detection unit 102, the region segmentation / motion compensation unit 103 performs the boundary determination process again. The motion vector detection processing by the motion vector detection unit 102 and the main boundary determination processing by the region division / motion compensation unit 103 are repeated until the main boundary coincides with the temporary boundary or becomes sufficiently close.

  Alternatively, as another method, the temporary region dividing unit 101 sets a plurality of temporary boundary candidates, and for each temporary boundary candidate, the first motion vector detection unit 108 and the second motion vector detection unit 102 detect a motion vector, The region dividing / motion compensating unit 103 performs the boundary determination process for each temporary boundary. Then, the region segmentation / motion compensation unit 103 selects the one having the best prediction efficiency of the combined motion compensated prediction image among the plurality of temporary boundaries.

  Regardless of which method is used, as a result, the main boundary finally determined by the main region dividing / motion compensating unit 103 is coincident with or sufficiently close to the temporary boundary set by the temporary region dividing unit 101. Efficiency is improved.

  Next, the syntax of a moving image bit stream encoded by the moving image encoding apparatus according to Embodiment 1 will be described.

  FIG. 13 shows a first syntax pattern based on the MPEG-4 AVC syntax. As shown in FIG. 13A, first, a flag (use_auto_mc_size) indicating whether or not the shape of motion compensated prediction is automatically determined on the decoding side using the feature amount of the predicted image in units of slices is transmitted. When the flag use_auto_mc_size is OFF, the motion compensation prediction is performed by dividing the macroblock fixedly based on the macroblock type mb_type shown in FIG. When the flag use_auto_mc_size is ON, motion compensated prediction is performed by automatically determining the shape of motion compensated prediction on the decoding side using the feature amount of the predicted image. In macroblock units, dec_me_flag is transmitted in addition to transmitting mb_type in the same way as MPEG-4 AVC.

  FIG. 14 shows the semantics of the macroblock type mb_type when the flag use_auto_mc_size is ON / OFF. When the macroblock type mb_type = 0, motion compensation prediction is performed without dividing a 16 × 16 macroblock into regions. When the macro block type mb_type = 1, when the flag use_auto_mc_size is OFF, motion compensation prediction is performed with 16 × 8. However, when the flag use_auto_mc_size is ON, the macro block is 2 × 16 × A and 16 × (16−A). Motion-compensated prediction is performed by automatically dividing into two regions. Here, when dec_me_flag is ON, a motion vector of a 16 × A region is not transmitted, and only a 16 × (16-A) motion vector is transmitted. Similarly, in the case of the macro block type mb_type = 2, when the flag use_auto_mc_size is OFF, motion compensation prediction is performed with 8 × 16 blocks. However, when the flag use_auto_mc_size is ON, the macro block is A × 16 (16−A). Motion-compensated prediction is performed by automatically dividing into two regions of × 16. Similarly, when dec_me_flag is ON, only the motion vector of (16−A) × 16 is transmitted without transmitting the motion vector of the region of A × 16. When the macro block type mb_type = 3, motion compensation prediction is performed with 8 × 8 blocks. In this way, when dec_me_flag is ON, the number of motion vectors to be transmitted is one and decreases compared to two when dec_me_flag is OFF, so the code amount of the motion vector is reduced and the entire coding is performed. Efficiency is improved.

(Embodiment 2)
A video encoding device and video decoding device according to Embodiment 2 of the present invention will be described. In Embodiment 1, by acquiring the first motion vector using the template region, the first motion vector is not required to be transmitted to the decoding side, while the second motion vector that is uniquely detected on the encoding side is decoded. Transmission required. In the second embodiment, by using two template areas, it is not necessary to transmit both the first and second motion vectors to the decoding side. By determining the shape of motion compensation prediction using two motion vectors obtained from two template shapes, the shape of motion compensation prediction can be automatically determined on the decoding side without transmitting any motion vector. .

  FIG. 15 is a block diagram showing a configuration of the moving picture coding apparatus according to the second embodiment. Unlike the first embodiment, the moving picture coding apparatus according to the second embodiment does not have the configuration of the temporary region dividing unit 101 and the second motion vector detecting unit 102, and the first and second motion vector detecting units 108. Detects the first and second motion vectors from the two template regions. In the second embodiment, if two template regions are defined for the encoding target block, two motion vectors can be detected, so that the process of dividing the encoding target block into temporary regions is unnecessary.

  The region segmentation / motion compensation unit 103 determines the boundary using the first and second motion vectors detected by the first and second motion vector detection units 108 and is stored in the reference image memory 107. Motion compensation prediction is performed from the reference image. The main region dividing / motion compensation unit 103 combines a plurality of motion compensated prediction images generated from the motion vectors of each main region to generate a combined motion compensated prediction image.

  FIG. 16 is a block diagram illustrating a configuration of the moving picture decoding apparatus according to the second embodiment. A configuration and operation different from those of the first embodiment will be described. The first and second motion vector detection units 208 have the same functions as the first and second motion vector detection units 108 of the moving image encoding device in FIG. 15, and are a group of pixels in a decoded area adjacent to the decoding target block. (“Template”) is used to detect the motion vector of the template region.

  The region segmentation / motion compensation unit 203 has the same function as the region segmentation / motion compensation unit 103 of the moving picture coding apparatus in FIG. 15 and is detected by the first and second motion vector detection units 208. Also, motion compensation prediction is performed from the reference image stored in the reference image memory 207 using the second motion vector. The region segmentation / motion compensation unit 203 performs a plurality of motion compensated prediction images generated from the motion vectors of each segment region in the same procedure as the region segmentation / motion compensation unit 103 of the video encoding device in FIG. A combined motion compensated prediction image is generated by combining.

  FIG. 17 is a flowchart showing the boundary determination procedure in the second embodiment.

  First, as shown in FIG. 18A, the first motion vector 310 is detected using the first template region 304 adjacent to the encoding target block 300 in the horizontal direction (S11). This is the same as the procedure described in the first embodiment. Similarly, as shown in FIG. 18B, the second motion vector 330 is detected using the second template region 306 that is adjacent to the encoding target block 300 in the vertical direction (S12). Here, the template regions 304 and 306 are not limited to the shapes shown in FIGS. It may be defined in any shape as long as it is an area adjacent to the encoding target block.

  Next, the main region division / motion compensation unit 103 performs main region division of the encoding target region based on the boundary activity of the image subjected to motion compensation prediction using the first motion vector and the second motion vector (S13). Boundary activity is obtained and evaluated for both the case where the main boundary 400 is set in the vertical direction as shown in FIG. 18C and the case where the main boundary 400 is set in the horizontal direction as shown in FIG. The book boundary with the better value is selected.

  Here, two motion vectors are obtained from two template regions, and the encoding target region is divided into two main regions. However, three or more motion vectors are obtained from three or more template regions, and three or more encoding target regions are obtained. You may divide into this book area. In this way, by calculating a plurality of motion vectors from a plurality of template shapes and automatically determining motion compensated prediction on the decoding side, motion compensated prediction in an arbitrary shape is possible without transmitting any motion vectors. Is possible.

  The syntax for determining the shape of motion compensated prediction using a plurality of motion vectors obtained when the shape of template matching is changed will be described. The syntax is the same as that of the first embodiment, and only the semantics of mb_type is different.

  FIG. 19 shows the semantics of the macroblock type mb_type when the flag use_auto_mc_size is ON / OFF. FIG. 19A shows a case where the flag use_auto_mc_size is OFF, which is the same as in the first embodiment. FIG. 19B shows a case where the flag use_auto_mc_size is ON, and the semantics of mb_type is divided into 16 × A and 16 × (16-A) horizontal directions, A × 16, and (16-A). ) × 16 vertical region division is handled without distinction as mb_type = 1.

  In the second embodiment, the first and second motion vectors obtained using the two template regions are obtained by the same process on the decoding side, so there is no need to transmit the motion vector to the decoding side.

(Embodiment 3)
A video encoding device and video decoding device according to Embodiment 3 of the present invention will be described. The third embodiment has the same configuration as that of the first embodiment, but the first motion vector detection unit 108 selects one motion vector from the first and second motion vectors obtained from the two template shapes. This is different from the first embodiment.

  FIG. 20 is a flowchart showing the boundary determination procedure in the third embodiment.

  First, as shown in FIG. 21A, the first motion vector 310 is detected using the first template region 304 that is adjacent to the encoding target block 300 in the horizontal direction (S21). This is the same as the procedure described in the first embodiment. Similarly, as shown in FIG. 21B, the second motion vector 330 is detected using the second template region 306 that is adjacent to the encoding target block 300 in the vertical direction (S22).

  Next, the region segmentation / motion compensation unit 103 detects an evaluation value when the first motion vector 310 is detected (for example, a prediction error evaluated by SAD or the like) and an evaluation value when the second motion vector 330 is detected. And the motion vector having the smaller SAD is regarded as a more reliable motion and is adopted as the first motion vector (S23). Here, if the areas (number of pixels) of the first template region 304 and the second template region 306 are the same, the evaluation values can be compared as they are, but if the number of pixels in each region is different, the difference in the number of pixels. Compare with weight to absorb. That is, the evaluation value is converted into a value per pixel and used. Further, the evaluation value at the time of motion vector detection is not limited to SAD, and SSD or SATD (Sum of Absolute Transfered Difference) can be used.

  When the first motion vector 310 is adopted, since the region adjacent to the first template region 304 is the left region of the encoding / decoding target block, the direction of this boundary is determined as the vertical direction. When the second motion vector 330 is adopted, since the region adjacent to the second template region 306 is the upper region of the encoding / decoding target block, the direction of the main boundary is determined in the horizontal direction.

  The subsequent processing is the same as in the first embodiment, and on the encoding side, a temporary boundary is determined (S24), and the region that is not adjacent to the template region or has few adjacent portions and has low prediction efficiency is used. On the other hand, the third motion vector 350 is detected by normal motion detection (S25). When the first motion vector 310 of FIG. 21A is employed, the third motion vector 350 is detected for the lower region obtained by dividing the temporary region, and the second motion vector 330 of FIG. 21B is employed. In this case, the third motion vector 350 is detected for the right region obtained by dividing the temporary region.

  Finally, using the motion vector employed in step S23 and the third motion vector 350, the boundary is determined by the same method as in the first embodiment (S26). When the first motion vector 310 of FIG. 21A is employed, the vertical boundary 400 is determined as shown in FIG. 21C, and the second motion vector 330 of FIG. 21B is employed. In such a case, the main boundary 400 in the horizontal direction is determined as shown in FIG.

  In the third embodiment, it is not necessary to transmit the first and second motion vectors obtained by using the two template regions, since the finally adopted motion vector is also obtained on the decoding side. Since the third motion vector detected by the normal motion detection on the encoding side is information that can be obtained only on the encoding side, it is transmitted to the decoding side. On the decoding side, out of the first and second motion vectors obtained using the two template regions, a motion vector selected by the same method as that on the encoding side, and a third motion vector transmitted from the encoding side, Motion compensation prediction.

  In the third embodiment, the direction of region division can be determined at the same time when automatically determining a motion vector to be adopted from two motion vector candidates, so that the syntax pattern shown in FIG. 21 is the same as in the second embodiment. Can be used.

(Embodiment 4)
A moving picture coding apparatus and a moving picture decoding apparatus according to Embodiment 4 of the present invention will be described. In the fourth embodiment, similarly to the second embodiment, by using two template regions, both the first and second motion vectors are not required to be transmitted to the decoding side. The third motion vector that is uniquely detected is transmitted to the decoding side to divide the region into three.

  FIG. 22 is a block diagram illustrating a configuration of the moving picture coding apparatus according to the fourth embodiment. Similar to the second embodiment, the first and second motion vector detectors 108 detect the first and second motion vectors from the two template regions. Unlike the second embodiment, the fourth embodiment has a configuration of a temporary area dividing unit 101 and a third motion vector detecting unit 102. The third motion vector detection unit 102 detects a third motion vector by normal motion detection for a third region (subregion) with low prediction efficiency that is not adjacent to any of the first and second template regions. .

  The temporary area dividing unit 101 determines a temporary boundary when the encoding target block is divided into three based on the first and second motion vectors obtained by the first and second motion vector detecting units.

  The region segmentation / motion compensation unit 103 receives the first and second motion vectors detected by the first and second motion vector detection units 108 and the third motion vector detected by the third motion vector detection unit 102. The boundary is determined by the same method as in the first embodiment, and motion compensation prediction is performed from the reference image stored in the reference image memory 107.

  FIG. 23 is a block diagram illustrating a configuration of the moving picture decoding apparatus according to the fourth embodiment. A configuration and operation different from those of the first embodiment will be described. The first and second motion vector detection units 208 have the same functions as the first and second motion vector detection units 108 of the moving image encoding device in FIG. 22 and are a group of pixels in a decoded area adjacent to the decoding target block. (“Template”) is used to detect the motion vector of the template region.

  The region segmentation / motion compensation unit 203 has the same function as the region segmentation / motion compensation unit 103 of the video encoding device in FIG. 22, and the third motion vector decoded by the variable length decoding unit 201, Using the first and second motion vectors detected by the first / second motion vector detection unit 208, motion compensation prediction is performed from the reference image stored in the reference image memory 207. The region segmentation / motion compensation unit 203 performs a plurality of motion compensated prediction images generated from the motion vectors of each segment region in the same procedure as the region segmentation / motion compensation unit 103 of the video encoding device in FIG. A combined motion compensated prediction image is generated by combining.

  FIG. 24 is a flowchart showing the boundary determination procedure in the fourth embodiment.

  First, as shown in FIG. 25A, the first motion vector 310 is detected using the first template region 304 that is adjacent to the encoding / decoding target block 300 in the horizontal direction (S31). This is the same as the procedure described in the first embodiment. Similarly, as shown in FIG. 25B, the second motion vector 330 is detected using the second template region 306 that is adjacent to the encoding target block 300 in the vertical direction (S32).

  Here, the first template region 304 is adjacent to only the lower left half of the encoding / decoding target block 300, and the second template region 306 is adjacent to only the upper right half of the encoding / decoding target block 300. Such setting of the template shape is important for calculating a highly accurate motion vector. That is, assuming that the region finally divided into three is shaped as shown in FIGS. 25C and 25D, it is expected that the region will not be adjacent to the region used for motion compensation prediction in advance. By excluding the region from the template region, the accuracy of the detected motion vector is increased.

  Next, the evaluation value when the first motion vector 310 is detected is compared with the evaluation value when the second motion vector 330 is detected by the same method as in the third embodiment, and the motion vector with the better evaluation value is compared. Is the main motion vector, and the motion vector with the worse evaluation value is the sub motion vector (S33). When the first motion vector 310 becomes the main motion vector, the region of the encoding / decoding target block 300 adjacent to the first template region 304 is on the left side, and thus the vertical main boundary 400 that divides the block 300 first. To decide. When the second motion vector 330 becomes the main motion vector, the region of the encoding / decoding target block 300 adjacent to the second template region 306 is on the upper side. The boundary 400 is determined (S34). Since the first division of the block 300 is performed based on two motion vectors obtained by using two templates, it is not necessary to define a temporary boundary as in the second embodiment, and the main boundary is determined from the beginning. Determined.

  Next, out of the regions divided by the first main boundary 400, the region not directly adjacent to the template region corresponding to the main motion vector, that is, the region corresponding to the sub motion vector, The direction of the main boundary is determined using the third motion vector (S35). In step S35, the region that has been motion compensation predicted by the main motion vector is more reliable than the region that has been motion compensation predicted by the sub motion vector, and therefore the region that has been motion compensation predicted by the sub motion vector is further divided into regions. As in the first embodiment, the encoding side first determines a temporary boundary, and performs normal motion detection for a region that is not adjacent to the template region or has a small number of adjacent portions and has low prediction efficiency. Thus, the third motion vector 350 is detected. When the block 300 is first divided at the main boundary 400 in the vertical direction, the third motion vector 350 is detected for the third lower right area obtained by further dividing the right area further vertically. When the block 300 is first divided at the main boundary 400 in the horizontal direction, the third motion vector 350 is detected for the third lower right area obtained by further dividing the lower area into the left and right temporary areas.

  Finally, using the sub motion vector and the third motion vector 350, the boundary is determined by the same method as in the first embodiment. When the block 300 is first divided at the first book boundary 400 in the vertical direction, as shown in FIG. 25C, the second book boundary 402 in the horizontal direction to be further divided up and down in the right region is determined. When the block 300 is first divided at the first book boundary 400 in the horizontal direction, as shown in FIG. 25D, the second book boundary in the vertical direction is further divided into the left and right regions as shown in FIG. 402 is determined.

  In the fourth embodiment, since the first and second motion vectors obtained using the two template regions are also obtained on the decoding side, they need not be transmitted, but are detected by normal motion detection on the coding side. Since the third motion vector is information that can be obtained only on the encoding side, it is transmitted to the decoding side.

  In the fourth embodiment, the first main boundary 400 is determined based on a more efficient motion vector among the two motion vectors, and motion compensation is performed using the remaining motion vectors having lower reliability than the reference motion vector. Since the region to be predicted is further divided into regions at the second main boundary 402 to perform motion compensation prediction, it is possible to perform motion compensation prediction with high efficiency.

  As described above, in the fourth embodiment, since motion compensation prediction can be performed with a more flexible shape than in the first embodiment, the coding efficiency is further improved.

  Up to this point, the embodiment of the present invention has been described for unidirectional prediction used in MPEG P pictures and the like. However, the embodiment of the present invention is bidirectional prediction (usually forward prediction used in B pictures). And backward prediction). Bidirectional prediction is a technique for obtaining a predicted image by averaging or weighted averaging two images obtained by performing motion compensation prediction from two reference images. Except for the point that there is a motion vector in each prediction direction (forward direction, backward direction) for each region, the embodiment of the present invention can be applied to bidirectional prediction as in the case of unidirectional prediction. . While reducing the prediction error by bidirectional prediction, the prediction error can be further reduced by appropriate region division.

  In addition, the embodiment of the present invention uses not only a motion vector automatically calculated using a template ME as a motion vector to be automatically calculated, but also a predicted motion vector (PMV) or uses motion vectors of surrounding blocks as they are. Of course it is also possible to do. In this case as well, it is possible to automatically determine the shape of the motion compensation prediction while reducing the code amount of the motion vector.

  Although a single image component (luminance) has been described so far, it can be applied to a plurality of components (luminance and color difference) such as YUV4: 4: 4 / YUV4: 2: 2 / YUV4: 2: 0. Is possible. However, since motion compensation prediction performs luminance and color difference at the same time, in a format with different luminance and color difference samples such as YUV4: 2: 2 and YUV4: 2: 0, the region is divided based on the luminance with a large number of samples. In this case, the area division position of the color difference may be ambiguous. For example, when the luminance is divided into 16 × 5 and 16 × 11 in the YUV 4: 2: 0 format, it is unknown whether the color difference area division is 8 × 2 or 8 × 3. As measures to prevent this ambiguity, the area is divided in advance based on the color difference with a small number of samples, and the division rule for the color difference at the ambiguous position is determined in advance (the block center is discarded). There are a method and a method using an average value (filtering) of both motion compensated prediction images for pixels on the boundary.

  As described above, according to the embodiment of the present invention, by using the feature amounts of a plurality of predicted images obtained from a plurality of motion vectors, the shape of motion compensated prediction is automatically determined on the decoding side. The shape of the motion compensation prediction can be made variable without transmitting the information of the motion compensation prediction shape pattern. Therefore, flexible motion compensation prediction in various shapes is possible. As a result, the prediction error of motion compensation prediction can be reduced without increasing the code amount of the additional information, and the coding efficiency is improved.

  Also, in the motion vector detection process on the encoding side, both the optimal motion vector and the optimal main boundary were realized simultaneously by evaluating the motion vector while calculating the main boundary determined on the decoding side. Thereby, the prediction efficiency of motion compensation prediction improves. On the decoding side, a moving image can be decoded only by performing motion compensation prediction using this boundary calculated from the transmitted motion vector.

  The motion compensation prediction shape is automatically determined on the decoding side using the motion vector automatically calculated on the decoding side using the decoded region adjacent to the encoding / decoding target region as a template. The amount of additional information code can be reduced. In addition, since there is no need to transmit a motion vector, the code amount of the motion vector can be reduced. Thereby, encoding efficiency can be improved.

  In addition, since a plurality of motion vectors are calculated from a plurality of template shapes and the shape of motion compensation prediction is automatically determined on the decoding side, motion compensation prediction in an arbitrary shape can be performed without transmitting any motion vectors. Is possible. Thereby, it is possible to further improve the encoding efficiency.

  Furthermore, a motion vector with high accuracy can be adopted from the shapes of a plurality of templates, and a motion vector can be transmitted for a region with low prediction efficiency of a region to be encoded / decoded. This makes it possible to apply a motion vector automatically calculated on the decoding side only to a region with high prediction efficiency, and further improve the encoding efficiency.

  The above processing relating to encoding and decoding can be realized as a transmission, storage, and reception device using hardware, and is stored in a ROM (Read Only Memory), a flash memory, or the like. It can also be realized by firmware or software such as a computer. The firmware program and software program can be recorded on a computer-readable recording medium, provided from a server through a wired or wireless network, or provided as a data broadcast of terrestrial or satellite digital broadcasting Is also possible.

  The present invention has been described based on the embodiments. The embodiments are exemplifications, and it will be understood by those skilled in the art that various modifications can be made to combinations of the respective constituent elements and processing processes, and such modifications are within the scope of the present invention. .

  101 Temporary region division unit, 102 Second motion vector detection unit, 103 Region division / motion compensation unit, 104 Orthogonal transformation / quantization unit, 105 Variable length coding unit, 106 Inverse quantization / inverse orthogonal transformation unit, 107 Image memory, 108 first motion vector detection unit, 109 addition unit, 110 subtraction unit, 201 variable length decoding unit, 203 region division / motion compensation unit, 206 inverse quantization / inverse orthogonal transform unit, 207 reference image memory, 208 A first motion vector detection unit, 209 addition unit.

Claims (6)

A first motion vector determination unit that determines a first motion vector of a predetermined region that has been locally decoded and is adjacent to an encoding target block;
A temporary area dividing unit that divides the encoding target block into two temporary areas at a predetermined temporary boundary;
A second motion vector detection unit that detects a second motion vector using an original image of an encoding target block for a temporary region having a smaller portion adjacent to the predetermined region of the two temporary regions;
Generating two prediction blocks corresponding to the coding target block from a reference image using the first motion vector and the second motion vector, determining a boundary based on activities of the two prediction blocks, and A motion compensation unit that generates a combined prediction block by combining regions obtained by dividing each prediction block at the main boundary between the prediction blocks;
A moving picture encoding apparatus comprising: a prediction difference block obtained by subtracting the combined prediction block from the encoding target block; and an encoding unit that encodes a motion vector of each temporary region. A first motion vector of a first predetermined region that has been locally decoded adjacent to the encoding target block in a horizontal direction, and a second motion vector of a second predetermined region that has been locally decoded and adjacent to the encoding target block in the vertical direction; A motion vector determination unit for determining
Generating two prediction blocks corresponding to the coding target block from a reference image using the first motion vector and the second motion vector, determining a boundary based on activities of the two prediction blocks, and A motion compensation unit that generates a combined prediction block by combining regions obtained by dividing each prediction block at the main boundary between the prediction blocks;
A moving picture encoding apparatus comprising: a prediction difference block obtained by subtracting the combined prediction block from the encoding target block; and an encoding unit that encodes a motion vector of each temporary region. Determine a first motion vector for a first predetermined region that has been locally decoded adjacent to the current block and a second motion vector for a second predetermined region that has been adjacent to the current block in the vertical direction A first and second motion vector determination unit that employs one of the first motion vector and the second motion vector;
A temporary area dividing unit that divides the encoding target block into two temporary areas at a predetermined temporary boundary;
Of the two provisional areas, the original image of the block to be encoded is used for the provisional area that is not adjacent to the predetermined area for which the motion vector adopted by the first and second motion vector determination units is determined. A third motion vector detection unit for detecting three motion vectors;
Using the motion vector adopted by the first and second motion vector determination units and the third motion vector, two prediction blocks corresponding to the coding target block are generated from a reference image, and two prediction blocks A motion compensation unit that determines a main boundary based on an activity and generates a combined prediction block by combining regions obtained by dividing the prediction blocks at the main boundary between the prediction blocks;
A moving picture encoding apparatus comprising: a prediction difference block obtained by subtracting the combined prediction block from the encoding target block; and an encoding unit that encodes a motion vector of each temporary region. The first motion vector of the first predetermined region that has been locally decoded adjacent to the block to be encoded in the horizontal direction, and the second motion vector of the second predetermined region that has been locally decoded and adjacent to the block to be encoded in the vertical direction. A first and second motion vector determination unit for selecting one of the first motion vector and the second motion vector as a main motion vector and the other motion vector as a sub motion vector;
Using the first motion vector and the second motion vector, first and second prediction blocks corresponding to the coding target block are generated from a reference image, and first and second prediction blocks are generated based on activities of the first and second prediction blocks. A motion compensator for determining one main boundary;
Of the two main regions divided by the first main boundary, the portion that is adjacent to the predetermined region in which the motion vector selected as the sub motion vector by the first / second motion vector determination unit is determined A temporary area dividing unit that selects the main area and divides the selected main area into two temporary areas at a predetermined temporary boundary;
Among the two temporary areas, a third motion vector is detected using the original image of the encoding target block for the temporary area that is not adjacent to the predetermined area for which the motion vector selected as the sub-motion vector is determined. A third motion vector detection unit,
The motion compensation unit generates a third and a fourth prediction block corresponding to the coding target block from a reference image using the motion vector selected as the sub motion vector and the third motion vector, And a region obtained by determining a second main boundary based on the activity of the fourth prediction block and dividing the first or second prediction block at the first main boundary; and third and fourth prediction blocks Are combined with each of the prediction blocks to generate a composite prediction block,
A moving picture coding apparatus comprising: a coding unit that codes a prediction difference block obtained by subtracting the combined prediction block from the coding target block and a motion vector of each temporary area. Determine a first motion vector for a first predetermined region that has been locally decoded adjacent to the current block and a second motion vector for a second predetermined region that has been adjacent to the current block in the vertical direction And adopting one of the first motion vector and the second motion vector,
Dividing the block to be encoded into two temporary regions at a predetermined temporary boundary;
Of the two provisional areas, the original image of the block to be encoded is used for the provisional area that is not adjacent to the predetermined area for which the motion vector adopted by the first and second motion vector determination units is determined. Detecting three motion vectors;
Using the motion vector adopted by the first and second motion vector determination units and the third motion vector, two prediction blocks corresponding to the coding target block are generated from a reference image, and two prediction blocks Determining a main boundary based on an activity, and generating a combined prediction block by combining regions obtained by dividing the prediction blocks at the main boundary between the prediction blocks;
A moving picture coding method comprising: a prediction difference block obtained by subtracting the synthesized prediction block from the coding target block; and a step of coding a motion vector of each temporary area. Determine a first motion vector for a first predetermined region that has been locally decoded adjacent to the current block and a second motion vector for a second predetermined region that has been adjacent to the current block in the vertical direction A function of adopting one of the first motion vector and the second motion vector;
A function of dividing the block to be encoded into two temporary areas at a predetermined temporary boundary;
Of the two provisional areas, the original image of the block to be encoded is used for the provisional area that is not adjacent to the predetermined area for which the motion vector adopted by the first and second motion vector determination units is determined. A function to detect three motion vectors;
Using the motion vector adopted by the first and second motion vector determination units and the third motion vector, two prediction blocks corresponding to the coding target block are generated from a reference image, and two prediction blocks A function of generating a composite prediction block by determining a main boundary based on an activity, and combining regions obtained by dividing the prediction blocks at the main boundary between the prediction blocks;
A moving picture encoding program that causes a computer to realize a prediction difference block obtained by subtracting the synthesized prediction block from the encoding target block and a function of encoding a motion vector of each temporary area.

Images