JP2012080213A - Moving image encoding apparatus, moving image decoding apparatus, moving image encoding method and moving image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To implement complicated block segmentation and represent the segmentation state with a small code amount.SOLUTION: For generation of encoding blocks by segmentation of a maximum encoding block, an encoding control section 1 selects the most efficient one from a plurality of segmentation patterns mixedly including a segmentation pattern of quad-tree segmentation and a segmentation pattern that cannot be represented by quad-tree segmentation, so that encoding blocks are generated according to the segmentation pattern.

Description

  The present invention relates to a moving image encoding apparatus and moving image encoding method for encoding a moving image with high efficiency, a moving image decoding apparatus and a moving image decoding method for decoding a moving image encoded with high efficiency, and It is about.

For example, in an international standard video coding system such as MPEG (Moving Picture Experts Group) or “ITU-T H.26x”, each frame of a video signal is equally divided into square blocks called macroblocks. By performing entropy coding by performing intra-frame / inter-frame prediction processing, orthogonal transform processing (for example, DCT) for the prediction difference signal, and quantization processing, a bit stream is obtained as final compressed data Yes.
AVC / H. Is an international standard system. In the system up to H.264 (ISO / IEC 14496-10 | ITU-T H.264), the macroblock has a block size of 16 pixels × 16 lines on the luminance signal. It has been reported that the encoding performance is greatly improved by adapting the sizes of the prediction target block and the transform block (refer to Non-Patent Documents 1 and 2 below).

In the conventional macroblock size expansion method, a configuration in which a macroblock is further divided and motion vectors are detected for each divided block, or a configuration in which the orthogonal transform size is adapted is disclosed. The unit for giving the coding mode information such as the selection of the inner coding or the inter-frame coding uses the quadtree division of FIG.
In FIG. 16, a coded block having a size of (M ⁰ , M ⁰ ) (the number on the upper right shoulder indicates a layer level) with a luminance component described as “0th layer” is a macroblock, With the macroblock as a starting point, the coded block division state is obtained by performing a hierarchical division to a predetermined depth determined separately in a quadtree structure.
Since quadtree partitioning is performed, (M ^{n + 1} , M ^{n + 1} ) = (M ⁿ / 2, M ⁿ / 2) always holds.

Quadtree partitioning is suitable for the purpose of expressing the partitioning state with a small amount of information, but conversely, the partitioning state is limited to a hierarchical quadtree structure and can express a partitioning state that deviates from it. It has a problem that it cannot be done.
The degree of freedom in expressing the divided state has a significant influence on the coding efficiency for the following two reasons.

Reason 1: In general, the division occurs when the optimum prediction method is different and the optimum prediction parameter needs to be changed.
By dividing the regions with different motions and obtaining the optimal motion vector for each, it is possible to improve the overall prediction efficiency.
On the other hand, in the video signal, it is generally not guaranteed that the boundary between the regions having different motions is always present at the boundary of the quadtree division.
Therefore, if the division is restricted to the boundary of the quadtree division, there is a possibility that some of the divided areas cannot perform the optimal motion prediction.

Reason 2: It is known that if the block size of the orthogonal transform can be increased, the encoding efficiency can be improved by a combination with appropriate quantization and entropy encoding.
Orthogonal transformation is a technique that improves the coding efficiency by utilizing the power of the image signal concentrated in the low frequency region. Therefore, when motion prediction is performed on a block basis and the prediction difference signal is orthogonally transformed, As a result, it is customary to avoid defining orthogonal transform blocks across motion prediction blocks so that unnecessary high-frequency components do not occur (pixel changes are generally discontinuous at the motion prediction block boundaries). Therefore, unnecessary high-frequency components are included).
When this concept is applied, the orthogonal transform block size is restricted by the size of the motion prediction block, and the opportunity to apply a large transform block size is limited.

S.Ma and C.-C. J.Kuo, “High-definition Video Coding with Super-macroblocks,” Visual Communications and Image Processing 2007, September 2007. P. Chen, Y. Ye and M. Karczewicz, “Video Coding Using Extended Block Sizes,” VCEG-AJ23, October 2008.

Since the conventional moving image coding apparatus is configured as described above, if the quadtree partitioning is used, the partition state can be expressed with a small amount of information. However, since the division state is limited to a hierarchical quadtree structure, there is a problem that it is not possible to express a division state that deviates therefrom.
Further, if quadtree partitioning is used, a block partition state with a higher degree of freedom can be described as the hierarchy is deepened. However, a large amount of code is required to express the division state, and there is a problem that the amount of code increases.

The present invention has been made to solve the above-described problems, and is capable of realizing complex block division and at the same time expressing a division state having a high degree of freedom with a small code amount. It is an object to obtain an encoding device and a moving image encoding method.
The present invention also provides a moving picture decoding apparatus and a moving picture decoding method capable of decoding information expressing a divided state having a high degree of freedom from a bit stream and reproducing a video signal. With the goal.

  In the moving picture coding apparatus according to the present invention, when the coding control unit divides the initial block and generates the coded block, a mixed pattern of the quadtree partitioning and a split pattern that cannot be expressed by the quadtree partitioning are mixed. The most efficient division pattern is selected from the plurality of division patterns, and an encoded block is generated according to the division pattern.

  According to the present invention, when the encoding control means divides the initial block to generate the encoded block, a plurality of divided patterns of quadtree division and division patterns that cannot be expressed by quadtree division are mixed. Since the most efficient division pattern is selected from the division patterns and the encoded block is generated according to the division pattern, it is possible to realize the division of a complicated block and at the same time, the code amount is small. Thus, it is possible to express a divided state with a high degree of freedom.

It is a block diagram which shows the moving image encoder by Embodiment 1 of this invention. It is a block diagram which shows the moving image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content of the moving image encoder by Embodiment 1 of this invention. 5 is a flowchart showing a coding mode selection process by the coding control unit 1. It is a flowchart which shows the processing content of the moving image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the decoding procedure of the division | segmentation state of the largest encoding block by the variable length decoding part. It is explanatory drawing which shows a mode that the encoding block of the largest size is divided | segmented into a some encoding block hierarchically. (A) shows the distribution of the partitions after the division, and (b) is an explanatory diagram showing a situation in which the encoding mode m (B ⁿ ) is assigned to the partition after the hierarchy division as a quadtree graph. It is explanatory drawing which shows the processing content at the time of determining the division | segmentation state of the largest encoding block. It is explanatory drawing which shows an example of inter division | segmentation state description information. It is explanatory drawing which shows the state from which the division | segmentation pattern of a current largest encoding block differs greatly from the state of a reference encoding block. It is explanatory drawing which shows the case where both the division | segmentation state of the current largest encoding block and the division | segmentation state of a reference encoding block are expressed by quadtree division | segmentation. It is explanatory drawing which shows the bit stream (especially data of a slice level) produced | generated by the variable length encoding part. It is explanatory drawing which shows the bit stream (especially data of a slice level) produced | generated by the variable length encoding part. It is explanatory drawing which shows a mode that the encoding block of the largest size is divided | segmented into a some encoding block hierarchically. It is explanatory drawing which shows the example which uses quadtree partitioning as a unit which provides encoding mode information.

Embodiment 1 FIG.
In the first embodiment, each frame image of a video is input, a prediction image is generated by performing intra prediction processing from an encoded neighboring pixel or motion compensation prediction processing between adjacent frames, and the prediction image A video encoding device that generates a bitstream by performing variable-length coding after compression processing by orthogonal transformation and quantization is performed on a prediction differential signal that is a difference image between a frame image and the video A moving picture decoding apparatus for decoding a bitstream output from an encoding apparatus will be described.

The moving picture coding apparatus according to the first embodiment adapts to local changes in the spatial and temporal directions of a video signal, divides the video signal into regions of various sizes, and performs intraframe / interframe adaptive coding. It is characterized by performing.
In general, a video signal has a characteristic that the complexity of the signal changes locally in space and time. When viewed spatially, on a particular video frame, there are patterns with uniform signal characteristics in a relatively large image area such as the sky and walls, and small images such as people and paintings with fine textures. A pattern having a complicated texture pattern may be mixed in the region.
Even when viewed temporally, the sky and the wall have small changes in the pattern in the time direction locally, but the moving person or object has a rigid or non-rigid motion in time, so the temporal change does not occur. large.

The encoding process reduces the overall code amount by generating a prediction difference signal with small signal power and entropy by temporal and spatial prediction, but the parameters for prediction are made uniform in as large an image signal region as possible. If applicable, the code amount of the parameter can be reduced.
On the other hand, if the same prediction parameter is applied to an image signal pattern having a large temporal and spatial change, the number of prediction differential signals cannot be reduced because prediction errors increase.
Therefore, for image signal patterns with large temporal and spatial changes, the prediction target signal power / entropy is reduced even if the prediction target area is reduced and the amount of parameter data for prediction is increased. Is preferable.
In order to perform coding adapted to the general properties of such a video signal, the moving picture coding apparatus of the first embodiment divides the video signal area hierarchically from a predetermined maximum block size, Prediction processing and prediction difference encoding processing are performed for each divided region.

The video signal to be processed by the moving image coding apparatus according to the first embodiment is a color in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to the video signal, the video frame is an arbitrary video signal such as a monochrome image signal or an infrared image signal, in which the video frame is composed of a horizontal and vertical two-dimensional digital sample (pixel) sequence.
The gradation of each pixel may be 8 bits, or may be gradation such as 10 bits or 12 bits.
However, in the following description, it is assumed that the input video signal is a YUV signal unless otherwise specified. In addition, it is assumed that the two color difference components U and V are subsampled 4: 2: 0 format signals with respect to the luminance component Y.
The processing data unit corresponding to each frame of the video is referred to as “picture”. In the first embodiment, “picture” is described as a signal of a video frame that has been sequentially scanned (progressive scan). However, when the video signal is an interlace signal, the “picture” may be a field image signal which is a unit constituting a video frame.

FIG. 1 is a block diagram showing a moving picture coding apparatus according to Embodiment 1 of the present invention.
In FIG. 1, the encoding control unit 1 has a maximum size of an encoding block (initial block size) that is a processing unit when intra prediction processing (intraframe prediction processing) or motion compensation prediction processing (interframe prediction processing) is performed. Size) and a process of determining the upper limit number of layers when the encoding block of the maximum size is hierarchically divided.
In addition, the encoding control unit 1 assigns each encoding block divided hierarchically from one or more available encoding modes (one or more intra encoding modes and one or more inter encoding modes). A process of selecting a suitable encoding mode is performed.
In addition, the encoding control unit 1 determines the quantization parameter and transform block size used when the difference image is compressed for each encoding block, and intra prediction used when the prediction process is performed. A parameter or an inter prediction parameter is determined, and a prediction differential encoding parameter including the quantization parameter and the transform block size is output to the transform / quantization unit 7, the inverse quantization / inverse transform unit 8, and the variable length coding unit 13. Then, the intra prediction parameter is output to the intra prediction unit 4 and the variable length coding unit 13, and the inter prediction parameter is output to the motion compensation prediction unit 5 and the variable length coding unit 13.
In addition, the encoding control unit 1 performs processing for outputting maximum coding block division state description information indicating the division state of the maximum size coding block to the variable length coding unit 13.
The encoding control unit 1 constitutes an encoding control unit.

  When the video signal indicating the input image is input, the block dividing unit 2 divides the input image indicated by the video signal into encoded blocks of the maximum size determined by the encoding control unit 1 and determined by the encoding control unit 1. The process of dividing the encoded block hierarchically is performed until the upper limit number of hierarchies is reached. The block dividing unit 2 constitutes block dividing means.

If the coding mode selected by the coding control unit 1 is the intra coding mode, the changeover switch 3 outputs the coding block divided by the block dividing unit 2 to the intra prediction unit 4, and the coding control unit 1 If the coding mode selected by (2) is the inter coding mode, a process of outputting the coding block divided by the block dividing unit 2 to the motion compensation prediction unit 5 is performed.
When the intra prediction unit 4 receives the encoded block divided by the block dividing unit 2 from the changeover switch 3, the intra prediction unit 4 converts the intra prediction parameter output from the encoding control unit 1 using the encoded image signal in the frame. Based on this, a process for generating a predicted image is performed by performing an intra-frame prediction process for the encoded block.

When the inter coding mode is selected by the coding control unit 1 as the coding mode suitable for the coding block divided by the block dividing unit 2, the motion compensated prediction unit 5 is stored by the motion compensated prediction frame memory 12. Based on the inter prediction parameter output from the encoding control unit 1, using a reference image of one or more frames, a process for generating a predicted image is performed by performing a motion compensation prediction process for the encoded block .
The changeover switch 3, the intra prediction unit 4, and the motion compensation prediction unit 5 constitute a predicted image generation unit.

The subtracting unit 6 subtracts the prediction image generated by the intra prediction unit 4 or the motion compensation prediction unit 5 from the encoded block divided by the block dividing unit 2, thereby obtaining a difference image (= encoded block−predicted image). The process to generate is performed. The subtracting unit 6 constitutes a difference image generating unit.
The transform / quantization unit 7 performs transform processing (for example, DCT (discrete) of the difference image generated by the subtraction unit 6 in units of transform block size included in the prediction difference encoding parameter output from the encoding control unit 1. Cosine transformation) and orthogonal transformation processing such as KL transformation in which a base design is made in advance for a specific learning sequence) and using the quantization parameter included in the prediction differential encoding parameter, By quantizing the transform coefficient of the difference image, a process of outputting the quantized transform coefficient as compressed data of the difference image is performed. The transform / quantization unit 7 constitutes an image compression unit.

  The inverse quantization / inverse transform unit 8 performs inverse quantization on the compressed data output from the transform / quantization unit 7 using the quantization parameter included in the prediction difference encoding parameter output from the encoding control unit 1. Inverse transform processing (for example, inverse DCT (Inverse Discrete Cosine Transform), inverse KL transform, etc.) of the inverse quantized compressed data in units of transform block sizes included in the prediction difference encoding parameter ), The process of outputting the compressed data after the inverse transform process as a local decoded prediction difference signal is performed.

The adding unit 9 adds the local decoded prediction difference signal output from the inverse quantization / inverse transform unit 8 and the prediction signal indicating the prediction image generated by the intra prediction unit 4 or the motion compensation prediction unit 5 to thereby perform local decoding. A process of generating a locally decoded image signal indicating an image is performed.
The intra prediction memory 10 is a recording medium such as a RAM that stores a local decoded image indicated by the local decoded image signal generated by the adding unit 9 as an image used in the next intra prediction process by the intra prediction unit 4.

The loop filter unit 11 compensates for the coding distortion included in the locally decoded image signal generated by the adder 9, and performs motion compensation prediction using the locally decoded image indicated by the locally decoded image signal after the coding distortion compensation as a reference image. A process of outputting to the frame memory 12 is performed.
The motion compensated prediction frame memory 12 is a recording medium such as a RAM that stores a locally decoded image after the filtering process by the loop filter unit 11 as a reference image used in the next motion compensated prediction process by the motion compensated prediction unit 5.

  The variable length coding unit 13 includes the compressed data output from the transform / quantization unit 7, the coding mode, the prediction differential coding parameter, and the maximum coding block division state description information (maximum) output from the coding control unit 1. Information indicating the division state of the coding block of size) and the intra prediction parameter output from the intra prediction unit 4 or the inter prediction parameter output from the motion compensation prediction unit 5 are variable-length encoded, and the compressed data, A process of generating a bitstream in which encoded data of an encoding mode, a prediction difference encoding parameter, maximum encoding block division state description information, and intra prediction parameter / inter prediction parameter is multiplexed is performed. The variable length encoding unit 13 constitutes variable length encoding means.

2 is a block diagram showing a moving picture decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 2, the variable length decoding unit 51 performs variable length decoding on the maximum encoded block division state description information indicating the division state of the maximum size encoded block from the encoded data multiplexed in the bitstream to obtain the maximum size. The coding block generated by hierarchically dividing the coding block is identified, and the compressed data, coding mode, and prediction difference relating to the coding block are coded from the coded data multiplexed in the bit stream. The encoding parameter and the intra prediction parameter / inter prediction parameter are subjected to variable length decoding, and the compressed data and the prediction differential encoding parameter are output to the inverse quantization / inverse transform unit 55, and the encoding mode and the intra prediction parameter / A process of outputting the inter prediction parameter to the changeover switch 52 is performed.
The variable length decoding unit 51 constitutes variable length decoding means.

  The changeover switch 52 outputs the intra prediction parameter output from the variable length decoding unit 51 to the intra prediction unit 53 when the coding mode related to the coding block output from the variable length decoding unit 51 is the intra coding mode. When the coding mode is the inter coding mode, a process of outputting the inter prediction parameter output from the variable length decoding unit 51 to the motion compensation prediction unit 54 is performed.

  The intra prediction unit 53 uses the decoded image signal in the frame to generate a prediction image by performing the intra-frame prediction process on the encoded block based on the intra prediction parameter output from the changeover switch 52. To implement.

The motion compensation prediction unit 54 performs motion compensation prediction processing for the encoded block based on the inter prediction parameter output from the changeover switch 52 using one or more reference images stored in the motion compensation prediction frame memory 59. The process which produces | generates an estimated image is implemented by implementing.
The changeover switch 52, the intra prediction unit 53, and the motion compensation prediction unit 54 constitute a predicted image generation unit.

  The inverse quantization / inverse transform unit 55 uses the quantization parameter included in the prediction difference encoding parameter output from the variable length decoding unit 51 to compress the encoded block output from the variable length decoding unit 51. Data is inversely quantized, and inverse transform processing (for example, inverse DCT (Inverse Discrete Cosine Transform), inverse KL transform, etc.) is performed on the transform block size unit included in the prediction differential encoding parameter. (Inverse transform process) is performed to output the compressed data after the inverse transform process as a decoded prediction difference signal (a signal indicating a difference image before compression). The inverse quantization / inverse conversion unit 55 constitutes a difference image generation unit.

The addition unit 56 adds the decoded prediction difference signal output from the inverse quantization / inverse conversion unit 55 and the prediction signal indicating the prediction image generated by the intra prediction unit 53 or the motion compensation prediction unit 54, thereby adding the decoded image. The process which produces | generates the decoded image signal shown is implemented. The adding unit 56 constitutes a decoded image generating unit.
The intra prediction memory 57 is a recording medium such as a RAM that stores a decoded image indicated by the decoded image signal generated by the adding unit 56 as an image used by the intra prediction unit 53 in the next intra prediction process.

The loop filter unit 58 compensates for the coding distortion included in the decoded picture signal generated by the adder 56, and uses the decoded picture indicated by the decoded picture signal after the coding distortion compensation as a reference picture as a motion compensated prediction frame memory 59. Execute the process to output to.
The motion compensated prediction frame memory 59 is a recording medium such as a RAM that stores a decoded image after the filtering process by the loop filter unit 58 as a reference image to be used by the motion compensation prediction unit 54 in the next motion compensation prediction process.

In FIG. 1, a coding control unit 1, a block division unit 2, a changeover switch 3, an intra prediction unit 4, a motion compensation prediction unit 5, a subtraction unit 6, and a transform / quantization unit 7, which are components of the moving image coding apparatus. , The inverse quantization / inverse transform unit 8, the adder unit 9, the loop filter unit 11 and the variable length coding unit 13 each have dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or a one-chip microcomputer) However, when the moving image encoding apparatus is configured by a computer, the encoding control unit 1, the block division unit 2, the changeover switch 3, the intra prediction unit 4, the motion compensation A program describing the processing contents of the prediction unit 5, subtraction unit 6, transformation / quantization unit 7, inverse quantization / inverse transformation unit 8, addition unit 9, loop filter unit 11, and variable length coding unit 13 Computer Stored in the memory, CPU of the computer may execute a program stored in the memory.
FIG. 3 is a flowchart showing the processing contents of the moving picture coding apparatus according to Embodiment 1 of the present invention.

In FIG. 2, a variable length decoding unit 51, a changeover switch 52, an intra prediction unit 53, a motion compensation prediction unit 54, an inverse quantization / inverse transformation unit 55, an addition unit 56, and a loop filter unit, which are components of the moving image decoding apparatus. 58 is assumed to be configured by dedicated hardware (for example, a semiconductor integrated circuit on which a CPU is mounted, or a one-chip microcomputer), but the moving image decoding apparatus is configured by a computer. A program describing the processing contents of the variable length decoding unit 51, the changeover switch 52, the intra prediction unit 53, the motion compensation prediction unit 54, the inverse quantization / inverse transformation unit 55, the addition unit 56, and the loop filter unit 58. May be stored in the memory of the computer, and the CPU of the computer may execute the program stored in the memory.
FIG. 5 is a flowchart showing the processing contents of the moving picture decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
First, the processing contents of the moving picture encoding apparatus in FIG. 1 will be described.
First, the encoding control unit 1 determines the maximum size of an encoding block that is a processing unit when intra prediction processing (intraframe prediction processing) or motion compensation prediction processing (interframe prediction processing) is performed, The upper limit number of layers when the coding block of the maximum size is divided hierarchically is determined (step ST1 in FIG. 3).

As a method of determining the maximum size of the encoded block, for example, a method of determining a size corresponding to the resolution of the input image for all the pictures can be considered.
In addition, the difference in complexity of local motion of the input image is quantified as a parameter, and the maximum size is determined to be a small value for pictures with intense motion, and the maximum size is determined to be a large value for pictures with little motion. And so on.
The upper limit of the number of hierarchies is set so that, for example, the more the input image moves, the deeper the number of hierarchies, so that finer motion can be detected. A way to do this is conceivable.

In addition, the encoding control unit 1 includes each encoding block divided hierarchically from one or more available encoding modes (M types of intra encoding modes and N types of inter encoding modes). Is selected (step ST2).
When selecting a coding mode, the coding control unit 1 selects a coding mode having the highest coding efficiency for each coding block from all available coding modes or a subset thereof. However, details of the encoding mode selection method by the encoding control unit 1 will be described later.

In addition, the encoding control unit 1 determines a quantization parameter and a transform block size that are used when the difference image is compressed for each encoding block, and an intra that is used when the prediction process is performed. A prediction parameter or an inter prediction parameter is determined.
Also, the encoding control unit 1 generates maximum encoded block division state description information (details will be described later) indicating the division state of the maximum size encoded block.
The encoding control unit 1 outputs the prediction difference encoding parameter including the quantization parameter and the transform block size to the transform / quantization unit 7, the inverse quantization / inverse transform unit 8, and the variable length coding unit 13, The maximum coded block division state description information is output to the variable length coding unit 13.

When a video signal indicating an input image is input, the block dividing unit 2 divides the input image indicated by the video signal into encoded blocks of the maximum size determined by the encoding control unit 1, and the encoding control unit 1 The encoded block is hierarchically divided until the determined upper limit number of layers is reached.
Here, FIG. 7 is an explanatory diagram showing a state in which a maximum-size encoded block is hierarchically divided into a plurality of encoded blocks.
In the example of FIG. 7, the maximum coding block when n = 0 is the coding block B ⁰ of the 0th layer, and has a size of (L ⁰ , M ⁰ ) as a luminance component.
Further, the example of FIG. 7 is characterized in that the division into eight division patterns is supported with the maximum coding block B ⁰ as the starting point (= 0th layer).
In FIG. 7, identification numbers SP1 to SP8 are assigned to the patterns. In a coding mode selection process by the coding control unit 1 to be described later, the pattern identification number is output to the variable length coding unit 13 as a part of maximum coding block division state description information.

SP1 is a quadtree structure pattern shown in the conventional example.
Other patterns make it possible to define coding blocks that cannot be represented by a quadtree structure, and the blocks surrounded by the solid lines of each pattern are coding blocks. The handling of the non-square area indicated by the dotted line will be described later.
SP2 is a horizontal center portion of the maximum coding block, SP3 is a vertical center portion of the maximum coding block, and a code having a size satisfying (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) This is a pattern for assigning generalized blocks.
SP4 enables the definition of a coding block that cannot be expressed by SP2 or SP3. These division patterns are effective in a case where motion activity and prediction differential signal power are concentrated at the center of the maximum coding block.
SP5 to SP8 make it possible to define a coding block that is more effective than quadtree partitioning by specializing in one of the top, bottom, left, and right areas in the maximum coding block.

By limiting the encoding block to the size of (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2), the orthogonal transform processing unit is raised to a power of 2 as in the case of the quadtree SP1. The maximum transform block size can be ensured in each layer while limiting to square blocks with one side as a side, and the number of division patterns can be limited.
By increasing the number of selectable division patterns, it is possible to improve the optimality of encoding block allocation, while increasing the processing load for selecting an optimal pattern on the moving image encoding device side.
By using the definition of the limited division pattern as shown in FIG. 7, it is possible to perform encoding with a good balance between encoding performance and encoding processing load.

Although the configuration of the moving image coding apparatus is complicated and the processing load is increased, the coding block size variation is increased to a rectangular shape in order to prioritize the improvement of coding efficiency, and more divisions than in FIG. You may comprise so that a pattern may be expressed.
In this case, with regard to the maximum transform block size, if the encoded block is configured with a multiple of the maximum transform block size that can be shared with the quadrature-structured SP1 encoded block, the orthogonal transform processing unit is 2 Therefore, it is possible to reduce the complexity of the apparatus related to the transformation / quantization processing by unifying the square block having a power of one side.
Of course, although it is premised that the device configuration is complicated, a transform block size having a size corresponding to an arbitrary rectangular encoding block may be newly defined and supported.

Using such a division pattern, a coded block ^Bn is obtained by dividing up to the number of division hierarchies determined separately. When the pattern of FIG. 7 is used, at the depth n, the coding block B ⁿ is an image area of size (L ⁿ , M ⁿ ).
However, L ⁿ and M ⁿ may be the same or different, but the example of FIG. 7 shows a case of L ⁿ = M ⁿ .
Later, the size of the encoded block ^{B n} is the size ^{(L n, M} n) in the luminance component of the encoded block ^{B n} is defined as always ^{(L n + 1, M n} + 1) = (L n / 2, M n / 2) is established.
However, in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ). When the 4: 2: 0 format is handled, the size of the corresponding color difference component coding block is (L ⁿ / 2, M ⁿ / 2).
Hereinafter, a coding mode that can be selected by the coding block B ⁿ is described as m (B ⁿ ), ^where B ⁿ is the coding block of the nth layer.

In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, but hereinafter, unless otherwise specified, YUV The description will be made on the assumption that it indicates the coding mode for the luminance component of the coding block of the signal 4: 2: 0 format.
The coding mode m (B ⁿ ) includes one or more intra coding modes (collectively “INTRA”), one or more inter coding modes (collectively “INTER”), As described above, the encoding control unit 1 selects an encoding mode having the highest encoding efficiency for the encoding block B ⁿ from all the encoding modes available for the picture or a subset thereof. .

The encoding control unit 1 hierarchically selects the division pattern shown in FIG. 7 for each maximum encoding block of the picture to be encoded (current picture), and specifies the encoding block ^Bn . To do.
FIG. 8 shows an example of the division state of the maximum coding block.
The example of FIG. 8 shows a case where the division pattern SP2 of FIG. 7 is selected at the level of the first hierarchy, and the division pattern SP3 of FIG. 7 is selected at the level of the third hierarchy.
A region surrounded by a dotted line in FIG. 8A is a region defined as a coding block ^Bn , and a shaded portion indicates a distribution of partitions that are units of prediction processing in the coding block ^Bn . ing.
FIG. 8B shows a situation in which the encoding mode m (B ⁿ ) is assigned to the partition after hierarchical division in a quadtree graph.
In FIG. 8B, white nodes surrounded by □ indicate nodes (encoded blocks B ⁿ ) to which the encoding mode m (B ⁿ ) is assigned, and black nodes surrounded by □. A filled node indicates a non-square area that is encoded by a method described later separately from the encoded block.

The coding block ^Bn is further divided into one or more prediction processing units (partitions).
Hereinafter, the partition belonging to the coding block B ⁿ is ^denoted as P _i ⁿ (i: the partition number in the nth layer).
How the partition P _i ⁿ belonging to the coding block B ⁿ is divided is included as information in the coding mode m (B ⁿ ).
All partitions P _i ⁿ are subjected to prediction processing according to the coding mode m (B ⁿ ), but individual prediction parameters can be selected for each partition P _i ⁿ .

Changeover switch 3, the encoding control unit 1 selects the optimal coding mode m (B ⁿ⁾ for the partition P _i ⁿ in each of the coding blocks B ^n, the encoding mode m (B ⁿ⁾ is If it is the intra coding mode (step ST3), the partition P _i ⁿ of the coding block B ⁿ divided by the block dividing unit 2 is output to the intra prediction unit 4.
On the other hand, if the coding mode m (B ⁿ ) is the inter coding mode (step ST 3), the partition P _i ⁿ of the coding block B ⁿ divided by the block dividing unit 2 is output to the motion compensation prediction unit 5. To do.

When the intra prediction unit 4 receives the partition P _i ⁿ of the coding block B ⁿ from the changeover switch 3, the intra prediction unit 4 uses the coded image signal in the frame as the intra prediction parameter output from the coding control unit 1. Based on this, intra-prediction processing for the partition P _i ⁿ of the coding block B ⁿ is performed to generate an intra prediction image P _i ⁿ (step ST4).
Intra prediction parameters used to generate the intra prediction image _P ^{i n,} for example, the intra prediction process, AVC / H. When performing adaptive spatial prediction having directionality as defined in the H.264 standard (ISO / IEC 14496-10), it includes information such as prediction mode information selected for each partition, and is exactly the same on the video decoding device side. In order to be able to generate an intra-predicted image, a variable-length encoding unit 13 described later multiplexes encoded data of intra-prediction parameters into a bitstream.
The intra prediction process is performed using AVC / H. Although not limited to the algorithm defined in the H.264 standard, the intra prediction parameter needs to include information necessary for generating exactly the same intra predicted image in the moving image encoding device and the moving image decoding device.

When the motion compensation prediction unit 5 receives the partition P _i ⁿ of the coding block B ⁿ from the changeover switch 3, the motion compensation prediction unit 5 uses the reference image of one or more frames stored in the motion compensation prediction frame memory 12 to use the coding control unit. based on the inter prediction parameter output from the 1, it generates an inter prediction image P _i ⁿ by performing motion compensation prediction processing on partition P _i ⁿ in the coded blocks B ⁿ (step ST5).
Incidentally, a technique for generating a prediction image by performing motion compensation prediction process and a detailed description thereof will be omitted because it is a known technique, the inter prediction parameters used for generating the inter prediction image P _i ⁿ is It is necessary to include the following information, etc., and in order to be able to generate exactly the same intra-predicted image on the video decoding device side, the variable-length encoding unit 13 described later converts the encoded data of the inter prediction parameter into bits. Multiplex to stream.

(1) Mode information describing partition division in coding block ^Bn (2) Motion vector of each partition (3) Any reference image in the case of a configuration including a plurality of reference images in the motion compensated prediction frame memory 12 Reference image indication index information indicating whether prediction processing is performed using (4) When there are a plurality of motion vector prediction value candidates, index information (5) indicating which motion vector prediction value is selected and used Index information indicating which filter is to be selected and used (6) The motion vector of the partition has a plurality of pixel accuracy (for example, half pixel, 1/4 pixel, 1 / Selection information indicating which pixel accuracy to use, if possible.

When the intra prediction unit 4 or the motion compensation prediction unit 5 generates a prediction image (intra prediction image P _i ⁿ , inter prediction image P _i ⁿ ), the subtraction unit 6 encodes the encoded block B ⁿ divided by the block division unit 2. A difference image is generated by subtracting a prediction image (intra prediction image P _i ⁿ , inter prediction image P _i ⁿ ) generated by the intra prediction unit 4 or the motion compensated prediction unit 5 from the partition P _i ^{n of} the prediction difference signal _e ^{i n} representing a difference image is output to the transform and quantization unit 7 (step ST6).

Transform and quantization unit 7 receives the prediction difference signal e _i ⁿ that indicates a difference image from the subtraction unit 6, conversion block size contained in the predictive differential coding parameter output from the coding controller 1 The differential image conversion process (for example, DCT (Discrete Cosine Transform) or orthogonal transform process such as KL transform in which a base design is made in advance for a specific learning sequence) is performed, and the prediction differential encoding is performed. The quantization coefficient included in the parameter is used to quantize the transform coefficient of the difference image, so that the quantized transform coefficient is converted into the compressed data of the difference image and the inverse quantization / inverse transform unit 8 and the variable length It outputs to the encoding part 13 (step ST7).

When the inverse quantization / inverse transform unit 8 receives the compressed data of the difference image from the transform / quantization unit 7, the inverse quantization / inverse transform unit 8 uses the quantization parameter included in the prediction difference encoding parameter output from the encoding control unit 1. Then, the compressed data of the difference image is inversely quantized, and the inverse quantization processing (for example, inverse DCT (Inverse Discrete Cosine Transform) is performed on the transform block size unit included in the prediction difference encoding parameter. ) Or inverse transformation processing such as inverse KL transformation), the compressed data after the inverse transformation processing is subjected to local decoded prediction difference signal e _i ⁿ hat ("^" Is expressed as a hat) to the adder 9 (step ST8).

Note that the prediction difference encoding parameter includes information on the quantization parameter and transform block size used for encoding the internal prediction difference signal e _i ⁿ for each region of the encoding block B ^n. As described above, the encoding parameter is determined when the encoding mode m (B ⁿ ) is selected by the encoding control unit 1.
The quantization parameter may be assigned in the unit of the maximum coding block, and may be used in common in the unit of the coding block obtained by dividing them, or the difference value from the value of the maximum coding block for each coding block May be expressed as:
The transform block size may be expressed by quadtree partitioning starting from the coding block ^Bn as in the case of partitioning the maximum coding block, and some selectable transform block sizes are index information. The format expressed as may be used.
The transform / quantization unit 7 and the inverse quantization / inverse transform unit 8 specify the block size of the transform / quantization process based on the information of the transform block size and perform processing. The information in this transform block size, the coding block B ⁿ no may be configured to determine the partition P _i ⁿ that divides the coded block B ⁿ units.

When the adder 9 receives the local decoded prediction difference signal e _i ⁿ hat from the inverse quantization / inverse transform unit 8, the adder 9 performs the local decoded prediction difference signal e _i ⁿ hat and the intra prediction unit 4 or the motion compensation prediction unit 5. By adding a prediction signal indicating the generated prediction image (intra prediction image P _i ⁿ , inter prediction image P _i ⁿ ), a local decoded partition image P _i ⁿ hat or a local decoded encoded block image as a collection thereof is used. A local decoded image is generated (step ST9).
When generating the locally decoded image, the adding unit 9 stores the locally decoded image signal indicating the locally decoded image in the intra prediction memory 10 and outputs the locally decoded image signal to the loop filter unit 11.

Processing in step ST3~ST9 is repeated until processing of all the coding blocks B ⁿ which are hierarchically split is completed, the process proceeds when the processing of all the coding blocks B ⁿ completion process in step ST12 (Steps ST10 and ST11).

The variable length encoding unit 13 includes the compressed data output from the transform / quantization unit 7, the encoding mode, the prediction differential encoding parameter, and the maximum encoding block division state description information output from the encoding control unit 1. The intra prediction parameter output from the intra prediction unit 4 or the inter prediction parameter output from the motion compensated prediction unit 5 is entropy encoded.
The variable-length encoding unit 13 is encoded data of compressed data, encoding mode, prediction differential encoding parameter, maximum encoding block division state description information, intra prediction parameter / inter prediction parameter, which is the encoding result of entropy encoding. Are multiplexed to generate a bit stream (step ST12).

It is no ^{(L n / 4, M n} / 4) indicated by a dotted line in SP2~SP8 7 non-square area of ^{(L n / 2, M n} / 2) , the following (1) to (3) Are encoded as follows.
(1) For these areas, subdivision as coding blocks is not performed. That is, it is assumed that further subdivision can be performed only when SP1 is selected.
(2) Each region performs intra / inter prediction individually.
(3) Transformation / quantization for the prediction difference signal is performed by dividing the region into block sizes of (L ⁿ / 4, M ⁿ / 4).

Further, when selecting the division pattern of SP2 to SP8, it is considered that the information to be encoded is concentrated in the selected encoding block area, and therefore, the dotted lines of SP2 to SP8 in FIG. 7 other than the encoding block. (L ⁿ / 2, M ⁿ / 4) to (L ⁿ / 4, M ⁿ / 2) non-square regions indicated by (1) may be provided with a mode for skipping encoding of the prediction difference signal. .

When receiving the locally decoded image signal from the adder 9, the loop filter unit 11 compensates for the encoding distortion included in the locally decoded image signal, and the locally decoded image indicated by the locally decoded image signal after the encoding distortion compensation Is stored in the motion compensated prediction frame memory 12 as a reference image (step ST13).
The filtering process by the loop filter unit 11 may be performed in units of the maximum coding block or individual coding block of the locally decoded image signal output from the adder 9, or local decoding corresponding to a macroblock for one screen. After the image signal is output, it may be performed for one screen.

Hereinafter, the encoding mode selection processing by the encoding control unit 1 will be described in detail.
FIG. 4 is a flowchart showing a coding mode selection process by the coding control unit 1.
When the coding control unit 1 selects a coding mode, the coding control unit 1 checks whether or not a reference image used for the current picture prediction process is stored in the motion compensated prediction frame memory 12 (step ST21).
In general, a moving image signal has a large degree of redundancy in the time direction, and when viewed locally, a moving image signal may have similar properties between a reference image and a current picture with respect to the pattern and movement.
In the first embodiment, if the reference picture used for the prediction process of the current picture is stored in the motion compensated prediction frame memory 12 using this, it is used when the reference picture is encoded. It is possible to select whether or not to use the division state of the maximum coding block as an initial state when determining the division state of the maximum coding block of the current picture.

FIG. 9 is an explanatory diagram showing the processing contents when determining the division state of the maximum coding block.
In the example of FIG. 9, the maximum encoding block to be encoded of the current picture is A, and the maximum encoding block in the same spatial position as the maximum encoding block A in the reference image that is closest in time is B. Yes.
As described above, the encoding control unit 1 checks whether or not the reference image used for the current picture prediction process is stored in the motion compensated prediction frame memory 12 when the encoding mode is selected.
When the reference picture used for the prediction process of the current picture is stored in the motion compensated prediction frame memory 12, the coding control unit 1 identifies a “reference coding block” (step ST22).
In the example of FIG. 9, the maximum coding block B is specified as a “reference coding block”.

Next, the encoding control unit 1 determines whether or not to use the division state used when the reference encoding block B is encoded as the initial state when determining the division state of the maximum encoding block A. Determine (step ST23).
As this determination method, for example, an evaluation value representing the similarity of the pattern between the maximum coding block A and the reference coding block B is calculated, and only when the evaluation value is a similarity equal to or higher than a predetermined threshold. A method of determining to use the division state of the reference coding block B as the initial state when determining the division state of the maximum coding block A is conceivable.
In addition, based on the division state of the reference coding block B, the maximum code without depending on the rate distortion cost when the predictive coding of the maximum coding block A is performed and the division state of the reference coding block B. The optimal division state of the coded block A is determined, and the rate distortion cost when the predictive coding of the maximum coding block A is performed based on the division state is compared, and the one with the smaller rate distortion cost is selected. A method is conceivable.

Regardless of which determination method is used, if the coding control unit 1 can use the division state of the reference coding block B, the code amount of information indicating the division state of the maximum coding block A can be reduced, and the efficiency can be improved. Encoding can be performed.
Since this determination also needs to be performed by the video decoding device, a division initial state instruction indicating whether or not to use the division state of the reference coding block B as the initial state when determining the division state of the maximum coding block A Information is included in a part of maximum coding block division state description information and multiplexed into a bit stream.

When the division initial state instruction information indicates that “the division state of the reference coding block B is set to the initial state” (step ST24), the final division state (starting from the division state of the reference coding block B) ( Maximum coding block division state description information), coding mode m (B ⁿ ), intra prediction parameter / inter prediction parameter, and prediction differential coding parameter for each coding block B ⁿ are determined (step ST25).
At this time, the final division state is transmitted as difference information from the division state of the reference coding block B. Such division state description information is referred to as “inter division state description information”.

FIG. 10 is an explanatory diagram showing an example of inter division state description information.
In the example of FIG. 10, “situation where quadtree partitioning is being executed” is represented by “1”, and “situation where quadtree partitioning is not being performed” is represented by “0”.
When the division state of the reference encoding block B in the example of FIG. 10 is confirmed, the 0th layer indicates that the division execution is “1”, and the 1st layer is the order of upper left, upper right, lower left, lower right (zigzag scan order ) If you look at each coding block, the first three indicate that the division is not executed “0”, the lower right is the division execution “1”, and the second layer is divided by all the coding blocks This indicates that it is non-execution “0”.
Assuming that three or more layers cannot be divided, the 0th layer is “1”, the 1st layer is “0001”, and the 2nd layer is “0000”.
Similarly, regarding the division state in the current maximum coding block, when viewed in hierarchical order and zigzag scan order, the 0th hierarchy is “1”, the 1st hierarchy is “0001”, and the 2nd hierarchy is “1000”.

When it is known that the reference encoded block B is the starting point, there is no difference between the 0th layer and the 1st layer, so no description information is necessary, and the difference “1000” − “0000” only for the 2nd layer. = “1000” can describe the division state of the current maximum coding block.
As is well known, when the division state is similar to the state of the reference coding block B, the difference of each layer may be biased near zero, or the neighboring coding blocks near the time / space Code amount can be further reduced by performing adaptive entropy coding with an appropriate conditional occurrence probability in consideration of the division state.

Alternatively, a pattern that can express the division state of the current maximum coding block as a difference from the division state of the reference coding block B is determined in advance, an index is assigned to each pattern, and only the index of the selected pattern is transmitted. Or a difference from the selected pattern may be encoded.
Here, FIG. 11 is an explanatory diagram showing a state where the division pattern of the current maximum coding block is significantly different from the state of the reference coding block.
In such a case, if a difference is accumulated for every hierarchy like FIG. 10, many codes | symbols are required.

Therefore, for each division state of the reference coding block B, a similar pattern that can be selected in an intermediate manner is limited, and a pattern closest to the division state of the current maximum coding block is selected from the pattern. The difference is encoded.
An index is assigned to the selected intermediate similar pattern, and the index information is encoded as a part of the inter-partition state description information (if the intermediate similar pattern itself represents a difference, only the index information is encoded) Is encoded).
As a result, even if the division state of the reference coding block B and the division state of the current maximum coding block are widely different, the number of cases where the difference can be efficiently expressed increases, and the coding efficiency of the inter division state description information is increased. Can be increased.
As shown in FIG. 12, it goes without saying that this idea can be applied even when both the division state of the current maximum coding block and the division state of the reference coding block are expressed by quadtree division.

When the division initial state instruction information indicates that “the division state is determined by the current maximum coding block” (step ST24), the procedure does not depend on the reference coding block B as in the procedure described in FIG. , Maximum coding block division state description information indicating a division state for the current maximum coding block, coding mode m (B ⁿ ) for each coding block B ⁿ , intra prediction parameter / inter prediction parameter, prediction differential coding parameter Is determined (step ST26).
The division state description information at this time is referred to as “intra division state description information” and is distinguished from the inter division state description information.

When the division initial state instruction information indicates that “the division state of the reference coding block is set to the initial state”, the division state of the current maximum coding block is set to be the same as the division state of the reference coding block. It may be configured.
Thereby, it is not necessary to encode the inter division state description information at all.
Furthermore, the coding mode m (B ⁿ ) used in each divided coding block B ⁿ may be configured to use the same information as that used in the reference coding block. In this case, it is not necessary to multiplex the encoding mode m (B ⁿ ) into the bit stream.
When there is no reference image, a reference encoding block cannot be determined, so that intra division state description information is always obtained and encoded.

FIG. 13 and FIG. 14 are explanatory diagrams showing bit streams (especially slice level data) generated by the variable length coding unit 13.
The example of FIG. 13 shows a state in which the slice encoded data is composed of the slice header and the encoded data of the maximum number of encoded blocks corresponding to the number in the subsequent slice.
The encoded data of each maximum encoded block includes maximum encoded block division state description information, and the maximum encoded block division state description information includes divided initial state instruction information and division state description information.

Although not shown, the encoded data of the maximum encoding block includes, in addition to this, an encoding mode m (B ⁿ ) for each encoding block B ⁿ , an intra prediction parameter / inter prediction parameter, and a prediction differential encoding parameter. , Including compressed data (quantized transform coefficients).
The format (a) of maximum coding block division state description information indicates a format for selecting whether intra division state description information or inter division state description information is transmitted.
In the format (b), whether the state of the reference coding block is used as it is as the division state of the current maximum coding block and no inter division state description information is transmitted or intra division state description information is transmitted. The format to select is shown.

In the example of FIG. 14, the division initial state indication information multiplexing flag is included in the slice header, and whether or not the division initial state indication information is multiplexed in units of each maximum coding block is specified based on the value of this flag. The configuration is shown.
Since the video signal is non-stationary, there may be a case where the temporal correlation between pictures is very low with respect to the division state (for example, when random motion dominates the entire screen).
In such a case, there is a possibility that the code amount of the divided initial state instruction information itself becomes overhead rather than multiplexing the divided initial state instruction information in units of the maximum coding block and adaptively determining the initial value. is there.
Therefore, a mechanism is provided that can select whether or not to use the divided initial state instruction information at a higher level such as a slice.
As a result, efficient division state encoding using the division initial state indication information can be performed only for slices in which the reference of the division state of the reference coding block by the division initial state indication information is valid.
The division initial state instruction information multiplexing flag may be configured to be multiplexed in a header information area defined in units of pictures, sequences, GOP (Group Of Pictures), and the like.

Next, processing contents of the moving picture decoding apparatus in FIG. 2 will be described.
When the variable length decoding unit 51 receives the bit stream output from the image encoding device in FIG. 1, the variable length decoding unit 51 performs a variable length decoding process on the bit stream and performs a sequence unit or a picture composed of one or more pictures. Information that defines the picture size (the number of horizontal pixels and the number of vertical lines) in units is decoded (step ST31 in FIG. 5).
The variable length decoding unit 51 is a unit of processing when intra prediction processing (intraframe prediction processing) or motion compensation prediction processing (interframe prediction processing) is performed in the same procedure as the coding control unit 1 in FIG. Is determined, and the upper limit number of layers when the maximum size encoded block (maximum encoded block) is hierarchically divided is determined (step ST32).

For example, in the video encoding apparatus, when the maximum size of the encoded block is determined according to the resolution of the input image, the maximum size of the encoded block is determined based on the previously decoded picture size. . When information indicating the maximum size of the encoded block and the upper limit number of layers is multiplexed in the bit stream, the information decoded from the bit stream is referred to.
Further, the variable length decoding unit 51 decodes the division state of the maximum coding block for each maximum coding block based on the division initial state instruction information included in the maximum coding block division state description information.

Hereinafter, the decoding procedure of the division state of the maximum coding block will be specifically described.
FIG. 6 is a flowchart showing the decoding procedure of the division state of the maximum coding block by the variable length decoding unit 51.
First, when decoding the current picture, the variable length decoding unit 51 checks whether or not a reference image used for the current picture prediction process is stored in the motion compensated prediction frame memory 59 (step ST51).
When the reference picture used for the prediction process of the current picture is stored in the motion compensated prediction frame memory 59, the variable length decoding unit 51 is included in the temporally nearest reference picture as shown in FIG. The largest coding block that exists in the same spatial position as the largest coding block to be decoded of a picture is set as a reference coding block (step ST52).

When the reference coding block is set, the variable length decoding unit 51 decodes the division initial state instruction information included in the maximum coding block division state description information of the reference coding block (step ST53), and the division initial state Based on the state instruction information, it is determined whether or not to use the division state of the reference coding block when decoding the division state of the current maximum coding block (step ST54).
That is, it is determined whether intra division state description information or inter division state description information is used when decoding the division state of the current maximum coding block.

The variable length decoding unit 51, when the division initial state instruction information indicates “decode the division state using the intra division state description information”, the intra division state multiplexed subsequent to the division initial state instruction information The description information is decoded, and the division state of the current maximum coding block is determined with reference to the intra division state description information (step ST55).
On the other hand, when the division initial state instruction information indicates “decoding the division state using the inter division state description information”, the inter division state description information (reference code) multiplexed subsequent to the division initial state instruction information (Difference information from the division state of the coded block) is decoded, and the division state of the current maximum coding block is determined with reference to the inter division state description information (step ST56).

As the encoding format of the inter-division state description information, for example, as described in the description of the processing content of the moving image encoding device, there are the following formats (1) and (2). The variable length decoding unit 51 is configured to perform the decoding process corresponding to the encoding format selected in (1).
(1) Adaptive entropy coding of difference information for each division layer (2) The division state of the current maximum coding block is determined in advance as a pattern that can be expressed as a difference from the division state of the reference coding block. As described in the description of the processing content of the moving picture coding apparatus that adaptively entropy-encodes only the index of the selected pattern or the difference information from the selected pattern by adding an index to the divided initial state instruction information, By providing the semantics of “using the reference coding block division state itself”, it is also possible to prevent the inter division state description information from being decoded from the bitstream.

When the bit stream having the configuration shown in FIG. 14 is input, the division initial state instruction information multiplexing flag is decoded from the slice header prior to decoding at the maximum coding block level, and the division initial state instruction information multiplexing flag is set. If it indicates that “the division initial state instruction information is multiplexed”, the division state of the maximum coding block is decoded by the procedure of FIG.
On the other hand, if the division initial state instruction information multiplexing flag indicates that “the division initial state instruction information is not multiplexed”, the division state of the maximum coding block is always decoded based on the intra division state description information. To.
If the division initial state instruction information multiplexing flag is a bit stream configured to be inserted into the sequence level header, GOP level header, or picture level header, decoding is performed from the header, and all the maximum contained below The coding state is subjected to the division state decoding process based on the division initial state instruction information multiplexing flag.

When the variable length decoding unit 51 decodes the division state of the maximum coding block, the variable length decoding unit 51 identifies each coding block ^Bn divided hierarchically based on the division state (step ST33 in FIG. 5).
When the variable length decoding unit 51 identifies each coding block B ⁿ , the variable length decoding unit 51 decodes the coding mode m (B ⁿ ) of the coding block B ⁿ and is included in the coding mode m (B ⁿ ). based on the information of the partition P _i ⁿ that there is partitioned P _i ⁿ a further one or a plurality of prediction processing unit coding block B ^n.
Next, the variable length decoding unit 51, for each partition P _i ^n, compressed data, coding mode, predictive differential coding parameters, decoding the intra prediction parameter / inter prediction parameters (step ST34).

That is, when the coding mode m (B ⁿ ) assigned to the coding block B ⁿ is the intra coding mode, the intra prediction parameter is decoded for each partition P _i ⁿ belonging to the coding block B ⁿ .
Decoding the intra prediction parameters, by the same procedure as the moving image encoding device of FIG. 1, on the basis of the intra prediction parameters near the decoded partition, calculates a predicted value of intra prediction parameters partitions P _i ⁿ is decoded Then, decoding is performed using the predicted value.
When the coding mode m (B ⁿ ) assigned to the coding block B ⁿ is the inter coding mode, the inter prediction parameter is decoded for each partition P _i ⁿ belonging to the coding block B ⁿ .
The partition serving as the prediction processing unit is further divided into one or a plurality of partitions serving as the transform processing unit based on the transform block size information included in the prediction differential encoding parameter, and compressed data ( (Transform coefficient after transform / quantization) is decoded.

When the coding mode m (B ⁿ ) of the partition P _i ⁿ belonging to the coding block B ⁿ is the intra coding mode from the variable length decoding unit 51, the changeover switch 52 is a variable length decoding unit (step ST35). The intra prediction parameter output from 51 is output to the intra prediction unit 53.
On the other hand, when the coding mode m (B ⁿ ) of the partition P _i ⁿ is the inter coding mode (step ST35), the inter prediction parameter output from the variable length decoding unit 51 is output to the motion compensation prediction unit 54.

When receiving the intra prediction parameter from the changeover switch 52, the intra prediction unit 53 uses the decoded image signal in the frame and encodes it based on the intra prediction parameter, as in the case of the intra prediction unit 4 in FIG. An intra-prediction image P _i ⁿ is generated by performing an intra-frame prediction process on the partition P _i ⁿ of the block B ⁿ (step ST36).
When the motion compensation prediction unit 54 receives the inter prediction parameter from the changeover switch 52, the motion compensation prediction unit 54 uses one or more reference images stored in the motion compensation prediction frame memory 59, and based on the inter prediction parameter, encodes a coding block. An inter-predicted image P _i ⁿ is generated by performing motion compensation prediction processing for the partition P _i ⁿ of B ⁿ (step ST37).

The inverse quantization / inverse transform unit 55 uses the quantization parameter included in the prediction difference encoding parameter output from the variable length decoding unit 51 to relate to the coding block output from the variable length decoding unit 51. The compressed data is inversely quantized, and inverse transform processing (for example, inverse DCT (inverse discrete cosine transform) or inverse KL transform) is performed on the transform block size unit included in the prediction differential encoding parameter. By performing the inverse transformation process, etc., the compressed data after the inverse transformation process is output to the adding unit 56 as a decoded prediction difference signal (a signal indicating a difference image before compression) (step ST38).
Note that the quantization parameter included in the prediction differential encoding parameter is restored in units of the encoding block B ⁿ from the encoded data multiplexed in the bit stream, and information on the transform block size is the encoding block. Starting from B ⁿ , in the same way as the division of the maximum coding block, the division information format expressed by quadtree division, the format of selectable transform block sizes expressed as index information, etc. Extract and restore.
Information transform block size, the coding block B ⁿ no may be configured to determine the partition P _i ⁿ that belong to coded blocks B ⁿ units.

  When the addition unit 56 receives the decoded prediction difference signal from the inverse quantization / inverse conversion unit 55, the addition unit 56 adds the decoded prediction difference signal and the prediction signal indicating the prediction image generated by the intra prediction unit 53 or the motion compensated prediction unit 54. Thus, a decoded image is generated, and a decoded image signal indicating the decoded image is stored in the intra prediction memory 57, and the decoded image signal is output to the loop filter unit 58 (step ST39).

The processes of steps ST33 to ST39 are repeatedly performed until the processes for all the coding blocks ^Bn divided hierarchically are completed (step ST40).
When receiving the decoded image signal from the adder 56, the loop filter unit 58 compensates for the encoding distortion included in the decoded image signal, and uses the decoded image indicated by the decoded image signal after the encoding distortion compensation as a reference image. It stores in the motion compensation prediction frame memory 59 (step ST41).
The filtering process by the loop filter unit 58 may be performed in units of maximum encoded blocks or individual encoded blocks of the decoded image signal output from the adder 56, or a decoded image signal corresponding to a macroblock for one screen. May be performed for one screen at a time after is output.

  As is apparent from the above, according to the first embodiment, when the encoding control unit 1 generates the encoded block by dividing the maximum encoded block, the division pattern and the quadtree of quadtree division are generated. Since the most efficient division pattern is selected from a plurality of division patterns that cannot be represented by division, and the coding block is generated according to the division pattern, the complicated block There is an effect that it is possible to realize a video encoding apparatus that can realize division and can express a division state with a high degree of freedom with a small amount of code.

  Further, according to the first embodiment, the variable length decoding unit 51 performs variable length decoding on the information indicating the division state of the maximum coding block from the coded data multiplexed in the bit stream, and performs maximum coding. A coding block generated by hierarchically dividing a block is specified, and compressed data, coding mode, intra prediction parameter, or inter prediction related to the coding block is coded from the coded data multiplexed in the bitstream. Since the parameters are configured to be variable-length decoded, a moving picture decoding apparatus capable of reproducing video signals by decoding information expressing a divided state having a high degree of freedom with a small amount of code from a bitstream is obtained. Has the effect.

In the first embodiment, the reference coding block is “the largest coding block spatially located at the same position as the current largest coding block in the temporally nearest reference image”. It may be configured such that the “maximum encoded block at a position shifted by the amount of motion representing the current maximum encoded block on the reference image closest in time” becomes the reference encoded block.
As the motion amount representing the current maximum coding block, for example, a motion amount obtained by adjusting the motion vector prediction value obtained for the current maximum coding block region to the grid position of the maximum coding block can be used. .
Also, since the motion vector prediction value can be calculated from the motion vectors of the peripheral coding blocks that have already been encoded, there is no need to separately multiplex and transmit them.
Thus, by referring to the division state in consideration of the motion assumed in the region of the current maximum coding block, a region having a higher correlation regarding the division state can be used as the reference coding block.

  In addition, the reference encoding block is set to “the maximum encoding block that is spatially the same position as the current maximum encoding block in the temporally nearest reference image” or “to the region of the current maximum encoding block. The motion vector prediction value obtained in this way is set to the maximum encoded block on the reference image at a position shifted by the amount of motion adjusted to the grid position of the maximum encoded block ”at the level of slice, picture, GOP, sequence, etc. You may make it select.

In the first embodiment, the reference encoding block is determined for each maximum encoding block. For example, a reference division state that becomes a template is determined at a higher level than a slice, a picture, a GOP, a sequence, or the like. The division state of each maximum coding block may be configured to perform encoding by transmitting difference information from the reference division state.
For example, when a characteristic subject / motion is fixed for each video scene, there is a possibility that the prediction / coding division state is made uniform in a relatively wide range in time and space. is there.
In such a case, instead of determining the reference coding block locally, it is possible to reduce the division initial state instruction information to be transmitted, and to determine the coding efficiency, by defining the global reference coding block region. Can be increased.
For example, the reference division state serving as the template may be configured to use the division state in the first picture of the scene change.

In the first embodiment, the configuration in which each maximum coding block is subjected to hierarchical division using the division pattern of FIG. 7 without depending on the division hierarchy has been described. However, the division pattern other than the quadtree is used. In the case where sufficient optimality of division can be obtained by quadtree division, an extra pattern expression code is required. Therefore, the division pattern of FIG. 7 is used, or the conventional example shown in FIG. Alternatively, it may be configured to adaptively select whether to use only quadtree partitioning.
For example, deeper hierarchy n of division, the size of the encoded block B ⁿ itself becomes smaller, to come no longer localized variation in signal characteristics of the coded block B ^n, the SP2~SP8 It is conceivable that the effect of using such a special division pattern is diminished.
In such a case, it is possible to select whether to use the pattern of SP2 to SP8 according to the size of the division hierarchy n or the coding block ^Bn itself, thereby making it possible to select extra division state description information. It is possible to obtain an effect that it is not necessary to encode.
Whether or not to describe such a selective division pattern may be configured such that identification information that is turned ON / OFF at the level of sequence, GOP, picture, slice, or the like is multiplexed into a bitstream.

Further, in the first embodiment, as shown in FIG. 7, the case of L ⁿ = M ⁿ is shown, but for example, as shown in FIG. 15, the maximum coding block size is set to L ⁿ = kM ⁿ . It can also be applied to.
At this time, in the 0th layer, it is assumed that only the division into the encoded blocks in which (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ , M ⁿ ) is performed, and the subsequent division is performed in the same manner as in FIG. Configure as follows.
With such a configuration, for example, by setting M ⁰ = 16, MPEG-2 (ISO / IEC 13818-2) and MPEG-4 AVC / H. H.264 (ISO / IEC 14496-10) can be defined as a maximum coding block having a configuration in which macroblocks composed of 16 × 16 pixels are horizontally connected, and a moving picture code maintaining compatibility with existing systems There is an effect that it is easy to configure the control device.
Here, the case of L ⁿ = kM ⁿ is shown, but it is needless to say that even if they are vertically connected as in kL ⁿ = M ⁿ , division is possible with the same idea.

In the present invention, any constituent element of the embodiment can be modified or any constituent element of the embodiment can be omitted within the scope of the invention.
For example, in the first embodiment, the coding control unit 1 uses a coding block that is a processing unit when intra prediction processing (intraframe prediction processing) or motion compensation prediction processing (interframe prediction processing) is performed. While determining the maximum size (initial block size) and determining the maximum number of layers when the maximum size encoded block is hierarchically divided, the initial block size and maximum size The upper limit number of layers when the encoded block is divided hierarchically is not determined by the encoding control unit 1, but the moving image encoding device is configured to operate based on information provided from outside. May be.
Similarly, when the variable length decoding unit 51 performs intra prediction processing (intraframe prediction processing) or motion compensation prediction processing (interframe prediction processing) in the same procedure as the coding control unit 1 in FIG. While determining the maximum size of the coding block as a processing unit and determining the upper limit number of layers when the maximum size coding block (maximum coding block) is hierarchically divided, The upper limit number of layers when the size or maximum size coding block is hierarchically divided is not determined by the variable length decoding unit 51, but is determined by another means with the corresponding moving image coding device. In addition, the video decoding device may be configured to operate based on uniquely determined information.

  In addition, when all the pictures of the input video signal are intra-coded so that random access to any picture is possible, the moving picture coding apparatus / moving picture decoding uses only the intra division state description information. The video encoding device and the video decoding device may be configured to handle a configuration of a bitstream in which information identifying that the device is configured or only intra-partition state description information is used is multiplexed.

  DESCRIPTION OF SYMBOLS 1 Coding control part (coding control means), 2 block division part (block division means), 3 changeover switch (prediction image generation means), 4 intra prediction part (prediction image generation means), 5 motion compensation prediction part (prediction) Image generation means), 6 subtraction section (difference image generation means), 7 transform / quantization section (image compression means), 8 inverse quantization / inverse transform section, 9 addition section, 10 intra prediction memory, 11 loop filter section , 12 motion compensated prediction frame memory, 13 variable length encoding unit (variable length encoding unit), 51 variable length decoding unit (variable length decoding unit), 52 changeover switch (predicted image generation unit), 53 intra prediction unit (prediction) Image generation means), 54 motion compensation prediction section (prediction image generation means), 55 inverse quantization / inverse transformation section (difference image generation means), 56 addition section (decoded image generation means), 57 inch RAM prediction memory, 58 loop filter section, 59 motion compensation prediction frame memory.

Claims (5)

In a moving picture coding apparatus that divides an input image into predetermined initial blocks and performs coding in units of coding blocks obtained by dividing the initial block.
Coding control means for selecting a coding mode that defines a coding method for each coding block generated by being divided hierarchically from among one or more available coding modes; A block dividing unit that divides the initial block into sizes and generates an encoded block by dividing the initial block hierarchically, and the encoding control unit for the encoded block generated by the block dividing unit A prediction image generation unit that generates a prediction image by performing a prediction process corresponding to the encoding mode determined by the above, a coding block generated by the block division unit, and a prediction generated by the prediction image generation unit A difference image generating means for generating a difference image with the image, and the difference image generated by the difference image generating means is compressed, Image compression means for outputting data, the compressed data output from the image compression means and the encoding mode selected by the encoding control means are variable length encoded and information indicating the division state of the initial block Variable length encoding means, and a variable length encoding means for generating a bit stream in which encoded data of information indicating the compressed data, the encoding mode, and the division state of the initial block is multiplexed, and
When the encoding control unit generates an encoded block by dividing the initial block, a plurality of divided patterns in which a divided pattern of quadtree division and a divided pattern that cannot be expressed by the quadtree division are mixed are included. A moving picture encoding apparatus, wherein a division pattern is selected based on a predetermined determination criterion, and an encoded block is generated according to the division pattern.   The moving image according to claim 1, wherein the division pattern that cannot be expressed by quadtree division is a pattern obtained by arranging encoded blocks having the same size as the encoded block size obtained by quadtree division. Image encoding device.   Variable length decoding is performed on the information indicating the division state of the initial block from the encoded data multiplexed in the bitstream, the encoded block generated by dividing the initial block hierarchically is identified, and the bit For variable length decoding means for variable length decoding compressed data and coding mode related to the encoded block from the encoded data multiplexed in the stream, and for the encoded block specified by the variable length decoding means, A prediction image generation unit that generates a prediction image by performing a prediction process corresponding to the encoding mode, and a differential image before compression from compressed data related to the encoded block that has been variable-length decoded by the variable-length decoding unit. Difference image generation means to be generated, difference image generated by the difference image generation means, and prediction image generated by the prediction image generation means Video decoding apparatus and a decoded image generating means for generating a decoded image by adding and. Each of the encoding control means is generated by being divided hierarchically from one or more available encoding modes for each encoded block obtained by hierarchically dividing an initial block of a predetermined size. An encoding control processing step for selecting an encoding mode for determining an encoding method of the encoding block, and a block dividing means for dividing the input image into initial blocks of a predetermined size and hierarchically dividing the initial block A block division processing step for generating an encoded block, and a prediction image generating unit sets the encoding mode determined in the encoding control processing step to the encoding block generated in the block division processing step. A prediction image generation processing step for generating a prediction image by executing a corresponding prediction processing, and a difference image generation means include the block division processing step. A difference image generation processing step for generating a difference image between the encoded block generated in the step and a prediction image generated in the prediction image generation processing step, and an image compression means generated in the difference image generation processing step. The image compression processing step for compressing the difference image and outputting the compressed data of the difference image, and the variable-length encoding means are selected in the compressed data output in the image compression processing step and the encoding control processing step. The encoding mode is variable-length encoded, the information indicating the division state of the initial block is variable-length encoded, and the compressed data, the encoding mode, and the encoded data of the information indicating the division state of the initial block are multiplexed. A variable-length encoding processing step for generating an encoded bitstream,
In the encoding control processing step, when the encoded block is generated by dividing the initial block, a plurality of division patterns in which a division pattern of quadtree division and a division pattern that cannot be expressed by the quadtree division are mixed A moving picture coding method comprising: selecting a division pattern based on a predetermined determination criterion and generating a coding block according to the division pattern.   A variable-length decoding unit performs variable-length decoding on the information indicating the division state of the initial block from the encoded data multiplexed in the bitstream, and the encoded block is generated by hierarchically dividing the initial block. A variable-length decoding processing step for variable-length decoding compressed data and a coding mode related to the encoded block from the encoded data multiplexed in the bitstream, and a predicted image generating means comprising the variable length A prediction image generation processing step for generating a prediction image by performing prediction processing corresponding to the encoding mode on the coding block specified in the decoding processing step, and the difference image generation means include the variable length decoding A difference image generation processing step for generating a difference image before compression from the compressed data relating to the encoded block subjected to variable length decoding in the processing step; The decoded image generation means includes a decoded image generation processing step for generating a decoded image by adding the difference image generated in the difference image generation processing step and the prediction image generated in the prediction image generation processing step. Video decoding method.

Images