JP6362370B2 - Image encoding device, image decoding device, image encoding method, and image decoding method - Google Patents

Description

  The present invention relates to an image encoding device and an image encoding method for encoding an image with high efficiency, and an image decoding device and an image decoding method for decoding an image encoded with high efficiency.

For example, in the image encoding device described in Non-Patent Document 1 below, an input color image is divided into maximum encoding blocks of a predetermined size, and the maximum encoding block is further divided into finer encoding blocks. Divide into hierarchies.
In addition, the encoded block is divided into finer prediction blocks (blocks of prediction processing units), and prediction images (prediction errors) are generated by performing intra prediction and motion compensation prediction on the prediction blocks.
Further, the prediction image is hierarchically divided into transform blocks in the coding block, and the transform coefficient of each transform block is entropy coded to achieve a high compression rate.

In Non-Patent Document 1, intra prediction is used as prediction using encoded pixels in a screen.
In this intra prediction, a pixel adjacent to a prediction block is used as a reference pixel, directional prediction is copied in a certain direction, and an average value (DC) in which a prediction value of a pixel in the prediction block is an average value of reference pixels. There are prediction and planar prediction in which a prediction block is interpolated from a reference pixel.
Since image signals generally have high spatial correlation, high prediction efficiency can be realized by appropriately switching prediction of various variations using adjacent pixels.

B. Bross, W.-J. Han, J.-R. Ohm, GJ Sullivan, Y.-K. Wang and T. Wiegand, “High Efficiency Video Coding (HEVC) text specification draft 10 (for FDIS & Consent)”, doc. JCTVC-L1003, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11, 12th Meeting, 2013

  Since the conventional image coding apparatus is configured as described above, if the intra prediction for the prediction block can be appropriately selected, the prediction accuracy can be improved and high coding efficiency can be realized. However, depending on the pattern in the prediction block, for example, when the pattern in the prediction block is a specific non-linear pattern or character, it cannot be predicted accurately even if any intra prediction prepared in advance is selected. There is a case. In such a case, there is a problem that prediction accuracy is lowered and high encoding efficiency cannot be realized.

The present invention has been made to solve the above-described problems. Even when the pattern in the prediction block is a specific non-linear pattern or character, the prediction accuracy is improved and high encoding efficiency is realized. An object of the present invention is to obtain an image encoding device and an image encoding method that can be used.
Another object of the present invention is to provide an image decoding apparatus and an image decoding method that can correctly decode an encoded bitstream generated by the image encoding apparatus and the image encoding method described above.

The image encoding device according to the present invention performs intra prediction processing for generating a prediction image of a luminance signal for each prediction block, which is a block of a prediction processing unit, using an intra prediction parameter of the luminance signal in the prediction block. And an intra-block copy prediction parameter for generating a block shift vector. The intra-prediction unit includes a flag for the color difference signal of the prediction block between color signals. If the non-execution of the prediction process is clearly indicated, the intra prediction process for generating a prediction image of the color difference signal is performed using the intra prediction parameter of the same color difference signal as the luminance signal, and the flag is used for the inter-color signal prediction. if explicit implementation of a process, with reference to the color difference signals encoded, in the prediction block The prediction processing between the color signal to generate a prediction image of the difference signal is obtained so as to implement.

According to this invention, if the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction process is not performed, the intra prediction unit uses the intra prediction parameter of the same color difference signal as the luminance signal, If intra prediction processing for generating a prediction image of the color difference signal is performed and the flag clearly indicates the execution of inter-color signal prediction processing, the encoded color difference signal is referred to and the color difference signal in the prediction block is referred to. Since it is configured to perform inter-color signal prediction processing that generates a prediction image, even if the pattern in the prediction block is a specific non-linear pattern or character, the prediction accuracy is improved and high coding efficiency is achieved. There is an effect that can be realized.

It is a block diagram which shows the image coding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (image coding method) of the image coding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the image decoding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the processing content (image decoding method) of the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the example by which the largest encoding block is divided | segmented into a some encoding block hierarchically. (A) shows the distribution of the encoding block and prediction block after a division | segmentation, (b) is explanatory drawing which shows the condition where encoding mode m ( ^Bn ) is allocated by hierarchy division | segmentation. Each prediction block P _i ⁿ the coded block B ⁿ is an explanatory diagram showing an example of a selectable intra prediction modes. It is explanatory drawing which shows an example of the pixel used when producing | generating the predicted value of the pixel in the prediction image production | generation block in the case of l _i ⁿ = m _i ⁿ = 4. It is explanatory drawing which shows the relative coordinate which makes the upper left pixel in a prediction image generation block the origin. It is explanatory drawing which shows an example of a quantization matrix. It is explanatory drawing which shows the structural example in the case of using a several loop filter process in the loop filter part of the image coding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the structural example in the case of using a several loop filter process in the loop filter part of the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows an example of an encoding bit stream. It is explanatory drawing which shows the index of the class classification method of a pixel adaptive offset process. It is explanatory drawing which shows the encoding order of the conversion factor in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows an example of distribution of the transformation coefficient in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows the switching area | region of the filter in the filter process at the time of average value prediction. It is explanatory drawing which shows the reference pixel arrangement | positioning of the filter process at the time of average value prediction. It is explanatory drawing which shows an example of the encoding order of the transform coefficient in the orthogonal transformation of the size of 4x4 pixel and 8x8 pixel. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color-difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the example of a response | compatibility of the intra prediction parameter of a color difference signal, and a color difference intra prediction mode. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between YUV4: 4: 4 format and YUV4: 2: 2 format. It is explanatory drawing which shows the example of the directionality prediction in YUV4: 2: 2 format equivalent to using the same directionality prediction with a luminance signal and a color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the prediction direction vector of directionality prediction with the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between directionality prediction and an angle. It is explanatory drawing which shows the relationship between the intra prediction mode index of a luminance signal, and the intra prediction mode index of a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between an intra prediction mode index and tan (theta). It is explanatory drawing which shows an example of an intra block copy. It is explanatory drawing which shows the effect at the time of embedding a specific pixel value in the pixel of an unencoded area | region in intra block copy. It is explanatory drawing which shows an example of the pixel pad process in an intra block copy. It is explanatory drawing of the Move-To-Front method used for the pixel pad process in an intra block copy. It is explanatory drawing which shows an example of the reference pixel of the filter process of the horizontal direction or the vertical direction implemented as a pixel filling process in an intra block copy. It is explanatory drawing which shows an example of the reference block in intra block copy. It is explanatory drawing of the L-shaped filter process implemented as a pixel pad process in an intra block copy. It is explanatory drawing which shows an example at the time of expanding a reference pixel with respect to the horizontal direction filter process implemented as a pixel pad process in an intra block copy. It is explanatory drawing which shows an example at the time of extending a reference pixel with respect to the vertical direction filter process implemented as a pixel pad process in an intra block copy. It is explanatory drawing at the time of extending a reference pixel with respect to the L-shaped filter process implemented as a pixel pad process in an intra block copy. In an intra block copy prediction process, it is explanatory drawing which shows the example in which the reference block which a block shift vector points out includes the uncoded area | region in a prediction object encoding block. It is a flowchart which shows the example of the process sequence of the process which calculates | requires the pixel value which fills the pixel of an unencoded area | region in an intra block copy prediction process. ₄₄ is a flowchart illustrating an example of a processing procedure of “L _max determination and S _max update” processing in the flowchart of FIG. 43. It is explanatory drawing which shows the example of the prediction between color signals in the signal whose input signal format is YUV4: 4: 4 format. It is explanatory drawing which shows the example of the pixel position referred when calculating the correlation parameter of the prediction between color signals in case a prediction image production | generation target block is 8x8 pixel. It is explanatory drawing which shows the example of the pixel position referred when calculating the correlation parameter of the prediction between color signals in case an encoding block is 8x8 pixel and a prediction image generation target block is a prediction block. It is explanatory drawing which shows the example of the intra prediction parameter of the color difference signal including the prediction between color signals.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image coding apparatus according to Embodiment 1 of the present invention.
The video signal to be processed by the image coding apparatus according to the first embodiment is a color video in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to signals, the video frame is an arbitrary video signal such as a monochrome image signal or an infrared image signal, in which a video frame is composed of a horizontal / vertical two-dimensional digital sample (pixel) sequence.
The gradation of each pixel may be 8 bits, or a gradation of 10 bits, 12 bits, or the like.
Further, it is natural that the input signal may be a still image signal instead of a video signal because the still image signal can be interpreted as a video signal composed of only one frame.

In the following description, for convenience, unless otherwise specified, the input video signal is YUV4: 2 in which the two color difference components U and V are subsampled in half both vertically and horizontally with respect to the luminance component Y: 0 format, YUV4: 2: 2 format in which two color difference components U and V are subsampled in half in the horizontal direction with respect to luminance component Y, or two color difference components U and V are luminance component Y And YUV 4: 4: 4 format signal having the same number of samples. In addition, regarding RGB 4: 4: 4 format color signals composed of signals of the three primary colors of red (R), green (G), and blue (B), each color signal is regarded as a color signal of YUV 4: 4: 4 format. The same encoding as in the YUV 4: 4: 4 format is performed. However, associating each color signal (RGB) of RGB 4: 4: 4 format with each color signal (YUV) of YUV 4: 4: 4 format is not limited (can be arbitrarily set).
This association may be encoded as an index information with a high-order header so that the image decoding apparatus can recognize it. In this way, the color of the decoded image decoded by the image decoding device can be recognized only from the decoded encoded data, and can be displayed correctly. In the case of a YUV 4: 4: 4 format signal or an RGB 4: 4: 4 format signal, each color signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV4: 0: 0) to generate a bitstream. May be.
In this way, encoding processing can be performed in parallel with each signal. At this time, information indicating which color signal each monochrome signal is may be encoded as index information in the upper header so that the image decoding apparatus can recognize it. By doing in this way, the color of the decoded image decoded by the image decoding apparatus can be correctly displayed.
In the above description, a case has been described in which each color signal of the 4: 4: 4 format signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV4: 0: 0). However, the monochrome image is actually targeted for the monochrome image signal. It is of course possible to encode.
In the above description, the YUV signal format and the RGB signal format have been described. However, other color signal formats (YCbCr, XYZ, etc.) are similarly 4: 2: 0, 4: 2: 2, 4: 4. : Any of the four formats can be encoded in the same manner as the YUV format. However, it is not limited (which can be arbitrarily set) which color signal in the target format corresponds to which color signal in the YUV format. Of course, as described above in the case of the RGB format, the color signal association may be encoded as index information in the upper header so that the image decoding apparatus can recognize it.
The processing data unit corresponding to each frame of the video is referred to as “picture”. In the first embodiment, “picture” is described as a signal of a video frame that has been sequentially scanned (progressive scan). However, when the video signal is an interlace signal, the “picture” may be a field image signal which is a unit constituting a video frame.

In FIG. 1, the encoding control unit 1 determines the maximum size of an encoding block that is a processing unit when the encoding process is performed, and the upper limit when the encoding block of the maximum size is hierarchically divided. By determining the number of layers, the process of determining the size of each encoded block is performed.
The encoding control unit 1 also has one or more selectable encoding modes (one or more intra encoding modes having different prediction block sizes indicating prediction processing units, etc., and different prediction block sizes indicating prediction processing units). Processing for selecting a coding mode to be applied to a coding block output from the block division unit 3 from one or more intra block copy coding modes and one or more inter coding modes having different prediction block sizes. To implement.
As an example of the selection method, there is a method of selecting a coding mode having the highest coding efficiency for the coding block output from the block dividing unit 3 from one or more selectable coding modes.

In addition, when the coding mode with the highest coding efficiency is the intra coding mode, the coding control unit 1 sets the intra prediction parameters used when performing the intra prediction process on the coding block in the intra coding mode. It is determined for each prediction block that is a prediction processing unit indicated by the intra coding mode, and when the coding mode having the highest coding efficiency is the intra block copy coding mode, the intra block copy coding mode is used for the coding block. An intra block copy prediction parameter used when performing the intra block copy prediction process is determined for each prediction block which is a prediction processing unit indicated by the intra block copy coding mode, and the coding mode having the highest coding efficiency is an inter code. The inter code The inter prediction parameters used in practicing the motion compensation prediction process on the coded blocks to implement the process of determining for each prediction block is a prediction processing unit indicated by the inter coding mode in the mode.
Note that, for the color difference signal, the encoding control unit 1 refers to the encoded color signal with respect to the prediction block, rather than the intra prediction process, the intra block copy prediction process, and the motion compensation prediction process. In the case where the encoding efficiency is higher when the inter-color-signal prediction process for performing the inter-color-signal prediction process for generating the predicted image of the color difference signal is applied to the encoded block output from the block dividing unit 3 Regardless of the encoding mode (intra encoding mode, intra block copy encoding mode, inter encoding mode) to be performed, the inter-color signal between the encoding blocks to be described later is performed so that inter-color signal prediction is performed as a prediction process. Set the prediction flag to ON.
Further, the inter-color-signal prediction mode may be defined as an encoding mode for executing the inter-color-signal prediction process, with the selection unit for determining whether or not the inter-color-signal prediction process is performed as an encoding block unit. By doing in this way, the code amount of the information regarding the selection of the prediction between color signals can be reduced rather than the case where the prediction signal between color signals is set per prediction block.

  Furthermore, the encoding control unit 1 performs a process of determining a prediction difference encoding parameter to be given to the transform / quantization unit 9 and the inverse quantization / inverse transform unit 10. The prediction difference coding parameter includes transform block partition information indicating transform block partition information that is a unit of orthogonal transform processing in the coded block, and a quantization parameter that specifies a quantization step size when transform coefficients are quantized. Etc. are included.

Here, FIG. 20 is an explanatory diagram showing a conversion block size when performing compression processing (conversion processing and quantization processing) of a luminance signal and a color difference signal in a YUV 4: 2: 0 format signal.
As shown in FIG. 20, the transform block size is determined by hierarchically dividing the encoded block into a quadtree.
For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining, it is possible to determine the optimal division shape of the transform block from the viewpoint of the trade-off between the code amount and the coding error.

For example, as shown in FIG. 20, the luminance signal is configured so that the coding block is hierarchically divided into one or a plurality of square transform blocks.
For the color difference signal, as shown in FIG. 20, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks in the same manner as the luminance signal. To be configured.
In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

  When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. Further, since the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction, the divided block is further divided into two parts in the vertical direction, so that the YUV 4: 2: 0 signal The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).

Further, when the input signal format is YUV 4: 4: 4 signal, as shown in FIG. 22, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block to be the same size conversion block. Configure as follows.
The division information of the conversion block of the luminance signal is output to the variable length encoding unit 15 as a conversion block division flag indicating whether or not to divide for each layer, for example.

  When a video signal is input as an input image, the slice division unit 2 performs a process of dividing the input image into one or more partial images called “slices” according to the slice division information determined by the encoding control unit 1. The slice division unit can be finely divided to the above-described coding block unit.

Each time the block dividing unit 3 inputs the slice divided by the slice dividing unit 2, the slice dividing unit 3 divides the slice into maximum coding blocks that are coding blocks of the maximum size determined by the coding control unit 1, and Until the maximum number of hierarchies determined by the coding control unit 1 is reached, a process of dividing the maximum coding block hierarchically into each coding block is performed.
That is, the block division unit 3 divides the slice into each coding block according to the division determined by the coding control unit 1, and performs a process of outputting the coding block. Each coding block is divided into one or a plurality of prediction blocks which are prediction processing units.

  If the coding mode determined by the coding control unit 1 is the intra coding mode, the changeover switch 4 outputs the coded block output from the block dividing unit 3 to the intra prediction unit 5, and the coding control unit 1 If the coding mode determined by is the intra block copy coding mode, the coding block output from the block division unit 3 is output to the intra block copy prediction unit 6, and the code determined by the coding control unit 1 is output. If the encoding mode is the inter encoding mode, a process of outputting the encoded block output from the block dividing unit 3 to the motion compensation prediction unit 7 is performed.

  When the intra control mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra prediction unit 5 performs local decoding stored in the intra memory 12. With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter determined by the encoding control unit 1 is performed to generate an intra predicted image.

That is, for the luminance signal, the intra prediction unit 5 performs an intra prediction process (intraframe prediction process) using an intra prediction parameter of the luminance signal, and generates a prediction image of the luminance signal.
On the other hand, for the color difference signal, when the intra prediction parameter of the color difference signal indicates that the same intra prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). When it is shown), the same intra-frame prediction as the luminance signal is performed to generate a predicted image of the color difference signal.
FIG. 24 shows a case where the same directionality prediction is used for the luminance signal and the color difference signal in the YUV 4: 2: 0 format signal, and FIG. 25 shows the same luminance signal and color difference signal in the YUV 4: 2: 2 format signal. The case where directionality prediction is used is shown.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal.

When the input signal format is a YUV 4: 2: 2 signal, as shown in FIG. 26, if the luminance signal is a square block, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 27, when a YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, YUV4: 2 is used to predict the luminance signal and the color difference signal in the same direction. : On two signals, in the case of directional prediction other than the vertical prediction and the horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal.
Specifically, as illustrated in FIG. 28, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 29, assuming that the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an intra prediction mode index used for prediction of the color difference signal, and the color difference signal prediction process is performed in the intra prediction mode corresponding to the converted index. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise.
With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  The intra prediction unit 5 always performs an intra prediction process for the generation of a prediction image of a luminance signal, but for the generation of a prediction image of a color difference signal, the encoding control unit 1 performs the prediction block on each prediction block. When the flag indicating whether or not to perform inter-color signal prediction (inter-color signal prediction flag) is set to valid (ON), the encoded color signal is referred to without performing intra prediction processing. Then, the inter-color signal prediction process for generating the predicted image of the color difference signal in the prediction block is performed. The intra prediction unit 5 constitutes an intra prediction unit.

When the intra block copy encoding mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra block copy predicting unit 6 predicts the encoded block. The pixel value of each pixel in an area that has not been subjected to local decoding is assumed in a predetermined method among the maximum size encoded block (maximum encoded block) to which the block (prediction processing unit block) belongs. Perform the process.
In addition, the intra block copy prediction unit 6 approximates the prediction block for each prediction block in the encoded block most closely to the prediction block from the regions that have already been encoded and locally decoded in the same picture. A process is performed for searching for a reference block that is an existing block and determining the reference block as a prediction image of the prediction block.
The intra block copy prediction unit 6 always performs the intra block copy prediction process for the generation of the prediction image of the luminance signal, but the encoding control unit 1 for each prediction block for the generation of the prediction image of the color difference signal. When the inter-color-signal prediction flag of the prediction block is set to ON by referring to the encoded color signal without performing the intra-block copy prediction process, the prediction image of the color difference signal in the prediction block is obtained. The prediction process between the color signals to be generated is performed. The intra block copy prediction unit 6 constitutes an intra block copy prediction unit.

When the inter coding mode is selected by the coding control unit 1 as the coding mode corresponding to the coding block output from the changeover switch 4, the motion compensated prediction unit 7 and the motion compensated prediction frame memory 14 A motion vector is searched by comparing locally decoded images of one frame or more stored in the image, and the motion vector and an inter prediction parameter such as a frame number to be determined determined by the encoding control unit 1 are used to encode the code. A process for generating an inter-predicted image by performing an inter prediction process (motion-compensated prediction process) on a block is performed.
The motion compensation prediction unit 7 always performs motion compensation prediction processing for the generation of a prediction image of a luminance signal, but the generation of a prediction image of a color difference signal is performed by the encoding control unit 1 for each prediction block. When the inter-color signal prediction flag of the prediction block is set to ON, the color that generates the prediction image of the color difference signal in the prediction block with reference to the encoded color signal without performing the motion compensation prediction process Perform inter-signal prediction processing. The motion compensation prediction unit 7 constitutes a motion compensation prediction unit.

In the first embodiment, for each prediction block, when the inter-color signal prediction flag of the prediction block is set to ON by the encoding control unit 1, the intra prediction unit 5, the intra block copy prediction unit 6 or Although it is assumed that any one of the motion compensation prediction units 7 performs the inter-color signal prediction process, a prediction unit that performs the inter-color signal prediction process may be provided separately.
Whether the intra prediction unit 5, the intra block copy prediction unit 6 or the motion compensation prediction unit 7 performs the inter-color signal prediction process is determined by, for example, using the inter-color signal prediction parameter output from the encoding control unit 1. Included inter-color signal prediction flag (ON / OFF information for each prediction block indicating whether or not to perform inter-color signal prediction processing. If ON, the execution of inter-color signal prediction processing is clearly indicated, and OFF. If so, the non-implementation of the inter-color signal prediction process is clearly indicated).

Therefore, the intra prediction unit 5 performs the inter-color signal prediction process if the inter-color signal prediction flag included in the inter-color signal prediction parameter is ON, and performs the intra prediction process if the flag is OFF. .
The intra block copy prediction unit 6 performs inter-color signal prediction processing if the inter-color signal prediction flag included in the inter-color signal prediction parameter is ON, and intra block copy prediction if the flag is OFF. Perform the process.
The motion compensation prediction unit 7 performs the inter-color signal prediction process if the inter-color signal prediction flag included in the inter-color signal prediction parameter is ON, and performs the motion compensation prediction process if the flag is OFF. carry out.
The inter-color signal prediction parameters may be included in these prediction parameters when the encoding control unit 1 outputs the intra prediction parameters, the intra block copy prediction parameters, or the inter prediction parameters, or separately from these prediction parameters. May be output.

The variable length encoding unit 15 described later encodes the inter-color signal prediction parameter output from the encoding control unit 1 and performs variable length encoding to generate encoded data.
Also, the intra prediction parameter, intra block copy prediction parameter, and inter prediction parameter of the color difference signal are encoded by variable length coding only when the inter-color signal prediction flag included in the inter-color signal prediction parameter is OFF. Generate data. Alternatively, the intra prediction parameter, intra block copy prediction parameter, and inter prediction parameter of the color difference signal may be the same as the luminance signal. That is, when the inter-color signal prediction flag is OFF, the color difference signal prediction process is performed using the intra prediction parameter, the intra block copy prediction parameter, and the inter prediction parameter of the luminance signal. By doing so, it is not necessary to separately encode the parameters necessary for predicting the color difference signal, so that the code amount can be reduced. In particular, for intra block copy prediction and motion compensation prediction, since the picture pattern and motion generally have a high correlation between color signals, the code amount can be increased by sharing the prediction parameters of the color difference signal and the prediction parameter of the luminance signal. High prediction efficiency can be realized while reducing.
Here, whether or not to perform the inter-color signal prediction process is selected for each prediction block, but whether or not to perform the inter-color signal prediction process is selected for each coding block unit or for each coding block of the maximum size. It may be. By doing so, the switching unit of the inter-color signal prediction process is increased and the prediction efficiency is lowered, but the code amount required for encoding the inter-color signal prediction flag can be reduced. Or you may select per conversion block. By doing so, control can be performed in units smaller than the encoded block, so that prediction performance is improved. At this time, when there are a plurality of conversion blocks in the prediction block, the inter-color signal prediction is performed only for the conversion block in which the inter-color signal prediction flag is ON, and the color difference is calculated in the conversion block in which the inter-color signal prediction flag is OFF Intra prediction, intra block copy prediction, or motion compensation prediction is performed on the signal.
Alternatively, the block size for switching the inter-color signal prediction flag may be set independently of the maximum size encoding block, encoding block, prediction block, and transform block. In this way, an optimal block size can be set for the inter-color signal prediction process. The block size set independently may be determined by the division information expressed by the quadtree, and in this way, the block size can be changed according to the local property of the image. The block size information may be encoded and decoded by an image decoding device, or a size common to the image encoding device and the image decoding device may be set as a fixed size in advance.
Further, the inter-color signal prediction flag may be common to the two color difference signals (U and V signals), or the U signal and the V signal may have flags independently. If the U signal and the V signal have the same highly correlated color signal (luminance signal or color difference signal), they can be used in common to realize high-precision prediction while reducing the code amount of the inter-color signal prediction flag. it can. On the other hand, when color signals (luminance signal or color difference signal) with high correlation are different between the U signal and the V signal, the U signal and the V signal have flags independently so that the color signals are more adaptively provided. The prediction process can be switched, and the prediction accuracy can be improved.

The subtracting unit 8 is an intra prediction image generated by the intra prediction unit 5, an intra block copy prediction image generated by the intra block copy prediction unit 6, or motion compensated prediction from the encoded block output from the block dividing unit 3. A process of subtracting the inter prediction image generated by the unit 7 and outputting a prediction difference signal indicating the difference image as a subtraction result to the transform / quantization unit 9 is performed.
The transform / quantization unit 9 refers to the transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 and performs orthogonal transform processing on the prediction difference signal output from the subtraction unit 8 (for example, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), orthogonal transform processing such as KL transform, which is previously designed for a specific learning sequence, is performed for each transform block, and transform coefficients are calculated. Then, referring to the quantization parameter included in the prediction differential encoding parameter, the transform coefficient of the transform block unit is quantized, and the compressed data that is the transform coefficient after quantization is dequantized / inverse transform unit 10 and variable A process of outputting to the long encoding unit 15 is performed.

When the transform / quantization unit 9 quantizes the transform coefficient, the transform / quantization unit 9 performs a transform coefficient quantization process using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient. You may do it.
Here, FIG. 10 is an explanatory diagram illustrating an example of a 4 × 4 DCT quantization matrix.
The numbers in the figure indicate the scaling value of the quantization step size of each transform coefficient.
For example, in order to suppress the encoding bit rate, as shown in FIG. 10, by scaling the quantization step size to a larger value for the higher frequency transform coefficient, the higher frequency generated in a complex image region or the like. Encoding can be performed without dropping information on low-frequency coefficients that greatly affect subjective quality while suppressing the amount of codes by suppressing transform coefficients.
Thus, when it is desired to control the quantization step size for each transform coefficient, a quantization matrix may be used.

In addition, the quantization matrix can use an independent matrix for each color signal and coding mode (intra coding or inter coding) with each orthogonal transform size. It is possible to select whether to use a quantization matrix that is commonly prepared in advance by the decoding apparatus or an already encoded quantization matrix or to use a new quantization matrix.
Accordingly, the transform / quantization unit 9 sets a flag indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, as a quantization matrix parameter to be encoded.

Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 10 is set as a quantization matrix parameter to be encoded.
On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance in the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Therefore, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, it is possible to select only the quantization matrix that is prepared in advance by the image encoding device and the image decoding device.

The inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1, and transforms and transforms the transform block 9 in units of transform blocks. A local decoded prediction difference corresponding to the prediction difference signal output from the subtracting unit 8 is obtained by dequantizing the output compressed data and performing an inverse orthogonal transform process on the transform coefficient that is the compressed data after the inverse quantization. A process of calculating a signal is performed. In addition, when the transform / quantization unit 9 uses the quantization matrix to perform the quantization process, the corresponding inverse quantization can be performed by referring to the quantization matrix even during the inverse quantization process. Perform the process.
The addition unit 11 includes a local decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 10, an intra prediction image generated by the intra prediction unit 5, and an intra block copy prediction image generated by the intra block copy prediction unit 6. Alternatively, the inter prediction image generated by the motion compensation prediction unit 7 is added to perform a process of calculating a local decoded image corresponding to the encoded block output from the block dividing unit 3.

The intra memory 12 is a recording medium that stores the locally decoded image calculated by the adding unit 11.
The loop filter unit 13 performs a predetermined filtering process on the local decoded image calculated by the adding unit 11 and outputs a local decoded image after the filter process.
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

However, the loop filter unit 13 determines whether or not to perform each of the deblocking filter process, the pixel adaptive offset process, and the adaptive filter process, and performs variable length coding using the valid flag of each process as header information. To the unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 11 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
Generally, the more types of filter processing that are used, the better the image quality, but the higher the processing load. That is, image quality and processing load are in a trade-off relationship. In addition, the image quality improvement effect of each filter process varies depending on the characteristics of the image to be filtered. Therefore, the filter processing to be used may be determined according to the processing load allowed by the image encoding device and the characteristics of the encoding target image. For example, when it is desired to reduce the processing load as compared with the configuration of FIG. 11, it may be configured only by the deblocking filter processing and the pixel adaptive offset processing.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are used. One classification method is selected from the methods.
Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating the coding distortion is calculated for each class.
Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.
Therefore, in the pixel adaptive offset processing, block division information, an index indicating a class classification method for each block, and offset information for specifying an offset value of each class in block units are output to the variable length coding unit 15 as header information.
In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

In adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for the distortion that is superimposed is designed for each region (local decoded image) belonging to each class. Filter the local decoded image.
Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.
As the class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of each block.
Further, the number of classes used in the adaptive filter processing may be set in advance as a value common to the image encoding device and the image decoding device, or may be a parameter to be encoded.
Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

  When performing the pixel adaptive offset process and the adaptive filter process, as shown in FIG. 11, the video signal needs to be referred to by the loop filter unit 13, so that the video signal is input to the loop filter unit 13. In addition, it is necessary to change the image encoding apparatus of FIG.

The motion compensated prediction frame memory 14 is a recording medium that stores a locally decoded image after the filter processing of the loop filter unit 13.
The variable length encoding unit 15 and the compressed data output from the transform / quantization unit 9 and the output signal of the encoding control unit 1 (block division information in the maximum encoding block, encoding mode, prediction difference encoding parameter, Intra-prediction parameters, intra-block copy prediction parameters, inter-prediction parameters, inter-color signal prediction parameters including inter-color signal prediction flags) and motion vectors output from the motion compensation prediction unit 7 (the encoding mode is the inter encoding mode) And a process of generating encoded data by variable length encoding.
Further, as illustrated in FIG. 13, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data. carry out. Note that the variable length coding unit 15 constitutes coding means.

However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.
The sequence level header includes the image size, the color signal format, the bit depth of the signal value of the luminance signal and the color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. In general, header information that is common to each sequence unit, such as a valid flag of () and a valid flag of a quantization matrix, is collected.
The picture level header is set on a picture-by-picture basis, such as the index of the sequence level header to be referenced, the number of reference pictures at the time of motion compensation, the entropy coding probability table initialization flag, and the quantization matrix parameter including the valid flag for the target picture. This is a summary of header information.

  The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a summary of parameters in units of slices such as flags indicating whether or not to perform each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the filter unit 13.

Here, an example is shown in which the variable length coding unit 15 performs variable length coding on the inter-color signal prediction parameter including the inter-color signal prediction flag, but at least of the sequence level header, the picture level header, and the slice level header. One header may be encoded including a flag indicating whether or not to perform inter-color signal prediction processing (higher header inter-color signal prediction valid flag). In this way, it can be effective only in a region where prediction can be performed with high accuracy by inter-color signal prediction processing (region where correlation between color signals is high) in units of sequences, pictures, or slices. On the other hand, by invalidating the region where the prediction accuracy of the inter-color signal prediction process is low, it is not necessary to encode the encoding parameter related to the inter-color signal prediction process, and the code amount can be reduced. The upper header color signal inter-prediction valid flag may be prepared for each of intra prediction, intra block copy prediction, and motion compensation prediction, or may be common.
For example, by enabling the inter-color signal prediction processing only when the intra prediction unit 5 performs the intra prediction processing of the luminance signal, the other prediction processing (intra block copy prediction processing, motion compensation prediction processing) is performed on the luminance signal. When performing, it is not necessary to encode the encoding parameter regarding the inter-color signal prediction process, the amount of code can be reduced, and the configurations of the intra block copy prediction unit 6 and the motion compensation prediction unit 7 are simplified. be able to. Alternatively, the high-order header color signal prediction valid flag is used as information common to intra prediction, intra block copy prediction, and motion compensation prediction. It is possible to avoid an increase in the amount of code due to the encoding of the inter-color signal prediction parameter for each block in a region where the prediction efficiency is low.

In addition, when the above-described upper header color signal prediction valid flag is encoded with two or more headers among a sequence level header, a picture level header, and a slice level header, the priority is a slice level header, a picture level header, a sequence Level header order. That is, when the flag is present in the slice level header and the picture level header, the flag in the slice level header has priority, and when the flag is present in the picture level header and sequence level header, the flag in the picture level header is set. have priority.
Furthermore, the upper header color signal inter-prediction valid flag may be provided in units of sub-pictures into which a picture is divided. Specifically, by providing the above-described upper header inter-color signal prediction valid flag in the sub-picture division unit of Tile or Wavefront Parallel Processing described in Non-Patent Document 1, only an area where inter-color signal prediction is effective is enabled. Therefore, it is possible to realize highly accurate prediction while suppressing an increase in code amount.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded. Supplemental information is also defined as a unique NAL unit if it exists. The picture data is defined as an access unit, and indicates a unit of data access of each picture in which NAL units related to the picture such as a NAL unit indicating slice data and a NAL unit indicating supplementary information are collected.

In the example of FIG. 1, a coding control unit 1, a slice division unit 2, a block division unit 3, a changeover switch 4, an intra prediction unit 5, an intra block copy prediction unit 6, and motion compensation prediction, which are components of the image coding apparatus. Unit 7, subtraction unit 8, transform / quantization unit 9, inverse quantization / inverse transform unit 10, addition unit 11, intra memory 12, loop filter unit 13, motion compensated prediction frame memory 14, and variable length coding unit 15 Are configured by dedicated hardware (components other than the intra memory 12 and the motion compensation prediction frame memory 14 are configured by, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like). Although the thing is assumed, the image coding apparatus may be configured by a computer.
When the image encoding device is configured by a computer, the intra memory 12 and the motion compensated prediction frame memory 14 are configured on the computer memory, and the encoding control unit 1, the slice dividing unit 2, the block dividing unit 3, and the changeover switch. 4, intra prediction unit 5, intra block copy prediction unit 6, motion compensation prediction unit 7, subtraction unit 8, transformation / quantization unit 9, inverse quantization / inverse transformation unit 10, addition unit 11, loop filter unit 13 and variable A program describing the processing contents of the long encoding unit 15 may be stored in a memory of a computer so that the CPU of the computer executes the program stored in the memory.
FIG. 2 is a flowchart showing the processing contents (image coding method) of the image coding apparatus according to Embodiment 1 of the present invention.

3 is a block diagram showing an image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 3, when the variable length decoding unit 31 receives the encoded bit stream generated by the image encoding apparatus of FIG. 1, each header information such as a sequence level header, a picture level header, a slice level header, and the like is extracted from the bit stream. A process of decoding picture data is performed.
However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data. At this time, the header information includes information indicating that each of the YUV 4: 4: 4 format signal and the RGB 4: 4: 4 format signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV 4: 0: 0). In the case where it is included, the decoding process can be performed independently on the encoded bit stream of each color signal.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the sequence level header), a picture parameter header (corresponding to the picture level header), and slice data are defined as unique NAL unit types, respectively, and identification information (index) of the NAL unit type is used. It is identified by decoding. The supplemental information is also identified as a unique NAL unit if it exists. Further, picture data is identified as an access unit in which NAL units related to the picture such as a NAL unit indicating slice data and a NAL unit indicating supplementary information are collected.

  When the valid flag of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding of the quantization matrix parameter to identify the quantization matrix. Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, a quantization matrix prepared in advance in the image encoding device and the image decoding device, or When indicating that the quantization matrix has already been decoded (not a new quantization matrix), the quantization matrix is identified with reference to the index information identifying which quantization matrix of the matrix, When the quantization matrix parameter indicates that a new quantization matrix is used, the quantization matrix is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter.

  In addition, the variable length decoding unit 31 refers to the slice level header, specifies the slice division state, decodes the slice data of each slice, specifies the encoded data of the maximum encoded block, and slice data The block division information included in the block, the encoding block that is a unit for performing decoding processing by hierarchically dividing the maximum encoding block, and specifying the compressed data, the encoding mode, Intra prediction parameters (when the encoding mode is the intra encoding mode), intra block copy prediction parameters (when the encoding mode is the intra block copy encoding mode), inter prediction parameters (the encoding mode is the inter encoding mode) ) Motion vector (if the coding mode is inter coding mode) If), carries out a process of variable length decoding the predictive parameters and predictive differential coding parameters between the color signal including a flag indicating whether or not to implement the inter-color signal prediction (between the color signal prediction flag).

Here, an example has been shown in which the variable length decoding unit 31 performs variable length decoding on the inter-color signal prediction parameters related to each coding block. However, the image coding apparatus in FIG. 1 includes a sequence level header, a picture level header, and a slice. In the case of encoding including a flag indicating whether or not to perform inter-color signal prediction processing (higher-header inter-color signal prediction valid flag) in at least one of the level headers, the variable length decoding unit 31 The corresponding intra prediction unit 34, intra block copy prediction unit 35, motion compensation prediction unit only when the upper header color signal inter prediction prediction flag is decoded and the flag is ON (when the inter color signal prediction process is performed). In 36, the inter-color signal prediction process is validated, and only the coding block indicating the coding mode in which the inter-color signal prediction process is valid is related to the coding block. Variable-length decoding prediction parameters between color signals. For a coding block indicating a coding mode in which inter-color signal prediction processing is not effective, the inter-color signal prediction parameter related to this coding block is not variable-length decoded, and the corresponding inter-color signal prediction flag is OFF (color Do not perform inter-signal prediction).
Here, when the image coding apparatus in FIG. 1 prepares the upper header color signal inter-prediction prediction valid flag for each of intra prediction, intra block copy prediction, and motion compensation prediction, the corresponding image decoding apparatuses are individually provided. If the prepared flag is configured to be decoded, and the image encoding device of FIG. 1 is prepared for intra prediction, intra block copy prediction, and motion compensation prediction, the corresponding image decoding device is the common Configure to decode the flag.
However, when the image encoding apparatus in FIG. 1 encodes each flag related to the validation of the inter-color signal prediction process with two or more headers among a sequence level header, a picture level header, and a slice level header, For the flag decoded by the variable length decoding unit 31, priority is set in the order of slice level header, picture level header, and sequence level header.
That is, if a flag exists in the slice level header and the picture level header, the flag in the slice level header takes precedence. If a flag exists in the picture level header and the sequence level header, the flag in the picture level header takes precedence. Is done.
In addition, regarding the above-described upper header color signal inter-prediction valid flag, when the image encoding apparatus of FIG. The image decoding apparatus that performs decoding is configured to decode in units of sub-pictures. In this way, it is possible to correctly decode the encoded bitstream of the image encoding apparatus that performs the validity / invalidity control of the inter-color signal prediction in units of sub-pictures.

The inverse quantization / inverse transform unit 32 refers to the quantization parameter and transform block division information included in the prediction difference encoding parameter variable length decoded by the variable length decoding unit 31, and the variable length decoding unit 31 performs variable length decoding. 1 is inversely quantized on a transform block basis, and inverse orthogonal transform processing is performed on transform coefficients that are compressed data after inverse quantization, and output from the inverse quantization / inverse transform unit 10 in FIG. A process of calculating the same decoded prediction difference signal as the local decoding prediction difference signal is performed.
Here, the division state of the transform block in the coding block is specified from the transform block partition information. For example, in the case of a YUV 4: 2: 0 format signal, the transform block size is determined by hierarchically dividing the encoded block into quadtrees as shown in FIG.

For example, as shown in FIG. 20, the luminance signal is configured so that the coding block is hierarchically divided into one or a plurality of square transform blocks.
For the color difference signal, as shown in FIG. 20, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks in the same manner as the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. In addition, the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction. The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
When the input signal format is a YUV 4: 4: 4 signal, as shown in FIG. 22, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block so as to be a conversion block of the same size. Configure.

In addition, when each header information variable-length decoded by the variable-length decoding unit 31 indicates that inverse quantization processing is to be performed using the quantization matrix in the slice, the inverse quantization is performed using the quantization matrix. Process.
Specifically, inverse quantization processing is performed using a quantization matrix specified from each header information.

  The changeover switch 33 outputs the intra-prediction parameter variable-length decoded by the variable-length decoding unit 31 to the intra-prediction unit 34 if the coding mode variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode. If the encoding mode variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode, the intra block copy prediction parameters variable length decoded by the variable length decoding unit 31 are output to the intra block copy prediction unit 35. If the coding mode variable length decoded by the variable length decoding unit 31 is the inter coding mode, the inter prediction parameter and the motion vector variable length decoded by the variable length decoding unit 31 are output to the motion compensation prediction unit 36. Perform the process.

  The intra prediction unit 34, when the coding mode related to the coding block identified from the block division information variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode, the decoding stored in the intra memory 38 With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter output from the changeover switch 33 is performed to generate an intra predicted image.

That is, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal, and generates a prediction image of the luminance signal.
On the other hand, for the color difference signal, if the inter-color signal prediction flag of the prediction block is OFF for each prediction block (indicating that the inter-color signal prediction process is not performed), intra prediction processing is performed. To do.
At this time, when the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (when the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal.

When the input signal format is a YUV 4: 2: 2 signal, as shown in FIG. 26, if the luminance signal is a square block, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 27, when a YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, YUV4: 2 is used to predict the luminance signal and the color difference signal in the same direction. : On two signals, in the case of directional prediction other than the vertical prediction and the horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal.
Specifically, as illustrated in FIG. 28, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 29, assuming that the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an intra prediction mode index used for prediction of the color difference signal, and the color difference signal prediction process is performed in the intra prediction mode corresponding to the converted index. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise.
With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  The intra prediction unit 34 always performs an intra prediction process for generating a predicted image of a luminance signal, but for generating a predicted image of a chrominance signal, for each predicted block, an inter-color signal prediction flag of the predicted block. Is ON (when the execution of the inter-color signal prediction process is specified), the intra-prediction process is not performed, the encoded color signal is referred to, and the prediction image of the color difference signal in the prediction block is obtained. The prediction process between the color signals to be generated is performed. The intra prediction unit 34 constitutes an intra prediction unit.

The intra block copy prediction unit 35, when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the intra block copy coding mode, In the maximum coding block to which the prediction block belongs, a process of assuming a pixel value of each pixel in a region not yet decoded by a predetermined method is performed.
In addition, the intra block copy prediction unit 35 selects, for each prediction block in the encoded block, a reference block that is a block that is closest to the prediction block from among regions that have already been decoded in the same picture. A process of searching and determining the reference block as a prediction image of the prediction block is performed.
The intra block copy prediction unit 35 always performs an intra block copy prediction process for the generation of a prediction image of a luminance signal, but for the generation of a prediction image of a color difference signal, the color of the prediction block is generated for each prediction block. When the inter-signal prediction flag is ON (when the execution of the inter-color signal prediction process is specified), the intra-block copy prediction process is not performed and the encoded color signal is referred to and An inter-color signal prediction process for generating a predicted image of the color difference signal is performed. The intra block copy prediction unit 35 constitutes an intra block copy prediction unit.

The motion compensation prediction unit 36 is stored in the motion compensation prediction frame memory 40 when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the inter coding mode. The inter prediction process (motion compensation prediction process) using the motion vector output from the changeover switch 33 and the inter prediction parameter is performed while referring to the decoded image, and the process of generating the inter prediction image is performed.
The motion compensation prediction unit 36 always performs motion compensation prediction processing for generating a prediction image of a luminance signal, but for generating a prediction image of a color difference signal, for each prediction block, between the color signals of the prediction block. When the prediction flag is ON (when the execution of the inter-color signal prediction process is clearly specified), the color difference signal in the prediction block is referred to by referring to the encoded color signal without performing the motion compensation prediction process. An inter-color signal prediction process for generating a predicted image is performed. The motion compensation prediction unit 36 constitutes a motion compensation prediction unit.

In the first embodiment, when the inter-color signal prediction flag included in the inter-color signal prediction parameter subjected to variable length decoding by the variable length decoding unit 31 is ON, the intra prediction unit 34 and the intra block copy prediction unit 35. Alternatively, it is assumed that one of the motion compensation prediction units 36 performs the inter-color signal prediction process, but a prediction unit that performs the inter-color signal prediction process may be separately provided.
In the first embodiment, whether or not the inter-color signal prediction process is performed is selected with reference to the inter-color signal prediction flag included in the inter-color signal prediction parameter. When the encoding device encodes the flag in units of the encoding block or the encoding block of the maximum size, the corresponding image decoding device decodes the flag in the same unit as the image encoding device in FIG. Whether or not the inter-color signal prediction process is performed is identified in the same unit as that of the image encoding device in FIG. In this way, it is possible to correctly decode the encoding parameter.
Alternatively, when the image encoding device in FIG. 1 is configured to encode the inter-color signal prediction flag in units of transform blocks, the corresponding image decoding device also decodes the inter-color signal prediction flag in units of transform blocks. Configure as follows. At this time, when there are a plurality of conversion blocks in the prediction block, the inter-color signal prediction is performed only for the conversion block in which the inter-color signal prediction flag is ON, and the color difference is calculated in the conversion block in which the inter-color signal prediction flag is OFF Intra prediction, intra block copy prediction, or motion compensation prediction is performed on the signal.
Further, when the image coding apparatus in FIG. 1 is configured to be common to two color difference signals (U and V signals), the image decoding apparatus uses two variable-length decoding units 31 to generate two inter-color signal prediction flags. The common flag is decoded with the color difference signals (U and V signals).
On the other hand, when the image coding apparatus of FIG. 1 is configured to have the inter-color signal prediction flag independently for the U signal and the V signal, in the image decoding apparatus, the variable length decoding unit 31 causes the U signal flag and the V signal Decrypt flags separately. In this way, the encoding parameter can be correctly decoded.

  The addition unit 37 includes a decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 32, an intra prediction image generated by the intra prediction unit 34, an intra block copy prediction image generated by the intra block copy prediction unit 35, Or the inter prediction image produced | generated by the motion compensation prediction part 36 is added, and the process which calculates the decoding image same as the local decoding image output from the addition part 11 of FIG. 1 is implemented.

The intra memory 38 is a recording medium that stores the decoded image calculated by the adding unit 37 as a reference image used in the intra prediction process and the intra block copy prediction process.
The loop filter unit 39 performs a predetermined filter process on the decoded image calculated by the adder unit 37 and performs a process of outputting the decoded image after the filter process.
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
However, the loop filter unit 39 performs each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing with reference to each header information variable-length decoded by the variable-length decoding unit 31 in the corresponding slice. Specify whether or not.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image encoding device is configured as shown in FIG. 11, a loop filter unit 39 is configured as shown in FIG. The Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing and pixel adaptive offset processing, the loop filter unit 39 is also configured by deblocking filter processing and pixel adaptive offset processing.

  Here, in the deblocking filter processing, with reference to the header information that has been subjected to variable length decoding by the variable length decoding unit 31, there is information for changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Based on the change information, deblocking filter processing is performed. When there is no change information, it is performed according to a predetermined method.

In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. If the index indicating the block classification method is not an index indicating that “offset processing is not performed”, each pixel in the block is classified according to the class classification method indicated by the index. To do.
Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance.
Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

  However, in the pixel adaptive offset processing of the loop filter unit 13 of the image encoding device, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit). When a class classification method is selected for each block and adaptive offset processing for each class is performed, the loop filter unit 39 also has a pixel adaptive offset in units of blocks having the same fixed size as the loop filter unit 13. Perform the process.

In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.
The motion compensation prediction frame memory 40 is a recording medium that stores the decoded image after the filter processing of the loop filter unit 39 as a reference image used in the inter prediction processing (motion compensation prediction processing).

In the example of FIG. 3, a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, an intra prediction unit 34, an intra block copy prediction unit 35, and a motion compensation prediction unit 36 that are components of the image decoding device. Each of the adder 37, the intra memory 38, the loop filter unit 39, and the motion compensated prediction frame memory 40 is configured by dedicated hardware (components other than the intra memory 38 and the motion compensated prediction frame memory 40 are, for example, It is assumed that the CPU is mounted on a semiconductor integrated circuit or a one-chip microcomputer), but the image decoding apparatus may be configured by a computer.
When the image decoding device is configured by a computer, the intra memory 38 and the motion compensated prediction frame memory 40 are configured on the computer memory, and a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, A program describing the processing contents of the intra prediction unit 34, the intra block copy prediction unit 35, the motion compensation prediction unit 36, the addition unit 37, and the loop filter unit 39 is stored in the memory of the computer, and the CPU of the computer stores the memory. The program stored in the program may be executed.
FIG. 4 is a flowchart showing the processing contents (image decoding method) of the image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
In the first embodiment, each frame image of a video is used as an input image, intra prediction from encoded neighboring pixels or motion compensation prediction between adjacent frames is performed, and an obtained prediction difference signal is obtained. An image encoding device that performs compression processing by orthogonal transform / quantization and then performs variable length encoding to generate an encoded bitstream, and an image that decodes the encoded bitstream output from the image encoding device A decoding apparatus will be described.

The image encoding apparatus in FIG. 1 performs intra-frame and inter-frame adaptive encoding by dividing a video signal into blocks of various sizes in response to local changes in the spatial and temporal directions of the video signal. It is characterized by.
In general, a video signal has a characteristic that the complexity of the signal changes locally in space and time. When viewed spatially, a small image, such as a picture with a uniform signal characteristic in a relatively wide image area such as the sky or a wall, or a picture containing a person or fine texture, on a video frame. A pattern having a complicated texture pattern in the region may be mixed.
Even when viewed temporally, the change in the pattern of the sky and the wall locally in the time direction is small, but because the outline of the moving person or object moves rigidly or non-rigidly in time, the temporal change Is big.

In the encoding process, a prediction difference signal with small signal power and entropy is generated by temporal and spatial prediction to reduce the overall code amount. However, the parameters used for the prediction are set as large as possible in the image signal region. If it can be applied uniformly, the code amount of the parameter can be reduced.
On the other hand, if the same prediction parameter is applied to a large image region with respect to an image signal pattern having a large temporal and spatial change, the number of prediction differential signals increases because prediction errors increase. .
Therefore, in a region where the temporal and spatial changes are large, the block size for performing the prediction process by applying the same prediction parameter is reduced, the amount of parameter data used for prediction is increased, and the power and entropy of the prediction difference signal are increased. It is desirable to reduce

  In the first embodiment, in order to perform coding adapted to the general characteristics of such a video signal, first, prediction processing or the like is started from a predetermined maximum block size, and the video signal region is divided hierarchically. In addition, the prediction process and the encoding process of the prediction difference are adapted for each divided area.

First, the processing contents of the image encoding device in FIG. 1 will be described.
First, the encoding control unit 1 determines the slice division state of a picture to be encoded (current picture), and the size of the maximum encoding block used for encoding the picture and the hierarchy for dividing the maximum encoding block into layers. The upper limit of the number is determined (step ST1 in FIG. 2).
As a method of determining the size of the maximum coding block, for example, the same size may be determined for all the pictures according to the resolution of the video signal of the input image, or the local motion of the video signal of the input image The size difference may be quantified as a parameter, and a small size may be determined for a picture with high motion, while a large size may be determined for a picture with little motion.

Examples of how to determine the upper limit of the number of division layers include, for example, a method of determining the same number of layers for all pictures according to the resolution of the video signal of the input image, or when the motion of the video signal of the input image is severe There is a method in which the number of hierarchies is increased so that finer movements can be detected, and when there are few movements, the number of hierarchies is set to be suppressed.
Note that the size of the maximum coding block and the upper limit of the number of hierarchies for dividing the maximum coding block into layers are coded by a sequence level header or the like. In that case, the minimum block size of the encoded block may be encoded instead of the upper limit of the number of division layers. That is, since the size of the block when the maximum encoded block is divided up to the upper limit of the number of division layers is the minimum block size of the encoded block, the size of the maximum encoded block and the size of the encoded block are determined on the image decoding device side. The upper limit of the number of divided hierarchies can be specified from the minimum block size.

Also, the encoding control unit 1 selects an encoding mode corresponding to each encoding block divided hierarchically from one or more available encoding modes (step ST2).
That is, the encoding control unit 1 divides the image area of the maximum encoding block size into encoded blocks having the encoding block size hierarchically until reaching the upper limit of the number of division layers defined above. A coding mode for each coding block is determined.
The coding modes include one or more intra coding modes (collectively referred to as “INTRA”), one or more intra block copy coding modes (collectively referred to as “ICOPY”), There are one or a plurality of inter coding modes (collectively referred to as “INTER”), and the coding control unit 1 can select from all the coding modes available in the picture or a subset thereof. The encoding mode corresponding to each encoding block is selected.

However, each coding block that is hierarchically divided by the block division unit 3 to be described later is further divided into one or a plurality of prediction blocks, which are units for performing prediction processing, and the division state of the prediction block is also coded mode. Is included as information. That is, the coding mode is an index for identifying what kind of prediction block division the intra coding mode, intra block copy coding mode, or inter coding mode is.
Since the encoding mode selection method by the encoding control unit 1 is a known technique, a detailed description thereof will be omitted. For example, an encoding process for an encoding block is performed using any available encoding mode. There is a method in which coding efficiency is verified by performing and a coding mode having the best coding efficiency is selected from among a plurality of available coding modes.

The encoding control unit 1 determines a quantization parameter and a transform block division state used when the differential image is compressed for each encoding block, and is used when the prediction process is performed. A prediction parameter (intra prediction parameter, intra block copy prediction parameter, inter prediction parameter, or inter-color signal prediction parameter) is determined.
However, when the encoded block is further divided into prediction block units for further prediction processing, a prediction parameter (intra prediction parameter, intra block copy prediction parameter, inter prediction parameter, or inter-color signal prediction parameter) is selected for each prediction block. To do.

Here, FIG. 20 is an explanatory diagram showing a conversion block size when performing compression processing (conversion processing, quantization processing) of luminance signals and color difference signals in a 4: 2: 0 format signal.
As shown in FIG. 20, the transform block size is determined by hierarchically dividing the encoded block into a quadtree.
For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining, it is possible to determine the optimal division shape of the transform block from the viewpoint of the trade-off between the code amount and the coding error.

For example, as shown in FIG. 20, the luminance signal is configured so that the coding block is hierarchically divided into one or a plurality of square transform blocks.
For the color difference signal, as shown in FIG. 20, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks in the same manner as the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. In addition, the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction. The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
Further, when the input signal format is YUV 4: 4: 4 signal, as shown in FIG. 22, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block to be the same size conversion block. Configure as follows.

The encoding control unit 1 includes predictive differential encoding including transform block partition information indicating transform block partition information in a block to be encoded, a quantization parameter that defines a quantization step size when transform coefficients are quantized, and the like. The parameter is output to the transform / quantization unit 9, the inverse quantization / inverse transform unit 10, and the variable length coding unit 15.
Further, when the intra prediction mode is selected as the encoding mode, the encoding control unit 1 outputs the intra prediction parameters of the luminance signal to the intra prediction unit 5 for the luminance signal, and the color signal for the color difference signal. An inter-color signal prediction parameter including an inter-color signal prediction flag indicating whether or not to perform inter prediction processing is output to the intra prediction unit 5. Further, when the inter-color signal prediction flag is OFF, the intra prediction parameter of the color difference signal is output to the intra prediction unit 5.
However, when the image coding apparatus is configured to always use a common intra prediction parameter for the luminance signal and the color difference signal, it is not necessary to output the intra prediction parameter of the color difference signal to the intra prediction unit 5.

In addition, when the intra block copy prediction mode is selected as the encoding mode, the encoding control unit 1 outputs the intra block copy prediction parameter of the luminance signal to the intra block copy prediction unit 6 for the luminance signal, and the color difference For signals, the inter-color signal prediction parameters including the inter-color signal prediction flag indicating whether or not to perform inter-color signal prediction are output to the intra block copy prediction unit 6. Further, when the inter-color signal prediction flag is OFF, the intra block copy prediction parameter of the color difference signal is output to the intra block copy prediction unit 6.
However, when the image coding apparatus is configured to always use a common intra block copy prediction parameter for the luminance signal and the color difference signal, it is not necessary to output the intra block copy prediction parameter of the color difference signal to the intra block copy prediction unit 6. .

Also, when the inter prediction mode is selected as the encoding mode, the encoding control unit 1 outputs the inter prediction parameters of the luminance signal to the motion compensation prediction unit 7 for the luminance signal, and the color difference signal for the color difference signal. An inter-color signal prediction parameter including an inter-color signal prediction flag indicating whether or not to perform inter-signal prediction is output to the motion compensation prediction unit 7. Further, when the inter-color signal prediction flag is OFF, the inter prediction parameter of the color difference signal is output to the motion compensation prediction unit 7.
However, when the image coding apparatus is configured to always use a common inter prediction parameter for the luminance signal and the color difference signal, it is not necessary to output the inter prediction parameter of the color difference signal to the motion compensation unit 7.

When a video signal is input as an input image, the slice division unit 2 divides the input image into slices that are one or more partial images according to the slice division information determined by the encoding control unit 1.
Each time each slice is input from the slice dividing unit 2, the block dividing unit 3 divides the slice into the maximum encoded block size determined by the encoding control unit 1, and further encodes the divided maximum encoded block. The coding block is hierarchically divided into coding blocks determined by the coding control unit 1, and the coding blocks are output.

Here, FIG. 5 is an explanatory diagram showing an example in which the maximum coding block is hierarchically divided into a plurality of coding blocks.
In FIG. 5, the maximum coding block is a coding block whose luminance component described as “0th layer” has a size of (L ⁰ , M ⁰ ).
Starting from the maximum encoding block, the encoding block is obtained by performing hierarchical division to a predetermined depth separately defined by a quadtree structure.
At depth n, the coding block is an image area of size (L ⁿ , M ⁿ ).
However, L ⁿ and M ⁿ may be the same or different, but FIG. 5 shows a case of L ⁿ = M ⁿ .

Hereinafter, the coding block size determined by the coding control unit 1 is defined as the size (L ⁿ , M ⁿ ) in the luminance component of the coding block.
Since quadtree partitioning is performed, (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) always holds.
Note that in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ), but 4: 2. : When the 0 format is handled, the encoding block size of the corresponding color difference component is (L ⁿ / 2, M ⁿ / 2).

Hereinafter, the coding block of the n hierarchy expressed in B ^n, denote the encoding modes selectable by the coding block B ⁿ with m (B ^n).
In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, or common to all color components. It may be configured to use a mode. Hereinafter, unless otherwise specified, description will be made assuming that it indicates a coding mode for a luminance component of a coding block of a YUV signal and 4: 2: 0 format.

As shown in FIG. 5, the encoded block B ⁿ is divided by the block dividing unit 3 into one or a plurality of prediction blocks representing a prediction processing unit.
Hereinafter, a prediction block belonging to the coding block B ⁿ is ^denoted as P _i ⁿ (i is a prediction block number in the n-th layer). FIG. 5 shows an example of P ₀ ⁰ and P ₁ ⁰ .
How the prediction block is divided in the coding block ^Bn is included as information in the coding mode m ( ^Bn ).
All prediction blocks P _i ⁿ are subjected to prediction processing according to the encoding mode m (B ⁿ ), but for each prediction block P _i ⁿ , individual prediction parameters (intra prediction parameters, intra block copy prediction parameters, or inter prediction parameters) are used. ) Can be selected.

For example, the encoding control unit 1 generates a block division state as illustrated in FIG. 6 for the maximum encoding block, and identifies the encoding block.
A rectangle surrounded by a dotted line in FIG. 6A represents each coding block, and a block painted with diagonal lines in each coding block represents a division state of each prediction block.
FIG. 6B shows, in a quadtree graph, a situation in which the encoding mode m (B ⁿ ) is assigned by hierarchical division in the example of FIG. 6A. Nodes surrounded by squares in FIG. 6B are nodes (encoding blocks) to which the encoding mode m (B ⁿ ) is assigned.
Information of the quadtree graph is output from the encoding control unit 1 to the variable length encoding unit 15 together with the encoding mode m (B ⁿ ), and is multiplexed into the bit stream.

The changeover switch 4 is output from the block dividing unit 3 when the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA). When the coding block B ⁿ is output to the intra prediction unit 5 and the coding mode m (B ⁿ ) determined by the coding control unit 1 is the intra block copy coding mode (m (B ⁿ ) ∈ICOPY The encoding block B ⁿ output from the block division unit 3 is output to the intra block copy prediction unit 6, and the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode. In some cases (when m (B ⁿ ) εINTER), the encoded block B ⁿ output from the block dividing unit 3 is output to the motion compensation prediction unit 7.

In the intra prediction unit 5, the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA), and the coding block B is changed from the changeover switch 4 to the coding block B. ⁿ (step ST3), using the intra prediction parameters determined by the encoding control unit 1 while referring to the local decoded image stored in the intra memory 12, the encoding block ^Bn and implementing intra prediction process for each of the prediction block _P ^{i n,} generates an intra prediction image _{P INTRAi} ⁿ (step ST4).

However, although details will be described later, since the encoded pixel adjacent to the prediction block is used when performing the process of generating the intra prediction image, the process of generating the intra prediction image is the prediction block used for the prediction process. Must always be performed in units of transform blocks so that pixels adjacent to are already encoded.
Therefore, in a coding block in which the coding mode is the intra coding mode, the block size of the selectable transform block is limited to the size of the prediction block or smaller and the transform block is smaller than the prediction block (in the prediction block). In the case where there are a plurality of transform blocks, the intra prediction process using the intra prediction parameters defined in the prediction block is performed for each transform block to generate an intra predicted image.
Since the image decoding apparatus needs to generate exactly the same intra prediction image and the intra prediction image P _INTRAi ^n, intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ is a variable from the coding controller 1 The data is output to the long encoding unit 15 and multiplexed into a bit stream.
Details of the intra prediction processing in the intra prediction unit 5 will be described later.

  The intra prediction unit 5 always performs an intra prediction process for the generation of a prediction image of a luminance signal, but for the generation of a prediction image of a color difference signal, the encoding control unit 1 performs the prediction block on each prediction block. When the inter-color signal prediction flag is set to ON, the intra-color signal prediction that generates the prediction image of the color difference signal in the prediction block with reference to the encoded color signal without performing the intra prediction process Perform the process. On the other hand, when the inter-color signal prediction flag of the prediction block is set to OFF by the encoding control unit 1, the color difference signal is also subjected to the intra prediction process.

The intra block copy prediction unit 6 is the intra block copy encoding mode when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is m (B ⁿ ) ∈ICOPY, and When the encoded block B ⁿ is received (step ST3), each prediction block P _i ⁿ in the encoded block B ⁿ is compared with the locally decoded image stored in the intra memory 12 to search for a block shift vector. .
That is, the intra block copy prediction unit 6, in the local decoded image stored in the intra-memory 12, to identify the block (reference block) in the region which is the most approximate to the prediction block P _i ^n, that Search for a block shift vector pointing to the reference block.
Intra block copy prediction unit 6, when searching for a block shift vector pointing to the reference block, the reference block whose block shift vector is pointing as the predicted image of the prediction block P _i ^n, generates an intra block copy predicted image P _ICOPYi ⁿ ( Step ST5).

However, as shown in FIG. 32, the prediction block P _i ⁿ a not closest to that region is performed is unencoded area (still encoding an area have not yet locally decoded, in FIG. 32 If shaded areas and the prediction block P _i ⁿ is as present in a position straddling the applicable), the reference blocks of the same size as the prediction block P _i ⁿ is, contains pixels which have not yet locally decoded Therefore, in the normal intra block copy prediction process, a block shift vector indicating the reference block cannot be selected.
Therefore, in the first embodiment, even when the reference block closest to the prediction block Pin includes a part of the uncoded region, it is possible to search for a block shift vector indicating the reference block. In order to achieve this, as shown in FIG. 33, the pixels of each pixel in the unencoded area in the maximum encoding block to be encoded to which the prediction block belongs (the maximum encoding block) in advance. A process of assuming a value by a predetermined method is performed (a process of filling a pixel in an unencoded area with a pixel value determined by a predetermined method).

Specifically, the pixel value of each pixel in the uncoded area is assumed by any one of the following methods (1) to (3). In the following description, the possible range of the pixel value is described as 0 to 255 having an 8-bit accuracy. However, the processing is performed in the same manner even when the 8-bit accuracy is not 10 bits or 12 bits. That is, if the accuracy is 10 bits, the possible range of pixel values is 0-1023, and if the accuracy is 12 bits, the possible range of pixel values is 0-4095. Further, the same processing is performed when the pixel value that can be taken is signed (for example, the value that can be taken with 8-bit accuracy is −123 to 122).
Further, the following method is used for each color signal (luminance signal (Y), color difference signal (U, V) or other color signal (red (R), green (G), blue (B) signal, etc.)). This is carried out to determine a pixel value for filling each pixel in the uncoded area.

(1) As shown in FIG. 34, a plurality of pixels adjacent to the left of the maximum coding block to which the prediction block belongs (the maximum coding block) (a column of pixels arranged in the vertical direction in the figure) The pixel value having the largest number of pixels among the pixel values (pixel values) is specified, and the pixel having the pixel value is embedded in the uncoded area.
In the example of FIG. 34, a pixel having a pixel value of “0” and a pixel having a pixel value of “64” are adjacent to the left of the maximum coding block and have a pixel value of “64”. Since the number of pixels is larger than the number of pixels having a pixel value of “0”, the pixels in the uncoded area are filled with a pixel value of “64”.
In general, since the pixel value with the largest number of pixels is likely to be the background color, there is an unnatural luminance change between the uncoded area where the pixels are filled and the background of the encoded area. It does not occur, and the decrease in prediction efficiency can be suppressed.

In the above example, the unencoded area is filled with the pixel value having the largest number of pixels among the pixel values of a plurality of pixels adjacent to the left of the maximum encoded block. You may make it fill an uncoded area | region with the pixel value with the largest number of pixels among the pixel values of the several pixel adjacent on a block. In this case, the pixel value that fills the uncoded region in the image having a high correlation in the vertical direction can be calculated with high accuracy. Furthermore, the unencoded area may be filled with the pixel value having the largest number of pixels among the pixel values of a plurality of pixels adjacent to the left and above the maximum encoded block. In this way, although the amount of calculation increases, an improvement in prediction accuracy can be expected due to an increase in the number of pixels to be referred to. At this time, the upper left pixel of the maximum coding block may also be referred to.
Alternatively, depending on the shape of the uncoded area in the reference block indicated by the block shift vector shown in FIG. 37, a plurality of pixels adjacent to the left of the maximum encoded block are used, or a plurality of adjacent pixels Whether to use pixels may be switched adaptively.

Specifically, when dh <dw where the uncoded portion in the vertical direction becomes shorter, a pixel value for filling the uncoded region is obtained from a plurality of adjacent pixels on the maximum coded block, and the other ( When dh ≧ dw), a pixel value that fills the uncoded region is obtained from a plurality of pixels adjacent to the left of the maximum coded block. By doing in this way, the encoded adjacent pixel whose distance (dh, dw) between the encoded pixel adjacent to the maximum encoded block and the right end or lower end pixel of the maximum encoded block is always shorter Therefore, the spatial correlation of the image can be used with high accuracy.
In addition, when some of the adjacent pixels to be referenced do not exist, a pixel to be referred to is generated by a predetermined method. For example, there are a method of substituting with a predetermined fixed value and a method of substituting with a pixel value of an adjacent pixel at another existing position. Further, when there is no adjacent pixel to be referred to, the pixels in the uncoded area are filled with a predetermined fixed value. The predetermined fixed value includes an intermediate value of possible pixel values.

(2) Fill the pixels in the uncoded area with a preset pixel value (fixed value).
The intra block copy prediction process is generally a prediction process that is highly effective for characters or lines drawn in graphics such as text images and CAD images. For example, a text image generally has a white background. For example, when a white pixel (a pixel having a pixel value of “255” when the pixel value can take 0 to 255) is embedded in an uncoded area of the text image, the intra block copy prediction process is performed. It becomes effective, and a performance improvement effect can be expected for many images. In addition, the pixel value to be set may be other than the above, such as an intermediate value of possible pixel values (a pixel having a pixel value of “128” when the pixel value can be 0 to 255).
The method of filling the pixels in the unencoded area with a preset pixel value (fixed value) may reduce the prediction accuracy depending on the content, but has an advantage that the processing load is very small.

As a method of extending the above method, a method of selecting an optimal pixel value for each maximum encoding block and encoding the pixel value can be considered.
At this time, the pixel value candidates that can be selected may be all existing pixel values, or a representative value of selectable pixel values may be determined so that only the representative value can be selected.
When the representative value is determined, if the variable length encoding unit 15 does not encode the representative value itself but encodes an index indicating the representative value, the code amount of the encoded bit stream is suppressed. can do.
Specifically, if the representative value is, for example, 0, 64, 128, 192, 256, the index is defined as 0, 1, 2, 3, 4 in that order, and the index indicating the selected representative value is Variable length coding.
As an example of determining such a representative value, a fixed value such as white (maximum value), gray (intermediate value), black (minimum value), etc. is determined as a representative value, and an arbitrary representative can be selected from these representative values. There are a method of selecting a value and a method of adaptively changing the representative value using a Move-To-Front method as shown in FIG.

The Move-To-Front method is a method of rearranging indexes according to the appearance frequency.
In the case of this example, as an initial table, if pixel values are arranged in the order of white (maximum value) to black (minimum value), for example, a pixel value of “255” is selected as shown in FIG. , The index of “6” indicating the pixel value of “255” is encoded, and rearranged so that the pixel value of “255” is at the top of the table (the index indicating the pixel value of “255” is “ The table is rearranged to be 1 ").
In each maximum coding block, one or more pixel values (for example, the upper eight) in the upper table are used as representative values, and an arbitrary representative value is selected from these representative values.

垂直方向フィルタ：

ここで、ｃ_ｎ（ｎ＝０、１、・・・、２Ｎｈ＋１），ｃ’_ｍ（ｍ＝０、１、・・・、２Ｎｖ＋１）はフィルタ係数、Ｎｈ，Ｎｖはフィルタの参照画素数を決める定数である。
即ち、水平方向フィルタはフィルタ処理対象画素の左に位置する隣接画素群によるフィルタ処理を示し、垂直方向フィルタはフィルタ処理対象画素の上に位置する隣接画素群によるフィルタ処理を示している。 (3) Generate pixels in the unencoded area by filtering.
Specifically, a filtering process referring to pixels as shown in FIG. 36 is performed (a solid circle indicates a reference pixel, and a dotted circle indicates a filtering target pixel).
If S (x, y) is the pixel value of the encoded pixel at the coordinate (x, y) (decoded pixel value) or the filter generation pixel value (in the case of an unencoded area), the coordinates of the pixel to be filtered Is set to (x, y), the filter processing is executed by the following equation.
Horizontal filter:

Vertical filter:

Here, c _n (n = 0, 1,..., 2Nh + 1), c ′ _m (m = 0, 1,..., 2Nv + 1) are filter coefficients, and Nh and Nv determine the number of reference pixels of the filter. It is a constant.
That is, the horizontal direction filter indicates a filter process by an adjacent pixel group located on the left side of the filter processing target pixel, and the vertical direction filter indicates a filter process by an adjacent pixel group positioned on the filter process target pixel.

In this _{case, c 0 + c 1 + ···} + c 2Nh = 1, c '0 + c' 1 + ··· + c '2Nv = 1 and when made, indicating the smoothing filter. Further, c _{2Nh + 1} and c ′ _{2Nv + 1} represent offset values. Examples of the filter include the following.
Horizontal filter:
_{Nh = 1, c 0 = 1} /4, c 1 = 1/2, c 2 = 1/4, c 3 = 0
(C ₃ are offset, in the present embodiment is that there is no offset)
Vertical filter:
Nv = 1, c ′ ₀ = 1/4, c ′ ₁ = 1/2, c ′ ₂ = 1/4, c ′ ₃ = 0
(C ′ ₃ is an offset value, and in this example, there is no offset)

  Nh, Nv and filter coefficients are defined in advance as values common to the image encoding device and the image decoding device. In this example, the filter coefficient of the pixel closer to the filter processing target pixel is set to a larger value. This is because, generally, the closer to the filter processing target pixel, the higher the correlation with the filter processing target pixel. By doing in this way, a highly accurate pixel can be produced | generated as a pixel in an unencoded area | region.

したがって、上記の例（Ｎｈ＝１、ｃ_０＝１／４、ｃ_１＝１／２、ｃ_２＝１／４、ｃ_３＝０の例）の場合、
ｃ_０＝１／４、ｃ_１＝３／４、ｃ_２＝０、ｃ_３＝０
となる。なお、このような参照不可能な画素が生じるフィルタ処理については、別途フィルタ係数やＮｈ，Ｎｖを定義するようにしてもよい。例えば、上記の水平方向フィルタの例では、Ｎｈ＝１のままとして、
ｃ_０＝１／２、ｃ_１＝１／２、ｃ_２＝０、ｃ_３＝０
としてもよい。この場合、フィルタ演算が２画素の単純平均となり、処理が簡易になるメリットがある。 Note that the filter processing procedure is as follows: in the case of horizontal filter processing, filter processing is performed one column at a time from the leftmost column to the rightmost column of the uncoded region, and in the case of vertical direction filter processing, Filtering is performed line by line from the top line to the bottom line. At this time, when a pixel that cannot be referred to is included as a reference pixel of the filter, the filter coefficient of the pixel that cannot be referred to is added to the filter coefficient of the closest pixel among the referenceable pixels (however, the closest pixel is When there are a plurality of pixels, the pixel closest to the coordinates (x, y) of the pixel to be filtered is targeted). For example, in the horizontal direction filter processing, when a pixel having coordinates (x−1, y + 1) or less cannot be referred to, the following expression is obtained.

Therefore, in the case of the above example (example of Nh = 1, c ₀ = 1/4, c ₁ = 1/2, c ₂ = 1/4, c ₃ = 0),
_{c 0 = 1/4, c} 1 = 3/4, c 2 = 0, c 3 = 0
It becomes. In addition, about the filter process which produces the pixel which cannot be referred to, you may make it define a filter coefficient, Nh, and Nv separately. For example, in the above example of the horizontal filter, Nh = 1 remains
c ₀ = 1/2, c ₁ = 1/2, c ₂ = 0, c ₃ = 0
It is good. In this case, the filter operation is a simple average of two pixels, and there is an advantage that the processing is simplified.

As examples of filters other than the filter examples described above,
Horizontal filter:
Nh = 0, c ₀ = 1, c ₁ = 0
(C ₁ is an offset value, and in this example, there is no offset)
Vertical filter:
Nv = 0, c ′ ₀ = 1, c ′ ₁ = 0
(C ′ ₁ is an offset value, in this example, no offset)
Also mentioned.
In this example, the horizontal filter copies the pixel value of the pixel to the left of the target pixel, and the vertical filter copies the pixel value of the pixel above the target pixel. In other words, the horizontal filter is equivalent to copying the pixel value of the pixel adjacent to the maximum coding block in the horizontal direction and the vertical filter in the vertical direction, reducing the amount of calculation required for the filter processing. In addition, all the pixels in the maximum coding block can be generated at the same time (all the pixels in the maximum coding block can be generated in parallel).

Which of the horizontal direction filter and the vertical direction filter is used is determined in advance for both the image encoding device and the image decoding device. For example, in a text image, since characters are often arranged in the horizontal direction, the image encoding device and the image decoding device are configured to always use a horizontal filter.
Alternatively, whether to use the horizontal filter or the vertical filter may be adaptively switched according to the shape of the uncoded region in the reference block indicated by the block shift vector shown in FIG. Specifically, a vertical filter is used when dh <dw when the uncoded portion in the vertical direction becomes shorter, and a horizontal filter is used when the other (dh ≧ dw). In this way, a filter that always shortens the distance (dh, dw) between the pixel to be filtered and the encoded pixel can be selected, so that the spatial correlation of the image can be used with high accuracy.

ここで、ｃ”_ｌ（ｌ＝０、１、・・・、３）はフィルタ係数を示している。また、ｃ”_０＋ｃ”_１＋ｃ”_２＝１となるとき、平滑化フィルタであることを示している。また、ｃ”_３はオフセット値を表している。上記フィルタの例としては下記が挙げられる。
ｃ”_０＝１／４、ｃ”_１＝３／８、ｃ”_２＝３／８、ｃ”_３＝０
（ｃ”_３はオフセット値、本例ではオフセットなしとしている）
本例ではフィルタ処理対象画素に近い画素程フィルタ係数が大きくなっており、これは一般にフィルタ処理対象画素に近い画素程、フィルタ処理対象画素との相関が高いためである。なお、フィルタ処理の手順としては、未符号化領域の左上の画素からフィルタ処理を実施し、隣接する未符号化領域の画素を順にフィルタ処理していくことで、常に参照画素Ｓ（ｘ−１，ｙ−１）、Ｓ（ｘ，ｙ−１）、Ｓ（ｘ−１，ｙ）が存在するようにフィルタ処理を行うことができる。 As another example of (3), the image encoding apparatus may be configured to perform the filtering process of the reference pixel arrangement shown in FIG. 38 (the solid circle indicates the reference pixel, and the dotted circle indicates the filtering target) Pixel). When the coordinates of the pixel to be filtered are (x, y), the filtering process is executed according to the following formula.

Here, c ″ _l (l = 0, 1,..., 3) indicates a filter coefficient. When c ″ ₀ + c ″ ₁ + c ″ ₂ = 1, the filter is a smoothing filter. Is shown. Also, c ″ ₃ represents an offset value. Examples of the filter include the following.
c ″ ₀ = 1/4, c ″ ₁ = 3/8, c ″ ₂ = 3/8, c ″ ₃ = 0
(C ″ ₃ is an offset value, and in this example, there is no offset)
In this example, the filter coefficient closer to the pixel to be filtered has a larger filter coefficient. This is because the pixel closer to the filter target pixel generally has a higher correlation with the filter target pixel. As a procedure of the filter process, the reference pixel S (x−1) is always obtained by performing the filter process from the upper left pixel of the uncoded area and sequentially filtering the pixels in the adjacent uncoded area. , Y-1), S (x, y-1), and S (x-1, y) can be filtered.

垂直方向フィルタ：

図３８のフィルタ：

ここで、“＞＞”は右ビットシフト演算を示している。これらの計算式は全て整数精度のため、元の式と比べて小数精度の誤差は生じるものの、整数演算とビットシフトのみで実行できるため、高速な計算を実現することができる。 For the following example described above,
Horizontal filter:
_{Nh = 1, c 0 = 1} /4, c 1 = 1/2, c 2 = 1/4, c 3 = 0
Vertical filter:
Nv = 1, c ′ ₀ = 1/4, c ′ ₁ = 1/2, c ′ ₂ = 1/4, c ′ ₃ = 0
The filter of FIG. 38:
c ″ ₀ = 1/4, c ″ ₁ = 3/8, c ″ ₂ = 3/8, c ″ ₃ = 0
These filters have significant filter coefficient denominators that are all powers of 2, and can be replaced by the following equations when implemented as integer operations by a computer.
Horizontal filter:

Vertical filter:

The filter of FIG. 38:

Here, “>>” indicates a right bit shift operation. Since these calculation formulas are all integer precision, an error of decimal precision occurs compared to the original formula, but can be executed only by integer arithmetic and bit shift, so that high-speed calculation can be realized.

In the above description, the method for performing the filtering process with reference to the pixels within the coordinate range in which either the horizontal direction or the vertical direction is ± 1 from the coordinates (x, y) of the pixel to be filtered has been described. As shown in FIGS. 40 and 41, the filter processing may be performed using pixels that are ± 2 or more in the horizontal and vertical directions from the coordinates (x, y) of the pixel to be filtered. 39, 40, and 41, a solid circle indicates a reference pixel, and a dotted circle indicates a filter processing target pixel. Similarly to Nh and Nv, Mh, Mv, Nl, and Ml indicate constants that determine the number of reference pixels of each filter.
In this case, similarly to the filter described above, the filter processing can be formulated from the filter coefficient and the offset value for each reference pixel. By increasing the reference pixels of the filter in this way, the calculation amount increases, but high-accuracy generation of uncoded area pixels can be realized.

  In the above example, the unit in which the pixels in the unencoded area are embedded is the maximum encoded block. However, the present invention is not limited to this. For example, the update is performed in encoded block units or predicted block units. You may do it. By doing so, the processing amount increases, but it is possible to realize pixel filling processing with high accuracy.

For example, in the methods (1) to (3) described above, the process of filling the pixels in the unencoded area in units of maximum encoded blocks has been described. However, the present process may be performed in units of encoded blocks.
That is, pixels are filled by the above-described methods (1) to (3) only in the prediction target coding block (the coding block) in the uncoded region of the maximum coding block. Specifically, it is as follows.
For method (1):
The pixels in the unencoded area in the coding block are filled with the pixel value having the largest number of pixels among the pixel values of the plurality of pixels adjacent to the left of the coding block to be predicted (the coding block). Like that.
In the case of method (2):
For the prediction target encoding block (the encoding block), a representative value for filling an unencoded area in the encoding block is determined. The representative value (or an index indicating the representative value) is encoded in units of encoded blocks.
In the case of method (3):
For an encoding block to be predicted (the encoding block), an unencoded area in the encoding block is filled by a filter process.
Therefore, as in the example shown in FIG. 42, in the intra block copy prediction process of each encoded block, the unencoded area that may be included in the reference block indicated by the block shift vector is only within the encoded block. Therefore, an area that can be referred to becomes narrower than in the case of performing the process of filling the pixels of the unencoded area in units of maximum encoded blocks. However, since it is possible to adaptively fill pixels in units of encoded blocks smaller than the maximum encoded block, the prediction accuracy of the pixels to be embedded is improved, and improvement in prediction efficiency of intra block copy prediction can be expected.

  In (1), the pixel value of the pixel number of the plurality of pixels adjacent on the encoding block is not the plurality of pixels adjacent to the left of the encoding block, but the pixel number You may make it fill the pixel of the unencoded area | region in the said encoding block with many pixel values. In this case, a pixel value that fills an uncoded area in an image having a high correlation in the vertical direction can be obtained with high accuracy. Furthermore, an unencoded area in the encoded block may be filled with a pixel value having the largest number of pixels among the pixel values of a plurality of pixels adjacent to the left and above the encoded block. . In this way, although the amount of calculation increases, an improvement in prediction accuracy can be expected due to an increase in the number of pixels to be referred to. At this time, the upper left pixel of the coding block may also be referred to.

Alternatively, depending on the shape of the uncoded region in the reference block indicated by the block shift vector shown in FIG. 37, a plurality of pixels adjacent to the left of the encoded block are used, or a plurality of pixels adjacent above It may be switched adaptively whether to use. Specifically, when dh <dw when the uncoded portion in the vertical direction becomes shorter, a pixel value for filling the uncoded region is obtained from a plurality of adjacent pixels on the coded block, and the other (dh When ≧ dw), a pixel value for filling the uncoded area is obtained from a plurality of pixels adjacent to the left of the coded block. By doing so, it is possible to always select an encoded adjacent pixel having a shorter distance (dh, dw) between the encoded pixel adjacent to the encoded block and the right end or lower end pixel of the encoded block. Therefore, the spatial correlation of images can be used with high accuracy.
In addition, when some of the adjacent pixels to be referenced do not exist, a pixel to be referred to is generated by a predetermined method. For example, there are a method of substituting with a predetermined fixed value and a method of substituting with a pixel value of an adjacent pixel at another existing position. Further, when there is no adjacent pixel to be referred to, the pixels in the uncoded area are filled with a predetermined fixed value. The predetermined fixed value includes an intermediate value of possible pixel values.

  Even in the case of the process of filling the pixels of the unencoded area in units of the maximum encoded block, the unencoded area that may be included in the reference block indicated by the block shift vector may be only within the encoded block to be predicted. . In this way, the search area is expanded so that only the area close to the encoding block to be predicted, that is, the area having a high spatial correlation, and the uncoded area can also be referred to can be searched. Compared with the case where the entire coding region can be searched, the prediction accuracy can be improved in the same manner while suppressing an increase in the calculation load required for the block shift vector search. Furthermore, by suppressing the search range, it is possible to suppress an increase in the size of the memory that stores the referenceable area required by the image decoding apparatus.

In the process of filling the pixels in the uncoded area (1), the pixel value for counting the number of pixels may be limited. By doing so, the number of necessary counters is reduced as compared with the method (1) in which counters corresponding to the number of values that the pixel values can take are required, so that it is possible to reduce the amount of memory required as a counter. . It should be noted that this processing is performed on the reference pixels set in each of the units of the maximum coding block, coding block, and prediction block as the unit for embedding the pixels in the uncoded region. .
An example of a method for limiting the pixel value for counting the number of pixels will be described below. First, an example of quantizing pixel values will be described. In this example, the pixel value is quantized by a predetermined quantizer, and the number of representative values after the quantization is counted. Then, the pixels in the uncoded area are filled with the representative value having the largest number of pixels. Here, the possible pixel values are 0 to 2 ^bit_depth −1, and the possible pixel values are quantized into N _Q representative values (bit_depth is an integer of 0 or more, and N _Q is an integer of 1 or more). As an example of the quantization processing at this time, an example performed by integer precision calculation is shown below.
pow = bit_depth-base
(Representative value) = (((pixel value) + (1 << (pow-1))) >> pow) << pow

However, if the representative value is greater than ^{2 bit_depth -1,} round the representative value to ^{2 bit_depth} -1. “<<” indicates a left bit shift operation, “>>” indicates a right bit shift operation, base is an integer greater than or equal to 0, and a relationship of N _Q = 2 ^base +1 is established. For example, when bit_depth = 8 and base = 1, the representative value of pixel values 0 to 63 is 0, the representative value of pixel values 64 to 191 is 128, and the representative value of pixel values 192 to 255 is 255. Note that the pixel value quantization method is not limited to the above method, and a method of quantizing at equal intervals such as not adding 1 << (pow-1) to the pixel value in the above formula, etc. Any method is acceptable.
As another example of the method of limiting the pixel value for counting the number of pixels, there is a method for invalidating a newly appearing pixel value when the number of pixel values for counting the number of pixels reaches a predetermined number.
Specifically, a predetermined number _Nc counters are prepared, and each time a new pixel value appears while counting adjacent pixels in sequence, an unused counter is assigned as a counter for the pixel value. To do. After reaching the state where all the predetermined number _Nc counters are used, when a new pixel value that has not appeared so far appears, the pixel value is invalidated. In this way, the pixel value that fills the uncoded region is determined only from the predetermined number _Nc of pixel values, and the pixel value for counting the number of pixels can be limited. In the case of this method, the order of counting adjacent pixels affects the result. Therefore, the order in which adjacent pixels are counted is determined in advance. For example, when referring to the pixels adjacent to the left and top of the maximum coding block, it is determined in advance such that the reference is made from the lower left pixel toward the upper left pixel and then the upper right pixel.
Note that the method for limiting the pixel value for counting the number of pixels is not limited to the above method, and other methods may be used.

In the method (1), a memory for a counter that counts the number of pixels for each pixel value is required. Therefore, it is possible to determine the pixel value that fills the unencoded area by using a continuous number of the same pixel value, which does not require memorizing the number of pixels of each pixel value. Specifically, the adjacent pixel to be referred to is the same as (1), and the number of pixels in which the same pixel value continues in a predetermined reference order is counted.
Then, the pixel value having the maximum number of consecutive pixels is determined as the pixel value that fills the pixels in the uncoded area. It should be noted that this processing is performed on the reference pixels set in each of the units of the maximum coding block, coding block, and prediction block as the unit for embedding the pixels in the uncoded region. .
In the case of this method, the maximum continuous pixel number L _max , the corresponding pixel value S _max , the currently continuous pixel value S _tmp , and the continuous pixel number L _tmp are stored, and the pixels are stored in a predetermined reference order. When referencing, if the pixel values S _cur and S _tmp of the currently referenced pixel are the same, the number of consecutive pixels L _tmp is increased by 1, and if they are different, L _tmp and L _max are compared to determine L _tmp If it is larger, after L _max = L _tmp and S _max = S _tmp , L _tmp is initialized and S _tmp = S _cur is set. After this is performed for all reference pixels, S _max is the pixel value that maximizes the number of consecutive pixels.

43 and 44 are flowcharts showing an example of the processing procedure of this method.
However, FIG. 44 shows a processing procedure example of “L _max determination and S _max update” in step ST 105 and step ST 106 shown in FIG. 43. In this example, in step ST100, L _max and S _max are initialized to 1 (the number of consecutive pixels is 1), and S _tmp and L _tmp are set to the pixel values of adjacent pixels to be referred to first. Then, with respect to adjacent pixels remaining references (step ST _101), carries out a process of updating the _{S max} and _{L max} (step ST102~ step ST 106).
First, let the pixel value of the pixel to be referred to be _Scur (step ST102), and compare whether _Scur and _Stmp are the same (step ST103). If S _cur and S _tmp are the same, the value of the continuous pixel number L _tmp is incremented by 1 (step ST104), and if different, the L _max determination and S _max update processing shown in FIG. 44 is performed (step ST105).

_{Specifically,} by comparing _{L tmp} and _{L max} (step ST _{200), L tmp} is _L max ₌ L _tmp is larger than _{L max,} and _S max ₌ S _tmp (Step ST 201). Then, it initializes to L _tmp = 1 and sets S _tmp = S _cur (step ST202). Then, after the above process is completed for all the adjacent pixels to be referred to, the L _max determination and the S _max update process shown in FIG. 44 are performed for the current continuous pixel number L _tmp (step ST106). Then, S _max when the above process is completed is set to a prime value that fills the pixels in the uncoded region (step ST107).
However, step ST106 may be omitted because the process of setting L _max = L _tmp in the process of step ST201 and the process of step ST202 are unnecessary.

As described above, a memory for a counter for counting the number of pixels of each pixel value required by the method (1) is not necessary, and the amount of memory can be reduced. Further, since the pixel value having the largest number of consecutive pixels is likely to be the pixel value having the largest number of pixels among the adjacent pixels to be referred to, the pixel value of the same level as (1) (fill the pixels in the uncoded region) Prediction accuracy (of pixel values) can be obtained.
In addition, since the adjacent pixel to which this technique refers is the same as (1), according to the shape of the uncoded area in the reference block indicated by the block shift vector shown in FIG. It is also possible to adaptively switch between using a plurality of pixels adjacent to the left of the pixel and a plurality of pixels adjacent to the upper side. The switching method is also the same as (1).

In the image decoding apparatus described later, it is necessary to generate exactly the same intra block copy predictive image and the intra block copy predicted image P _ICOPYi ^n, intra block copy predicted image P _ICOPYi ⁿ intra block copy prediction parameters used to generate the Is output from the encoding control unit 1 to the variable length encoding unit 15 and multiplexed into a bit stream.
Examples of the intra block copy prediction parameter include a block shift vector searched by the intra block copy prediction unit 6 and a target parameter when there is a parameter to be encoded by pixel value padding processing in an unencoded area. Also, the block shift vector is encoded by using the block shift vector of the previous encoded prediction block or the difference value with the block shift vector of the encoded prediction block around the prediction block as a part of the intra block copy prediction parameter. You may make it do.

For the color difference signal, prediction using the same vector as the luminance signal is performed. Alternatively, the search may be performed based on the luminance signal vector, and the difference vector of the luminance signal vector may be encoded. By doing so, the prediction accuracy of the color difference signal can be improved.
This difference vector may be determined independently for each of the U signal and the V signal. By doing so, the degree of freedom is further increased, so that the prediction accuracy of the color difference signal can be further improved.
The difference vector may be limited to within ± M pixels (M: natural number). By limiting the search range in this way, the calculation amount required for the search can be reduced. Moreover, the memory amount of a reference image can be reduced by limiting the reference image prepared also in the corresponding image decoding apparatus.
At this time, decimal pixel accuracy may be prohibited. In this way, it is possible to reduce the amount of memory required for the reference image while further reducing the amount of calculation required for the search.
Alternatively, it may be limited to decimal pixel accuracy within ± 1 pixel. Also in this case, by limiting the search range, it is possible to reduce the amount of calculation required for the search and the amount of memory required for the reference image.

  The intra block copy prediction unit 6 always performs the intra block copy prediction process for the generation of the prediction image of the luminance signal, but the encoding control unit 1 for each prediction block for the generation of the prediction image of the color difference signal. When the inter-color-signal prediction flag of the prediction block is set to ON by referring to the encoded color signal without performing the intra-block copy prediction process, the prediction image of the color difference signal in the prediction block is obtained. The prediction process between the color signals to be generated is performed. On the other hand, if the inter-color signal prediction flag of the prediction block is set to OFF by the encoding control unit 1, the color difference signal is also subjected to intra block copy prediction processing.

The motion compensated prediction unit 7 uses the encoding switch 4 from the changeover switch 4 when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode (when m (B ⁿ ) ∈INTER). Upon receiving the B ⁿ (step ST3), the motion vector by comparing the locally decoded image after the filtering process stored in the prediction block P _i ⁿ and the motion compensated prediction frame memory 14 of the encoding block B ⁿ Using the motion vector and the inter prediction parameter determined by the encoding control unit 1, the motion compensated prediction process for each prediction block P _i ⁿ in the encoding block B ⁿ is performed, and the inter prediction image generating a P _INTERi ⁿ (step ST6).
Since the image decoding apparatus must generate an identical inter prediction image and the inter-predicted image P _INTERi ^n, inter prediction parameters used for generating the inter prediction image P _INTERi ⁿ is a variable from the coding controller 1 The data is output to the long encoding unit 15 and multiplexed into a bit stream.
The motion vector searched by the motion compensation prediction unit 7 is also output to the variable length encoding unit 15 and multiplexed into the bit stream.

For the color difference signal, prediction using the same vector as the luminance signal is performed. Alternatively, the search may be performed based on the luminance signal vector, and the difference vector of the luminance signal vector may be encoded. By doing so, the prediction accuracy of the color difference signal can be improved.
This difference vector may be determined independently for each of the U signal and the V signal. By doing so, the degree of freedom is further increased, so that the prediction accuracy of the color difference signal can be further improved.
The difference vector may be limited to within ± M pixels (M: natural number). By limiting the search range in this way, the calculation amount required for the search can be reduced. Moreover, the memory amount of a reference image can be reduced by limiting the reference image prepared also in the corresponding image decoding apparatus.
At this time, decimal pixel accuracy may be prohibited. In this way, it is possible to reduce the amount of memory required for the reference image while further reducing the amount of calculation required for the search.
Alternatively, it may be limited to only pixels with decimal pixel accuracy. For example, by limiting to pixels with decimal pixel accuracy within ± 1 pixel, it is possible to reduce the calculation amount required for search and the memory amount required for the reference image while realizing high-precision prediction with decimal pixel accuracy.

  The motion compensation prediction unit 7 always performs motion compensation prediction processing for the generation of a prediction image of a luminance signal, but the generation of a prediction image of a color difference signal is performed by the encoding control unit 1 for each prediction block. When the inter-color signal prediction flag of the prediction block is set to ON, the color that generates the prediction image of the color difference signal in the prediction block with reference to the encoded color signal without performing the motion compensation prediction process Perform inter-signal prediction processing. On the other hand, if the inter-color signal prediction flag of the prediction block is set to OFF by the encoding control unit 1, the color difference signal is also subjected to motion compensation prediction processing.

Here, the processing contents when the intra prediction unit 5, the intra block copy prediction unit 6, or the motion compensation prediction unit 7 performs the inter-color signal prediction process and generates the predicted image of the color difference signal will be specifically described. .
When the flag included in the inter-color signal prediction parameter is ON, the intra-color signal vector is not performed on the color difference signal without performing the intra prediction process, the intra block copy prediction process, and the motion compensation prediction process, as shown in FIG. The prediction image is generated with reference to the block of the encoded color difference signal indicated by.

FIG. 46 is an explanatory diagram showing an example of inter-color signal prediction when the input signal format is a YUV 4: 4: 4 format signal and the predicted image generation target block is 8 × 8 pixels.
The prediction image generation target block indicated by gray pixels in FIG. 46 and a plurality of pixels adjacent to the upper side and the left side of the encoded color signal block to be referred to A correlation parameter indicating the correlation between the pixel value and the pixel value of the predicted image of the color difference signal to be predicted is calculated, and the pixel value of the corresponding encoded color difference signal block corrected by the correlation parameter is predicted for the color difference signal. The pixel value of the image. The correction formula of the pixel value by the correlation parameter is as follows.
Correction formula: (pixel value of predicted image) = α × (pixel value of reference block) + β
In the correction formula, α and β indicate correlation parameters, and correction may be made with only one of α and β. Specifically, it is as follows.
(Pixel value of predicted image) = α × (pixel value of reference block)
Or
(Pixel value of predicted image) = (Pixel value of reference block) + β
By doing so, it is possible to reduce the amount of calculation required for calculating the correlation parameter.

However, the unit of the prediction image generation target block may be any of an encoding block unit, a prediction block unit, and a transform block unit. In general, prediction accuracy is improved by performing prediction in units of small blocks. However, since the amount of calculation required for correlation parameter calculation processing increases, it may be configured according to the calculation performance of the image encoding device to be configured. For example, when there is no constraint on the calculation time and complex calculations can be tolerated, high accuracy can be achieved by performing prediction image generation processing in units of prediction blocks or transform blocks that have a block size that is smaller than the block size of the coding block. Can be realized. For example, the intra prediction unit 5 can realize a highly accurate prediction by using a transform block as a unit for generating a predicted image, the intra block copy prediction unit 6 and the motion compensation prediction unit 7 as a prediction block.
In addition, in the intra block copy prediction unit 6 and the motion compensation prediction unit 7, when the unit of the prediction image generation target block is a prediction block unit, the pixels in the coding block are not coded. The correlation parameter is calculated with reference to pixels adjacent to the coding block to which each prediction block belongs. FIG. 47 shows an example of a prediction block in which an encoded block is divided into left and right parts. At this time, the correlation parameter of the prediction block is calculated using pixels adjacent to the encoded block indicated by gray pixels. Therefore, the correlation parameter of each prediction block in the encoded block is the same regardless of the division state of the prediction block.
Or you may make it calculate a correlation parameter using only the pixel adjacent to the said prediction block (prediction image production | generation object block) among the gray pixels of FIG. By doing in this way, since the correlation parameter is calculated only from the pixel adjacent to the prediction block and having high correlation with the pixel in the prediction block, the parameter calculation accuracy can be improved. Further, the number of gray pixels referred to in the parameter calculation process may always be a power of two. By doing in this way, the calculation formula of a correlation parameter can be simplified and the amount of calculation can be reduced. Note that all of the above-described correlation parameter calculation examples are powers of 2.

In the above example, the correlation parameter is calculated to correct the pixel value of the block of the encoded chrominance signal, but the pixel value of the block of the encoded chrominance signal is directly used as the predicted image without calculating the correlation parameter. The image encoding device may be configured to have a pixel value of. By doing in this way, the calculation process which calculates a correlation parameter becomes unnecessary, and the amount of calculations can be reduced.
When the input signal format is YUV4: 2: 0 format or YUV4: 2: 2 format, the block size is scaled so that the encoded color signal to be referenced has the same size as the prediction image generation target block. refer.
Specifically, in the YUV 4: 2: 0 format, when referring to the luminance (Y) signal, a luminance signal in which the vertical and horizontal sizes are reduced to ½ is used. Similarly, in the case of the YUV 4: 2: 2 format, when referring to the luminance (Y) signal, a luminance signal in which the horizontal size is reduced to ½ is used.
Further, the color signal vector may be limited to within ± M pixels (M: natural number). By limiting the search range in this way, the calculation amount required for the search can be reduced.
Moreover, the memory amount of a reference image can be reduced by limiting the reference image prepared also in the corresponding image decoding apparatus. At this time, decimal pixel accuracy may be prohibited. In this way, it is possible to reduce the amount of memory required for the reference image while further reducing the amount of calculation required for the search.
Alternatively, it may be limited to only pixels with decimal pixel accuracy. For example, by limiting to pixels with decimal pixel accuracy within ± 1 pixel, it is possible to reduce the calculation amount required for search and the memory amount required for the reference image while realizing high-precision prediction with decimal pixel accuracy.

In the above example, as the inter-color signal prediction process, the prediction image is generated with reference to the block of the encoded color signal indicated by the inter-color signal vector, but is always at the same position as the prediction image generation target block. You may comprise so that the block of this may be referred. In general, the blocks of the encoded color signal at the same position are likely to have the same pattern, and thus, it is not necessary to encode the inter-color signal vector while suppressing a decrease in prediction accuracy. Therefore, the code amount can be reduced.
When the encoded color signal to be referenced is encoded in the order of Y (luminance) signal, U signal, and V signal, the U signal refers to the Y signal, and the V signal refers to the Y signal or U signal. At this time, with respect to the V signal, it may be fixedly configured to refer to either the Y signal or the U signal in common to the image encoding device and the image decoding device (for example, the U signal is always referred to). The reference signal information may be encoded for each switching unit.
The unit of switching may be a sequence, a picture, a slice unit, or a block unit such as an encoded block, a prediction block, or a transform block. In general, the smaller the switching unit, the higher the prediction efficiency. On the other hand, the amount of code generated when encoding the reference signal information to be encoded can be suppressed by roughly setting the switching unit.

Further, when the reference signal can be switched in units of blocks, a flag indicating whether or not the switching processing in units of blocks is valid may be included in the sequence level header, picture level header, and slice level header. In this way, only when this switching process is effective, it is effective. When this switching process is not effective, it is not necessary to encode the reference signal information for each block, and the encoding efficiency is improved. Can be increased.
In the above example, the reference signal information has been described as information for determining whether to refer to the Y signal or the U signal, but the reference block is further determined by weighted averaging with reference to both the Y signal and the U signal. You may make it also include the case where it produces | generates. That is, for each set switching unit, a predicted image is generated with reference to only the Y signal, a predicted image is generated with reference to only the U signal, or a predicted image is referred to with reference to both the Y signal and the U signal. Select whether to generate. In this way, more accurate prediction can be realized.

In the inter-color signal prediction process, the presence or absence of execution is selected for each prediction block, and a flag indicating the presence or absence of the execution is encoded as an inter-color signal prediction parameter. The prediction block in which the inter-color signal prediction process is OFF is predicted by the same prediction unit (intra prediction unit 5, intra block copy prediction unit 6 or motion compensation prediction unit 7) as the luminance signal. Therefore, among the intra prediction parameters of the color difference signal of the intra prediction unit 5, only the intra prediction parameters of the prediction block that does not perform inter-color signal prediction are encoded.
Similarly, when the intra block copy prediction unit 6 or the motion compensation prediction unit 7 is configured to encode the difference vector for the color difference signal, the difference vector for the color difference signal of only the prediction block whose flag is OFF. Is encoded. However, when the image coding apparatus is configured so that the intra block copy prediction parameter or the inter prediction parameter is shared between the luminance signal and the color difference signal, the block shift vector or the motion vector is always the same for the luminance signal and the color difference signal. Therefore, the difference vector is not encoded. The corresponding image decoding apparatus is configured to use the intra block copy prediction parameter or the inter prediction parameter of the luminance signal also for the color difference signal. Alternatively, the color difference signal difference vector is always interpreted as zero.
Further, the intra prediction parameter of the color difference signal of the intra prediction unit 5 and the inter-color signal prediction flag may be combined and encoded as the intra prediction parameter of the color difference signal as shown in FIG. In this case, when the intra prediction parameter indicates the inter-color signal prediction process, the inter-color signal prediction process is performed. In other cases, the intra prediction is performed according to the intra prediction parameter.

In the above example, the presence / absence of execution of the inter-color signal prediction process is selected for each prediction block. However, the presence / absence of the execution may be selected for each coding block or each coding block having the maximum size. By doing so, the switching unit of the inter-color signal prediction becomes large and the prediction efficiency decreases, but the amount of code required for encoding the inter-color signal prediction flag can be reduced.
Further, the reference signal information of the encoded color signal and the inter-color signal prediction flag may be one information. That is, the reference signal information may include a case where the inter-color signal prediction process is not performed (the encoded color signal is not referred to).

Subtracting unit 8 receives the encoded block ^{B n} from the block dividing unit 3, from its prediction block _P ^{i n} the coded block ^{B n,} the intra prediction image _{P INTRAi} ⁿ generated by the intra prediction unit 5, the intra block copy prediction unit 6 intra block copy predicted image P _{ICOPYi n} generated ^by, or subtracts one of the inter-prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 7, which is the subtraction result difference image the prediction difference signal _e ^{i n} that indicates the output to transform and quantization unit 9 (step ST7).

When the transform / quantization unit 9 receives the prediction difference signal e _i ⁿ from the subtraction unit 8, the transform / quantization unit 9 refers to the transform block division information included in the prediction difference encoding parameter determined by the encoding control unit 1, and performs the prediction. orthogonal transform processing with respect to the difference signal e _i ⁿ (e.g., DCT (discrete cosine transform) or DST (discrete sine transform), the orthogonal transform for KL conversion and the base design have been made in advance to the particular learning sequence) to transform This is performed for each block, and a conversion coefficient is calculated.
Further, the transform / quantization unit 9 refers to the quantization parameter included in the prediction differential encoding parameter, quantizes the transform coefficient of the transform block unit, and reverses the compressed data that is the transform coefficient after quantization. It outputs to the quantization / inverse transform unit 10 and the variable length coding unit 15 (step ST8). At this time, the quantization process may be performed using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient.

As the quantization matrix, an independent matrix can be used for each color signal and coding mode (intra coding or inter coding) at each orthogonal transform size. As an initial value, an image coding device and an image are used. In the decoding apparatus, it is possible to select whether to use a previously prepared quantization matrix or an already encoded quantization matrix or to use a new quantization matrix.
Accordingly, the transform / quantization unit 9 sets a flag indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, as a quantization matrix parameter to be encoded.
Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 10 is set as a quantization matrix parameter to be encoded.
On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance in the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Therefore, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, only the quantization matrix prepared in advance can be selected in advance in the image encoding device and the image decoding device.
Then, the transform / quantization unit 9 outputs the set quantization matrix parameter to the variable length coding unit 15.

When the inverse quantization / inverse transform unit 10 receives the compressed data from the transform / quantization unit 9, the inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 Then, the compressed data is inversely quantized for each transform block.
When the transform / quantization unit 9 uses a quantization matrix for the quantization process, the corresponding inverse quantization process is performed with reference to the quantization matrix even during the inverse quantization process.
Further, the inverse quantization / inverse transform unit 10 performs inverse orthogonal transform processing (for example, inverse DCT, inverse DST, inverse KL transform, etc.) on transform coefficients that are compressed data after inverse quantization for each transform block. Then, a local decoded prediction difference signal corresponding to the prediction difference signal e _i ⁿ output from the subtraction unit 8 is calculated and output to the addition unit 11 (step ST9).

Addition unit 11, when the inverse quantization and inverse transform unit 10 receives the local decoded prediction difference signal, and the local decoded prediction difference signal, an intra prediction image P _{INTRAi n} generated by the intra prediction unit ^5, the intra block copy prediction part 6 intra block copy predicted image _{P ICOPYi} ⁿ generated by, or by adding one of the inter-prediction image _{P INTERi} ⁿ generated by the motion compensation prediction unit 7 calculates a local decoded image (step ST10 ).
The adding unit 11 outputs the locally decoded image to the loop filter unit 13 and stores the locally decoded image in the intra memory 12.
This locally decoded image becomes an encoded image signal used in the subsequent intra prediction process and intra block copy prediction process.

When the loop filter unit 13 receives the local decoded image from the adder unit 11, the loop filter unit 13 performs a predetermined filter process on the local decoded image, and stores the filtered local decoded image in the motion compensated prediction frame memory 14. (Step ST11).
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

However, the loop filter unit 13 determines whether or not to perform processing for each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and sets a valid flag of each processing as a part of the sequence level header and It outputs to the variable length encoding part 15 as a part of slice level header. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 11 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
Generally, the more types of filter processing that are used, the better the image quality, but the higher the processing load. That is, image quality and processing load are in a trade-off relationship. In addition, the image quality improvement effect of each filter process varies depending on the characteristics of the image to be filtered. Therefore, the filter processing to be used may be determined according to the processing load allowed by the image encoding device and the characteristics of the encoding target image.

  Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.

In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are provided. One classification method is selected from the methods.
Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating for the coding distortion is calculated for each class.
Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.

As a classifying method, a method of classifying by the size of a pixel value of a locally decoded image (referred to as a BO method), or a classification according to a situation around each pixel (whether it is an edge portion or the like) for each edge direction. There is a technique (referred to as EO technique).
These methods are prepared in advance by the image encoding device and the image decoding device in advance. For example, as shown in FIG. 14, when no offset processing is performed, these methods are defined as one of the class classification methods. Of the methods, an index indicating which method is used for class classification is selected for each block.

Therefore, the pixel adaptive offset processing outputs the block division information, the index indicating the class classification method for each block, and the offset information for each block to the variable length encoding unit 15 as header information.
In the pixel adaptive offset processing, the adaptive offset processing may be performed for each class by always dividing the block into fixed-size blocks such as a maximum coding block and selecting a class classification method for each block. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

Also, in adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for superimposed distortion is designed for each region (local decoded image) belonging to each class. Then, the local decoded image is filtered.
Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.
Here, as a class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of the image in units of blocks. Further, the number of classes used in the adaptive filter process may be set in advance to a common value in the image encoding device and the image decoding device, or may be one of the parameters to be encoded.
Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

The processes of steps ST3 to ST10 are repeatedly performed until the processes for all the coding blocks ^Bn divided hierarchically are completed, and when the processes for all the coding blocks ^Bn are completed, the process proceeds to the process of step ST14. (Steps ST12 and ST13).

The variable length encoding unit 15 uses the compressed data output from the transform / quantization unit 9 and the block division information (FIG. 6B) in the maximum encoded block output from the encoding control unit 1 as an example. (Quadrant tree information), coding mode m (B ⁿ ), prediction differential coding parameter, intra prediction parameter output from coding control unit 1 (when coding mode is intra coding mode), intra block A copy prediction parameter (when the coding mode is an intra block copy coding mode) or an inter prediction parameter (when the coding mode is an inter coding mode), and an inter-color signal prediction parameter including an inter-color signal prediction flag; , Variable-length coding the motion vector output from the motion compensation prediction unit 7 (when the coding mode is the inter coding mode) Generating encoded data indicating the encoding result (step ST14).

At this time, as a method of encoding compressed data that is a quantized orthogonal transform coefficient, the transform block is further divided into blocks of 4 × 4 pixel units (encoding sub-blocks) called Coefficient Group (CG), and CG Coding of coefficients is performed for each unit.
FIG. 15 shows the coding order (scan order) of coefficients in a 16 × 16 pixel transform block.
In this way, 16 CGs in units of 4 × 4 pixels are encoded in order from the lower right CG, and each CG encodes 16 coefficients in the CG in order from the lower right coefficient.

Specifically, first, a flag indicating whether or not a significant (nonzero) coefficient exists in 16 coefficients in the CG is encoded, and then a significant (nonzero) coefficient exists in the CG. Only in the case, whether each coefficient in the CG is a significant (non-zero) coefficient is encoded in the above order, and finally, the coefficient value information is encoded in order for the significant (non-zero) coefficient. This is performed in the above order in units of CG.
In this case, the encoding efficiency by entropy encoding can be increased by using a biased scan order so that significant (non-zero) coefficients are generated as continuously as possible.
Since the coefficient after orthogonal transformation represents the lower coefficient of the lower frequency component as it is closer to the upper left, starting with the DC component located at the upper left, generally, the closer to the upper left, the more significant as the example is shown in FIG. Since many (non-zero) coefficients are generated, as shown in FIG. 15, encoding can be performed efficiently by encoding sequentially from the lower right.
In the above description, a 16 × 16 pixel conversion block has been described. However, in a block size other than 16 × 16 pixels, such as an 8 × 8 pixel conversion block or a 32 × 32 pixel conversion block, a code in CG (encoding sub-block) unit is used. It shall be implemented.
Further, the 4 × 4 pixel and 8 × 8 pixel conversion blocks for which intra prediction is selected are processed in the scan order shown in FIG. 19 instead of the scan order shown in FIG. 15 according to the intra prediction mode index. . This is because the frequency component distribution of the prediction difference signal tends to be different depending on the direction of intra prediction.

Further, as illustrated in FIG. 13, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data.
However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.

The sequence level header includes the image size, the color signal format, the bit depth of the signal value of the luminance signal and the color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. In general, header information that is common to each sequence unit, such as a valid flag of () and a valid flag of a quantization matrix, is collected.
The picture level header is a collection of header information set in units of pictures such as an index of a sequence level header to be referenced, the number of reference pictures at the time of motion compensation, an entropy encoding probability table initialization flag, and the like.
The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a summary of parameters in units of slices such as a flag indicating whether or not each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the filter unit 13 is performed.

Each header information and picture data is identified by a NAL unit.
Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded.
If supplementary information also exists, it is defined as a unique NAL unit. The picture data is defined as an access unit, and indicates a unit of data access of each picture in which NAL units related to the picture such as a NAL unit indicating slice data and a NAL unit indicating supplementary information are collected.

Next, the intra prediction process in the intra prediction unit 5 will be described in detail.
The intra prediction unit 5, as described above, with reference to the intra prediction parameters of the prediction block P _i ^n, to implement intra prediction processing for the prediction block P _i ^n, but to generate an intra prediction image P _INTRAi ⁿ , will be described here intra process for generating an intra prediction image predicted block P _i ⁿ in the luminance signal.

Figure 7 is an explanatory diagram showing an example of the prediction block P _i ^n-selectable intra prediction modes for intra-coded blocks B ^n, and the index value of the intra prediction mode, the prediction direction vector indicated by the intra-prediction mode Show. The index value of the intra prediction mode indicates the intra prediction parameter.
In addition, you may comprise so that the number of intra prediction modes may differ according to the size of the block used as a process target.
Intra-prediction efficiency decreases for large-sized blocks, so the number of selectable intra-prediction directions is reduced, and for small-sized blocks, the number of selectable intra-prediction directions is increased to reduce the amount of computation. can do.

First, since the process which produces | generates an intra estimated image uses the encoded pixel adjacent to the block of a process target, as above-mentioned, it must be performed per conversion block.
Here, the transform block that generates the intra predicted image is referred to as a predicted image generation block. Therefore, the intra prediction unit 5 may implement an intra-prediction image generation processing described below to the predicted image generation block generates an intra prediction image predicted block P _i ^n.
The size of the predicted image generation block is assumed to be l _i ⁿ × m _i ⁿ pixels.
FIG. 8 is an explanatory diagram illustrating an example of a pixel used when generating a predicted value of a pixel in a predicted image generation block in the case of l _i ⁿ = m _i ⁿ = 4.
In FIG. 8, the encoded pixels (2 × l _i ⁿ +1) and the left encoded pixels (2 × m _i ⁿ ) on the predicted image generation block are used as pixels for prediction. The number of pixels used for prediction may be more or less than the pixels shown in FIG.
Further, in FIG. 8, pixels for one row or one column in the vicinity of the predicted image generation block are used for prediction, but two or two or more pixels may be used for prediction.

If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs is 0 (plane (Planar) prediction) includes a coded pixels adjacent to the top of the predicted image generation block, the predicted image generation Using the encoded pixels adjacent to the left of the block, a predicted image is generated using a value interpolated according to the distance between these pixels and the prediction target pixel in the predicted image generation block as a predicted value.

・領域Ａ（Ｐ_ｉ ^ｎの左上の画素）
ａ_０＝１／２，ａ_１＝１／４，ａ_２＝１／４
・領域Ｂ（領域Ａ以外のＰ_ｉ ^ｎの上端の画素）
ａ_０＝３／４，ａ_２＝１／４，（ａ_１＝０）
・領域Ｃ（領域Ａ以外のＰ_ｉ ^ｎの左端の画素）
ａ_０＝３／４，ａ_１＝１／４，（ａ_２＝０） When the index value of the intra prediction mode for the prediction block P _i ⁿ to which the prediction image generation block belongs is 1 (average (DC) prediction), encoded pixels adjacent to the prediction image generation block and the prediction image A predicted image is generated using the average value of the encoded pixels adjacent to the left of the generated block as the predicted value of the pixels in the predicted image generating block.
Further, the region A, B, C in FIG. 17 located at the upper end and the left end of the predicted image generation block is subjected to filter processing for smoothing the block boundary to obtain a final predicted image. For example, according to the following formula (1), the filter processing is performed using the following filter coefficients in the reference pixel arrangement of the filter of FIG.

· Area A _(the upper left pixel of the _P ^{i n)}
a ₀ = 1/2, a ₁ = ¼, a ₂ = ¼
- region B (the upper end of the pixel of _P ^{i n} other than the region A)
a ₀ = 3/4, a ₂ = ¼, (a ₁ = 0)
· Area C (the leftmost pixel of the _P ^{i n} other than the region A)
a ₀ = 3/4, a ₁ = ¼, (a ₂ = 0)

In equation (1), a _n (n = 0, 1, 2) is a filter coefficient applied to the reference pixel, and p _n (n = 0, 1, 2) is a reference to a filter including the pixel to be filtered p ₀ . The pixel, S ′ (p ₀ ) is the predicted value after the filtering process in the filtering target pixel p ₀ , and S (p _n ) (n = 0, 1, 2) is the filter of the reference pixel including the filtering target pixel p ₀ It represents the predicted value before processing.

Furthermore, the block size of the predicted image generation block that performs the filtering process may be limited.
In general, when the prediction value is changed by filtering only at the block edge, the prediction difference caused by this change in the prediction value is small because the proportion of the area where the prediction value changes due to the filter processing is small in the block having a large block size. The change in the signal is represented by a very high frequency component, and the encoding efficiency tends to be deteriorated in order to encode the high frequency component. Also, by giving priority to encoding efficiency and not encoding this high-frequency component, the change in the prediction difference signal at the block end cannot be restored, and there is a tendency that the block boundary is distorted.

On the other hand, in a block having a small block size, since the ratio of the area where the prediction value changes due to the filter processing is large, the change in the prediction difference signal caused by the change in the prediction value is as in the case of a block having a large block size. The prediction difference signal can be appropriately encoded without being represented by a high frequency component, and the quality of the decoded image can be improved by the increase in the continuity of the block boundary by this filter processing.
Therefore, for example, in a predicted image generation block having a block size of 32 × 32 pixels or more, the above-described filter processing is not applied, and the above-described filter processing is applied only to blocks smaller than 32 × 32 pixels, thereby obtaining a conventional average value. It is possible to suppress an increase in the calculation amount while improving the prediction performance rather than the prediction.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図９を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の画素値（復号された画素値）である。また、算出した予測値が画素値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ belonging to the prediction image generation block 26 (vertical prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (2) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 9) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is a pixel value (decoded pixel value) of an encoded pixel at coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the pixel value can take, the value is rounded so that the predicted value falls within the range.

  Note that the equation in the first line of equation (2) is MPEG-4 AVC / H. The amount of change S (−1, y) −S (−1, −1) in the vertical direction of adjacent encoded pixels with respect to S (x, −1), which is the predicted value of the vertical direction prediction in H.264. ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (2) is , MPEG-4 AVC / H. The same prediction formula as the vertical direction prediction in H.264 is shown.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図９を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の画素値（復号された画素値）である。また、算出した予測値が画素値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs 10 (horizontal prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (3) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 9) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is a pixel value (decoded pixel value) of an encoded pixel at coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the pixel value can take, the value is rounded so that the predicted value falls within the range.

  Note that the expression on the first line of Expression (3) is MPEG-4 AVC / H. The amount of change S (x, −1) −S (−1, −1) in the horizontal direction of adjacent encoded pixels with respect to S (−1, y), which is the predicted value of the horizontal direction prediction in H.264. ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (3) is , MPEG-4 AVC / H. The same prediction formula as the horizontal prediction in H.264 is shown.

However, the block size of the predicted image generation block that performs the vertical direction prediction of Expression (2) and the horizontal direction prediction of Expression (3) may be limited.
In general, when the prediction value is changed by performing a filter process that adds a value proportional to the amount of change in the pixel value in the prediction direction only at the block edge, the block edge filter of the predicted image generation block described above is used in a large block size block. Since the ratio of the region where the predicted value changes due to processing is small, the change in the prediction difference signal caused by the change in the predicted value is represented by a very high frequency component, and this high frequency component is encoded. Therefore, there is a tendency that the encoding efficiency is deteriorated. Also, by giving priority to encoding efficiency and not encoding this high-frequency component, there is a tendency that the change in the prediction difference signal at the block end cannot be restored and distortion occurs at the block boundary.

On the other hand, in a block with a small block size, since the ratio of the region where the prediction value changes due to the above filter processing is large, the change in the prediction difference signal caused by the change in the prediction value is as in the case of a block with a large block size. Therefore, the prediction difference signal can be appropriately encoded, and the quality of the decoded image can be improved by increasing the continuity of the block boundary by this filter processing.
Therefore, for example, in a prediction image generation block having a block size of 32 × 32 pixels or more, the expressions in the second row of Expression (2) and Expression (3) are always used regardless of the coordinates of the prediction target pixel (prediction). By applying the equations (2) and (3) for performing the above-described filter processing only to blocks smaller than 32 × 32 pixels, the filter edge of the image generation block is not performed. It is possible to suppress an increase in the calculation amount while improving the prediction performance compared to the direction prediction and the horizontal direction prediction.

ただし、ｋは負の実数である。 When the index value in the intra prediction mode is other than 0 (plane prediction), 1 (average value prediction), 26 (vertical direction prediction), and 10 (horizontal direction prediction), the prediction direction vector υ _p = ( Based on (dx, dy), a predicted value of a pixel in the predicted image generation block is generated.
As shown in FIG. 9, when the upper left pixel of the predicted image generation block is the origin and the relative coordinates in the predicted image generation block are set to (x, y), the position of the reference pixel used for prediction is adjacent to the following L This is the intersection of pixels.

However, k is a negative real number.

When the reference pixel is at the integer pixel position, the integer pixel is set as the prediction value of the prediction target pixel. When the reference pixel is not at the integer pixel position, an interpolation pixel generated from the integer pixel adjacent to the reference pixel is selected. Estimated value.
In the example of FIG. 8, since the reference pixel is not located at the integer pixel position, a value interpolated from two pixels adjacent to the reference pixel is set as the predicted value. Note that an interpolation pixel may be generated not only from two adjacent pixels but also from two or more adjacent pixels, and used as a predicted value.
While increasing the number of pixels used in the interpolation process has the effect of improving the interpolation accuracy of the interpolated pixels, it increases the complexity of the calculation required for the interpolation process, requiring high coding performance even when the calculation load is large. In the case of an image encoding device, it is better to generate interpolation pixels from a larger number of pixels.

The processing described above, to generate a predicted pixel for all the pixels of the luminance signals of the prediction block P _i ⁿ in the predicted image generation block, and outputs an intra prediction image P _INTRAi ^n.
Incidentally, the intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ (intra prediction mode) is output to the variable length coding unit 15 for multiplexing the bitstream.

Note that the MPEG-4 AVC / H. Similarly to the smoothing process performed on the reference pixels at the time of intra prediction of the 8 × 8 pixel block in H.264, the intra prediction unit 5 predicts the reference pixels when generating the predicted image of the predicted image generation block. Even when the encoded pixels adjacent to the image generation block are configured to be the smoothed pixels, it is possible to perform the same filtering process on the predicted image as in the above example. By doing in this way, the noise of the reference pixel by the filter process to a reference pixel is removed, and prediction accuracy can be improved by performing prediction using this.
Alternatively, the filtering process on the reference pixel may be performed only in the prediction other than the average value prediction, the vertical direction prediction, and the horizontal direction prediction for performing the filtering process on the predicted image. By doing in this way, it is only necessary to perform at most one filter process for each prediction mode, and an increase in the amount of calculation can be suppressed.

In the above description, the predicted image generation process of the luminance signal has been described, but the predicted image for the color difference component is generated as follows.
The color difference signal of the prediction block P _i ^n, conduct intra prediction processing based on the intra prediction parameter of the color difference signal (intra prediction mode), the variable length coding unit intra prediction parameter used to generate the intra-prediction image 15 is output.

FIG. 23 is an explanatory diagram showing an example of correspondence between intra prediction parameters (index values) of color difference signals and color difference intra prediction modes.
When the intra prediction parameter of the chrominance signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (when the intra prediction parameter indicates the luminance / chrominance common intra prediction mode (DM mode)), the luminance signal The same intra-frame prediction is performed to generate a prediction image of the color difference signal.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal.

When the input signal format is a YUV 4: 2: 2 signal, as shown in FIG. 26, if the luminance signal is a square block, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 27, when a YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, YUV4: 2 is used to predict the luminance signal and the color difference signal in the same direction. : On two signals, in the case of directional prediction other than the vertical prediction and the horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal.
Specifically, as illustrated in FIG. 28, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 29, assuming that the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an intra prediction mode index used for prediction of the color difference signal, and the color difference signal prediction process is performed in the intra prediction mode corresponding to the converted index.
FIG. 30 shows an example of conversion of the intra prediction mode index in the intra prediction mode of FIG.
In the conversion table of FIG. 30, when the angle of the prediction direction is θ (see FIG. 29), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to.
As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted.
With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. It is good also as the prediction method similar to H.264. By not performing the filtering process in this way, it is possible to reduce the prediction process.

Next, the processing content of the image decoding apparatus in FIG. 3 will be specifically described.
When the encoded bit stream generated by the image encoding device in FIG. 1 is input, the variable length decoding unit 31 performs a variable length decoding process on the bit stream (step ST21 in FIG. 4), and more than one frame. Each header information and picture data such as header information (sequence level header) in units of a sequence composed of pictures and header information (picture level header) in units of pictures are decoded. However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.

At this time, when the valid flag of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding on the quantization matrix parameter to identify the quantization matrix.
Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, and a quantization matrix prepared in advance by the image encoding device and the image decoding device, or When indicating that the quantization matrix is already decoded (not a new quantization matrix), refer to the index information for specifying which quantization matrix among the matrices included in the quantization matrix parameter. When the quantization matrix is specified, and the quantization matrix parameter indicates that a new quantization matrix is to be used, it is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter.
Then, slice unit header information (slice level header) such as slice division information is decoded from slice data constituting picture unit data, and encoded data of each slice is decoded.

In addition, the variable length decoding unit 31 specifies the maximum encoding block size and the upper limit of the number of division layers from the header information (step ST22).
However, when the minimum block size of the encoded block is encoded instead of the upper limit of the number of division layers, the upper limit of the number of division layers is determined by decoding this. That is, the upper limit of the number of division layers is obtained when the maximum encoded block is divided to the minimum block size.
The variable length decoding unit 31 decodes the division state of the maximum encoded block as shown in FIG. 6 for each determined maximum encoded block. Based on the decoded division state, coding blocks are identified hierarchically (step ST23).

  Next, the variable length decoding unit 31 decodes the encoding mode assigned to the encoding block. Based on the information included in the decoded coding mode, the coded block is further divided into one or more prediction blocks which are prediction processing units, and the prediction parameters assigned to the prediction block units are decoded (step ST24). .

  That is, when the encoding mode assigned to the encoding block is the intra encoding mode, the variable length decoding unit 31 is included in the encoding block, and each of one or more prediction blocks serving as a prediction processing unit When the intra prediction parameter is decoded and the coding mode assigned to the coding block is the intra block copy coding mode, one or more prediction blocks included in the coding block and serving as a prediction processing unit The intra block copy prediction parameter is decoded every time. Further, when the coding mode assigned to the coding block is the inter coding mode, the inter prediction parameter and the motion vector are included in the coding block and each of one or more prediction blocks serving as a prediction processing unit. Is decrypted. Further, the inter-color signal prediction parameters including the inter-color signal prediction flag are decoded (step ST24).

Further, the variable length decoding unit 31 decodes the compressed data (transformed / transformed transform coefficients) for each transform block based on the transform block division information included in the prediction difference coding parameter (step ST24).
At that time, similarly to the encoding process of the compressed data in the variable length encoding unit 15 of the image encoding apparatus of FIG.
Therefore, as shown in FIG. 15, 16 CGs in units of 4 × 4 pixels are decoded in order from the lower right CG, and each CG decodes 16 coefficients in the CG in order from the lower right coefficient. Will do.

Specifically, first, a flag indicating whether or not a significant (non-zero) coefficient exists in 16 coefficients in the CG is decoded, and then the decoded flag is a significant (non-zero) coefficient in the CG. Only when it is shown that the coefficient is present, whether each coefficient in the CG is a significant (non-zero) coefficient is decoded in the above order, and finally, the coefficient value information for the coefficient indicating the significant (non-zero) coefficient is sequentially Decrypt. This is performed in the above order in units of CG.
However, with regard to the scan order, in the case of a 4 × 4 pixel and 8 × 8 pixel conversion block for which intra prediction is selected, the scan shown in FIG. 19 is used instead of the scan order of FIG. 15 according to the index of the intra prediction mode. Process in order.

If the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is an intra-encoding mode (when m (B ⁿ ) ∈INTRA), the changeover switch 33 is changed by the variable-length decoding unit 31. The intra-prediction parameter for each prediction block subjected to variable-length decoding is output to the intra-prediction unit 34, and the coding mode m (B ⁿ ) subjected to variable-length decoding by the variable-length decoding unit 31 is the intra-block copy coding mode. (In the case of m (B ⁿ ) εICOPY), the intra block copy prediction parameter of the prediction block unit variable length decoded by the variable length decoding unit 31 is output to the intra block copy prediction unit 35 and variable by the variable length decoding unit 31 If the length decoded coding mode m ^{(B n)} is an inter coding mode (m (the case of ^{B n)} ∈INTER), variable recovery And it outputs the inter prediction parameter and the motion vector variable length decoding prediction block to the motion compensation prediction unit 36 by the part 31.

When the coding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the intra coding mode (m (B ⁿ ) ∈INTRA) (step ST25), the intra prediction unit 34 selects the changeover switch 33. 1 is received, and the intra prediction parameter is used while referring to the decoded image stored in the intra memory 38 in the same procedure as the intra prediction unit 5 in FIG. intra prediction processing for each of the prediction block _P ^{i n} the stomach coded block ^{B n} to implement, generates an intra prediction image _{P INTRAi} ⁿ (step ST26).

In addition, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal to generate a prediction image of the luminance signal.
On the other hand, for the color difference signal, if the inter-color signal prediction flag is OFF (indicating non-execution of the inter-color signal prediction process), the intra-prediction process based on the intra prediction parameter of the color difference signal is performed. A predicted image of the signal is generated.

FIG. 23 is an explanatory diagram showing an example of correspondence between intra prediction parameters (index values) of color difference signals and color difference intra prediction modes.
When the intra prediction parameter of the chrominance signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (when the intra prediction parameter indicates the luminance / chrominance common intra prediction mode (DM mode)), the luminance signal The same intra-frame prediction is performed to generate a prediction image of the color difference signal.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal.

When the input signal format is a YUV 4: 2: 2 signal, as shown in FIG. 26, if the luminance signal is a square block, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 27, when a YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, YUV4: 2 is used to predict the luminance signal and the color difference signal in the same direction. : On two signals, in the case of directional prediction other than the vertical prediction and the horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal.
Specifically, as illustrated in FIG. 28, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 29, assuming that the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an intra prediction mode index used for prediction of the color difference signal, and the color difference signal prediction process is performed in the intra prediction mode corresponding to the converted index.
FIG. 30 shows an example of conversion of the intra prediction mode index in the intra prediction mode of FIG.
In the conversion table of FIG. 30, when the angle of the prediction direction is θ (see FIG. 29), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to.
As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted.
With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. When an image encoding apparatus is configured as a prediction method similar to H.264, the image decoding apparatus has the same configuration so that an encoded bit stream generated from the image encoding apparatus can be decoded.
By not performing the filtering process in this way, it is possible to reduce the prediction process.

  The intra prediction unit 34 always performs an intra prediction process for generating a predicted image of a luminance signal, but for generating a predicted image of a chrominance signal, for each predicted block, an inter-color signal prediction flag of the predicted block. Is ON (when the execution of inter-color signal prediction processing is clearly specified), the prediction image of the color difference signal in the prediction block is generated by referring to the decoded color signal without performing the intra prediction processing. The inter-color signal prediction process is performed.

The intra block copy prediction unit 35, when the encoding mode m (B ⁿ ) that has been variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode (m (B ⁿ ) εICOPY) (step ST25). The intra block copy prediction parameter including the block shift vector of the prediction block unit output from the changeover switch 33 is received, and the intra block copy prediction parameter is used while referring to the decoded image stored in the intra memory 38. There was then conducted intra block copy prediction processing for each of the prediction block _P ^{i n} the coded block ^{B n,} generates an intra block copy predicted image _{P ICOPYi} ⁿ (step ST27).

In the image encoding device of FIG. 1, the block shift vector is the difference between the block shift vector of the previous encoded (decoded) predicted block or the block shift vector of the encoded (decoded) predicted block around the predicted block. When the value is encoded as a part of the intra block copy prediction parameter, the difference value included in the intra block copy prediction parameter and the block shift vector are added to calculate the block shift vector of the prediction block. .
At this time, the pixels in the region before decoding (undecoded region) in the maximum coding block to be decoded are set to the image of FIG. 1 so that the block indicated by the block shift vector can be referred to beyond the decoded region. A process of filling with pixel values determined by the same method as the intra block copy prediction unit 6 in the encoding apparatus is performed. At this time, when a parameter necessary for the process of filling the undecoded area with the pixel value is included in the bitstream as a part of the intra block copy prediction parameter, the parameter decoded by the variable length decoding unit 31 is used. Then, a process of filling the undecoded area with the pixel value is performed.
For the color difference signal, prediction using the same vector as the luminance signal is performed. Alternatively, when the image encoding apparatus of FIG. 1 is configured to search based on the luminance signal vector and encode the difference vector of the luminance signal vector, the variable length decoding unit 31 decodes the difference vector. To calculate a color difference signal vector.

  The intra block copy prediction unit 35 always performs an intra block copy prediction process for the generation of a prediction image of a luminance signal, but for the generation of a prediction image of a color difference signal, the color of the prediction block is generated for each prediction block. When the inter-signal prediction flag is ON (when the execution of the inter-color signal prediction process is specified), the color difference in the prediction block is referred to by referring to the decoded color signal without performing the intra block copy prediction process. An inter-color signal prediction process for generating a predicted image of the signal is performed.

When the coding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the inter coding mode (m (B ⁿ ) εINTER) (step ST25), the motion compensation prediction unit 36 performs the changeover switch. The motion vector and the inter prediction parameter of the prediction block unit output from 33 are received, and the motion vector and the inter prediction parameter are used while referring to the decoded image stored in the motion compensated prediction frame memory 40. and performing motion compensation prediction process generates an inter prediction image _{P INTERi} ⁿ for each of the prediction block _P ^{i n} the stomach coded block ^{B n} (step ST28).
For the color difference signal, prediction using the same vector as the luminance signal is performed. Alternatively, when the image encoding apparatus of FIG. 1 is configured to search based on the luminance signal vector and encode the difference vector of the luminance signal vector, the variable length decoding unit 31 decodes the difference vector. To calculate a color difference signal vector.

  The motion compensation prediction unit 36 always performs motion compensation prediction processing for generating a prediction image of a luminance signal, but for generating a prediction image of a color difference signal, for each prediction block, between the color signals of the prediction block. When the prediction flag is ON (when the execution of the inter-color signal prediction process is clearly specified), the prediction of the color difference signal in the prediction block is performed by referring to the decoded color signal without performing the motion compensation prediction process. An inter-color signal prediction process for generating an image is performed.

Here, the processing contents when the intra prediction unit 34, the intra block copy prediction unit 35, or the motion compensation prediction unit 36 performs the inter-color signal prediction process to generate the prediction image of the color difference signal will be specifically described. .
When the inter-color signal prediction flag included in the inter-color signal prediction parameter is ON, the color difference signal does not perform the intra prediction process, the intra block copy prediction process, and the motion compensation prediction process, as shown in FIG. A predicted image is generated with reference to the block of the decoded color signal indicated by the inter-color signal vector.
At this time, the correlation parameter indicating the correlation between the pixel value of the block of the decoded color signal to be referenced and the pixel value of the prediction image of the color difference signal to be predicted is calculated by the same method as the image encoding device of FIG. The pixel value of the block of the corresponding decoded color signal corrected by the correlation parameter is set as the pixel value of the predicted image of the color difference signal to be predicted. The unit of the prediction image generation target block is the same unit as the image encoding device (any one of the encoding block unit, the prediction block unit, and the transform block unit).

In the above example, the correlation parameter is calculated and the pixel value of the block of the decoded color signal is corrected. However, the image encoding apparatus in FIG. When the pixel value is used as the pixel value of the predicted image as it is, the image decoding apparatus directly uses the pixel value of the block of the decoded color signal as the pixel value of the predicted image without calculating the correlation parameter.
When the input signal format is YUV4: 2: 0 format or YUV4: 2: 2 format, the decoded color signal to be referenced is the same size as the predicted image generation target block, as in the image encoding device of FIG. The block size is scaled so that

The inter-color signal prediction parameters that have been variable-length decoded by the variable-length decoding unit 31 include information indicating whether or not inter-color signal prediction is effective, information on a sequence, a picture, and a slice level, and execution of inter-color signal prediction processing in units of prediction blocks. It includes an inter-color signal prediction flag indicating the presence or absence.
A block for which the inter-color signal prediction process is OFF is predicted by the same prediction unit (intra prediction unit 34, intra block copy prediction unit 35, or motion compensation prediction unit 36) as the luminance signal. Therefore, only the intra prediction parameters of the prediction block that does not perform inter-color signal prediction among the intra prediction parameters of the color difference signal of the intra prediction unit 34 are decoded, and the intra prediction process is performed according to the intra prediction parameters.
Similarly, when the image encoding apparatus in FIG. 1 is configured to encode the difference vector for the color difference signal, the intra block copy prediction unit 35 or the motion only when the inter-color signal prediction flag is OFF. The difference vector for the color difference signal used by the compensation prediction unit 36 is decoded.

In addition, when the image coding apparatus in FIG. 1 combines the intra prediction parameter of the color difference signal and the inter-color signal prediction flag of the intra prediction unit 34 and encodes it as the intra prediction parameter of the color difference signal in FIG. The intra prediction unit 34 of the apparatus determines whether to perform inter-color signal prediction or intra prediction from the intra prediction parameters of the color difference signals variable-length decoded by the variable-length decoding unit 31.
In the above example, the case where the presence / absence of the execution of the inter-color signal prediction process is selected for each prediction block has been described. However, the image coding apparatus in FIG. 1 performs the coding block unit or the maximum size coding block unit. When configured to perform selection, the image decoding apparatus is configured to decode the inter-color signal prediction flag in the same block unit. In this way, the encoding parameter can be correctly decoded.
Further, when the inter-color signal prediction flag is ON as the inter-color signal prediction parameter that has been variable-length decoded by the variable-length decoding unit 31, the variable-length decoding unit 31 uses the inter-color signal vector indicating the block position of the reference signal. The reference signal information indicating which color signal is referred to is decoded. However, when the image encoding apparatus of FIG. 1 is configured not to use the inter-color signal vector and the reference signal information, the image decoding apparatus does not decode these parameters and is the same as the image encoding apparatus. The prediction image is generated with reference to the block at the same position of the reference signal.

When receiving the compressed data and the prediction difference encoding parameter from the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32 performs the prediction difference encoding in the same procedure as the inverse quantization / inverse conversion unit 10 of FIG. With reference to the quantization parameter and transform block division information included in the parameters, the compressed data is inversely quantized in transform block units.
At this time, when referring to each header information variable-length decoded by the variable-length decoding unit 31, each header information indicates that the inverse quantization process is performed using the quantization matrix in the slice. Inverse quantization processing is performed using a quantization matrix.

At this time, referring to each header information variable length decoded by the variable length decoding unit 31, for each color signal and coding mode (intra coding, intra block copy coding, inter coding) with each orthogonal transform size Specify the quantization matrix to be used.
Further, the inverse quantization / inverse transform unit 32 performs inverse orthogonal transform processing on transform coefficients that are compressed data after inverse quantization in units of transform blocks, and outputs from the inverse quantization / inverse transform unit 10 in FIG. The same decoded prediction difference signal as the local decoding prediction difference signal thus obtained is calculated (step ST29).

Addition unit 37, an intra block generated by the decoded prediction difference signal calculated by the inverse quantization and inverse transform unit 32, an intra prediction image P _{INTRAi ^n,} intra block copy prediction unit 35 generated by the intra prediction unit 34 copy predicted image _{P ICOPYi} ^n, or, together with by adding one of generated by the motion compensation prediction unit 36 inter-prediction image _{P INTERi} ⁿ calculates a decoded image, and outputs the decoded image to the loop filter unit 39, The decoded image is stored in the intra memory 38 (step ST30).
This decoded image becomes a decoded image signal used in the subsequent intra prediction processing and intra block copy prediction processing.

Loop filter unit 39, the process of step ST23~ST30 of all the coding blocks ^{B n} is completed (step ST31), the output has been decoded image from the adder 37, and performs a predetermined filtering process, The decoded image after the filter process is stored in the motion compensated prediction frame memory 40 (step ST32).
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
However, the loop filter unit 39 refers to each header information variable-length decoded by the variable-length decoding unit 31 for each of the above-described deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and performs processing in the corresponding slice. Specify whether or not to perform.
At this time, when two or more filter processes are performed, for example, when the loop filter unit 13 of the image encoding device is configured as shown in FIG. 11, the loop filter unit 39 as shown in FIG. Composed.

  Here, in the deblocking filter processing, when there is information for referring to the header information that has been variable-length decoded by the variable-length decoding unit 31 and changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Performs a deblocking filter process based on the change information. When there is no change information, it is performed according to a predetermined method.

In the pixel adaptive offset processing, the block is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the block unit of variable-length decoded by the variable-length decoding unit 31 is divided into the blocks. When an index indicating a class classification method is referred to and the index is not an index indicating that “offset processing is not performed”, each pixel in the block is classified into blocks in accordance with the class classification method indicated by the index.
Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance.

  Then, the loop filter unit 39 performs processing for adding the offset to the pixel value of the decoded image with reference to the offset information that has been variable-length decoded by the variable-length decoding unit 31 that identifies the offset value of each class in block units. .

In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.
The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

  As is apparent from the above, according to the first embodiment, the intra prediction unit 5, the intra block copy prediction unit 6 or the motion compensated prediction unit 7 of the image coding apparatus performs the prediction block for each prediction block. If the inter-color signal prediction flag clearly indicates that the inter-color signal prediction process is performed, the encoded color signal is referred to without performing the intra prediction process, intra block copy prediction process, and motion compensation prediction process. Therefore, even if the pattern in the prediction block is a specific non-linear pattern or character, the prediction accuracy is improved. There is an effect that it is possible to achieve high encoding efficiency.

  In addition, according to the first embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above effects can be obtained. Play.

Embodiment 2. FIG.
In the second embodiment, an image encoding device and an image decoding device that perform a correlation removal process between color signals of a prediction difference signal will be described.
That is, for the prediction difference signal of the chrominance signal in the image coding apparatus of the first embodiment, a difference value between the encoded chrominance signal and the locally decoded prediction difference signal is input to the transform / quantization unit 9; An image coding apparatus that obtains a final locally decoded prediction difference signal by adding the prediction difference signal of the encoded signal to the local decoded prediction difference signal of the color difference signal obtained by the inverse quantization / inverse transform unit 10 Configure.
Also, the decoded prediction difference signal of the chrominance signal obtained by the inverse quantization / inverse transform unit 32 in the image decoding apparatus of the first embodiment is encoded in the same way as the image coding apparatus of the second embodiment. An image decoding apparatus that obtains a final decoded prediction difference signal by adding the decoded prediction difference signals of color signals is configured.
By doing in this way, the correlation of the prediction difference between color signals can be reduced, and the code amount required for encoding the prediction difference signal of a color difference signal can be reduced.

  At this time, the local decoded prediction difference signal to be referred to in the image coding apparatus calculates the correlation parameter using the surrounding coded pixels in the same manner as in the inter-color signal prediction process of the first embodiment, and performs the main correlation. You may make it refer to the local decoding prediction difference signal processed with the parameter. By doing in this way, the correlation removal precision of the prediction difference between color signals can be improved, and encoding efficiency can be improved. In this case, the decoded prediction difference signal to be referred to in the image decoding apparatus also calculates a correlation parameter using surrounding decoded pixels, and refers to the decoded prediction difference signal processed with this correlation parameter.

In the above example, the correlation parameter is calculated using surrounding encoded (decoded) pixels. However, for the correlation parameter, a plurality of candidates are prepared in advance, and selected and encoded for each transform block. Also good. In this way, the optimum correlation parameter can be used for each transform block, and the coding efficiency can be improved. In this case, the image decoding apparatus decodes the correlation parameter in units of transform blocks.
Further, ON / OFF information indicating whether or not to perform the correlation removal processing between color signals of the prediction difference signal may be encoded as flag information for each transform block. By doing in this way, since the one where encoding efficiency is higher can be selected from the two cases where the inter-color signal correlation removal processing is performed in units of transform blocks and when it is not performed, the encoding efficiency can be improved. it can.
The ON / OFF information may be encoded as part of the correlation parameter. Specifically, when the correlation parameter is a specific value (for example, 0), it may be indicated that the correlation removal process between color signals is not performed.

  Further, similarly to the inter-color signal prediction process of the first embodiment, an inter-color signal vector may be prepared and a block indicated by this vector may be referred to. At this time, the inter-color signal vector may be common to the inter-color signal prediction process of the first embodiment, or may be provided independently. When prepared independently, the value of the prediction difference signal can be further reduced because more adaptive control can be performed, but encoding is performed separately from the inter-color signal vector of the inter-color signal prediction process of the first embodiment. Depending on the characteristics of the image, it is different whether the coding efficiency is higher when it is independent or when it is shared. Therefore, whether the inter-color signal vector is used in at least one of the sequence level header, the picture level header, and the slice level header and whether the inter-color signal prediction process of the first embodiment is independent or common. May be prepared, and the above-described flag may be encoded by the image encoding device. At this time, the image decoding apparatus decodes these flags, thereby identifying the presence / absence and the value of the inter-color signal vector in the inter-color signal correlation removal process of the prediction difference signal.

Furthermore, the encoded color signal to be referred to may be switched statically or dynamically, similarly to the inter-color signal prediction process of the first embodiment. Specifically, when encoding in the order of Y (luminance) signal, U signal, and V signal, the U signal refers to the Y signal, and the V signal refers to the Y signal or U signal. At this time, with respect to the V signal, it may be fixedly configured to refer to either the Y signal or the U signal in common to the image encoding device and the image decoding device (for example, the U signal is always referred to). The reference signal information may be encoded for each switching unit.
The unit of switching may be a sequence, a picture, a slice unit, or a block unit such as an encoded block, a prediction block, or a transform block. In general, the smaller the switching unit, the higher the prediction efficiency. On the other hand, the amount of code generated when encoding the reference signal information to be encoded can be suppressed by roughly setting the switching unit.
Further, when the reference signal can be switched in units of blocks, a flag indicating whether or not the switching processing in units of blocks is valid may be included in the sequence level header, picture level header, and slice level header. In this way, only when this switching process is effective, it is effective. When this switching process is not effective, it is not necessary to encode the reference signal information for each block, and the encoding efficiency is improved. Can be increased.

  In the above example, the reference signal information has been described as information for determining whether to refer to the Y signal or the U signal, but the reference block is further determined by weighted averaging with reference to both the Y signal and the U signal. You may make it also include the case where it produces | generates. That is, for each set switching unit, a predicted image is generated with reference to only the Y signal, a predicted image is generated with reference to only the U signal, or a predicted image is referred to with reference to both the Y signal and the U signal. Select whether to generate. By doing in this way, a prediction difference signal can be made smaller and encoding efficiency can be improved.

  The color of the first embodiment is used regardless of whether the reference color signal is fixedly set in common with the image encoding device and the image decoding device, or when the reference signal information is encoded. It may be common with the inter-signal prediction process or may be set independently. When prepared independently, since the adaptive control can be performed, the value of the prediction difference signal can be further reduced. However, when the reference signal information is configured to be encoded, it is necessary to encode the reference signal information separately from the reference signal information of the inter-color signal prediction process of the first embodiment. Which of the independent case and the common case has higher encoding efficiency is different. Therefore, a flag indicating whether to be independent or common to the inter-color signal prediction process of the first embodiment is prepared in at least one place of the sequence level header, the picture level header, and the slice level header. You may comprise so that the said flag may be encoded. At this time, the image decoding apparatus identifies the reference signal information in the color signal correlation removal processing of the prediction difference signal by decoding this flag.

In addition, in the block that performs the inter-color signal prediction process of the first embodiment as a combination of the inter-color signal prediction process of the first embodiment and the inter-color signal correlation removal process of the prediction difference signal, the prediction The difference signal intercolor signal correlation removal processing may not be performed. In this way, in the block that performs the inter-color signal prediction process, it is not necessary to encode the encoding parameters (correlation parameters, ON / OFF information, etc.) related to the inter-color signal correlation removal process of the prediction difference signal. Therefore, the encoding amount can be reduced. In this case, in the image decoding apparatus, only when the inter-color signal prediction parameter variable-length decoded by the variable-length decoding unit 31 indicates that the inter-color signal prediction process is not performed, the variable-length decoding unit 31 performs the prediction difference signal. The coding parameters related to the correlation removal process between color signals are decoded.
In this case, the ON / OFF information in units of blocks of the inter-color signal prediction process of Embodiment 1 and the ON / OFF information in units of blocks of the inter-color signal correlation removal process of the prediction difference signal may be integrated. Specifically, when performing the inter-color signal prediction process of the first embodiment, performing the inter-color signal correlation removal process of the prediction difference signal, and not performing both processes as index information (for example, in order, Index 2, 1, 0). At this time, the unit for switching ON / OFF information for both processes must be unified. That is, in this case, the ON / OFF processing is unified to any one of a coding block, a prediction block, and a transform block. Or it is good also as an original block unit. In this way, an optimal block size can be set for both processes.

As a parameter for controlling the inter-color signal correlation removal processing of the prediction difference signal, an effective flag (higher header residual color signal inter-prediction effective flag) of this processing is set as at least one of a sequence level header, a picture level header, and a slice level header. You may make it encode with one header. By doing in this way, when the correlation removal process between color signals of a prediction difference signal is not effective, it can be made invalid, and the fall of encoding efficiency can be suppressed. In this case, in the image decoding apparatus, only when the variable length decoding unit 31 decodes the upper header residual color signal inter-prediction valid flag and the upper header residual color signal inter-prediction valid flag indicates valid, the prediction difference signal The color signal correlation removal processing is enabled, and the ON / OFF information in units of transform blocks is decoded.
Furthermore, the upper header residual color signal inter-prediction valid flag may be provided in units of sub-pictures that divide a picture. Specifically, by providing the upper header residual color signal prediction effective flag in the division unit of the Tile or Wavefront Parallel Processing sub-picture described in Non-Patent Document 1, only the region where the color signal prediction is effective is effective. Therefore, highly accurate prediction can be realized while suppressing an increase in the code amount.

  Further, the sequence level, picture level, slice level, and sub-picture level flags indicating whether or not to enable the inter-color signal correlation removal processing of the prediction difference signal and the inter-color signal prediction processing of the first embodiment are enabled. Sequence level, picture level, slice level, and sub-picture level flags indicating whether or not to be used may be combined. Specifically, it shows the combination of valid / invalid of the inter-color signal correlation removal processing of the prediction difference signal and the inter-color signal prediction processing (valid / valid, valid / invalid, invalid / valid, invalid / invalid) Encode as an index. In this case, in the image decoding apparatus, the variable length decoding unit 31 decodes the above-described index, and determines whether the inter-color signal correlation removal process and the inter-color signal prediction process of the prediction difference signal are valid / invalid based on the index. .

Further, the orthogonal transform process of the transform block that performs the inter-color signal correlation removal process of the prediction difference signal may be always DST. The prediction difference signal of the transform block that has been subjected to the process of removing the correlation between the color signals of the prediction difference signal has a very low correlation between the signals. Therefore, it is highly likely that DST is more efficient than DCT, and DCT is used. Thus, the encoding efficiency can be increased.
Alternatively, DCT and DST may be switched according to the block size, coding mode, intra prediction parameter, intra block copy prediction parameter, motion compensation prediction parameter, and the like. For example, when the size of the target conversion block is smaller than a specific block size (threshold value), DST may be performed. By doing in this way, the conversion according to the characteristic of the conversion block made into object can be realized, and coding efficiency can be improved. In addition, the DCT and DST that have higher encoding efficiency may be selected, and the selection information may be encoded. By doing in this way, it can convert using DCT and DST for every conversion block, and can improve encoding efficiency.
As a configuration other than the above, an appropriate KL transform may be designed in advance for the prediction difference signal of the transform block that has been subjected to the inter-color signal correlation removal processing of the prediction difference signal, or may be designed by learning dynamically. By doing in this way, the conversion by the base more suitable for the prediction difference signal of the conversion block which performed the correlation removal process between color signals of a prediction difference signal can be implement | achieved, and encoding efficiency can be improved.
In addition to the matters described above, the second embodiment implements the same configuration and processing as the first embodiment.

Embodiment 3 FIG.
In the third embodiment, the correlation parameter of the inter-color signal prediction process is selected from a plurality of candidates as compared with the first embodiment or the second embodiment. The unit of selection may be a sequence, a picture, a slice unit, or a block unit such as an encoded block, a prediction block, or a transform block. In general, the smaller the switching unit, the higher the prediction efficiency. On the other hand, the amount of code generated when encoding the reference signal information to be encoded can be suppressed by roughly setting the switching unit.
Further, the correlation parameter may be shared with the correlation parameter of the inter-color signal correlation removal process of the prediction difference signal of the second embodiment. By doing in this way, the code amount regarding a correlation parameter can be reduced.
Except for the matters described above, the third embodiment performs the same configuration and processing as the first and second embodiments.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Encoding control part, 2 slice division part, 3 block division part, 4 changeover switch, 5 intra prediction part (intra prediction means), 6 intra block copy prediction part (intra block copy prediction means), 7 motion compensation prediction part ( Motion compensation prediction means), 8 subtraction unit, 9 transformation / quantization unit, 10 inverse quantization / inverse transformation unit, 11 addition unit, 12 intra memory, 13 loop filter unit, 14 motion compensation prediction frame memory, 15 variable length Encoding unit (encoding unit), 31 variable length decoding unit, 32 inverse quantization / inverse conversion unit, 33 changeover switch, 34 intra prediction unit (intra prediction unit), 35 intra block copy prediction unit (intra block copy prediction unit) ), 36 motion compensation prediction unit (motion compensation prediction means), 37 addition unit, 38 intra memory, 39 loops Filter unit, 40 motion-compensated prediction frame memory.

Claims (9)

Intra block copy prediction means for performing an intra block copy prediction process for generating a prediction image of the luminance signal using a block shift vector of the luminance signal in the prediction block for each prediction block that is a block of a prediction processing unit; Variable length encoding means for variable length encoding intra block copy prediction parameters for generating the block shift vector,
If the flag for the color difference signal of the prediction block clearly indicates that inter-color signal prediction processing is not performed, the intra block copy prediction means uses the same block shift vector as the luminance signal in the prediction block, and uses the color difference signal. If intra block copy prediction processing for generating a prediction image of a signal is performed and the flag clearly indicates execution of inter-color signal prediction processing, the color difference signal in the prediction block is referred to by referring to the encoded color difference signal An image encoding device that performs inter-color signal prediction processing for generating a predicted image of the image. For each prediction block that is a block of a prediction processing unit, using the motion vector of the luminance signal in the prediction block, a motion compensation prediction unit that performs motion compensation prediction processing for generating a prediction image of the luminance signal, and the motion vector Variable length coding means for variable length coding inter prediction parameters for generating
If the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction process is not performed, the motion compensation prediction means uses the same motion vector as the luminance signal in the prediction block, and and performing motion compensation prediction process for generating a predicted image, if the flag if explicitly the implementation of inter-color signal prediction processing with reference to the color difference signals encoded, predictive image of the color difference signal in the prediction block An image encoding device that performs inter-color signal prediction processing for generating a color signal.   3. The encoding device according to claim 1, further comprising: an encoding unit configured to encode a flag indicating whether or not to perform the inter-color signal prediction process when generating the predicted image of the color difference signal. 4. Image encoding device. Variable length decoding means for variable length decoding intra block copy prediction parameters for generating a block shift vector for each prediction block which is a block of a prediction processing unit; and a luminance signal generated from the intra block copy prediction parameters for each prediction block Intra block copy prediction means for performing intra block copy prediction processing for generating a prediction image of the luminance signal using a block shift vector,
If the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction process is not performed, the intra block copy prediction means has the same block shift vector as the luminance signal generated from the intra block copy prediction parameter. Is used to perform an intra block copy prediction process for generating a prediction image of the color difference signal, and if the flag clearly indicates the execution of the inter-color signal prediction process, the decoded color difference signal is referred to An image decoding apparatus that performs inter-color signal prediction processing for generating a prediction image of a color difference signal in a prediction block. Using variable-length decoding means for variable-length decoding inter prediction parameters for generating a motion vector for each prediction block, which is a block of a prediction processing unit, and a motion vector of a luminance signal generated from the inter-prediction parameters for each prediction block A motion compensation prediction means for performing a motion compensation prediction process for generating a prediction image of the luminance signal,
If the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction process is not performed, the motion compensation prediction unit uses the same motion vector as the luminance signal generated from the inter prediction parameter, If a motion compensation prediction process for generating a prediction image of the color difference signal is performed and the flag clearly indicates the execution of the inter-color signal prediction process, the color difference signal in the prediction block is referred to with reference to the decoded color difference signal An image decoding apparatus that performs an inter-color signal prediction process for generating a predicted image of the image. An intra block copy prediction unit performs an intra block copy prediction process for generating a prediction image of the luminance signal, using a block shift vector of the luminance signal in the prediction block, for each prediction block that is a block of a prediction processing unit. An intra-block copy prediction processing step, and a variable-length encoding means, wherein the variable-length encoding means variable-length encodes an intra-block copy prediction parameter for generating the block shift vector,
In the intra block copy prediction processing step, if the flag for the color difference signal of the prediction block clearly indicates that inter-color signal prediction processing is not performed, the same block shift vector as the luminance signal in the prediction block is used. If intra block copy prediction processing for generating a prediction image of a color difference signal is performed, and the flag clearly indicates execution of the inter-color signal prediction processing, the color difference in the prediction block is referred to by referring to the encoded color difference signal. An image coding method, comprising: performing inter-color signal prediction processing for generating a predicted image of a signal. Motion compensation prediction means for performing motion compensation prediction processing for generating a prediction image of the luminance signal by using a motion vector of the luminance signal in the prediction block for each prediction block which is a block of a prediction processing unit. A processing step, and a variable-length coding means, a variable-length coding step for variable-length coding an inter prediction parameter for generating the motion vector,
In the motion compensation prediction processing step, if the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction processing is not performed, the color difference signal is calculated using the same motion vector as the luminance signal in the prediction block. When the motion compensated prediction process for generating the predicted image is performed and the flag clearly indicates the execution of the inter-color signal prediction process, the encoded color difference signal is referred to and the prediction of the color difference signal in the prediction block is performed. An image encoding method comprising performing inter-color signal prediction processing for generating an image. A variable length decoding step, wherein the variable length decoding means variable length decodes an intra block copy prediction parameter for generating a block shift vector for each prediction block which is a block of a prediction processing unit;
Intra block copy prediction means for generating a prediction image of the luminance signal by using a block shift vector of the luminance signal generated from an intra block copy prediction parameter for each prediction block for each prediction block. An intra block copy prediction processing step for performing the processing;
In the intra block copy prediction processing step, if the flag for the color difference signal of the prediction block clearly indicates that inter-color signal prediction processing is not performed, the same block shift as the luminance signal generated from the intra block copy prediction parameter Perform intra block copy prediction processing for generating a prediction image of the color difference signal using a vector, and if the flag clearly indicates the execution of the inter-color signal prediction processing, refer to the decoded color difference signal, An image decoding method comprising: performing inter-color signal prediction processing for generating a predicted image of a color difference signal in the prediction block. A variable length decoding step, wherein the variable length decoding means variable length decodes an inter prediction parameter for generating a motion vector for each prediction block which is a block of a prediction processing unit;
Motion in which motion compensation prediction means performs motion compensation prediction processing for generating a prediction image of the luminance signal using a motion vector of the luminance signal generated from the inter prediction parameter for each prediction block for each prediction block. A compensation prediction processing step,
In the motion compensation prediction processing step, if the flag for the color difference signal of the prediction block clearly indicates that the inter-color signal prediction processing is not performed, the same motion vector as the luminance signal generated from the inter prediction parameter is used. If the motion compensation prediction process for generating the predicted image of the color difference signal is performed and the flag clearly indicates the execution of the inter-color signal prediction process, the color difference in the prediction block is referred to by referring to the decoded color difference signal An image decoding method comprising: performing inter-color signal prediction processing for generating a predicted image of a signal.

Images