JP2015177342A - Image coding device, image decoding device, image coding method, and image decoding method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To sufficiently reduce a coding distortion occurring in a local decoded image of a screen content where a histogram is sparse.SOLUTION: An image coding device comprises: a histogram calculator 21 which calculates the histogram of a pixel value in a local decoded image obtained from compression data of a coding target image; and a pixel value correction unit 22 which identifies, from the histogram calculated by the histogram calculator 21, a pixel with noise associated with the coding distortion superimposed thereon, and corrects the pixel value of the pixel with the noise superimposed thereon.

Description

  The present invention relates to an image encoding device and an image encoding method for encoding an image with high efficiency, and an image decoding device and an image decoding method for decoding an image encoded with high efficiency.

  When transmitting and accumulating moving images, MPEG (Moving Picture Expert Groups) or ITU-T H.264. It is common practice to compress the amount of information using an international standard video coding scheme such as 26x. In particular, HEVC (High Efficiency Video Coding), which has been standardized in January 2013, is a broadcasting / communication service (HD or HD) using UHDTV (Ultra High Definition Television) such as 4K and SHV (Super Hi-Vision), and mobile lines (HD). High Definition) This technology is indispensable for realizing video transmission.

When moving image coding of an encoding target image that is an original image is performed, a decoded image obtained by decoding compressed data of the encoding target image includes noise that does not originally exist in the original image (encoding distortion). Accompanying noise) may occur.
The presence of this noise not only lowers the image quality of the decoded image, but also reduces the image quality of the reference image used when the image coding apparatus performs coding. A reduction in the image quality of the reference image causes a reduction in encoding efficiency, and therefore noise accompanying encoding distortion must be removed as much as possible.

As a technique for reducing noise accompanying coding distortion, there is a loop filter that applies a filter to a decoded image.
For example, the image encoding device disclosed in Patent Document 1 below uses a filter called a pixel adaptive offset as one of the loop filters. This is a method of classifying each pixel of a decoded image into a plurality of classes according to a predetermined rule, and adding a certain offset to the pixel value for each pixel belonging to the same class. The offset value is determined by the image encoding device and transmitted to the image decoding device.
For example, if the difference average value of the pixel values of the original image and the decoded image is used as an offset, the decoded image approaches the original image by applying a pixel adaptive offset, and noise associated with encoding distortion can be reduced.

The pixel classification method in the pixel adaptive offset is roughly divided into two types, a band offset and an edge offset (see, for example, Non-Patent Document 1).
The band offset divides the pixel value from the minimum value to the maximum value at equal intervals, and classifies the pixel according to which region the pixel value of the target pixel belongs. On the other hand, the edge offset compares the pixel value of the target pixel with the pixel values of its neighboring pixels to determine whether the target pixel is an edge, and when the target pixel is an edge, The pixels are classified according to the direction. The image encoding apparatus determines whether to use the band offset or the edge offset.

By the way, there is a growing need to record PC / control terminal screens as moving images for screen sharing, FA / building displays, signage, media archives, etc. ing. These are called screen contents in order to distinguish them from moving images taken by a normal camera.
The screen content has several properties that are different from those of a moving image shot by a camera.

FIG. 42 is an explanatory diagram of a characteristic example of screen content.
When a pixel value histogram (appearance frequency for each pixel value) in an image photographed by a camera is calculated, a dense histogram is generally obtained as shown in FIG.
On the other hand, when a histogram of pixel values in the screen content is calculated, only a very limited pixel value appears as shown in FIG. 42B, resulting in a very sparse histogram.

When screen content having such characteristics is encoded, a change occurs in the histogram due to the encoding distortion described above.
FIG. 43 is an explanatory diagram showing changes in the histogram before and after encoding in the screen content.
In the histogram of the screen content before encoding, there is no peak around the point where there is one sharp peak, but when the screen content is encoded and then decoded, coding distortion occurs, In the histogram of the decoded screen content, there are a plurality of peaks (noise due to encoding distortion) around a portion with a single sharp peak.
Noise due to coding distortion causes a reduction in the image quality of the decoded image, and thus needs to be removed by the loop filter described above.

JP2012-5113A (paragraph number [0005])

H.265 Recommendation, p. 158-160

  Since the conventional image coding apparatus is configured as described above, if the image to be coded is an image taken by a camera, if the pixel value histogram is dense, pixel adaptive offset, etc. By applying this loop filter, it is possible to reduce noise accompanying coding distortion. However, even when a loop filter such as a pixel adaptive offset is applied to a locally decoded image of screen content having a sparse histogram, there is a problem that noise accompanying coding distortion cannot be sufficiently reduced.

The present invention has been made to solve the above-described problems, and an image encoding apparatus capable of sufficiently reducing noise accompanying encoding distortion generated in a locally decoded image of a screen content having a sparse histogram. The object is to obtain an image coding method.
Another object of the present invention is to provide an image decoding apparatus and an image decoding method capable of sufficiently reducing noise accompanying encoding distortion occurring in a decoded image of screen content having a sparse histogram.

  The image coding apparatus according to the present invention includes a histogram calculation unit that calculates a histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image, and encoding distortion from the histogram calculated by the histogram calculation unit. A pixel value correcting means for specifying a pixel on which noise is superimposed and correcting a pixel value of the pixel on which the noise is superimposed is provided.

  According to this invention, the histogram calculation means for calculating the histogram of the pixel values in the locally decoded image obtained from the compressed data of the encoding target image, and the noise accompanying the encoding distortion are superimposed from the histogram calculated by the histogram calculation means. And a pixel value correction unit that corrects the pixel value of the pixel on which the noise is superimposed, so that the encoding that occurs in the locally decoded image of the screen content with a sparse histogram There is an effect that noise accompanying distortion can be sufficiently reduced.

It is a block diagram which shows the image coding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color-difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between YUV4: 4: 4 format and YUV4: 2: 2 format. It is explanatory drawing which shows the example of the directionality prediction in YUV4: 2: 2 format equivalent to using the same directionality prediction with a luminance signal and a color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the prediction direction vector of directionality prediction with the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between directionality prediction and an angle. It is explanatory drawing which shows an example of the quantization matrix of 4x4 DCT. It is explanatory drawing which shows the structural example of the loop filter part 13 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 13 which implements a histogram correction process. It is explanatory drawing which shows the histogram of the pixel value before and behind correction | amendment. It is explanatory drawing which shows an example of an encoding bit stream. It is a flowchart which shows the processing content of the image coding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the structural example of the loop filter part 39 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 39 which implements a histogram correction process. It is a flowchart which shows the processing content of the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the example by which the largest encoding block is divided | segmented into a some encoding block hierarchically. (A) shows the distribution of the encoding block and prediction block after a division | segmentation, (b) is explanatory drawing which shows the condition where encoding mode m ( ^Bn ) is allocated by hierarchy division | segmentation. It is explanatory drawing which shows the index of the class classification method of a pixel adaptive offset process. It is a flowchart which shows the content of the histogram correction process in the image coding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the content of the histogram correction process in the image coding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the encoding order of the conversion factor in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows an example of distribution of the transformation coefficient in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows an example of the encoding order of the transform coefficient in the orthogonal transformation of the size of 4x4 pixel and 8x8 pixel. Each prediction block P _i ⁿ the coded block B ⁿ is an explanatory diagram showing an example of a selectable intra prediction modes. It is explanatory drawing which shows an example of the pixel used when producing | generating the predicted value of the pixel in the predicted image generation block in the case of l _i ⁿ = m _i ⁿ = 4. It is explanatory drawing which shows the switching area | region of the filter in the filter process at the time of average value prediction. It is explanatory drawing which shows the reference pixel arrangement | positioning of the filter process at the time of average value prediction. It is explanatory drawing which shows the relative coordinate which makes the upper left pixel in a prediction image generation block the origin. It is explanatory drawing which shows the example of a response | compatibility with the intra prediction parameter (index value) of a color difference signal, and color difference intra prediction mode. It is explanatory drawing which shows the relationship between the intra prediction mode index of a luminance signal, and the intra prediction mode index of a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between an intra prediction mode index and tan (theta). It is explanatory drawing which shows the example of a response | compatibility of the intra prediction parameter of a color difference signal, and a color difference intra prediction mode. It is a flowchart which shows the content of the histogram correction process in the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the structural example of the loop filter part 13 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 39 which implements a histogram correction process. It is explanatory drawing which shows the example of a response | compatibility of an index and an offset application method. It is a flowchart which shows the procedure which selects the offset application method to be used from the some offset application methods in FIG. It is explanatory drawing of the example of a characteristic of a screen content. It is explanatory drawing which shows the change of the histogram before and behind encoding in a screen content. In the pixel adaptive offset process, it is explanatory drawing which shows the position which encodes the parameter in a pixel adaptive offset process, and inserts it in a bit stream, taking as an example the case where it always divides | segments in the unit of Coding Tree Unit (CTU) which is a largest encoding block. is there. It is the flowchart which showed how the pixel adaptive offset process parameter was encoded in the variable length encoding part 15, and it added to a bit stream. 12 is a flowchart illustrating a decoding procedure of pixel value adaptive offset processing parameters by the variable length decoding unit 31. 10 is a flowchart showing the operation of the loop filter unit 13 in the third embodiment. 10 is a flowchart showing the operation of the loop filter unit 13 in the third embodiment. It is explanatory drawing which shows the block division by a histogram correction process. It is explanatory drawing which shows the diversion method of a parameter. It is explanatory drawing which shows the diversion method of a parameter. It is explanatory drawing which shows the index table of the parameter encoding method. 3 is a flowchart showing the operation of the variable length coding unit 15. It is a flowchart which shows operation | movement of the variable length decoding part 31 regarding a histogram correction process. It is explanatory drawing which shows the index table of the parameter encoding method. 10 is a flowchart illustrating an operation related to offset processing of the variable length coding unit 15 when the third embodiment is combined with the second embodiment. 12 is a flowchart illustrating an operation related to an offset process of the variable length decoding unit 31 when the third embodiment is combined with the second embodiment. It is explanatory drawing which shows the relationship between the pixel value of correction object, and appearance frequency. It is explanatory drawing which shows the relationship between the pixel value of correction object, and appearance frequency. It is explanatory drawing which shows the index table of an offset application method.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image coding apparatus according to Embodiment 1 of the present invention.
The video signal to be processed by the image coding apparatus according to the first embodiment is a color video in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to signals, the video frame is an arbitrary video signal such as a monochrome image signal or an infrared image signal, in which a video frame is composed of a horizontal / vertical two-dimensional digital sample (pixel) sequence. The gradation of each pixel may be 8 bits, or a gradation of 10 bits, 12 bits, or the like. Further, it is natural that the input signal may be a still image signal instead of a video signal because the still image signal can be interpreted as a video signal composed of only one frame.

  In the following description, for convenience, unless otherwise specified, the input video signal is YUV4: 2 in which the two color difference components U and V are subsampled in half both vertically and horizontally with respect to the luminance component Y: 0 format, YUV4: 2: 2 format in which two color difference components U and V are subsampled in half in the horizontal direction with respect to luminance component Y, or two color difference components U and V are luminance component Y And YUV 4: 4: 4 format signal having the same number of samples. In addition, for RGB 4: 4: 4 format signals composed of signals of the three primary colors of red (R), green (G), and blue (B), each signal is regarded as a YUV 4: 4: 4 format signal and YUV 4: Performs the same encoding as the 4: 4 format. However, the correspondence of each signal (RGB) in RGB 4: 4: 4 format to each signal (YUV) in YUV 4: 4: 4 format is not limited (can be arbitrarily set). This association may be encoded as an index information with a high-order header so that the image decoding apparatus can recognize it. By doing in this way, the color of the decoded image decoded by the image decoding apparatus can be correctly displayed.

In the case of a YUV 4: 4: 4 format signal or an RGB 4: 4: 4 format signal, each signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV 4: 0: 0) to generate a bit stream. May be. In this way, encoding processing can be performed in parallel with each signal. At this time, information indicating which color signal each monochrome signal is may be encoded as an index information with a high-order header so that the image decoding apparatus can recognize it. By doing in this way, the color of the decoded image decoded by the image decoding apparatus can be correctly displayed.
The processing data unit corresponding to each frame of the video is referred to as “picture”. In the first embodiment, “picture” is described as a signal of a video frame that has been sequentially scanned (progressive scan). However, when the video signal is an interlace signal, the “picture” may be a field image signal which is a unit constituting a video frame.

  In FIG. 1, the encoding control unit 1 determines the maximum size of an encoding block that is a processing unit when the encoding process is performed, and the upper limit when the encoding block of the maximum size is hierarchically divided. By determining the number of layers, the process of determining the size of each encoded block is performed. The encoding control unit 1 also has one or more selectable encoding modes (one or more intra encoding modes having different prediction block sizes indicating prediction processing units, etc., and different prediction block sizes indicating prediction processing units). Processing for selecting a coding mode to be applied to a coding block output from the block division unit 3 from one or more intra block copy coding modes and one or more inter coding modes having different prediction block sizes. To implement. As an example of the selection method, there is a method of selecting a coding mode having the highest coding efficiency for the coding block output from the block dividing unit 3 from one or more selectable coding modes.

  In addition, when the coding mode with the highest coding efficiency is the intra coding mode, the coding control unit 1 sets the intra prediction parameters used when performing the intra prediction process on the coding block in the intra coding mode. It is determined for each prediction block that is a prediction processing unit indicated by the intra coding mode, and when the coding mode having the highest coding efficiency is the intra block copy coding mode, the intra block copy coding mode is used for the coding block. An intra block copy prediction parameter used when performing the intra block copy prediction process is determined for each prediction block which is a prediction processing unit indicated by the intra block copy coding mode, and the coding mode having the highest coding efficiency is an inter code. The inter code The inter prediction parameters used in practicing the inter prediction process for encoding blocks to implement the process of determining for each prediction block is a prediction processing unit indicated by the inter coding mode in the mode.

  Furthermore, the encoding control unit 1 performs a process of determining a prediction difference encoding parameter to be given to the transform / quantization unit 9 and the inverse quantization / inverse transform unit 10. The prediction difference coding parameter includes transform block partition information indicating transform block partition information that is a unit of orthogonal transform processing in the coded block, and a quantization parameter that specifies a quantization step size when transform coefficients are quantized. Etc. are included.

  Here, FIG. 2 is an explanatory diagram showing the conversion block size when the compression processing (conversion processing and quantization processing) of the luminance signal and the color difference signal in the YUV 4: 2: 0 format signal is performed. As shown in FIG. 2, the transform block size is determined by hierarchically dividing the encoded block into a quadtree. For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining, it is possible to determine the optimal division shape of the transform block from the viewpoint of the trade-off between the code amount and the coding error.

  For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

  When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. Further, since the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction, the divided block is further divided into two parts in the vertical direction, so that the YUV 4: 2: 0 signal The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).

  When the input signal format is a YUV 4: 4: 4 signal, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block as shown in FIG. Configure as follows. The division information of the conversion block of the luminance signal is output to the variable length encoding unit 15 as a conversion block division flag indicating whether or not to divide for each layer, for example.

When a video signal is input as an input image, the slice division unit 2 performs a process of dividing the input image into one or more partial images called “slices” according to the slice division information determined by the encoding control unit 1. The slice division unit can be finely divided to the above-described coding block unit.
Each time the block dividing unit 3 inputs the slice divided by the slice dividing unit 2, the slice dividing unit 3 divides the slice into maximum coding blocks that are coding blocks of the maximum size determined by the coding control unit 1, and Until the maximum number of hierarchies determined by the coding control unit 1 is reached, a process of dividing the maximum coding block hierarchically into each coding block is performed. That is, the block division unit 3 divides the slice into each coding block according to the division determined by the coding control unit 1, and performs a process of outputting the coding block. Each coding block is divided into one or a plurality of prediction blocks which are prediction processing units.

  If the coding mode determined by the coding control unit 1 is the intra coding mode, the changeover switch 4 outputs the coded block output from the block dividing unit 3 to the intra prediction unit 5, and the coding control unit 1 If the coding mode determined by is the intra block copy coding mode, the coding block output from the block division unit 3 is output to the intra block copy prediction unit 6, and the code determined by the coding control unit 1 is output. If the encoding mode is the inter encoding mode, a process of outputting the encoded block output from the block dividing unit 3 to the motion compensation prediction unit 7 is performed.

  When the intra control mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra prediction unit 5 performs local decoding stored in the intra memory 12. With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter determined by the encoding control unit 1 is performed to generate an intra predicted image.

  That is, for the luminance signal, the intra prediction unit 5 performs an intra prediction process (intraframe prediction process) using an intra prediction parameter of the luminance signal to generate a prediction image of the luminance signal. On the other hand, for the color difference signal, the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal. FIG. 5 shows a case where the same directionality prediction is used for a luminance signal and a color difference signal in a YUV 4: 2: 0 format signal, and FIG. 6 shows that the luminance signal and the color difference signal are the same in a YUV 4: 2: 2 format signal. The case where directionality prediction is used is shown.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  When the intra block copy encoding mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra block copy predicting unit 6 predicts the encoded block. The pixel value of each pixel in an area that has not been subjected to local decoding is assumed in a predetermined method among the maximum size encoded block (maximum encoded block) to which the block (prediction processing unit block) belongs. Perform the process. In addition, the intra block copy prediction unit 6 approximates the prediction block for each prediction block in the encoded block most closely to the prediction block from the regions that have already been encoded and locally decoded in the same picture. A process is performed for searching for a reference block that is an existing block and determining the reference block as a prediction image of the prediction block.

  When the inter coding mode is selected by the coding control unit 1 as the coding mode corresponding to the coding block output from the changeover switch 4, the motion compensated prediction unit 7 and the motion compensated prediction frame memory 14 A motion vector is searched by comparing locally decoded images of one frame or more stored in the image, and the motion vector and an inter prediction parameter such as a frame number to be determined determined by the encoding control unit 1 are used to encode the code. A process for generating an inter-predicted image by performing an inter prediction process (motion-compensated prediction process) on a block is performed.

  The subtracting unit 8 is an intra prediction image generated by the intra prediction unit 5, an intra block copy prediction image generated by the intra block copy prediction unit 6, or motion compensated prediction from the encoded block output from the block dividing unit 3. A process of subtracting the inter prediction image generated by the unit 7 and outputting a prediction difference signal indicating the difference image as a subtraction result to the transform / quantization unit 9 is performed.

The transform / quantization unit 9 refers to the transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 and performs orthogonal transform processing on the prediction difference signal output from the subtraction unit 8 (for example, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), orthogonal transform processing such as KL transform, which is previously designed for a specific learning sequence, is performed for each transform block, and transform coefficients are calculated. Then, referring to the quantization parameter included in the prediction differential encoding parameter, the transform coefficient of the transform block unit is quantized, and the compressed data that is the transform coefficient after quantization is dequantized / inverse transform unit 10 and variable A process of outputting to the long encoding unit 15 is performed.
When the transform / quantization unit 9 quantizes the transform coefficient, the transform / quantization unit 9 performs a transform coefficient quantization process using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient. You may do it.

  FIG. 11 is an explanatory diagram showing an example of a 4 × 4 DCT quantization matrix. The numbers in the figure indicate the scaling value of the quantization step size of each transform coefficient. For example, in order to suppress the encoding bit rate, as shown in FIG. 11, by scaling the quantization step size to a larger value for a higher frequency transform coefficient, a higher frequency generated in a complex image region or the like. It is possible to perform coding without dropping information on low-frequency coefficients that greatly affect subjective quality while suppressing the amount of codes by suppressing conversion coefficients. Thus, when it is desired to control the quantization step size for each transform coefficient, a quantization matrix may be used.

  In addition, the quantization matrix can use an independent matrix for each color signal and coding mode (intra coding or inter coding) with each orthogonal transform size. It is possible to select whether to use a quantization matrix that is commonly prepared in advance by the decoding apparatus or an already encoded quantization matrix or to use a new quantization matrix. Accordingly, the transform / quantization unit 9 sets flag information indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, in a quantization matrix parameter to be encoded. .

  Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 11 is set as a quantization matrix parameter to be encoded. On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance by the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Thus, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, it is possible to select only the quantization matrix that is prepared in advance by the image encoding device and the image decoding device.

  The inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1, and transforms and transforms the transform block 9 in units of transform blocks. A local decoded prediction difference corresponding to the prediction difference signal output from the subtracting unit 8 is obtained by dequantizing the output compressed data and performing an inverse orthogonal transform process on the transform coefficient that is the compressed data after the inverse quantization. A process of calculating a signal is performed. In addition, when the transform / quantization unit 9 uses the quantization matrix to perform the quantization process, the corresponding inverse quantization can be performed by referring to the quantization matrix even during the inverse quantization process. Implement the process.

The addition unit 11 includes a local decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 10, an intra prediction image generated by the intra prediction unit 5, and an intra block copy prediction image generated by the intra block copy prediction unit 6. Alternatively, the inter prediction image generated by the motion compensation prediction unit 7 is added to perform a process of calculating a local decoded image corresponding to the encoded block output from the block dividing unit 3.
The intra memory 12 is a recording medium that stores the locally decoded image calculated by the adding unit 11.

The loop filter unit 13 performs a predetermined filter process on the local decoded image calculated by the adding unit 11 and performs a process of outputting the local decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 13 determines whether or not to perform each of the deblocking filter process, the pixel adaptive offset process, the adaptive filter process, and the histogram correction process, and uses a variable length code with the valid flag of each process as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 12 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 12 are merely examples, and the types and application order of the loop filters to be applied are not necessarily as shown in FIG. For example, the loop filter unit 13 is configured by only three types of deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, and the application order is in order of deblocking filter processing, histogram correction processing, and pixel adaptive offset processing. And so on.

In the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are used. One classification method is selected from the methods. Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating the coding distortion is calculated for each class. Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.

Therefore, in the pixel adaptive offset processing, block division information, an index indicating a class classification method for each block, and offset information for specifying an offset value of each class in block units are output to the variable length coding unit 15 as header information.
In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  In adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for the distortion that is superimposed is designed for each region (local decoded image) belonging to each class. Filter the local decoded image. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information. As the class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of each block. Further, the number of classes used in the adaptive filter processing may be set in advance as a value common to the image encoding device and the image decoding device, or may be a parameter to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

FIG. 13 is a configuration diagram illustrating a processing unit in the loop filter unit 13 that performs the histogram correction processing.
In FIG. 13, the histogram calculation unit 21 divides an input image (encoding target image) indicated by the video signal into a plurality of blocks, divides the local decoded image calculated by the addition unit 11 into a plurality of blocks, A process of calculating a histogram of pixel values is performed for each block of the decoded image. The histogram calculator 21 constitutes a histogram calculator.
The pixel value correcting unit 22 specifies a pixel on which noise accompanying coding distortion is superimposed from the histogram calculated by the histogram calculating unit 21, and performs processing for correcting the pixel value of the pixel on which the noise is superimposed. To do. The pixel value correcting unit 22 constitutes a pixel value correcting unit.

The first error calculation unit 23 of the pixel value correction unit 22 performs a process of calculating an error between the input image in units of blocks and the locally decoded image.
The second error calculation unit 24 is a local decoded image in block units on the assumption that the pixel value within the correction pixel value range including the peak pixel value in the histogram calculated by the histogram calculation unit 21 is replaced with the peak pixel value. And processing for calculating an error between the input image and the block unit.

If the error calculated by the second error calculation unit 24 is smaller than the error calculated by the first error calculation unit 23, the corrected pixel value range storage unit 25 calculates the error calculated by the first error calculation unit 23. The correction pixel value range is overwritten and stored in place of the error calculated by the second error calculation unit 24.
The iterative processing unit 26 updates the correction pixel value range until the correction pixel value range reaches a preset maximum range, and repeatedly instructs the second error calculation unit 24 to re-execute the error calculation process. Perform the process.
After the corrected pixel value range updated by the iterative processing unit 26 reaches a preset maximum range, the pixel value replacement unit 27 stores pixel values within the corrected pixel value range stored by the corrected pixel value range storage unit 25. A process of substituting for the peak pixel value is performed.

That is, in the histogram correction process, first, the histogram calculation unit 21 divides the input image indicated by the video signal into a plurality of blocks, and also divides the local decoded image calculated by the addition unit 11 into a plurality of blocks. A histogram of pixel values is calculated for each block of the locally decoded image.
The pixel value correction unit 22 specifies the peak pixel value of the histogram calculated by the histogram calculation unit 21 by a predetermined method. Since noise due to encoding distortion exists in the vicinity of the peak pixel value, the pixel value is set to the peak pixel value for a pixel having a pixel value close to the peak pixel value (a pixel value within the correction pixel value range). The encoding distortion is reduced by replacing with.
Here, FIG. 14 is an explanatory diagram showing a histogram of pixel values before and after correction.
In FIG. 14, a correction width (correction pixel value range) indicating how much of the pixel value is to be corrected is represented by w, and this correction width w indicates the position of the peak pixel value or the image. Since the optimum value changes according to the characteristics, the pixel value correction unit 22 calculates.

In FIG. 1, a motion compensated prediction frame memory 14 is a recording medium that stores a locally decoded image after the filter processing of the loop filter unit 13.
The variable length encoding unit 15 and the compressed data output from the transform / quantization unit 9 and the output signal of the encoding control unit 1 (block division information in the maximum encoding block, encoding mode, prediction difference encoding parameter, A loop filter parameter, an intra prediction parameter, an intra block copy prediction parameter, an inter prediction parameter), a motion vector output from the motion compensation prediction unit 7 (when the encoding mode is an inter encoding mode), and a pixel value correction unit. A process for generating encoded data by performing variable length encoding on the optimal correction width w (correction pixel value range) calculated in Step 22 is performed.
Further, as illustrated in FIG. 15, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data. carry out. Note that the variable length coding unit 15 constitutes coding means.

However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.
The sequence level header includes an image size, a color signal format, a bit depth of a signal value of a luminance signal or a color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. , Histogram correction processing) effective flag information, quantization matrix effective flag information, and the like, which are header information that is generally common to each sequence unit.

The picture level header is a collection of header information set in units of pictures, such as an index of a sequence level header to be referenced, the number of reference pictures at the time of motion compensation, an entropy coding probability table initialization flag, and a quantization matrix parameter. .
The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a collection of parameters in units of slices such as flag information indicating whether or not each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process, and adaptive offset process) in the filter unit 13 is performed.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded. Supplemental information is also defined as a unique NAL unit if it exists. The picture data is defined as an access unit and indicates a unit of data access including encoded data of one picture.

  In the example of FIG. 1, a coding control unit 1, a slice division unit 2, a block division unit 3, a changeover switch 4, an intra prediction unit 5, an intra block copy prediction unit 6, and motion compensation prediction, which are components of the image coding apparatus. Unit 7, subtraction unit 8, transform / quantization unit 9, inverse quantization / inverse transform unit 10, addition unit 11, intra memory 12, loop filter unit 13, motion compensated prediction frame memory 14, and variable length coding unit 15 Are configured by dedicated hardware (components other than the intra memory 12 and the motion compensation prediction frame memory 14 are configured by, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like). Although the thing is assumed, the image coding apparatus may be configured by a computer.

When the image encoding device is configured by a computer, the intra memory 12 and the motion compensated prediction frame memory 14 are configured on the computer memory, and the encoding control unit 1, the slice dividing unit 2, the block dividing unit 3, and the changeover switch. 4, intra prediction unit 5, intra block copy prediction unit 6, motion compensation prediction unit 7, subtraction unit 8, transformation / quantization unit 9, inverse quantization / inverse transformation unit 10, addition unit 11, loop filter unit 13 and variable A program describing the processing contents of the long encoding unit 15 may be stored in a memory of a computer so that the CPU of the computer executes the program stored in the memory.
FIG. 16 is a flowchart showing the processing contents of the image coding apparatus according to Embodiment 1 of the present invention.

FIG. 17 is a block diagram showing an image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 17, when the variable length decoding unit 31 receives the encoded bit stream generated by the image encoding device of FIG. A process of decoding picture data is performed.
However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data. At this time, the header information includes information indicating that each of the YUV 4: 4: 4 format signal and the RGB 4: 4: 4 format signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV 4: 0: 0). In the case where it is included, the decoding process can be performed independently on the encoded bit stream of each color signal.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the sequence level header), a picture parameter header (corresponding to the picture level header), and slice data are defined as unique NAL unit types, respectively, and identification information (index) of the NAL unit type is used. It is identified by decoding. The supplemental information is also identified as a unique NAL unit if it exists. The picture data is identified as an access unit in which NAL units indicating slice data are collected.

  When the valid flag information of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding of the quantization matrix parameter, and specifies the quantization matrix. Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, a quantization matrix prepared in advance in the image encoding device and the image decoding device, or When indicating that the quantization matrix has already been decoded (not a new quantization matrix), the quantization matrix is identified with reference to the index information identifying which quantization matrix of the matrix, When the quantization matrix parameter indicates that a new quantization matrix is used, the quantization matrix is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter.

  In addition, the variable length decoding unit 31 refers to the slice level header, specifies the slice division state, decodes the slice data of each slice, specifies the encoded data of the maximum encoded block, and slice data The block division information included in the block, the encoding block that is a unit for performing decoding processing by hierarchically dividing the maximum encoding block, and specifying the compressed data, the encoding mode, Intra prediction parameters (when the encoding mode is the intra encoding mode), intra block copy prediction parameters (when the encoding mode is the intra block copy encoding mode), inter prediction parameters (the encoding mode is the inter encoding mode) ) Motion vector (if the coding mode is inter coding mode) If), carries out a process of variable length decoding optimum correction width w (corrected pixel value range) and predictive differential coding parameters. The variable length decoding unit 31 constitutes decoding means.

  The inverse quantization / inverse transform unit 32 refers to the quantization parameter and transform block division information included in the prediction difference encoding parameter variable length decoded by the variable length decoding unit 31, and the variable length decoding unit 31 performs variable length decoding. 1 is inversely quantized on a transform block basis, and inverse orthogonal transform processing is performed on transform coefficients that are compressed data after inverse quantization, and output from the inverse quantization / inverse transform unit 10 in FIG. A process of calculating the same decoded prediction difference signal as the local decoding prediction difference signal is performed.

Here, the division state of the transform block in the coding block is specified from the transform block partition information. For example, in the case of a YUV 4: 2: 0 format signal, the transform block size is determined by hierarchically dividing the encoded block into quadtrees as shown in FIG.
For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. In addition, the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction. The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
When the input signal format is a YUV 4: 4: 4 signal, as shown in FIG. 4, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block so that the conversion block has the same size. Configure.

  In addition, when each header information variable-length decoded by the variable-length decoding unit 31 indicates that inverse quantization processing is to be performed using the quantization matrix in the slice, inverse quantization is performed using the quantization matrix. Process. Specifically, inverse quantization processing is performed using a quantization matrix specified from each header information.

  The changeover switch 33 outputs the intra-prediction parameter variable-length decoded by the variable-length decoding unit 31 to the intra-prediction unit 34 if the coding mode variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode. If the encoding mode variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode, the intra block copy prediction parameters variable length decoded by the variable length decoding unit 31 are output to the intra block copy prediction unit 35. If the coding mode variable length decoded by the variable length decoding unit 31 is the inter coding mode, the inter prediction parameter and the motion vector variable length decoded by the variable length decoding unit 31 are output to the motion compensation prediction unit 36. Perform the process.

  The intra prediction unit 34, when the coding mode related to the coding block identified from the block division information variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode, the decoding stored in the intra memory 38 With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter output from the changeover switch 33 is performed to generate an intra predicted image.

  That is, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal, and generates a prediction image of the luminance signal. On the other hand, for the color difference signal, the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  The intra block copy prediction unit 35, when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the intra block copy coding mode, In the maximum coding block to which the prediction block belongs, a process of assuming a pixel value of each pixel in a region not yet decoded by a predetermined method is performed. In addition, the intra block copy prediction unit 35 selects, for each prediction block in the encoded block, a reference block that is a block that is closest to the prediction block from among regions that have already been decoded in the same picture. A process of searching and determining the reference block as a prediction image of the prediction block is performed.

  The motion compensation prediction unit 36 is stored in the motion compensation prediction frame memory 40 when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the inter coding mode. The inter prediction process (motion compensation prediction process) using the motion vector output from the changeover switch 33 and the inter prediction parameter is performed while referring to the decoded image, and the process of generating the inter prediction image is performed.

The addition unit 37 includes a decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 32, an intra prediction image generated by the intra prediction unit 34, an intra block copy prediction image generated by the intra block copy prediction unit 35, Or the inter prediction image produced | generated by the motion compensation prediction part 36 is added, and the process which calculates the decoding image same as the local decoding image output from the addition part 11 of FIG. 1 is implemented.
The intra memory 38 is a recording medium that stores the decoded image calculated by the adding unit 37 as a reference image used in the intra prediction process and the intra block copy prediction process.

The loop filter unit 39 performs a predetermined filter process on the decoded image calculated by the adder unit 37 and performs a process of outputting the decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, adaptive filter processing, and histogram correction processing. Specify whether to do this in slices.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image coding apparatus is configured as shown in FIG. 12, a loop filter unit 39 is configured as shown in FIG. The Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, the loop filter unit 39 also performs deblocking filter processing, pixel adaptive offset processing, and histogram correction processing. Consists of.

In the deblocking filter processing, when there is information for changing various parameters used for selecting the filter strength applied to the block boundary from the initial value with reference to the header information variable-length decoded by the variable-length decoding unit 31, the change information Based on the above, deblocking filtering is performed. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. If the index indicating the block classification method is not an index indicating that “offset processing is not performed”, each pixel in the block is classified according to the class classification method indicated by the index. To do. Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

However, in the pixel adaptive offset processing of the loop filter unit 13 of the image encoding device, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit). When a class classification method is selected for each block and adaptive offset processing for each class is performed, the loop filter unit 39 also has a pixel adaptive offset in units of blocks having the same fixed size as the loop filter unit 13. Perform the process.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

FIG. 19 is a configuration diagram illustrating a processing unit in the loop filter unit 39 that performs the histogram correction processing.
In FIG. 19, the histogram calculation unit 41 divides the decoded image calculated by the addition unit 11 into a plurality of blocks, and performs a process of calculating a pixel value histogram for each block of the local decoded image. The histogram calculator 41 constitutes a histogram calculator.
The pixel value correction unit 42 identifies a pixel on which noise accompanying coding distortion is superimposed from the histogram calculated by the histogram calculation unit 41, and performs a process of correcting the pixel value of the pixel on which the noise is superimposed. To do.
That is, the pixel value correction unit 42 performs a process of replacing the pixel value within the correction width w (correction pixel value range) including the peak pixel value in the histogram calculated by the histogram calculation unit 41 with the peak pixel value. The pixel value correction unit 42 constitutes a pixel value correction unit.

That is, in the histogram correction process, the histogram calculation unit 41 divides the decoded image based on the block division information of the histogram correction process variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 for each block. Referring to the flag indicating whether or not to perform histogram correction that has been subjected to variable length decoding in accordance with the above, if the flag is not a value indicating that “histogram correction processing is not performed”, a histogram of pixel values is calculated in the block .
The pixel value correction unit 42 specifies the peak pixel value of the histogram calculated by the histogram calculation unit 41 by a predetermined method. Since noise due to coding distortion exists in the vicinity of the peak pixel value, a pixel having a pixel value close to the peak pixel value (a pixel value within the correction width w that has been variable-length decoded by the variable-length decoding unit 31). On the other hand, encoding distortion is reduced by replacing the pixel value with a peak pixel value.

  In FIG. 17, the motion compensation prediction frame memory 40 is a recording medium that stores the decoded image after the filter processing of the loop filter unit 39 as a reference image used in the inter prediction processing (motion compensation prediction processing).

  In the example of FIG. 17, a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, an intra prediction unit 34, an intra block copy prediction unit 35, and a motion compensation prediction unit 36 that are components of the image decoding device. Each of the adder 37, the intra memory 38, the loop filter unit 39, and the motion compensated prediction frame memory 40 is configured by dedicated hardware (components other than the intra memory 38 and the motion compensated prediction frame memory 40 are, for example, It is assumed that the CPU is mounted on a semiconductor integrated circuit or a one-chip microcomputer), but the image decoding apparatus may be configured by a computer.

When the image decoding device is configured by a computer, the intra memory 38 and the motion compensated prediction frame memory 40 are configured on the computer memory, and a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, A program describing the processing contents of the intra prediction unit 34, the intra block copy prediction unit 35, the motion compensation prediction unit 36, the addition unit 37, and the loop filter unit 39 is stored in the memory of the computer, and the CPU of the computer stores the memory. The program stored in the program may be executed.
FIG. 20 is a flowchart showing the processing contents of the image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
In the first embodiment, each frame image of a video is used as an input image, intra prediction from encoded neighboring pixels or motion compensation prediction between adjacent frames is performed, and an obtained prediction difference signal is obtained. An image encoding device that performs compression processing by orthogonal transform / quantization and then performs variable length encoding to generate an encoded bitstream, and an image that decodes the encoded bitstream output from the image encoding device A decoding apparatus will be described.

  The image encoding apparatus in FIG. 1 performs intra-frame and inter-frame adaptive encoding by dividing a video signal into blocks of various sizes in response to local changes in the spatial and temporal directions of the video signal. It is characterized by. In general, a video signal has a characteristic that the complexity of the signal changes locally in space and time. When viewed spatially, a small image, such as a picture with a uniform signal characteristic in a relatively wide image area such as the sky or a wall, or a picture containing a person or fine texture, on a video frame. A pattern having a complicated texture pattern in the region may be mixed. Even when viewed temporally, the change in the pattern of the sky and the wall locally in the time direction is small, but because the outline of the moving person or object moves rigidly or non-rigidly in time, the temporal change Is big.

In the encoding process, a prediction difference signal with small signal power and entropy is generated by temporal and spatial prediction to reduce the overall code amount. However, the parameters used for the prediction are set as large as possible in the image signal region. If it can be applied uniformly, the code amount of the parameter can be reduced. On the other hand, if the same prediction parameter is applied to a large image region with respect to an image signal pattern having a large temporal and spatial change, the number of prediction differential signals increases because prediction errors increase. . Therefore, in a region where the temporal and spatial changes are large, the block size for performing the prediction process by applying the same prediction parameter is reduced, the amount of parameter data used for prediction is increased, and the power and entropy of the prediction difference signal are increased. It is desirable to reduce
In the first embodiment, in order to perform coding adapted to the general characteristics of such a video signal, first, prediction processing or the like is started from a predetermined maximum block size, and the video signal region is divided hierarchically. In addition, the prediction process and the encoding process of the prediction difference are adapted for each divided area.

First, the processing contents of the image encoding device in FIG. 1 will be described.
First, the encoding control unit 1 determines a slice division state of a picture to be encoded (current picture), and the size of a maximum encoding block used for encoding a picture and a hierarchy for dividing the maximum encoding block into layers. The upper limit of the number is determined (step ST1 in FIG. 16).
As a method of determining the size of the maximum coding block, for example, the same size may be determined for all the pictures according to the resolution of the video signal of the input image, or the local motion of the video signal of the input image The size difference may be quantified as a parameter, and a small size may be determined for a picture with high motion, while a large size may be determined for a picture with little motion.

  For example, the upper limit of the number of division layers can be determined by, for example, determining the same number of layers for all pictures according to the resolution of the video signal of the input image, or when the motion of the video signal of the input image is severe There is a method in which the number of hierarchies is increased so that finer movements can be detected, and when there are few movements, the number of hierarchies is set to be suppressed. Note that the size of the maximum coding block and the upper limit of the number of hierarchies for dividing the maximum coding block into layers are coded by a sequence level header or the like. In that case, the minimum block size of the encoded block may be encoded instead of the upper limit of the number of division layers. That is, since the size of the block when the maximum encoded block is divided up to the upper limit of the number of division layers is the minimum block size of the encoded block, the size of the maximum encoded block and the size of the encoded block are determined on the image decoding device side. The upper limit of the number of divided hierarchies can be specified from the minimum block size.

  Also, the encoding control unit 1 selects an encoding mode corresponding to each encoding block divided hierarchically from one or more available encoding modes (step ST2 in FIG. 16). That is, the encoding control unit 1 divides the image area of the maximum encoding block size into encoded blocks having the encoding block size hierarchically until reaching the upper limit of the number of division layers defined above. A coding mode for each coding block is determined.

The coding modes include one or more intra coding modes (collectively referred to as “INTRA”), one or more intra block copy coding modes (collectively referred to as “ICOPY”), There are one or a plurality of inter coding modes (collectively referred to as “INTER”), and the coding control unit 1 can select from all the coding modes available in the picture or a subset thereof. The encoding mode corresponding to each encoding block is selected.
However, each coding block that is hierarchically divided by the block division unit 3 to be described later is further divided into one or a plurality of prediction blocks, which are units for performing prediction processing, and the division state of the prediction block is also coded mode. Is included as information. That is, the coding mode is an index for identifying what kind of prediction block division the intra coding mode, intra block copy coding mode, or inter coding mode is.
Since the encoding mode selection method by the encoding control unit 1 is a known technique, a detailed description thereof will be omitted. For example, an encoding process for an encoding block is performed using any available encoding mode. There is a method in which coding efficiency is verified by performing and a coding mode having the best coding efficiency is selected from among a plurality of available coding modes.

  The encoding control unit 1 determines a quantization parameter and a transform block division state used when the differential image is compressed for each encoding block, and is used when the prediction process is performed. A prediction parameter (intra prediction parameter, intra block copy prediction parameter, or inter prediction parameter) is determined. However, when the encoded block is further divided into prediction block units for performing prediction processing, a prediction parameter (intra prediction parameter, intra block copy prediction parameter, or inter prediction parameter) is selected for each prediction block.

Here, FIG. 2 is an explanatory diagram showing the conversion block size when performing compression processing (conversion processing, quantization processing) of the luminance signal and the color difference signal in the 4: 2: 0 format signal. As shown in FIG. 2, the transform block size is determined by hierarchically dividing the encoded block into a quadtree.
For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining this, it is possible to determine the optimal transform block division shape from the viewpoint of the trade-off between the code amount and the coding error.

  For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. In addition, the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction. The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
When the input signal format is a YUV 4: 4: 4 signal, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block as shown in FIG. Configure as follows.

The encoding control unit 1 includes predictive differential encoding including transform block partition information indicating transform block partition information in a block to be encoded, a quantization parameter that defines a quantization step size when transform coefficients are quantized, and the like. The parameter is output to the transform / quantization unit 9, the inverse quantization / inverse transform unit 10, and the variable length coding unit 15.
The encoding control unit 1 outputs intra prediction parameters to the intra prediction unit 5 as necessary.
Also, the encoding control unit 1 outputs the intra block copy prediction parameters to the intra block copy prediction unit 6 as necessary.
Also, the encoding control unit 1 outputs inter prediction parameters to the motion compensation prediction unit 7 as necessary.

When a video signal is input as an input image, the slice division unit 2 divides the input image into slices that are one or more partial images according to the slice division information determined by the encoding control unit 1.
Each time each slice is input from the slice dividing unit 2, the block dividing unit 3 divides the slice into the maximum encoded block size determined by the encoding control unit 1, and further encodes the divided maximum encoded block. The coding block is hierarchically divided into coding blocks determined by the coding control unit 1, and the coding blocks are output.

Here, FIG. 21 is an explanatory diagram showing an example in which the maximum coding block is hierarchically divided into a plurality of coding blocks.
In FIG. 21, the maximum coding block is a coding block whose luminance component described as “0th layer” has a size of (L ⁰ , M ⁰ ). Starting from the maximum encoding block, the encoding block is obtained by performing hierarchical division to a predetermined depth separately defined by a quadtree structure. At depth n, the coding block is an image area of size (L ⁿ , M ⁿ ). However, L ⁿ and M ⁿ may be the same or different, but FIG. 21 shows a case of L ⁿ = M ⁿ .

Hereinafter, the coding block size determined by the coding control unit 1 is defined as the size (L ⁿ , M ⁿ ) in the luminance component of the coding block. Since quadtree partitioning is performed, (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) always holds. Note that in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ), but 4: 2. : When the 0 format is handled, the encoding block size of the corresponding color difference component is (L ⁿ / 2, M ⁿ / 2).

Hereinafter, the coding block of the n hierarchy expressed in B ^n, denote the encoding modes selectable by the coding block B ⁿ with m (B ^n). In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, or common to all color components. It may be configured to use a mode. Hereinafter, unless otherwise specified, description will be made assuming that it indicates a coding mode for a luminance component of a coding block of a YUV signal and 4: 2: 0 format.

As shown in FIG. 21, the coding block B ⁿ is divided into one or a plurality of prediction blocks representing a prediction processing unit by the block dividing unit 3. Hereinafter, a prediction block belonging to the coding block B ⁿ is ^denoted as P _i ⁿ (i is a prediction block number in the n-th layer). FIG. 21 shows an example of P ₀ ⁰ and P ₁ ⁰ . How the prediction block is divided in the coding block ^Bn is included as information in the coding mode m ( ^Bn ). Prediction blocks P _i ⁿ are all subjected to prediction processing according to the encoding mode m (B ⁿ ). For each prediction block P _i ⁿ , individual prediction parameters (intra prediction parameters, intra block copy prediction parameters, or inter prediction parameters) are used. ) Can be selected.

For example, the encoding control unit 1 generates a block division state as illustrated in FIG. 22 for the maximum encoding block, and specifies the encoding block. A rectangle surrounded by a dotted line in FIG. 22A represents each coding block, and a block painted with diagonal lines in each coding block represents a division state of each prediction block. FIG. 22B shows a situation in which the coding mode m (B ⁿ ) is assigned by hierarchical division in the quadtree graph for the example of FIG. Nodes surrounded by □ in FIG. 22B are nodes (encoding blocks) to which the encoding mode m (B ⁿ ) is assigned. Information of the quadtree graph is output from the encoding control unit 1 to the variable length encoding unit 15 together with the encoding mode m (B ⁿ ), and is multiplexed into the bit stream.

The changeover switch 4 is output from the block dividing unit 3 when the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA). When the coding block B ⁿ is output to the intra prediction unit 5 and the coding mode m (B ⁿ ) determined by the coding control unit 1 is the intra block copy coding mode (m (B ⁿ ) ∈ICOPY The encoding block B ⁿ output from the block division unit 3 is output to the intra block copy prediction unit 6, and the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode. In some cases (when m (B ⁿ ) εINTER), the encoded block B ⁿ output from the block dividing unit 3 is output to the motion compensation prediction unit 7.

In the intra prediction unit 5, the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA), and the coding block B is changed from the changeover switch 4 to the coding block B. ^{When n} is received (step ST3 in FIG. 16), the encoding block B is determined using the intra prediction parameter determined by the encoding control unit 1 while referring to the local decoded image stored in the intra memory 12. and implementing intra prediction process for each of the prediction block _P ^{i n} in the ^n, generates an intra prediction image _{P INTRAi} ⁿ (step ST4 in FIG. 16).

However, although details will be described later, since the encoded pixel adjacent to the prediction block is used when performing the process of generating the intra prediction image, the process of generating the intra prediction image is the prediction block used for the prediction process. Must always be performed in units of transform blocks so that pixels adjacent to are already encoded. Therefore, in a coding block in which the coding mode is the intra coding mode, the block size of the selectable transform block is limited to the size of the prediction block or smaller and the transform block is smaller than the prediction block (in the prediction block). In the case where there are a plurality of transform blocks, the intra prediction process using the intra prediction parameters defined in the prediction block is performed for each transform block to generate an intra predicted image.
Incidentally, since it is necessary to image decoding apparatus of FIG. 17 to generate exactly the same intra prediction image and the intra prediction image P _INTRAi ^n, intra prediction parameters used for generating the intra prediction image P _INTRAi ^n, the encoding control unit 1 is output to the variable length encoding unit 15 and multiplexed into the bit stream. Details of processing contents of the intra prediction unit 5 will be described later.

The intra block copy prediction unit 6 is the intra block copy encoding mode when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is m (B ⁿ ) ∈ICOPY, and When the encoded block B ⁿ is received (step ST3 in FIG. 16), each prediction block P _i ⁿ in the encoded block B ⁿ is compared with the locally decoded image stored in the intra memory 12, and a block shift vector is obtained. Explore. That is, the intra block copy prediction unit 6, in the local decoded image stored in the intra-memory 12, to identify the block (reference block) in the region which is the most approximate to the prediction block P _i ^n, that Search for a block shift vector pointing to the reference block.
Intra block copy prediction unit 6, when searching for a block shift vector pointing to the reference block, the reference block whose block shift vector is pointing as the predicted image of the prediction block P _i ^n, generates an intra block copy predicted image P _ICOPYi ⁿ ( Step ST5 in FIG.

In the image decoding apparatus of FIG. 17, it is necessary to generate exactly the same intra block copy predictive image and the intra block copy predicted image P _ICOPYi ^n, intra block copy prediction is used for generating the intra block copy predicted image P _ICOPYi ⁿ The parameters are output from the encoding control unit 1 to the variable length encoding unit 15 and multiplexed into the bit stream.
Examples of the intra block copy prediction parameter include a block shift vector searched by the intra block copy prediction unit 6. Also, the block shift vector is encoded by using the block shift vector of the previous encoded prediction block or the difference value with the block shift vector of the encoded prediction block around the prediction block as a part of the intra block copy prediction parameter. You may make it do.

The motion compensated prediction unit 7 uses the encoding switch 4 from the changeover switch 4 when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode (when m (B ⁿ ) ∈INTER). When receiving B ⁿ (step ST3 in FIG. 16), each prediction block P _i ⁿ in the encoded block B ⁿ is compared with the locally decoded image after filtering stored in the motion compensated prediction frame memory 14. It searches a motion vector, by using the inter prediction parameters determined by the motion vector and the encoding control unit 1, to implement the inter-prediction processing for each of the prediction block P _i ⁿ in the encoding block B ^n, inter generating a predicted image _{P INTERi} ⁿ (step ST6 in FIG. 16).

Since the image decoding apparatus of FIG. 17 needs to generate exactly the same inter prediction image and the inter-predicted image P _INTERi ^n, inter prediction parameters used for generating the inter prediction image P _INTERi ⁿ from the coding controller 1 The data is output to the variable length encoding unit 15 and multiplexed into the bit stream.
The motion vector searched by the motion compensation prediction unit 7 is also output to the variable length encoding unit 15 and multiplexed into the bit stream.

Subtracting unit 8 receives the encoded block ^{B n} from the block dividing unit 3, from its prediction block _P ^{i n} the coded block ^{B n,} the intra prediction image _{P INTRAi} ⁿ generated by the intra prediction unit 5, the intra block copy prediction unit 6 intra block copy predicted image P _{ICOPYi n} generated ^by, or subtracts one of the inter-prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 7, which is the subtraction result difference image the prediction difference signal _e ^{i n} outputs the transform and quantization unit 9 shown (step ST7 in FIG. 16).

When the transform / quantization unit 9 receives the prediction difference signal e _i ⁿ from the subtraction unit 8, the transform / quantization unit 9 refers to the transform block division information included in the prediction difference encoding parameter determined by the encoding control unit 1, and performs the prediction. orthogonal transform processing with respect to the difference signal e _i ⁿ (e.g., DCT (discrete cosine transform) or DST (discrete sine transform), the orthogonal transform for KL conversion and the base design have been made in advance to the particular learning sequence) to transform This is performed for each block, and a conversion coefficient is calculated.
Also, the transform / quantization unit 9 refers to the quantization parameter included in the prediction differential encoding parameter, quantizes the transform coefficient of the transform block unit, and reverses the compressed data that is the transform coefficient after quantization. The data is output to the quantization / inverse transform unit 10 and the variable length coding unit 15 (step ST8 in FIG. 16). At this time, the quantization process may be performed using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient.

  As the quantization matrix, an independent matrix can be used for each color signal and coding mode (intra coding or inter coding) at each orthogonal transform size. As an initial value, an image coding device and an image are used. In the decoding apparatus, it is possible to select whether to use a previously prepared quantization matrix or an already encoded quantization matrix or to use a new quantization matrix. Accordingly, the transform / quantization unit 9 sets flag information indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, in a quantization matrix parameter to be encoded. .

  Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 11 is set as a quantization matrix parameter to be encoded. On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance by the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Thus, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, only the quantization matrix prepared in advance can be selected in advance in the image encoding device and the image decoding device. Then, the transform / quantization unit 9 outputs the set quantization matrix parameter to the variable length coding unit 15.

When the inverse quantization / inverse transform unit 10 receives the compressed data from the transform / quantization unit 9, the inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 Then, the compressed data is inversely quantized for each transform block. When the transform / quantization unit 9 uses a quantization matrix for the quantization process, the corresponding inverse quantization process is performed with reference to the quantization matrix even during the inverse quantization process.
Further, the inverse quantization / inverse transform unit 10 performs inverse orthogonal transform processing (for example, inverse DCT, inverse DST, inverse KL transform, etc.) on transform coefficients that are compressed data after inverse quantization for each transform block. calculates a local decoded prediction difference signal corresponding to the prediction difference signal e _i ⁿ output from the subtraction unit 8 outputs to the adder 11 (step ST9 of FIG. 16).

Addition unit 11, when the inverse quantization and inverse transform unit 10 receives the local decoded prediction difference signal, and the local decoded prediction difference signal, an intra prediction image P _{INTRAi n} generated by the intra prediction unit ^5, the intra block copy prediction part 6 intra block copy predicted image _{P ICOPYi} ⁿ generated by, or by adding one of the inter-prediction image _{P INTERi} ⁿ generated by the motion compensation prediction unit 7 calculates a local decoded image (FIG. 16 Step ST10).
The adding unit 11 outputs the locally decoded image to the loop filter unit 13 and stores the locally decoded image in the intra memory 12. This locally decoded image becomes an encoded image signal used in the subsequent intra prediction process and intra block copy prediction process.

When the loop filter unit 13 receives the local decoded image from the adder unit 11, the loop filter unit 13 performs a predetermined filter process on the local decoded image, and stores the filtered local decoded image in the motion compensated prediction frame memory 14. (Step ST11 in FIG. 16). Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 13 determines whether or not to perform each of the deblocking filter process, the pixel adaptive offset process, the adaptive filter process, and the histogram correction process, and uses a variable length code with the valid flag of each process as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 12 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 12 are merely examples, and the types and application order of the loop filters to be applied are not necessarily as shown in FIG. For example, the loop filter unit 13 is configured by only three types of deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, and the application order is in order of deblocking filter processing, histogram correction processing, and pixel adaptive offset processing. And so on.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are used. One classification method is selected from the methods. Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating the coding distortion is calculated for each class. Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.

As a classifying method, a method of classifying by the size of a pixel value of a locally decoded image (referred to as a BO method), or a classification according to a situation around each pixel (whether it is an edge portion or the like) for each edge direction. There is a technique (referred to as EO technique). These methods are prepared in advance by the image encoding device and the image decoding device in advance. For example, as shown in FIG. 23, when no offset processing is performed, these methods are defined as one of the class classification methods. An index indicating which method is used for class classification is selected for each block.
Therefore, the pixel adaptive offset processing outputs the block division information, the index indicating the class classification method for each block, and the offset information for each block to the variable length encoding unit 15 as header information.

In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.
Also, in adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for superimposed distortion is designed for each region (local decoded image) belonging to each class. Then, the local decoded image is filtered. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.

  Here, as a class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying the image in units of blocks according to local characteristics (dispersion, etc.). Further, the number of classes used in the adaptive filter process may be set in advance to a common value in the image encoding device and the image decoding device, or may be one of the parameters to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

24A and 24B are flowcharts showing the contents of histogram correction processing in the image coding apparatus according to Embodiment 1 of the present invention.
Hereinafter, the flow of the histogram correction process will be described with reference to FIGS. 24A and 24B.
In the histogram correction process, first, the histogram calculation unit 21 in FIG. 13 divides the input image indicated by the video signal into a plurality of blocks and also divides the local decoded image calculated by the addition unit 11 into a plurality of blocks.
The first error calculation unit 23 of the pixel value correction unit 22 calculates an error E _min between the input image in units of blocks divided by the histogram calculation unit 21 and the locally decoded image (steps ST100 to ST101 in FIG. 24A).

As a calculation method of the error _Emin , for example, SAD (Sum of Absolute Differences) for summing absolute value differences of pixel values, SSD (Sum of Squared Differences) for summing values obtained by squaring pixel value differences, or the like may be used. Conceivable. The smaller these indexes, the more similar the input image and the local decoded image, and the smaller the distortion due to encoding.
In addition, an index representing image quality instead of error may be calculated. As an index representing image quality, PSNR (Peak Signal-to-Noise Ratio) representing the ratio of the maximum power that a signal can take and the noise that causes deterioration can be considered. When an index representing image quality is used, the larger the value, the smaller the encoding distortion.

Further, the histogram calculation unit 21 calculates a histogram of pixel values for each block of the locally decoded image (step ST102 in FIG. 24A), selects a pixel value with the maximum histogram value, and sets the pixel value to p ( Hereinafter, it is referred to as “peak pixel value”) (step ST103 in FIG. 24A).
Note that the step width of the histogram is not necessarily set to 1, and may be 1 or more. Increasing the step width for generating the histogram reduces the accuracy of the histogram, but it is possible to reduce the memory required for generating the histogram. When the histogram step width is 1 or more, the peak pixel value p may be set to the median value of the step width.

  Further, it is not always necessary to calculate the histogram using all the pixels in the block, and the histogram may be obtained using only some of the pixels. By reducing the number of pixels used for the histogram calculation, it is possible to reduce the amount of calculation required for the histogram calculation.

  In addition to directly using the obtained histogram to select the pixel value that maximizes the value, it is also conceivable to obtain the maximum value after differentiating the histogram, for example. By doing in this way, the effect which can specify only the remarkable peak with a big difference from an adjacent peak can be anticipated.

Next, the second error calculation unit 24, the correction pixel value range storage unit 25, and the iterative processing unit 26 perform a process of determining the correction width w in FIG.
First, as the initial value of the correction width w, after the correction width w _temp is set to 1, for example (step ST104 in FIG. 24A), the second error calculation unit 24 uses the pixel value ((peak) in the correction width w _temp Pixel value p-correction width w _temp ) to (pixel value of peak pixel value p + correction width w _temp ) are replaced with peak pixel value p (steps ST 105 to ST 108 in FIG. 24A). However, the replacement here is only temporary replacement to determine the correction width w, and the actual replacement (correction of the pixel value) is performed by the pixel value replacement unit 27 after the optimum correction width w is determined. Done.

ここで、（ｘ，ｙ）はブロック単位の局所復号画像における画素の座標を表し、Ｌ（ｘ，ｙ）は置換前の画素値、Ｌ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ_ｔｅｍｐが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ_ｔｅｍｐが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。
ここでは、補正幅ｗ_ｔｅｍｐをピーク画素値ｐよりも大きい側と小さい側で同一としているが、必ずしも両者を一致させる必要はなく、それぞれに対して異なる補正幅ｗ_ｔｅｍｐを使用してもよい。 Specifically, pixel value replacement is performed as follows.

Here, (x, y) represents the pixel coordinates in the local decoded image in block units, L (x, y) represents the pixel value before replacement, and L ′ (x, y) represents the pixel value after replacement. ing.
The above correction is performed on all or some of the pixels in the block. In the above equation, when p-w _temp falls below the lower limit of the pixel value, this is replaced with the lower limit of the pixel value, and when p + w _temp exceeds the upper limit of the pixel value, it is replaced with the upper limit of the pixel value. Also good.
Here, although the correction width w _temp is the same on the side larger than the peak pixel value p and the smaller side, it is not always necessary to match both, and different correction widths w _temp may be used for each.

When the pixel value in the correction width w _temp is replaced with the peak pixel value p, the second error calculation unit 24 calculates an error E between the block-unit local decoded image and the block-unit input image (FIG. 24A). Step ST109).
When the second error calculation unit 24 calculates the error E, the corrected pixel value range storage unit 25 compares the error E with the error E _min calculated by the first error calculation unit 23 (step of FIG. 24A). ST110) If the error E is smaller than the error E _min (E <E _min ), the error E _min calculated by the first error calculation unit 23 is replaced with the error E, and the correction width w _{temp is changed} to the current peak pixel. The optimum correction width w ⁱ (initial value is 0) for the value is overwritten and saved (step ST111 in FIG. 24A).
Here, i is an index indicating the peak pixel value being processed, and the initial value is 0. When the error E is larger than the error E _min , the error is increased by replacing the pixel value, and therefore the error E _min and the correction width w _temp are not updated. When an index representing image quality such as PSNR is used as an error, the inequality sign in step ST110 is reversed.

Then, repetitive processing unit 26 for the correction width _{w temp,} for example, by adding 1, to update the correction width _{w temp} (step of FIG. 24A ST 112), substituted to the peak pixel value p After the pixel value in the corrected correction width w _temp is returned to the original pixel value (step ST113 in FIG. 24A), the updated correction width w _temp is compared with the preset maximum correction width w _max (in FIG. 24A). Step ST114).
If the updated correction width w _temp is equal to or smaller than the maximum correction width w _max (w _temp ≦ w _max ), the iterative processing unit 26 instructs the second error calculation unit 24 to perform the error E calculation process again. Then, the process returns to step ST105.
On the other hand, if the updated correction width w _temp is larger than the maximum correction width w _max (w _temp > w _max ), the process proceeds to step ST115.

The number of repetitions of the processes of steps ST105 to ST114 depends on the maximum correction width w _max . The value of the maximum correction width w _max may be freely set by the image coding apparatus. However, if a large value is set, the image quality improvement effect by the histogram correction processing is enhanced, but the calculation time increases because the number of repetitions increases. Conversely, if a small value is set, the image quality improvement effect is reduced, but there is a trade-off relationship that the calculation time can be suppressed. A specific example of the maximum correction width w _max is 15 or the like.

Note that the value of w _max does not always have to be a fixed value, and may be changed according to the encoding parameter during the encoding process. For example, when the quantization parameter is large and the coding distortion is large, w _max is increased to improve the image quality improvement effect, and when the quantization parameter is small and the coding distortion is small, w _max is decreased to reduce the calculation time. It is possible to do.

  Further, the correction width w may be limited to a value such as a power of 2, for example. In this case, since only the exponent part needs to be encoded when encoding the correction width w, it is possible to reduce the code amount necessary for encoding the correction width w.

ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。 When the updated correction width w _temp becomes larger than the maximum correction width w _max (w _temp > w _max ), the pixel value replacement unit 27 optimizes that is stored by the correction pixel value range storage unit 25 as follows. The pixel values in the correction width w ⁱ ((peak pixel value p−correction width w ⁱ ) to (pixel value of peak pixel value p + correction width w ⁱ )) are replaced with the peak pixel value p (step ST115 in FIG. 24B). ST118).

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.

Next, the pixel value replacement unit 27 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculation unit 21 in ST102 ( Steps ST119 to ST120 in FIG. 24B).
The pixel value replacement unit 27 compares the index i with a preset maximum peak number P _max (step ST121 in FIG. 24B), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 21 is instructed to reselect values, and the process returns to step ST103.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.

In the first embodiment, the pixel value correction unit 22 obtains an optimum correction width w ⁱ for each peak pixel value of the histogram, and the variable length coding unit 15 obtains the optimum correction width w ⁱ for each peak pixel value. Encode.
Therefore, it is necessary to encode the correction width w ⁱ by the number of peaks P _max to be corrected (encoding all of w ⁰ , w ¹ , w ² ,..., W ^Pmax−1 ), and the maximum peak When the number P _max increases, the image quality improvement effect increases, but the code amount increases. Conversely, when the maximum peak number P _max decreases, the image quality improvement effect decreases but the code amount can be suppressed.
Naturally, the calculation time increases as the maximum number of peaks P _max increases. As a specific example of the maximum peak number P _max , 4 or the like can be considered.

For example, when the correction width w ⁱ for all the peak pixel values is 0 (when no effect is seen even if the histogram correction process is performed), the histogram correction process is not applied to the block. It is also possible to do. By doing in this way, it can avoid encoding unnecessary information and can improve encoding efficiency by reducing code amount.
Various other methods can be used to determine whether or not to apply the histogram correction processing. For example, how much encoding efficiency is achieved when encoding is performed by actually performing the histogram correction processing. It may be determined whether or not it can be improved, or may be determined using image characteristics such as texture and edges, and the result of determining whether or not the screen content.
Therefore, in the histogram correction process, first, the block division information is output as header information to the variable length encoding unit 15, and a flag indicating whether or not to perform correction for each block. An appropriate correction width w ⁱ (i = 0, 1,..., P _max −1) is output to the variable length encoding unit 15 as header information.
Whether or not to perform the histogram correction process may be determined in units of video, slices, or pictures. For example, if a flag is added to the header unit (sequence parameter set) in units of video and it can be determined in advance that the video is not screen content and histogram correction processing is not effective, this flag should be set to 0 or the like. Thus, the histogram correction process may not be performed. Of course, it is also conceivable to add a flag to the header (picture parameter set, slice parameter set) in units of pictures and slices, and switch in units of pictures and slices. By doing so, it is not necessary to encode parameters relating to the histogram correction process for a video, picture, or slice that does not have the effect of the histogram correction process, so that the code amount can be reduced.
In the histogram correction process, for example, the block is always divided into fixed-size block units such as the maximum coding block, a histogram is calculated for each block, and the correction width w ⁱ for each peak pixel value is obtained to perform the correction process. May be. In this case, the above-described block division information becomes unnecessary and the code amount is reduced by the amount of code required for the block division information, so that the coding efficiency can be improved.

Further, the correction width w ^{i of the} block being processed is predicted using the correction width of the neighboring block that has already undergone the histogram correction processing, and only the prediction error is output to the variable length encoding unit 15 for encoding. May be. In this way, the code amount of the correction width w ⁱ can be reduced.

The processes in steps ST3 to ST10 in FIG. 16 are repeatedly performed until the processes for all the encoding blocks ^Bn divided hierarchically are completed. When the processes for all the encoding blocks ^Bn are completed, the process of step ST14 is performed. The process proceeds (steps ST12 and ST13 in FIG. 16).

The variable length encoding unit 15 uses the compressed data output from the transform / quantization unit 9 and the block division information (FIG. 22B) in the maximum encoded block output from the encoding control unit 1 as an example. (Quadrant tree information), encoding mode m (B ⁿ ), prediction differential encoding parameter, loop filter parameter, and intra prediction parameter output from the encoding control unit 1 (the encoding mode is an intra encoding mode). ), An intra block copy prediction parameter (when the encoding mode is the intra block copy encoding mode) or an inter prediction parameter (when the encoding mode is the inter encoding mode), and the motion compensated prediction unit 7 Motion vectors (when the encoding mode is the inter encoding mode) and all the peaks calculated by the pixel value correction unit 22. A correction width w ⁱ with respect to the pixel value to the variable length coding to generate a coded data indicating their encoding result (step ST14 in FIG. 16).

At this time, as a method of encoding compressed data that is a quantized orthogonal transform coefficient, the transform block is further divided into blocks of 4 × 4 pixel units (encoding sub-blocks) called Coefficient Group (CG), and CG Coding of coefficients is performed for each unit.
FIG. 25 shows the coding order (scan order) of coefficients in a 16 × 16 pixel transform block. In this way, 16 CGs in units of 4 × 4 pixels are encoded in order from the lower right CG, and each CG encodes 16 coefficients in the CG in order from the lower right coefficient.

Specifically, first, flag information indicating whether or not a significant (non-zero) coefficient exists in 16 coefficients in the CG is encoded, and then a significant (non-zero) coefficient exists in the CG. Only in this case, whether each coefficient in the CG is a significant (non-zero) coefficient is encoded in the above order, and finally, the coefficient value information is encoded in order for the significant (non-zero) coefficient. This is performed in the above order in units of CG. In this case, the encoding efficiency by entropy encoding can be increased by using a biased scan order so that significant (non-zero) coefficients are generated as continuously as possible.
Since the coefficient after the orthogonal transformation represents the lower coefficient of the low frequency component as it is closer to the upper left, including the DC component located at the upper left, generally, the closer to the upper left, the more significant as the example is shown in FIG. Since many (non-zero) coefficients are generated, as shown in FIG. 25, encoding can be performed efficiently by encoding sequentially from the lower right.

  In the above description, a 16 × 16 pixel conversion block has been described. However, in a block size other than 16 × 16 pixels, such as an 8 × 8 pixel conversion block or a 32 × 32 pixel conversion block, a code in CG (encoding sub-block) unit is used. It shall be implemented. In addition, for 4 × 4 pixel and 8 × 8 pixel conversion blocks for which intra prediction is selected, processing is performed in the scan order shown in FIG. 27 instead of the scan order of FIG. 25 according to the index of the intra prediction mode. . This is because the frequency component distribution of the residual signal tends to differ depending on the direction of intra prediction.

Further, as illustrated in FIG. 15, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data. However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.
The sequence level header includes an image size, a color signal format, a bit depth of a signal value of a luminance signal or a color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. , Histogram correction processing) effective flag information, quantization matrix effective flag information, and the like, which are header information that is generally common to each sequence unit.

The picture level header is a collection of header information set in units of pictures such as an index of a sequence level header to be referenced, the number of reference pictures at the time of motion compensation, an entropy encoding probability table initialization flag, and the like.
The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a collection of parameters in units of slices such as flag information indicating whether or not to perform each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process, histogram correction process) in the filter unit 13.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded. If supplementary information also exists, it is defined as a unique NAL unit. The picture data is defined as an access unit and indicates a unit of data access including encoded data of one picture.

Next, the processing content of the intra estimation part 5 is demonstrated in detail.
The intra prediction unit 5, as described above, with reference to the intra prediction parameters of the prediction block P _i ^n, to implement intra prediction processing for the prediction block P _i ^n, but to generate an intra prediction image P _INTRAi ⁿ , will be described here intra process for generating an intra prediction image predicted block P _i ⁿ in the luminance signal.

Figure 28 is an explanatory diagram showing an example of the prediction block P _i ^n-selectable intra prediction modes for intra-coded blocks B ^n, and the index value of the intra prediction mode, the prediction direction vector indicated by the intra-prediction mode Show. The index value of the intra prediction mode indicates the intra prediction parameter. In addition, you may comprise so that the number of intra prediction modes may differ according to the size of the block used as a process target. Intra-prediction efficiency decreases for large-sized blocks, so the number of selectable intra-prediction directions is reduced, and for small-sized blocks, the number of selectable intra-prediction directions is increased to reduce the amount of computation. can do.

First, since the process which produces | generates an intra estimated image uses the encoded pixel adjacent to the block of a process target, as above-mentioned, it must be performed per conversion block. Here, the transform block that generates the intra predicted image is referred to as a predicted image generation block. Therefore, the intra prediction unit 5 may implement an intra-prediction image generation processing described below to the predicted image generation block generates an intra prediction image predicted block P _i ^n. The size of the predicted image generation block is assumed to be l _i ⁿ × m _i ⁿ pixels.
FIG. 29 is an explanatory diagram illustrating an example of a pixel used when generating a predicted value of a pixel in a predicted image generation block in the case of l _i ⁿ = m _i ⁿ = 4. In FIG. 29, the encoded pixels (2 × l _i ⁿ +1) and the left encoded pixels (2 × m _i ⁿ ) on the predicted image generation block are used as pixels for prediction. The number of pixels used for prediction may be more or less than the pixels shown in FIG. In FIG. 29, pixels for one row or one column in the vicinity of the predicted image generation block are used for prediction. However, two or two or more pixels may be used for prediction.

If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs is 0 (plane (Planar) prediction) includes a coded pixels adjacent to the top of the predicted image generation block, the predicted image generation Using the encoded pixels adjacent to the left of the block, a predicted image is generated using a value interpolated according to the distance between these pixels and the prediction target pixel in the predicted image generation block as a predicted value.
When the index value of the intra prediction mode for the prediction block P _i ⁿ to which the prediction image generation block belongs is 1 (average (DC) prediction), encoded pixels adjacent to the prediction image generation block and the prediction image A predicted image is generated using the average value of the encoded pixels adjacent to the left of the generated block as the predicted value of the pixels in the predicted image generating block.

・領域Ａ（Ｐ_ｉ ^ｎの左上の画素）
ａ_０＝１／２，ａ_１＝１／４，ａ_２＝１／４
・領域Ｂ（領域Ａ以外のＰ_ｉ ^ｎの上端の画素）
ａ_０＝３／４，ａ_２＝１／４，（ａ_１＝０）
・領域Ｃ（領域Ａ以外のＰ_ｉ ^ｎの左端の画素）
ａ_０＝３／４，ａ_１＝１／４，（ａ_２＝０） Further, filter processing for smoothing the block boundary is performed on the regions A, B, and C in FIG. For example, according to the following formula (1), the filter processing is performed using the following filter coefficients with the reference pixel arrangement of the filter of FIG.

· Area A _(the upper left pixel of the _P ^{i n)}
a ₀ = 1/2, a ₁ = ¼, a ₂ = ¼
- region B (the upper end of the pixel of _P ^{i n} other than the region A)
a ₀ = 3/4, a ₂ = ¼, (a ₁ = 0)
· Area C (the leftmost pixel of the _P ^{i n} other than the region A)
a ₀ = 3/4, a ₁ = ¼, (a ₂ = 0)

In equation (1), a _n (n = 0, 1, 2) is a filter coefficient applied to the reference pixel, and p _n (n = 0, 1, 2) is a reference to a filter including the pixel to be filtered p ₀ . The pixel, S ′ (p ₀ ) is the predicted value after the filtering process in the filtering target pixel p ₀ , and S (p _n ) (n = 0, 1, 2) is the filter of the reference pixel including the filtering target pixel p ₀ It represents the predicted value before processing.

  Furthermore, the block size of the predicted image generation block that performs the filtering process may be limited. In general, when the prediction value is changed by filtering only at the block edge, since the ratio of the area where the prediction value changes due to the filter processing is small in a block having a large block size, the prediction residual caused by the change in the prediction value is generated. The change of the difference signal is expressed by a very high frequency component, and the encoding efficiency tends to be deteriorated because the high frequency component is encoded. Also, by giving priority to encoding efficiency and not encoding this high frequency component, the change in the prediction residual signal at the block end cannot be restored, and the block boundary tends to be distorted. .

  On the other hand, in a block having a small block size, since the ratio of the area where the prediction value changes due to the filter processing is large, the change in the prediction residual signal caused by the change in the prediction value is as in the case of the block having a large block size. Therefore, the residual signal can be appropriately encoded, and the quality of the decoded image can be improved by increasing the continuity of the block boundary by this filter processing. Therefore, for example, in a predicted image generation block having a block size of 32 × 32 pixels or more, the above-described filter processing is not applied, and the above-described filter processing is applied only to blocks smaller than 32 × 32 pixels, thereby obtaining a conventional average value. It is possible to suppress an increase in the calculation amount while improving the prediction performance rather than the prediction.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図３２を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の輝度値（復号された輝度値）である。また、算出した予測値が輝度値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ belonging to the prediction image generation block 26 (vertical prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (2) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 32) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is the luminance value (decoded luminance value) of the encoded pixel at the coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the luminance value can take, the value is rounded so that the predicted value falls within the range.

  Note that the equation in the first line of equation (2) is MPEG-4 AVC / H. The amount of change S (−1, y) −S (−1, −1) in the vertical direction of adjacent encoded pixels with respect to S (x, −1), which is the predicted value of the vertical direction prediction in H.264. ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (2) is , MPEG-4 AVC / H. The same prediction formula as the vertical direction prediction in H.264 is shown.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図３２を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の輝度値（復号された輝度値）である。また、算出した予測値が輝度値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs 10 (horizontal prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (3) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 32) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is the luminance value (decoded luminance value) of the encoded pixel at the coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the luminance value can take, the value is rounded so that the predicted value falls within the range.

  Note that the expression on the first line of Expression (3) is MPEG-4 AVC / H. The amount of change in luminance value S (x, −1) −S (−1, −1) in the horizontal direction of an adjacent encoded pixel with respect to S (−1, y), which is the predicted value of the horizontal direction prediction in H.264. ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (3) is , MPEG-4 AVC / H. The same prediction formula as the horizontal prediction in H.264 is shown.

  However, the block size of the predicted image generation block that performs the vertical direction prediction of Expression (2) and the horizontal direction prediction of Expression (3) may be limited. In general, when the prediction value is changed by performing a filter process that adds a value proportional to the amount of change in the luminance value in the prediction direction only at the block edge, the block edge filter of the prediction image generation block described above is used in a block having a large block size. Since the ratio of the area where the predicted value changes due to processing is small, the change in the prediction residual signal caused by the change in the predicted value is represented by a very high frequency component. Therefore, the encoding efficiency tends to deteriorate. Also, by giving priority to encoding efficiency and not encoding this high frequency component, the change in the prediction residual signal at the block end cannot be restored, and the block boundary tends to be distorted.

  On the other hand, in a block with a small block size, since the ratio of the region where the prediction value changes due to the above filtering process is large, the change in the prediction residual signal caused by the change in the prediction value is large when the block size is large. The residual signal can be appropriately encoded without being represented by such high frequency components, and the quality of the decoded image can be improved by the increase in the continuity of the block boundary by this filter processing. Therefore, for example, in a prediction image generation block having a block size of 32 × 32 pixels or more, the expressions in the second row of Expression (2) and Expression (3) are always used regardless of the coordinates of the prediction target pixel (prediction). By applying the equations (2) and (3) for performing the above-described filter processing only to blocks smaller than 32 × 32 pixels, the filter edge of the image generation block is not performed. It is possible to suppress an increase in the calculation amount while improving the prediction performance compared to the direction prediction and the horizontal direction prediction.

ただし、ｋは負の実数である。 When the index value in the intra prediction mode is other than 0 (plane prediction), 1 (average value prediction), 26 (vertical direction prediction), and 10 (horizontal direction prediction), the prediction direction vector υ _p = ( Based on (dx, dy), a predicted value of a pixel in the predicted image generation block is generated.
As shown in FIG. 32, when the upper left pixel of the predicted image generation block is the origin and the relative coordinates in the predicted image generation block are set to (x, y), the position of the reference pixel used for prediction is adjacent to the following L This is the intersection of pixels.

However, k is a negative real number.

  When the reference pixel is at the integer pixel position, the integer pixel is set as the prediction value of the prediction target pixel. When the reference pixel is not at the integer pixel position, an interpolation pixel generated from the integer pixel adjacent to the reference pixel is selected. Estimated value. In the example of FIG. 29, since the reference pixel is not located at the integer pixel position, a value interpolated from two pixels adjacent to the reference pixel is set as a predicted value. Note that an interpolation pixel may be generated not only from two adjacent pixels but also from two or more adjacent pixels, and used as a predicted value. While increasing the number of pixels used in the interpolation process has the effect of improving the interpolation accuracy of the interpolated pixels, it increases the complexity of the calculation required for the interpolation process, requiring high coding performance even when the calculation load is large. In the case of an image encoding device, it is better to generate interpolation pixels from a larger number of pixels.

The processing described above, to generate a predicted pixel for all the pixels of the luminance signals of the prediction block P _i ⁿ in the predicted image generation block, and outputs an intra prediction image P _INTRAi ^n. Incidentally, the intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ (intra prediction mode) is output to the variable length coding unit 15 for multiplexing the bitstream.

  Note that the MPEG-4 AVC / H. Similarly to the smoothing process performed on the reference pixels at the time of intra prediction of the 8 × 8 pixel block in H.264, the intra prediction unit 5 predicts the reference pixels when generating the predicted image of the predicted image generation block. Even when the encoded pixels adjacent to the image generation block are configured to be the smoothed pixels, it is possible to perform the same filtering process on the predicted image as in the above example. By doing in this way, the noise of the reference pixel by the filter process to a reference pixel is removed, and prediction accuracy can be improved by performing prediction using this. Alternatively, the filtering process on the reference pixel may be performed only in the prediction other than the average value prediction, the vertical direction prediction, and the horizontal direction prediction for performing the filtering process on the predicted image. By doing in this way, it is only necessary to perform at most one filter process for each prediction mode, and an increase in the amount of calculation can be suppressed.

In the above description, the predicted image generation process of the luminance signal has been described, but the predicted image for the color difference component is generated as follows. The color difference signal of the prediction block P _i ^n, conduct intra prediction processing based on the intra prediction parameter of the color difference signal (intra prediction mode), the variable length coding unit intra prediction parameter used to generate the intra-prediction image 15 is output.

FIG. 33 is an explanatory diagram showing an example of correspondence between intra prediction parameters (index values) of color difference signals and color difference intra prediction modes. When the intra prediction parameter of the chrominance signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (when the intra prediction parameter indicates the luminance / chrominance common intra prediction mode (DM mode)), the luminance signal The same intra-frame prediction is performed to generate a prediction image of the color difference signal.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.
Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed.

FIG. 34 shows a conversion example of the intra prediction mode index in the intra prediction mode of FIG. In the conversion table of FIG. 34, when the angle of the prediction direction is θ (see FIG. 10), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to.
As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

Further, the LM mode may not be performed on the color difference signal. FIG. 34 shows a correspondence example between the intra prediction parameter (index value) of the color difference signal and the color difference intra prediction mode at this time. By not using the LM mode in this way, the dependency of the luminance signal and the color difference signal of the pixel to be predicted is eliminated, so that the prediction processing of the luminance signal and the color difference signal can be parallelized, and high-speed calculation processing is performed. Can be realized.
Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. It is good also as the prediction method similar to H.264. By not performing the filtering process in this way, it is possible to reduce the prediction process.

Next, the processing content of the image decoding apparatus in FIG. 17 will be specifically described.
When the encoded bit stream generated by the image encoding device in FIG. 1 is input, the variable length decoding unit 31 performs a variable length decoding process on the bit stream (step ST21 in FIG. 20), and performs one or more frames. Each header information such as header information (sequence level header) and header information (picture level header) in units of a sequence composed of pictures is decoded. However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.

  At this time, when the valid flag information of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding of the quantization matrix parameter to identify the quantization matrix. Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, and a quantization matrix prepared in advance by the image encoding device and the image decoding device, or When indicating that the quantization matrix is already decoded (not a new quantization matrix), refer to the index information for specifying which quantization matrix among the matrices included in the quantization matrix parameter. When the quantization matrix is specified, and the quantization matrix parameter indicates that a new quantization matrix is to be used, it is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter. Then, slice unit header information (slice level header) such as slice division information is decoded from slice data constituting picture unit data, and encoded data of each slice is decoded.

Further, the variable length decoding unit 31 specifies the maximum encoding block size and the upper limit of the number of division layers from the header information (step ST22 in FIG. 20). However, when the minimum block size of the encoded block is encoded instead of the upper limit of the number of division layers, the upper limit of the number of division layers is determined by decoding this. That is, the upper limit of the number of division layers is obtained when the maximum encoded block is divided to the minimum block size.
The variable length decoding unit 31 decodes the division state of the maximum encoded block as shown in FIG. 22 for each determined maximum encoded block. Based on the decoded division state, coding blocks are identified hierarchically (step ST23 in FIG. 20).

Next, the variable length decoding unit 31 decodes the encoding mode assigned to the encoding block. Based on the information included in the decoded coding mode, the coded block is further divided into one or more prediction blocks which are prediction processing units, and the prediction parameters assigned to the prediction block units are decoded (FIG. 20). Step ST24).
That is, when the encoding mode assigned to the encoding block is the intra encoding mode, the variable length decoding unit 31 is included in the encoding block, and each of one or more prediction blocks serving as a prediction processing unit When the intra prediction parameter is decoded and the coding mode assigned to the coding block is the intra block copy coding mode, one or more prediction blocks included in the coding block and serving as a prediction processing unit The intra block copy prediction parameter is decoded every time. Further, when the coding mode assigned to the coding block is the inter coding mode, the inter prediction parameter and the motion vector are included in the coding block and each of one or more prediction blocks serving as a prediction processing unit. Is decoded (step ST24 in FIG. 20). Also, the correction width w ⁱ for all peak pixel values is decoded.

  Furthermore, the variable length decoding unit 31 decodes the compressed data (transformed and transformed transform coefficients) for each transform block based on the transform block division information included in the prediction difference encoding parameter (step ST24 in FIG. 20). . At that time, similarly to the encoding process of the compressed data in the variable length encoding unit 15 of the image encoding apparatus of FIG. Accordingly, as shown in FIG. 25, 16 CGs in units of 4 × 4 pixels are decoded in order from the lower right CG, and each CG decodes 16 coefficients in the CG in order from the lower right coefficient. Will do.

  Specifically, first, flag information indicating whether or not a significant (non-zero) coefficient exists in 16 coefficients in the CG is decoded, and then the decoded flag information is significant (non-zero) in the CG. Only when it indicates that a coefficient exists, whether each coefficient in the CG is a significant (non-zero) coefficient is decoded in the above order, and finally, coefficient value information for the coefficient indicating the significant (non-zero) coefficient Are sequentially decoded. This is performed in the above order in units of CG. However, with regard to the scan order, in the case of a 4 × 4 pixel and 8 × 8 pixel conversion block for which intra prediction is selected, the scan shown in FIG. 27 is used instead of the scan order of FIG. 25 according to the index of the intra prediction mode. Process in order.

If the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is an intra-encoding mode (when m (B ⁿ ) ∈INTRA), the changeover switch 33 is changed by the variable-length decoding unit 31. outputs the intra prediction parameters of the variable length decoded prediction block to the intra prediction unit 34, if the variable-length decoded coding mode m (B ⁿ⁾ is an intra block copy coding mode by the variable-length decoder 31 (In the case of m (B ⁿ ) εICOPY), the intra block copy prediction parameter of the prediction block unit variable length decoded by the variable length decoding unit 31 is output to the intra block copy prediction unit 35 and variable by the variable length decoding unit 31 If the length decoded coding mode m ^{(B n)} is an inter coding mode (m (the case of ^{B n)} ∈INTER), variable recovery And it outputs the inter prediction parameter and the motion vector variable length decoding prediction block to the motion compensation prediction unit 36 by the part 31.

When the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode (m (B ⁿ ) εINTRA) (Step ST25 in FIG. 20), the intra prediction unit 34 The intra prediction parameter output from the selector switch 33 is received, and the intra prediction is performed with reference to the decoded image stored in the intra memory 38 in the same procedure as the intra prediction unit 5 in FIG. and implementing intra prediction process for each of the prediction block _P ^{i n} the coded block ^{B n} using a parameter to generate an intra prediction image _{P INTRAi} ⁿ (step ST26 in FIG. 20).
In addition, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal to generate a prediction image of the luminance signal.
On the other hand, for the color difference signal, intra prediction processing based on the intra prediction parameter of the color difference signal is performed to generate a predicted image of the color difference signal.

As shown in FIG. 33, when the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed.

In the conversion table of FIG. 34, when the angle of the prediction direction is θ (see FIG. 10), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to. As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

In addition, when the image encoding device is configured not to perform the LM mode on the color difference signal, the image decoding device is also configured so that the encoded bitstream generated from the image encoding device can be decoded. The configuration is as follows. FIG. 34 shows a correspondence example between the intra prediction parameter (index value) of the color difference signal and the color difference intra prediction mode at this time. By not using the LM mode in this way, the dependency of the luminance signal and the color difference signal of the prediction target pixel is eliminated, so that the prediction process of the luminance signal and the color difference signal can be parallelized, and high-speed calculation processing is possible. Can be realized.
Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. When an image encoding apparatus is configured as a prediction method similar to H.264, the image decoding apparatus has the same configuration so that an encoded bit stream generated from the image encoding apparatus can be decoded. By not performing the filtering process in this way, it is possible to reduce the prediction process.

The intra block copy prediction unit 35, when the encoding mode m (B ⁿ ) that has been variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode (m (B ⁿ ) εICOPY) (step ST25). The intra block copy prediction parameter including the block shift vector of the prediction block unit output from the changeover switch 33 is received, and the intra block copy prediction parameter is used while referring to the decoded image stored in the intra memory 38. There was then conducted intra block copy prediction processing for each of the prediction block _P ^{i n} the coded block ^{B n,} generates an intra block copy predicted image _{P ICOPYi} ⁿ (step ST27). In the image coding apparatus, the block shift vector is obtained by calculating the difference value between the block shift vector of the previous encoded (decoded) predicted block or the block shift vector of the encoded (decoded) predicted block around the predicted block. When encoding is performed as a part of the block copy prediction parameter, the difference value included in the intra block copy prediction parameter and the block shift vector are added to calculate the block shift vector of the prediction block.

The motion compensation prediction unit 36, when the encoding mode m (B ⁿ ) that has been variable length decoded by the variable length decoding unit 31 is the inter encoding mode (m (B ⁿ ) εINTER) (step ST25 in FIG. 20). The motion vector and inter prediction parameters output from the changeover switch 33 are received and the inter prediction parameters are received while referring to the decoded image after filtering stored in the motion compensated prediction frame memory 40. by carrying out inter-prediction processing for each of the prediction block _P ^{i n} the coded block ^{B n} using a parameter to generate an inter prediction image _{P INTERi} ⁿ (step ST28 in FIG. 20).

  When receiving the compressed data and the prediction difference encoding parameter from the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32 performs the prediction difference encoding in the same procedure as the inverse quantization / inverse conversion unit 10 of FIG. With reference to the quantization parameter and transform block division information included in the parameters, the compressed data is inversely quantized in transform block units. At this time, when referring to each header information variable-length decoded by the variable-length decoding unit 31, each header information indicates that the inverse quantization process is performed using the quantization matrix in the slice. Inverse quantization processing is performed using a quantization matrix.

  At this time, referring to each header information variable length decoded by the variable length decoding unit 31, for each color signal and coding mode (intra coding, intra block copy coding, inter coding) with each orthogonal transform size Specify the quantization matrix to be used. Further, the inverse quantization / inverse transform unit 32 performs inverse orthogonal transform processing on transform coefficients that are compressed data after inverse quantization in units of transform blocks, and outputs from the inverse quantization / inverse transform unit 10 in FIG. The same decoded prediction difference signal as the local decoding prediction difference signal thus obtained is calculated (step ST29 in FIG. 20).

Addition unit 37, an intra block generated by the decoded prediction difference signal calculated by the inverse quantization and inverse transform unit 32, an intra prediction image P _{INTRAi ^n,} intra block copy prediction unit 35 generated by the intra prediction unit 34 copy predicted image _{P ICOPYi} ^n, or, together with by adding one of generated by the motion compensation prediction unit 36 inter-prediction image _{P INTERi} ⁿ calculates a decoded image, and outputs the decoded image to the loop filter unit 39, The decoded image is stored in the intra memory 38 (step ST30 in FIG. 20). This decoded image becomes a decoded image signal used in the subsequent intra prediction processing and intra block copy prediction processing.

Loop filter unit 39, the process of step ST23~ST30 of all the coding blocks ^{B n} is completed (step ST31 in FIG. 20), with respect to output decoded image from the adder 37, a predetermined filter process performed Then, the decoded image after filter processing is stored in the motion compensated prediction frame memory 40 (step ST32 in FIG. 20).
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, adaptive filter processing, and histogram correction processing. Specify whether to do this in slices.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image encoding device of FIG. 1 is configured as shown in FIG. 12, the loop filter unit 39 as shown in FIG. Is configured. Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, the loop filter unit 39 also performs deblocking filter processing, pixel adaptive offset processing, and histogram correction processing. Consists of.

Here, in the deblocking filter processing, when there is information for referring to the header information that has been variable-length decoded by the variable-length decoding unit 31 and changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Performs a deblocking filter process based on the change information. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the block is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the block unit of variable-length decoded by the variable-length decoding unit 31 is divided into the blocks. When an index indicating a class classification method is referred to and the index is not an index indicating that “offset processing is not performed”, each pixel in the block is classified into blocks in accordance with the class classification method indicated by the index. Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance.

Then, the loop filter unit 39 performs processing for adding the offset to the pixel value of the decoded image with reference to the offset information that has been variable-length decoded by the variable-length decoding unit 31 that identifies the offset value of each class in block units. .
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

  In the histogram correction process, the histogram calculation unit 41 divides the decoded image based on the block division information of the histogram correction process variable-length decoded by the variable-length decoding unit 31, and variable by the variable-length decoding unit 31 for each block. With reference to a flag indicating whether or not to perform long-decoded histogram correction, if the flag is not a value indicating that “no histogram correction processing is performed”, the histogram correction processing is executed for the block.

FIG. 37 is a flowchart showing the contents of the histogram correction processing in the image decoding apparatus according to Embodiment 1 of the present invention.
Hereinafter, the flow of the histogram correction process will be described with reference to FIG.
In the histogram correction processing, first, the histogram calculation unit 41 in FIG. 19 divides the decoded image calculated by the addition unit 37 into a plurality of blocks, and calculates a histogram of pixel values for each block of the decoded image (FIG. 19). 37 steps ST200 to ST201). Further, the pixel value having the maximum histogram value is selected, and the pixel value is set to p (hereinafter referred to as “peak pixel value”) (step ST202 in FIG. 37).
Note that the step width of the histogram does not necessarily have to be 1, but the output image of the loop filter unit 39 needs to match the output image of the loop filter unit 13 in the image encoding device of FIG. The same step width as that of the loop filter unit 13 is used. When the step width of the histogram is set to 1 or more, it is conceivable that p is set to the median value of the step width or the like, but this also needs to be matched with the loop filter unit 13 for the same reason as described above.

  Further, it is not always necessary to calculate the histogram using all the pixels in the block, and the histogram may be obtained using only some of the pixels. By reducing the number of pixels used for the histogram calculation, it is possible to reduce the amount of calculation required for the histogram calculation. This also needs to be matched with the loop filter unit 13 for the same reason as described above.

  In addition to directly using the obtained histogram to select the pixel value that maximizes the value, for example, it may be possible to obtain the maximum value after differentiating the histogram, for the same reason as above. It is necessary to match with the loop filter unit 13.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。 The pixel value correction unit 42 includes pixel values ((peak pixel value p−correction width w ⁱ ) to (peak pixel value p + correction) within the correction width w ⁱ for the i-th peak variable-length decoded by the variable length decoding unit 31. The pixel value of the width w ⁱ ) is replaced with the peak pixel value p (steps ST203 to ST206 in FIG. 37).

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 in the image encoding device of FIG. 1, a method similar to that for the loop filter unit 13 must be used.

Next, the pixel value correcting unit 42 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculating unit 41 in ST202 ( Steps ST207 to ST208 in FIG. 37).
The pixel value correcting unit 42 compares the index i with a preset maximum peak number P _max (step ST209 in FIG. 37), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 41 is instructed to reselect the value, and the process returns to step ST202.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.
The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

As is apparent from the above, according to the first embodiment, the histogram calculation unit 21 that calculates a histogram of pixel values in the local decoded image obtained from the compressed data of the input image, and the histogram calculated by the histogram calculation unit 21 And the pixel value correcting unit 22 for correcting the pixel value of the pixel on which the noise is superimposed and specifying the pixel on which the noise due to the coding distortion is superimposed. This produces an effect of sufficiently reducing noise accompanying encoding distortion occurring in the locally decoded image of the screen content.
That is, the pixel value correction unit 22 of the loop filter unit 13 in the image encoding device obtains an optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, and the correction value w ⁱ within the correction width w ⁱ By replacing the pixel value with the peak pixel value p so as to remove noise from the local decoded image, as shown in FIG. 42, for screen content where each peak of the histogram is sharp and sparse, There is an effect that the image quality of the locally decoded image can be significantly improved. However, this effect is not limited to the screen content, and the same effect can be obtained as long as it is a video signal having a similar histogram characteristic.

Further, according to the first embodiment, the variable length encoding unit 15 of the image encoding device encodes and outputs the correction width w ⁱ for each peak pixel value, while the variable length decoding unit 31 of the image decoding device. However, the correction width w ⁱ for each peak pixel value is decoded, and the pixel value correction unit 42 of the loop filter unit 39 sets the pixel value within the correction width w ⁱ decoded by the variable length decoding unit 31 to the peak pixel value p. Since it is configured to remove noise from the local decoded image by replacing, as shown in FIG. 42, the image quality of the decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and the same effect can be obtained as long as it is a video signal having a similar histogram characteristic.
In addition, according to the first embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above effects can be obtained. Play.

Embodiment 2. FIG.
In the first embodiment, histogram correction processing is added to the loop filter unit 13 of the image encoding device of FIG. 1 as shown in FIG. 12, and the loop filter unit 39 of the image decoding device of FIG. On the other hand, as shown in FIG. 18, the histogram correction processing is added. However, by configuring the histogram correction processing as a part of the existing loop filter, the amount of information to be encoded can be reduced. You may make it reduce.

The image coding apparatus according to the second embodiment has the same configuration as the image coding apparatus of FIG. 1 in the first embodiment, but only the internal configuration of the loop filter unit 13 shown in FIG. Will be changed as follows.
Here, only the configuration of the modified loop filter unit 13 shown in FIG. 38 will be described.

  The loop filter unit 13 in FIG. 38 performs a predetermined filtering process on the local decoded image calculated by the adding unit 11, and performs a process of outputting the local decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable.

The loop filter unit 13 in FIG. 38 determines whether or not to perform each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and uses a variable length code with the valid flag of each processing as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 38 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 38 are merely examples, and the types and application order of the loop filters to be applied do not necessarily have to be as shown in FIG. For example, it is conceivable that the loop filter unit 13 is configured with only two types of deblocking filter processing and pixel adaptive offset processing, or that the application order is in order of pixel adaptive offset processing and deblocking filter processing.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and one of a plurality of offset application methods is selected for each block including the case where the offset process is not performed.
In the second embodiment, histogram correction processing can be selected as one method of applying this offset. The loop filter unit 13 in FIG. 38 classifies each pixel in the block according to the selected offset application method, and calculates an offset value for compensating for coding distortion for each class, or performs histogram correction. An optimum correction width W ⁱ is calculated for each peak pixel value.
Finally, processing for adding an offset value to the pixel value of the locally decoded image is performed, or the image quality of the locally decoded image is improved by correcting the histogram of the locally decoded image.

ここで、（ｘ，ｙ）はブロック単位の局所復号画像における画素の座標を表し、Ｌ（ｘ，ｙ）は置換前の画素値、Ｌ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。
ここでは、補正幅ｗ^ｉをピーク画素値ｐよりも大きい側と小さい側で同一としているが、必ずしも両者を一致させる必要はなく、それぞれに対して異なる補正幅ｗ^ｉを使用してもよい。 When the correction width is w ⁱ , the histogram correction process is performed as follows.

Here, (x, y) represents the pixel coordinates in the local decoded image in block units, L (x, y) represents the pixel value before replacement, and L ′ (x, y) represents the pixel value after replacement. ing.
The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each.

In the histogram correction process, after correcting the first peak as described above, the process moves to the next peak, and the same process is repeated until a predetermined number of peaks is reached.
In the pixel adaptive offset process, header information includes block division information, an index indicating a class classification method for each block, offset information for specifying an offset value of each class in block units, or a correction width for each peak of a histogram for histogram correction. The data is output to the variable length coding unit 15.

  In the first embodiment, it is necessary to encode information for the histogram correction process in addition to the information for the pixel adaptive offset process. However, in the second embodiment, as a part of the pixel adaptive offset process, By performing the histogram correction process, only one of the information for the offset process and the information for the histogram correction process needs to be encoded, so that the code amount in the entire loop filter can be reduced.

  In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  In adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for the distortion that is superimposed is designed for each region (local decoded image) belonging to each class. Filter the local decoded image. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information. As the class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of each block. Further, the number of classes used in the adaptive filter processing may be set in advance as a value common to the image encoding device and the image decoding device, or may be a parameter to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

The image decoding apparatus according to the second embodiment has the same configuration as that of the image decoding apparatus in FIG. 17 in the first embodiment, but only the internal configuration of the loop filter unit 39 shown in FIG. 18 is as shown in FIG. Be changed.
Here, only the configuration of the modified loop filter unit 39 shown in FIG. 39 will be described.

  The loop filter unit 39 in FIG. 39 performs a predetermined filter process on the decoded image calculated by the adder unit 37, and performs a process of outputting the decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable. However, the combination method needs to be the same as that of the loop filter unit 13 of the image encoding device.

The loop filter unit 39 in FIG. 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing described above, and Specify whether to do it.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image encoding device is configured as shown in FIG. 38, the loop filter unit 39 is configured as shown in FIG. The Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing and pixel adaptive offset processing, the loop filter unit 39 is also configured by deblocking filter processing and pixel adaptive offset processing.

Here, in the deblocking filter processing, with reference to the header information that has been subjected to variable length decoding by the variable length decoding unit 31, there is information for changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Based on the change information, deblocking filter processing is performed. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. Referring to an index indicating a block classification method, if the index is not an index indicating that “offset processing is not performed” and is not an index indicating that “histogram correction processing is performed”, the index Accordingly, each pixel in the block is classified according to the class classification method indicated by the index. As the class classification method candidates, the same class classification method candidates for the pixel adaptive offset process of the loop filter unit 13 of FIG. 38 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

  On the other hand, the index indicating the block class classification method variable-length decoded by the variable-length decoding unit 31 is not an index indicating that “offset processing is not performed” and is an index indicating that “histogram correction processing is performed”. In some cases, a histogram of pixel values in the block is calculated, and the position of the peak pixel value in the histogram is determined by a predetermined method.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。 The pixel value correction unit 42 includes pixel values ((peak pixel value p−correction width w ⁱ ) to (peak pixel value p + correction) within the correction width w ⁱ for the i-th peak variable-length decoded by the variable length decoding unit 31. Replace the pixel value of width w ⁱ ) with the peak pixel value p.

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.
The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.

Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 in the image encoding device, the same method as the loop filter unit 13 must be used.
In the histogram correction process, after correcting the first peak as described above, the process moves to the next peak, and the same process is repeated until a predetermined number of peaks is reached.

However, in the pixel adaptive offset processing of the loop filter unit 13 of the image encoding device, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit). When a class classification method is selected for each block and adaptive offset processing for each class is performed, the loop filter unit 39 also has a pixel adaptive offset in units of blocks having the same fixed size as the loop filter unit 13. Perform the process.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

Next, in the image coding apparatus according to the second embodiment, the operation of the loop filter unit 13 having a configuration different from that of the image coding apparatus according to the first embodiment will be described. Other operations are the same as those in the first embodiment.
When the loop filter unit 13 in FIG. 38 receives the local decoded image from the adder unit 11, the loop filter unit 13 performs a predetermined filter process on the local decoded image, and the local decoded image after the filter process is subjected to motion compensation prediction frame memory. 14 (step ST11 in FIG. 16). Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.
In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable.

The loop filter unit 13 in FIG. 38 determines whether or not to perform each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and uses a variable length code with the valid flag of each processing as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 38 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 38 are merely examples, and the types and application order of the loop filters to be applied do not necessarily have to be as shown in FIG. For example, it is conceivable that the loop filter unit 13 is configured with only two types of deblocking filter processing and pixel adaptive offset processing, or that the application order is in order of pixel adaptive offset processing and deblocking filter processing.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and one of a plurality of offset application methods is selected for each block including the case where the offset process is not performed.
In the second embodiment, histogram correction processing can be selected as one method of applying this offset. The loop filter unit 13 in FIG. 38 classifies each pixel in the block according to the selected offset application method, and calculates an offset value for compensating for coding distortion for each class, or performs histogram correction. An optimum correction width W ⁱ is calculated for each peak pixel value.
Finally, processing for adding an offset value to the pixel value of the locally decoded image is performed, or the image quality of the locally decoded image is improved by correcting the histogram of the locally decoded image.

As an offset application method, a method of classifying by the size of a pixel value of a locally decoded image (referred to as a BO method), a situation around each pixel (whether or not an edge portion, etc.) for each edge direction, etc. There is a method of classifying (called an EO method).
In the second embodiment, unlike the first embodiment, a histogram correction process is added here. These methods are prepared in advance by the image encoding device and the image decoding device in advance. For example, as shown in FIG. 40, when the offset process is not performed, it is defined as one of the offset application methods. Among the methods, an index indicating which method is used is selected for each block.

  Also, the offset application method shown in FIG. 40 may be prepared by preparing several sets in advance and switching them. For example, a flag (herein referred to as an SC flag) indicating whether or not the input video is screen content is prepared in a header (sequence parameter set) assigned for each sequence, and two flags shown in FIG. It is possible to switch the set. In FIG. 59, when the input video is not screen content, the BO method or EO method is used, and when the input video is screen content, the BO method or histogram correction processing is used. By doing so, it is possible to reduce the number of bits of the index necessary for expressing the offset application method. Note that the SC flag does not necessarily have to be placed in the sequence parameter set. The SC flag is placed in a header (picture parameter set) given for each picture to switch to a picture unit, or a header (slice that is given to each slice to switch in a slice unit). It may be placed in the parameter set. Furthermore, switching to a block unit is naturally conceivable. Of course, the SC flag can be generalized to a flag that can be changed under an arbitrary condition.

FIG. 41 is a flowchart showing a procedure for selecting an offset application method to be used from among a plurality of offset application methods in FIG.
First, the loop filter unit 13 in FIG. 38 obtains an encoding cost C _off when encoding is performed without applying the pixel adaptive offset processing (step ST300 in FIG. 41).
There are various methods for calculating the encoding cost, but it is generally expressed in the form of d + λr, where d is the error from the original image and r is the amount of code required to send the header information. Is. Here, λ is a coefficient determined by the image coding apparatus according to the bit rate or the like.

Next, the loop filter unit 13 of FIG. 38 classifies each pixel in the block for each of the BO method and the EO method shown in FIG. 40, and calculates an offset value for compensating the coding distortion for each class. Since the class classification method and the offset calculation method are known techniques, description thereof is omitted here.
And the loop filter part 13 of FIG. 38 calculates | requires the encoding cost at the time of adding these offset values with respect to the pixel value of a local decoded image (step ST301-ST305 of FIG. 41).

In the second embodiment, in addition to the above, the loop filter unit 13 of FIG. 38 calculates a correction width w ⁱ for each peak pixel value of the histogram for performing histogram correction. It is conceivable that the correction width w ⁱ is calculated in the same manner as in the first embodiment.
The loop filter unit 13 in FIG. 38 obtains the coding cost when the histogram correction of the locally decoded image is performed using the correction width w ⁱ for each peak pixel value (step ST306 in FIG. 41).
The loop filter unit 13 in FIG. 38 selects a technique indicating the lowest coding cost from the plurality of offset application techniques in FIG. 40 (step ST307 in FIG. 41).

As a result of the above comparison, if no improvement in coding efficiency is observed even when pixel adaptive offset processing is applied (when C _off is minimized), index 0 is output to the variable length coding unit 15 as header information. (Steps ST308 to ST309 in FIG. 41). This means that pixel adaptive offset processing is not applied.
On the other hand, when the BO method or the EO method is selected as a method for minimizing the coding cost, an index representing the selected method is output to the variable length coding unit 15 as header information according to FIG. The offset for the class is output as header information to the variable length coding unit 15 and the process ends (steps ST310 to ST311 in FIG. 41).
Further, when histogram correction is selected as a method for minimizing the coding cost, an index representing the selected method is output to the variable length coding unit 15 as header information according to FIG. 40, and each peak of the histogram is further displayed. Is output to the variable length encoding unit 15 as header information, and the process ends (step ST312 in FIG. 41).

As is clear from FIG. 41, in the second embodiment, only one of the offset for applying the BO method or the EO method or the correction width w ⁱ for applying the histogram correction is encoded. Is done. Therefore, it is possible to reduce the code amount as compared with the first embodiment in which both need to be encoded.
However, in contrast to the first embodiment in which both the BO method or the EO method and the histogram correction can be applied, in the second embodiment, only either the BO method or the EO method or the histogram correction is performed. Since it cannot be applied, the image quality improvement effect may be inferior to that of the first embodiment.

If the number of classes classified by the BO method or EO method is the same as the number of histogram peaks to which the histogram correction processing is applied (maximum peak number P _max in the first embodiment), the offset or the correction width is encoded. A common syntax can be simplified. Of course, the number of classes and the number of peaks in the histogram may be different from each other.
Therefore, the pixel adaptive offset processing outputs block division information, an index indicating an offset application technique in units of blocks, offset information in units of blocks, or correction width w ⁱ for histogram correction to the variable length encoding unit 15 as header information. .

  In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  FIG. 44 shows an example of a case where division is always performed in units of Coding Tree Unit (CTU) which is the maximum coding block in the pixel adaptive offset process, and the position where the parameter in the pixel adaptive offset process is encoded and inserted into the bitstream It is illustrated. Note that FIG. 44 is merely an example, and it is not always necessary to perform such encoding. In FIG. 44, an index (FIG. 40) indicating an offset application method is first encoded and inserted at the head of each CTU data following the slice header, and then, when using the EO method or the BO method, an offset and a histogram for each class. When performing the correction process, the correction width corresponding to each peak of the pixel value histogram is encoded and inserted. When the pixel adaptive offset process is not performed, nothing is encoded after the index.

  FIG. 45 is a flowchart showing how the variable-length encoding unit 15 encodes the pixel adaptive offset processing parameter and adds it to the bitstream. First, the variable length encoding unit 15 encodes an index corresponding to each offset application method shown in FIG. 40 (step ST400 in FIG. 45). Then, when no offset is applied (step ST401 in FIG. 45), the processing ends as it is, and when the EO method or the BO method is used, all the offsets for each class are encoded (steps ST402 and ST403 in FIG. 45). On the other hand, when the histogram correction process is performed, all correction widths corresponding to the respective peaks of the pixel value histogram are encoded (step ST404 in FIG. 45). In FIG. 45, the number of classes in the EO method or the BO method and the number of peaks of the pixel value histogram to be corrected in the histogram correction are both 4.

  Also, in adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for superimposed distortion is designed for each region (local decoded image) belonging to each class. Then, the local decoded image is filtered. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.

  Here, as a class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying the image in units of blocks according to local characteristics (dispersion, etc.). Further, the number of classes used in the adaptive filter process may be set in advance to a common value in the image encoding device and the image decoding device, or may be one of the parameters to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

Next, in the image decoding apparatus according to the second embodiment, the operation of the loop filter unit 39 having a configuration different from that of the image decoding apparatus according to the first embodiment will be described. Other operations are the same as those in the first embodiment.
In the flowchart representing the operation of the image decoding apparatus shown in FIG. 20, the loop filter unit 39 in FIG. 39 adds the processing after completing the processing of steps ST23 to ST30 for all the coding blocks ^Bn (step ST31 in FIG. 20). A predetermined filter process is performed on the decoded image output from the unit 37, and the decoded image after the filter process is stored in the motion compensated prediction frame memory 40 (step ST32 in FIG. 20).

Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

However, the loop filter unit 39 in FIG. 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and Specify whether to perform processing in slices. At this time, when two or more filter processes are performed, for example, when the loop filter unit 13 of the image encoding device is configured as shown in FIG. 38, the loop filter unit 39 is configured as shown in FIG. Made.
Here, in the deblocking filter processing, when there is information for referring to the header information that has been variable-length decoded by the variable-length decoding unit 31 and changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Performs a deblocking filter process based on the change information. When there is no change information, it is performed according to a predetermined method.

In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. Referring to an index indicating a block classification method, if the index is not an index indicating that “offset processing is not performed” and is not an index indicating that “histogram correction processing is performed”, the index Accordingly, each pixel in the block is classified according to the class classification method indicated by the index.
As the class classification method candidates, the same class classification method candidates for the pixel adaptive offset processing of the loop filter unit 13 of FIG. 38 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

  On the other hand, the index indicating the block class classification method variable-length decoded by the variable-length decoding unit 31 is not an index indicating that “offset processing is not performed” and is an index indicating that “histogram correction processing is performed”. If there is, the histogram correction processing is executed for the block.

  FIG. 46 is a flowchart showing the decoding procedure of the pixel value adaptive offset processing parameter by the variable length decoding unit 31.

  The processing content of the histogram correction process in the image decoding apparatus according to the second embodiment is the same as the histogram correction process in the image decoding apparatus according to the first embodiment.

That is, the histogram calculation unit 41 divides the decoded image calculated by the addition unit 37 into a plurality of blocks, and calculates a histogram of pixel values for each block of the decoded image (steps ST200 to ST201 in FIG. 37). Also, the pixel value with the maximum histogram value is selected, and the pixel value is set as the peak pixel value p (step ST202 in FIG. 37).
Note that the step width of the histogram does not necessarily have to be 1, but the output image of the loop filter unit 39 needs to match the output image of the loop filter unit 13 in FIG. Use the same step width. When the step width of the histogram is set to 1 or more, it is conceivable that p is set to the median value of the step width or the like, but this also needs to be matched with the loop filter unit 13 for the same reason as described above.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。 The pixel value correction unit 42 includes pixel values ((peak pixel value p−correction width w ⁱ ) to (peak pixel value p + correction) within the correction width w ⁱ for the i-th peak variable-length decoded by the variable length decoding unit 31. The pixel value of the width w ⁱ ) is replaced with the peak pixel value p (steps ST203 to ST206 in FIG. 37).

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 of FIG. 38, the same method as the loop filter unit 13 must be used.

Next, the pixel value correcting unit 42 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculating unit 41 in ST202 ( Steps ST207 to ST208 in FIG. 37).
The pixel value correcting unit 42 compares the index i with a preset maximum peak number P _max (step ST2091 in FIG. 37), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 41 is instructed to reselect the value, and the process returns to step ST202.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.

However, in the pixel adaptive offset processing of the loop filter unit 13 in FIG. 38, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit), When the class classification method is selected and adaptive offset processing is performed for each class, the loop filter unit 39 also performs pixel adaptive offset processing in units of blocks having the same fixed size as the loop filter unit 13. carry out.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.
The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the second embodiment, as in the first embodiment, the loop filter unit 13 of the image encoding device obtains the optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, Since the configuration is such that noise is removed from the locally decoded image by correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  Furthermore, in the second embodiment, the above-described histogram correction is executed as a part of an existing loop filter (for example, pixel adaptive offset), and one method is selected and applied by the image coding apparatus (for example, 40, only one of the methods shown in FIG. 40 is selected and applied), and header information to be encoded with respect to the loop filter can be reduced as compared with the first embodiment. There is an effect that the amount of codes can be reduced.

In the second embodiment, similarly to the first embodiment, the variable length coding unit 15 of the image coding apparatus codes and outputs the correction width w ⁱ of each peak necessary for the histogram correction. Then, the image decoding apparatus is configured to perform variable length decoding on the correction width w ⁱ of each peak and to perform a histogram correction of the decoded image in the loop filter unit 39 to remove noise from the decoded image, as shown in FIG. There is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the second embodiment, the image decoding apparatus and the image decoding method capable of correctly decoding the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above effects can be obtained. Play.

Embodiment 3 FIG.
In Embodiment 1 described above, the image encoding apparatus always calculates and encodes a new histogram correction parameter (such as a correction width for each peak of the pixel value histogram) for each block. In the third embodiment, a case will be described in which the amount of information to be encoded can be reduced by configuring so that parameters of other blocks calculated in the past can be used.

  The image coding apparatus according to Embodiment 3 has basically the same configuration and operation as the image coding apparatus according to Embodiment 1 shown in FIG. However, only the operations related to the histogram correction processing of the loop filter unit 13 and the variable length coding unit 15 shown in FIG. 12 are different. Hereinafter, first, an operation related to the histogram correction processing of the loop filter unit 13 will be described.

47A and 47B are flowcharts showing the operation of the loop filter unit 13 in the third embodiment, and the operation of the loop filter unit 13 will be described based on this. First, the loop filter unit 13 obtains the encoding cost C _OFF when the histogram correction processing is not performed (step ST600 in FIG. 47A), and then the pixel value histogram is obtained by the same procedure as in the first embodiment (FIG. 24). An optimum correction width for each peak is obtained, and an encoding cost C _NEW for encoding this parameter is obtained (step ST601 in FIG. 47A).

  Here, it is assumed that block division as shown in FIG. 48 is performed by the histogram correction processing, and the lower right block is a block during the histogram correction processing. At this time, it is assumed that the histogram correction processing has already been completed for the surrounding blocks 1, 2, and 3. In the third embodiment, a newly calculated parameter may be used in the block whose histogram is being corrected, or any of the parameters of blocks 1, 2, and 3 that have already been processed may be used. . Note that FIG. 48 merely shows examples of blocks that can be referred to, and various other variations such as referring to just above and to the left of the block being processed can be considered.

FIG. 49 shows a parameter diversion method. For example, it is assumed that the histogram correction processing is performed as shown in the upper part of FIG. 49 in the histogram correction processed block 1 shown in FIG. In the upper part of FIG. 49, p ⁰ , p ¹ , p ² , and p ³ represent peak positions that have been subjected to correction processing, and the values of the histogram are p ⁰ > p ¹ > p ² > p ³ . Further, the correction width for each peak is set to w ⁰ , w ¹ , w ² , and w ³ . At this time, in the block during histogram correction, a histogram of pixel values is obtained as shown in the lower part of FIG. 49, and the correction width w ⁰ is determined with reference to the histogram corrected block 1 and the peak p ⁰ ′ at which the histogram value is maximum is obtained. Apply to the above to correct the peak. Subsequently, correction is performed with a correction width of w ¹ for p ¹ ′ having the histogram value next to p ⁰ ′. Thereafter, this is repeated, and correction of p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ is performed using the parameters of the block 1 that has been subjected to the histogram correction processing.

Further, as a method of diverting different parameters, it is conceivable to always perform correction on the same peak position as shown in FIG. In FIG. 49, only the correction widths w ⁰ , w ¹ , w ² , and w ^{3 of} the histogram-corrected blocks are referenced, and the peak positions p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ are being corrected. Was newly calculated. On the other hand, in the diversion method shown in FIG. 50, the corrected block is also referred to for the peak positions p ⁰ , p ¹ , p ² , and p ³ in addition to the correction width, regardless of the histogram shape of the block being corrected, The correction of the widths w ⁰ , w ¹ , w ² , and w ³ is always applied to p ⁰ , p ¹ , p ² , and p ³ .

An advantage of the parameter diversion method shown in FIG. 49 is that the amount of parameters to be held can be minimized. Only the correction widths w ⁰ , w ¹ , w ² , and w ³ in each block need to be retained in order to use the parameters. However, a histogram of pixel values must be calculated in order to obtain the peak position in the block being corrected. On the other hand, in the parameter diversion method shown in FIG. 50, since the peak position is also diverted in addition to the correction width, it is not necessary to calculate a histogram in the block being corrected. However, since it is necessary to hold not only the correction width but also the peak position, the memory size for holding the parameter becomes larger than the method shown in FIG.

  Now, as shown in FIG. 48, if the blocks that can be referred to are upper left, right above, and left, the histogram correction processing parameters in these blocks are diverted, newly calculated parameters are used, or histogram correction processing is not performed. Can be expressed by the index shown in FIG. That is, when the histogram correction process is not performed, index 0 is encoded. When a newly calculated parameter is used, index 1 is encoded and then the calculated histogram correction process parameter is encoded. On the other hand, when the parameters of the surrounding blocks are used, only the index indicating the block position needs to be encoded, and there is no need to separately encode the parameters. Therefore, it is possible to reduce the amount of codes compared to the case of newly encoding parameters. Note that FIG. 51 is merely an example of an index, and an index different from that in FIG. 51 may be used.

When the parameters of the surrounding blocks are diverted in this way, the encoding cost C when the histogram correction processing parameters of the upper left, right above, and left blocks are diverted in consideration that only the index needs to be encoded. _TL, calculating _C T, the _{C L,} respectively (ST 604 from step ST602 of FIG. 47A).

Then, the respective coding costs are compared, and when C _OFF is the smallest, index 0 is output to the variable length coding unit 15 based on FIG. 51 and the process ends (steps ST605 and ST606 in FIG. 47B). On the other hand, when C _NEW is the smallest, index 1 which means that a parameter is newly encoded is output to the variable length encoding unit 15 (steps ST607 and ST608 in FIG. 47B), and subsequently the histogram correction processing parameters are set. It outputs to the variable-length encoding part 15 and complete | finishes (step ST609 of FIG. 47B).

If any one of C _TL , C _T , and C _L is the minimum cost, only the index indicating the position of the corresponding block is output to the variable length coding unit 15 and the process ends (step ST610 in FIG. 47B). ST614).

  FIG. 52 is a flowchart showing the operation of the variable length coding unit 15. The operation of the variable length coding unit 15 in the histogram correction processing according to the third embodiment will be described with reference to FIG. The variable length encoding unit 15 receives an index representing the parameter encoding method shown in FIG. 51 for the histogram correction processing from the loop filter unit 13, and first encodes this (step ST700 in FIG. 52).

  Subsequently, when the parameter is newly encoded (index is 1 in FIG. 51) (step ST701), the parameter for the histogram correction processing (correction width for each peak) received from the loop filter unit 13 after the index is encoded. And finishes (step ST702 in FIG. 52). On the other hand, when a parameter is not newly encoded (in FIG. 51, the index is other than 1), only the index is encoded and the process is terminated.

  The image decoding apparatus according to the third embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, only the operations relating to the histogram correction processing of the loop filter unit 39 and the variable length decoding unit 31 shown in FIG. 18 are different. Hereinafter, first, an operation related to the histogram correction processing of the variable length decoding unit 31 will be described.

  FIG. 53 is a flowchart showing the operation of the variable length decoding unit 31 relating to the histogram correction process. First, the variable length decoding unit 31 decodes an index indicating a parameter encoding method for the histogram correction processing of the block from the bit stream (step ST800 in FIG. 52). When the decoded index is a value meaning that a parameter is newly encoded (index 1 in FIG. 51), the histogram correction processing parameter is further decoded (steps ST801 and ST802 in FIG. 52).

  The processing content of the histogram correction processing in the image decoding device according to the third embodiment is basically the same as the histogram correction processing in the image decoding device according to the first embodiment, but the operation when the parameters of other blocks are diverted. Is different.

  That is, when the index of FIG. 51 obtained by referring to the variable length decoding unit 31 is 0 or 1, the operation is the same as the histogram correction processing in the image decoding apparatus according to the first embodiment. On the other hand, when the index is any of 2, 3, and 4, the block corresponding to the index among the blocks that have been subjected to the histogram correction processing shown in FIG. 48 is referred to, and parameters relating to the histogram correction processing are used.

FIG. 49 shows a parameter diversion method. For example, it is assumed that the histogram correction processing is performed as shown in the upper part of FIG. 49 in the histogram correction processed block 1 shown in FIG. In the upper part of FIG. 49, p ⁰ , p ¹ , p ² , and p ³ represent peak positions that have been subjected to correction processing, and the values of the histogram are p ⁰ > p ¹ > p ² > p ³ . Further, the correction width for each peak is set to w ⁰ , w ¹ , w ² , and w ³ . At this time, in the block during histogram correction, a histogram of pixel values is obtained as shown in the lower part of FIG. 49, and the correction width w ⁰ is determined with reference to the histogram corrected block 1 and the peak p ⁰ ′ at which the histogram value is maximum is obtained. Apply to the above to correct the peak. Subsequently, correction is performed with a correction width of w ¹ for p ¹ ′ having the histogram value next to p ⁰ ′. Thereafter, this is repeated, and correction of p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ is performed using the parameters of the block 1 that has been subjected to the histogram correction processing.

Further, as a method of diverting different parameters, it is conceivable to always perform correction on the same peak position as shown in FIG. In FIG. 49, only the correction widths w ⁰ , w ¹ , w ² , and w ^{3 of} the histogram-corrected blocks are referenced, and the peak positions p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ are being corrected. Was newly calculated. On the other hand, in the diversion method shown in FIG. 50, the corrected block is also referred to for the peak positions p ⁰ , p ¹ , p ² , and p ³ in addition to the correction width, regardless of the histogram shape of the block being corrected, The correction of the widths w ⁰ , w ¹ , w ² , and w ³ is always applied to p ⁰ , p ¹ , p ² , and p ³ .

An advantage of the parameter diversion method shown in FIG. 49 is that the amount of parameters to be held can be minimized. Only the correction widths w ⁰ , w ¹ , w ² , and w ³ in each block need to be retained in order to use the parameters. However, a histogram of pixel values must be calculated in order to obtain the peak position in the block being corrected. On the other hand, in the parameter diversion method shown in FIG. 50, since the peak position is also diverted in addition to the correction width, it is not necessary to calculate a histogram in the block being corrected. However, since it is necessary to hold not only the correction width but also the peak position, the memory size for holding the parameter becomes larger than the method shown in FIG.

  Since the output image of the loop filter unit 39 must match the output image of the loop filter unit 13 in FIG. 38, the parameter diversion method must be the same as that of the loop filter unit 13.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the third embodiment, as in the first embodiment, the loop filter unit 13 of the image encoding device obtains the optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, Since the configuration is such that noise is removed from the locally decoded image by correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  Furthermore, in the third embodiment, since the parameters relating to the above histogram correction can be referred to from the blocks that have already been subjected to the histogram correction processing, the header information that is always encoded as compared with the case where the above parameters are encoded. Can be reduced as compared with the first embodiment, and the amount of code can be reduced.

In the third embodiment, similarly to the first embodiment, the variable length coding unit 15 of the image coding apparatus performs the correction width w ^{i of} each peak necessary for the histogram correction or the block to be referred to. The image decoding apparatus encodes and outputs the index, and the image decoding apparatus performs variable length decoding on the correction width w ^{i of} each peak or the index of the block to be referred to, and performs a histogram correction of the decoded image in the loop filter unit 39 to generate noise from the decoded image. Since it is configured to be removed, as shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the third embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

  Further, the third embodiment can be easily combined with the second embodiment. For example, the index table shown in FIG. 40 can be replaced with one using two types of indexes as shown in FIG.

  FIG. 55 illustrates an operation related to the offset processing of the variable length coding unit 15 when the third embodiment is combined with the second embodiment. First, the variable length encoding unit 15 encodes index 1 shown in FIG. 54 (step ST900 in FIG. 55). That is, 0 is performed when no offset processing (including histogram correction processing) is performed in the block, 1 is performed when offset processing is performed using a newly encoded parameter, and offset processing is performed using the parameters of the processed block. When 2 is implemented, 2 is encoded.

  Subsequently, when the offset process is not performed in the block, the process ends (step ST901 in FIG. 55), and when it is performed, index 2 is encoded (step ST902 in FIG. 55). That is, when a parameter is newly encoded (index 1 is 1), a value indicating an offset application method is encoded, and when a parameter of a processed block is used, a value indicating a block position is encoded.

  Subsequently, when the parameters of the processed block are diverted, the processing ends. When the parameters are newly encoded, the parameters are encoded (steps ST903 and ST904 in FIG. 55).

  FIG. 56 illustrates an operation related to the offset process of the variable length decoding unit 31 when the third embodiment is combined with the second embodiment. First, the variable length decoding unit 31 decodes index 1 shown in FIG. 54 (step ST1000 in FIG. 56).

  Subsequently, the decoded index 1 is referred to, and if the offset process is not performed in the block, the process ends (step ST1001 in FIG. 56), and if performed, the index 2 is decoded (step ST1002 in FIG. 56).

  Subsequently, the decoded index 2 is referred to, and when the parameter of the processed block is diverted, the process ends. When the parameter is newly encoded, the parameter is decoded (steps ST1003 and ST1004 in FIG. 56).

Embodiment 4 FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable-length encoded and added to the bitstream. In Embodiment 4, however, A case will be described in which the amount of information to be encoded can be reduced by encoding the number of peaks for which noise removal is performed instead of the correction width.

  The image coding apparatus according to the fourth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter necessary for the histogram correction processing, not the correction width of each peak of the histogram but the number of peaks for which noise removal is performed is encoded.

Loop filter unit 13 in FIG. 1, and all w correction width for each peak pixel value histogram as shown in FIG. 57, determines the number N _P of the peak of implementing the correction. FIG. 57 shows an example of N _P = 4. p ⁰ , p ¹ , p ² , p ³ , p ⁴ , and p ⁵ represent the height of each peak, and p ⁰ > p ¹ > p ² > p ³ > p ⁴ > p ⁵ . That is, the width w is corrected in order from the peak having the largest value.

In this case, the loop filter unit 13, in the same manner as that for determining the optimum value of the correction width for each peak in the first embodiment, so that the image quality improving effect and the coding efficiency is maximized while changing the N _P seek such _{N P.} For the correction width w, a constant such as 15 may be set in advance, or an optimal value may be obtained for each block on which the histogram correction processing is performed.

The encoding outputs an optimum value N _P of the number of peaks found in the variable length coding unit 15. When the correction width w is changed for each block, the correction width w is also encoded.

The image decoding apparatus according to the fourth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes the number N _P of the peak of implementing noise reduction as a parameter necessary for histogram correction process, the loop filter unit 39 with reference to this implementing histogram correction process. Further, when the correction width w is encoded, this is also decoded by the variable length decoding unit 31 and is referenced by the loop filter unit 39 to perform the histogram correction processing.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the fourth embodiment, the loop filter unit 13 of the image encoding device sets the optimum number N _P (or correction width w) of the histogram peak for performing noise removal for each block of the locally decoded image. Since the configuration is such that noise is removed from the locally decoded image by obtaining and correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved with respect to screen content in which each peak of the histogram is sharp and sparse. The effect which can implement | achieve improvement is produced. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

In the fourth embodiment, to encode one type of number N _P of the peak, or two of the number N _P and the width w only as a parameter necessary for histogram correction process, as compared to the first embodiment codes The number of parameters to be reduced is reduced, header information to be encoded can be reduced, and the amount of code can be reduced.

In the fourth embodiment, the variable length coding unit 15 of the image coding apparatus codes and outputs the number of peaks N _P (or correction width w) in addition to the above-described histogram correction. The image decoding apparatus is configured to perform variable-length decoding on the number of peaks N _P (or in addition to the correction width w) and to perform noise correction on the decoded image in the loop filter unit 39 to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the fourth embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

In addition, the fourth embodiment can be easily combined with other embodiments. For example, in addition to obtaining and encoding the optimum correction width for each peak of the histogram as in the first to third embodiments, the number of peaks to be corrected is also encoded (N _P and w ⁱ (Encoding all of i = 0 to _{N P} −1) can further enhance the image quality improvement effect. However, it should be noted that the amount of code increases as the header information to be encoded increases.

  Note that the combination with the other embodiments described above may be adaptively changed in units of video, or in units of pictures, slices, or blocks. By doing so, the image quality improvement effect and the encoding efficiency are further improved. At this time, a flag indicating a combination may be added to the header (sequence parameter set, picture parameter set, slice parameter set, etc.) according to the change unit, and may be encoded by the image encoding device.

Embodiment 5 FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable-length encoded and added to the bitstream. In Embodiment 5, however, A case will be described in which the calculation amount in the decoding process can be reduced by encoding the peak position where noise removal is performed instead of the correction width.

  The image coding apparatus according to the fifth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter necessary for the histogram correction processing, the peak position where noise removal is performed is used instead of the correction width of each peak of the histogram.

As shown in FIG. 58, the loop filter unit 13 in FIG. 1 sets w as the correction width for each peak of the pixel value histogram, and determines peak positions v ⁰ , v ¹ , v ² , and v ³ to be corrected. . In FIG. 57, the number of peaks to be corrected is four.

At this time, the loop filter unit 13 performs the correction value v ⁱ (i = 0, 1, 2, in the example of FIG. 57) in the same manner as in the case of obtaining the optimum value of the correction width for each peak in the first embodiment. While changing 3), v ⁱ that maximizes the image quality improvement effect and coding efficiency is obtained. For the correction width w, a constant such as 15 may be set in advance, or an optimal value may be obtained for each block on which the histogram correction processing is performed.

Then, the obtained optimum value v ⁱ of the peak position is output to the variable length encoding unit 15 and encoded. When the correction width w is changed for each block, the correction width w is also encoded.

The image decoding apparatus according to the fifth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes the position v ⁱ of the peak of implementing noise reduction as a parameter necessary for histogram correction process, the loop filter unit 39 with reference to this implementing histogram correction process. Further, when the correction width w is encoded, this is also decoded by the variable length decoding unit 31 and is referenced by the loop filter unit 39 to perform the histogram correction processing.

Since the peak position v ^{i to} be corrected is known by the loop filter unit 39, it is not actually necessary to obtain a histogram by the loop filter unit 39. That is, in the fifth embodiment, it is possible to reduce the calculation amount in the loop filter unit 39 as compared with the first embodiment.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the fifth embodiment, the loop filter unit 13 of the image encoding device sets the optimum histogram peak position v ⁱ (or correction width w) for performing noise removal for each block of the locally decoded image. Since the configuration is such that noise is removed from the locally decoded image by obtaining and correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved with respect to screen content in which each peak of the histogram is sharp and sparse. The effect which can implement | achieve improvement is produced. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

In the fifth embodiment, since the peak position v ⁱ (or the correction width w) is encoded as a parameter necessary for the histogram correction process, it is necessary to perform histogram calculation by the loop filter unit 39 at the time of decoding. As a result, the amount of calculation for the decoding process can be reduced as compared with the first embodiment.

Further, in the fifth embodiment, the variable length coding unit 15 of the image encoding apparatus, the position of the peak required in the above histogram correction v ^{i (or} in addition to the correction width w) and outputs the encoded The image decoding apparatus is configured to perform variable-length decoding on the peak position v ⁱ (or correction width w in addition to this) and to perform histogram correction of the decoded image in the loop filter unit 39 to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the fifth embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

In addition, the fifth embodiment can be easily combined with other embodiments. For example, in addition to obtaining and encoding the optimum correction width for each peak of the histogram as in the first to third embodiments, the number of peaks to be corrected as in the fourth embodiment is also encoded. (Encoding all of N _P , v ⁱ , and w ⁱ (i = 0 to _{N P} −1)) can further enhance the image quality improvement effect. However, it should be noted that the amount of code increases as the header information to be encoded increases.

  Note that the combination with the other embodiments described above may be adaptively changed in units of video, or in units of pictures, slices, or blocks. By doing so, the image quality improvement effect and the encoding efficiency are further improved. At this time, a flag indicating a combination may be added to the header (sequence parameter set, picture parameter set, slice parameter set, etc.) according to the change unit, and may be encoded by the image encoding device.

Embodiment 6 FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable length encoded and added to the bitstream. In Embodiment 6, however, A case will be described in which the code amount related to the encoding of the parameters necessary for the histogram correction process can be reduced by encoding only the flag indicating whether or not to perform the histogram correction for each block.

  The image coding apparatus according to the sixth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter for histogram correction processing, only a flag indicating whether or not to perform histogram correction on the block is encoded.

  The loop filter unit 13 in FIG. 1 corrects a predetermined number of peaks with a predetermined correction width, and obtains an encoding cost at that time. Note that the number of peaks may always be a fixed value, or may be changed in an arbitrary unit such as a slice or a block. As for the correction width, the same fixed value may always be used for all peaks, or a different value may be used for each peak.

  If the obtained encoding cost is smaller than the encoding cost when the histogram correction processing is not performed, for example, 1 is substituted into a flag indicating whether or not the histogram correction processing is performed in the block. The data is output to the long encoding unit 15. In addition, when the obtained coding cost is larger than the coding cost when the histogram correction processing is not performed, it is variable by substituting, for example, 0 for a flag indicating whether or not the histogram correction processing is performed in the block. The data is output to the long encoding unit 15.

  The image decoding apparatus according to the sixth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes a flag indicating whether or not the histogram correction processing is performed in the block as a parameter necessary for the histogram correction processing, and the loop filter unit 39 refers to this to perform the histogram correction processing. To implement.

  That is, the loop filter unit 39 refers to the flag decoded by the variable length decoding unit 31, and when this is 1, the loop filter unit 39 corrects the predetermined number of peaks with a predetermined correction width. The number of peaks may always be a fixed value, or may be changed in arbitrary units such as slices and blocks, but the output image of the loop filter unit 39 is used as the loop filter unit 13 in the image encoding device of FIG. Therefore, it is necessary to make the same as the loop filter unit 13. As for the correction width, the same fixed value may be always used for all the peaks, or a different value may be used for each peak, which is also the same as the loop filter unit 13. There is a need.

  When the flag decoded by the variable length decoding unit 31 is 0, the loop filter unit 39 does not perform the histogram correction process on the block.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

  In the sixth embodiment, the loop filter unit 13 of the image encoding device removes noise from the local decoded image by correcting the histogram with reference to a predetermined number of peaks and correction width for each block of the local decoded image. Since it is configured, as shown in FIG. 42, there is an effect that a significant improvement in the image quality of the locally decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In the sixth embodiment, since only the flag indicating whether or not the histogram correction processing is to be performed is encoded in the block, it is necessary for encoding parameters related to the histogram correction processing as compared with the first embodiment. There is an effect that the amount of codes can be reduced.

  In the sixth embodiment, the variable length encoding unit 15 of the image encoding device encodes and outputs a flag indicating whether or not the above-described histogram correction is performed in the block, and the image decoding device Since the above-mentioned flag is variable-length decoded and the flag is a value indicating that the histogram correction processing is performed, the loop filter unit 39 performs histogram correction of the decoded image to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the sixth embodiment, an effect of obtaining an image decoding apparatus and an image decoding method capable of correctly decoding an encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Encoding control part, 2 slice division part, 3 block division part, 4 changeover switch, 5 intra prediction part, 6 intra block copy prediction part, 7 motion compensation prediction part, 8 subtraction part, 9 conversion and quantization part, 10 Inverse quantization / inverse transform unit, 11 addition unit, 12 intra memory, 13 loop filter unit, 14 motion compensated prediction frame memory, 15 variable length coding unit (encoding unit), 21 histogram calculation unit (histogram calculation unit) , 22 Pixel value correction unit (pixel value correction means), 23 First error calculation unit, 24 Second error calculation unit, 25 Correction pixel value range storage unit, 26 Repeat processing unit, 27 Pixel value replacement unit, 31 Variable Long decoding unit (decoding means), 32 inverse quantization / inverse conversion unit, 33 changeover switch, 34 intra prediction unit, 35 intra block copy prediction unit, 3 Motion compensation prediction unit, 37 adder unit, 38 an intra memory, 39 a loop filter unit, 40 motion-compensated prediction frame memory, 41 histogram calculation unit (histogram calculating means), 42 a pixel value correcting unit (pixel value correcting unit).

Claims (7)

A histogram calculating means for calculating a histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image;
An image comprising: a pixel value correcting unit that identifies a pixel on which noise accompanying coding distortion is superimposed from a histogram calculated by the histogram calculating unit, and corrects a pixel value of the pixel on which the noise is superimposed. Encoding device. The pixel value correcting means includes
A first error calculation unit for calculating an error between the encoding target image and the locally decoded image;
An error between the local decoded image and the encoding target image when it is assumed that a pixel value within a correction pixel value range including a peak pixel value in the histogram calculated by the histogram calculation unit is replaced with the peak pixel value. A second error calculation unit for calculating
If the error calculated by the second error calculator is smaller than the error calculated by the first error calculator, the error calculated by the first error calculator is converted by the second error calculator. A corrected pixel value range storage unit that overwrites and stores the corrected pixel value range in place of the calculated error;
A repetitive processing unit that updates the correction pixel value range until the correction pixel value range reaches a preset range and repeatedly instructs the second error calculation unit to re-execute an error calculation process;
After the correction pixel value range updated by the iterative processing unit reaches a preset range, the pixel value in the correction pixel value range stored by the correction pixel value range storage unit is replaced with the peak pixel value The image encoding device according to claim 1, further comprising: a pixel value replacement unit that performs the processing.   An encoding unit configured to encode the correction pixel value range stored by the correction pixel value range storage unit after the correction pixel value range updated by the iterative processing unit reaches a preset range; The image encoding device according to claim 2. Histogram calculating means for calculating a histogram of pixel values in a decoded image obtained from compressed data of an encoding target image;
An image comprising: a pixel value correcting unit that identifies a pixel on which noise accompanying coding distortion is superimposed from a histogram calculated by the histogram calculating unit, and corrects a pixel value of the pixel on which the noise is superimposed. Decoding device. A decoding means for decoding the corrected pixel value range encoded by the image encoding device;
5. The image according to claim 4, wherein the pixel value correcting unit replaces a pixel value within the correction pixel value range including a peak pixel value in the histogram calculated by the histogram calculating unit with the peak pixel value. Decoding device. A histogram calculation processing step in which a histogram calculation means calculates a histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image;
A pixel value for correcting a pixel value of a pixel on which the noise is superimposed by specifying a pixel on which noise associated with encoding distortion is superimposed from the histogram calculated in the histogram calculation processing step. An image encoding method comprising: a correction processing step. A histogram calculation processing step in which a histogram calculation means calculates a histogram of pixel values in a decoded image obtained from compressed data of an encoding target image;
A pixel value for correcting a pixel value of a pixel on which the noise is superimposed by specifying a pixel on which noise associated with encoding distortion is superimposed from the histogram calculated in the histogram calculation processing step. An image decoding method comprising: a correction processing step.

Images