JP6318729B2 - Terminal device and data management device - Google Patents

Description

  The present invention relates to a terminal device and a data management device including an encoding unit that encodes an image with high efficiency and a decoding unit that decodes an image encoded with high efficiency.

  When transmitting and accumulating moving images, MPEG (Moving Picture Expert Groups) or ITU-T H.264. It is common practice to compress the amount of information using an international standard video coding scheme such as 26x. In particular, HEVC (High Efficiency Video Coding), which has been standardized in January 2013, is a broadcasting / communication service (HD or HD) using UHDTV (Ultra High Definition Television) such as 4K and SHV (Super Hi-Vision), and mobile lines (HD). High Definition) This technology is indispensable for realizing video transmission.

When moving image coding of an encoding target image that is an original image is performed, a decoded image obtained by decoding compressed data of the encoding target image includes noise that does not originally exist in the original image (encoding distortion). Accompanying noise) may occur.
The presence of this noise not only lowers the image quality of the decoded image, but also reduces the image quality of the reference image used when the image coding apparatus performs coding. A reduction in the image quality of the reference image causes a reduction in encoding efficiency, and therefore noise accompanying encoding distortion must be removed as much as possible.

As a technique for reducing noise accompanying coding distortion, there is a loop filter that applies a filter to a decoded image.
For example, the image encoding device disclosed in Patent Document 1 below uses a filter called a pixel adaptive offset as one of the loop filters. This is a method of classifying each pixel of a decoded image into a plurality of classes according to a predetermined rule, and adding a certain offset to the pixel value for each pixel belonging to the same class. The offset value is determined by the image encoding device and transmitted to the image decoding device.
For example, if the difference average value of the pixel values of the original image and the decoded image is used as an offset, the decoded image approaches the original image by applying a pixel adaptive offset, and noise associated with encoding distortion can be reduced.

The pixel classification method in the pixel adaptive offset is roughly divided into two types, a band offset and an edge offset (see, for example, Non-Patent Document 1).
The band offset divides the pixel value from the minimum value to the maximum value at equal intervals, and classifies the pixel according to which region the pixel value of the target pixel belongs. On the other hand, the edge offset compares the pixel value of the target pixel with the pixel values of its neighboring pixels to determine whether the target pixel is an edge. If the target pixel is an edge, The pixels are classified according to the direction. The image encoding apparatus determines whether to use the band offset or the edge offset.

By the way, there is a growing need to record PC / control terminal screens as moving images in screen sharing, FA / building displays, signage, media archives, etc., where the needs are expanding with the spread of smartphones and tablets. ing. These are called screen contents in order to distinguish them from moving images taken by a normal camera.
The screen content has several properties that are different from those of a moving image shot by a camera.

FIG. 42 is an explanatory diagram of a characteristic example of screen content.
When a pixel value histogram (appearance frequency for each pixel value) in an image photographed by a camera is calculated, a dense histogram is generally obtained as shown in FIG.
On the other hand, when a histogram of pixel values in the screen content is calculated, only a very limited pixel value appears as shown in FIG. 42B, resulting in a very sparse histogram.

When screen content having such characteristics is encoded, a change occurs in the histogram due to the encoding distortion described above.
FIG. 43 is an explanatory diagram showing changes in the histogram before and after encoding in the screen content.
In the histogram of the screen content before encoding, there is no peak around the point where there is one sharp peak, but when the screen content is encoded and then decoded, coding distortion occurs, In the histogram of the decoded screen content, there are a plurality of peaks (noise due to encoding distortion) around a portion with a single sharp peak.
Noise due to coding distortion causes a reduction in the image quality of the decoded image, and thus needs to be removed by the loop filter described above.

JP2012-5113A (paragraph number [0005])

H.265 Recommendation, p. 158-160

  Since the conventional image coding apparatus is configured as described above, if the image to be coded is an image taken by a camera, if the pixel value histogram is dense, pixel adaptive offset, etc. By applying this loop filter, it is possible to reduce noise accompanying coding distortion. However, even when a loop filter such as a pixel adaptive offset is applied to a locally decoded image of screen content having a sparse histogram, there is a problem that noise accompanying coding distortion cannot be sufficiently reduced.

The present invention has been made to solve the above-described problems, and includes an encoding unit that can sufficiently reduce noise accompanying encoding distortion generated in a locally decoded image of a screen content having a sparse histogram. An object is to obtain a terminal device and a data management device.
It is another object of the present invention to obtain a terminal device and a data management device including a decoding unit that can sufficiently reduce noise accompanying coding distortion generated in a decoded image of screen content having a sparse histogram.

A terminal device according to the present invention includes a receiving unit that receives data, an arithmetic processing unit that processes data received by the receiving unit, and compressed data of an encoding target image with respect to the data processed by the arithmetic processing unit. The pixel value histogram in the locally decoded image obtained from the above is calculated, the pixel on which noise accompanying coding distortion is superimposed is identified from the calculated histogram, and the pixel value of the pixel on which the noise is superimposed is determined. An encoding unit that corrects and encodes the data; and a data storage unit that stores data encoded by the encoding unit, wherein the encoding unit is within a correction pixel value range including a peak pixel value in the histogram. In the pixel value, a first error between the encoding target image and the locally decoded image, and a pixel value within the correction target image and the correction pixel value range are set as the peak pixel value. Calculating a second error between where the decoded image, smaller than said second error the first error, and to so that to replace the pixel values in the correction pixel value range to the peak pixel value Is.

According to the present invention, a receiving unit that receives data, an arithmetic processing unit that processes the data received by the receiving unit, and the data processed by the arithmetic processing unit are obtained from the compressed data of the encoding target image. Calculating a histogram of pixel values in the locally decoded image, specifying a pixel on which noise accompanying coding distortion is superimposed from the calculated histogram, and correcting the pixel value of the pixel on which the noise is superimposed A data storage unit that stores data encoded by the encoding unit, and the encoding unit includes pixel values within a correction pixel value range including a peak pixel value in the histogram. The first error between the encoding target image and the local decoded image, and local decoding using the encoding target image and a pixel value within the correction pixel value range as the peak pixel value Calculating a second error between the image, if the second error is smaller than said first error, since a pixel value in the corrected pixel value range it has been configured so that to replace the peak pixel value , The effect of sufficiently reducing the noise caused by the coding distortion generated in the locally decoded image of the screen content in which the histogram is sparse, and the need for using a protocol for remotely operating a remote computer on the network, There is an effect that security can be improved.

It is a block diagram which shows the image coding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color-difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the conversion block size at the time of implementing the compression process of the luminance signal and color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 0 format. It is explanatory drawing which shows the case where the same directionality prediction is used with a luminance signal and a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between YUV4: 4: 4 format and YUV4: 2: 2 format. It is explanatory drawing which shows the example of the directionality prediction in YUV4: 2: 2 format equivalent to using the same directionality prediction with a luminance signal and a color difference signal in the signal of YUV4: 4: 4 format. It is explanatory drawing which shows the prediction direction vector of directionality prediction with the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between directionality prediction and an angle. It is explanatory drawing which shows an example of the quantization matrix of 4x4 DCT. It is explanatory drawing which shows the structural example of the loop filter part 13 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 13 which implements a histogram correction process. It is explanatory drawing which shows the histogram of the pixel value before and behind correction | amendment. It is explanatory drawing which shows an example of an encoding bit stream. It is a flowchart which shows the processing content of the image coding apparatus by Embodiment 1 of this invention. It is a block diagram which shows the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the structural example of the loop filter part 39 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 39 which implements a histogram correction process. It is a flowchart which shows the processing content of the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the example by which the largest encoding block is divided | segmented into a some encoding block hierarchically. (A) shows the distribution of the encoding block and prediction block after a division | segmentation, (b) is explanatory drawing which shows the condition where encoding mode m ( ^Bn ) is allocated by hierarchy division | segmentation. It is explanatory drawing which shows the index of the class classification method of a pixel adaptive offset process. It is a flowchart which shows the content of the histogram correction process in the image coding apparatus by Embodiment 1 of this invention. It is a flowchart which shows the content of the histogram correction process in the image coding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the encoding order of the conversion factor in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows an example of distribution of the transformation coefficient in the orthogonal transformation of the size of 16x16 pixels. It is explanatory drawing which shows an example of the encoding order of the transform coefficient in the orthogonal transformation of the size of 4x4 pixel and 8x8 pixel. Each prediction block P _i ⁿ the coded block B ⁿ is an explanatory diagram showing an example of a selectable intra prediction modes. It is explanatory drawing which shows an example of the pixel used when producing | generating the predicted value of the pixel in the predicted image generation block in the case of l _i ⁿ = m _i ⁿ = 4. It is explanatory drawing which shows the switching area | region of the filter in the filter process at the time of average value prediction. It is explanatory drawing which shows the reference pixel arrangement | positioning of the filter process at the time of average value prediction. It is explanatory drawing which shows the relative coordinate which makes the upper left pixel in a prediction image generation block the origin. It is explanatory drawing which shows the example of a response | compatibility with the intra prediction parameter (index value) of a color difference signal, and color difference intra prediction mode. It is explanatory drawing which shows the relationship between the intra prediction mode index of a luminance signal, and the intra prediction mode index of a color difference signal in the signal of YUV4: 2: 2 format. It is explanatory drawing which shows the relationship between an intra prediction mode index and tan (theta). It is explanatory drawing which shows the example of a response | compatibility of the intra prediction parameter of a color difference signal, and a color difference intra prediction mode. It is a flowchart which shows the content of the histogram correction process in the image decoding apparatus by Embodiment 1 of this invention. It is explanatory drawing which shows the structural example of the loop filter part 13 in the case of using a several filter process. It is a block diagram which shows the process part in the loop filter part 39 which implements a histogram correction process. It is explanatory drawing which shows the example of a response | compatibility of an index and an offset application method. It is a flowchart which shows the procedure which selects the offset application method to be used from the some offset application methods in FIG. It is explanatory drawing of the example of a characteristic of a screen content. It is explanatory drawing which shows the change of the histogram before and behind encoding in a screen content. In the pixel adaptive offset process, it is an explanatory view showing the position where the parameter in the pixel adaptive offset process is encoded and inserted into the bit stream, taking as an example a case where the unit is always divided in units of Coding Tree Unit (CTU) which is the maximum coding block. is there. It is the flowchart which showed how the pixel adaptive offset process parameter was encoded in the variable length encoding part 15, and it added to a bit stream. 12 is a flowchart illustrating a decoding procedure of pixel value adaptive offset processing parameters by the variable length decoding unit 31. 10 is a flowchart showing the operation of the loop filter unit 13 in the third embodiment. 10 is a flowchart showing the operation of the loop filter unit 13 in the third embodiment. It is explanatory drawing which shows the block division by a histogram correction process. It is explanatory drawing which shows the diversion method of a parameter. It is explanatory drawing which shows the diversion method of a parameter. It is explanatory drawing which shows the index table of the parameter encoding method. 3 is a flowchart showing the operation of the variable length coding unit 15. It is a flowchart which shows operation | movement of the variable length decoding part 31 regarding a histogram correction process. It is explanatory drawing which shows the index table of the parameter encoding method. 10 is a flowchart illustrating an operation related to offset processing of the variable length coding unit 15 when the third embodiment is combined with the second embodiment. 12 is a flowchart illustrating an operation related to an offset process of the variable length decoding unit 31 when the third embodiment is combined with the second embodiment. It is explanatory drawing which shows the relationship between the pixel value of correction object, and appearance frequency. It is explanatory drawing which shows the relationship between the pixel value of correction object, and appearance frequency. It is explanatory drawing which shows the index table of an offset application method. FIG. 20 is an explanatory diagram illustrating an example of a system configuration in a seventh embodiment. It is a block diagram which shows the example of a terminal device. It is a block diagram which shows the example of a maintenance worker terminal. FIG. 20 is an explanatory diagram illustrating another example of a system configuration according to the seventh embodiment. It is a block diagram which shows the example of a data management center. It is a block diagram which shows the example of an information device.

Embodiment 1 FIG.
FIG. 1 is a block diagram showing an image coding apparatus according to Embodiment 1 of the present invention.
The video signal to be processed by the image coding apparatus according to the first embodiment is a color video in an arbitrary color space such as a YUV signal composed of a luminance signal and two color difference signals, or an RGB signal output from a digital image sensor. In addition to signals, the video frame is an arbitrary video signal such as a monochrome image signal or an infrared image signal, in which a video frame is composed of a horizontal and vertical two-dimensional digital sample (pixel) sequence. The gradation of each pixel may be 8 bits, or a gradation of 10 bits, 12 bits, or the like. Further, it is natural that the input signal may be a still image signal instead of a video signal because the still image signal can be interpreted as a video signal composed of only one frame.

  In the following description, for convenience, unless otherwise specified, the input video signal is YUV4: 2 in which the two color difference components U and V are subsampled in half both vertically and horizontally with respect to the luminance component Y: 0 format, YUV4: 2: 2 format in which two color difference components U and V are subsampled in half in the horizontal direction with respect to luminance component Y, or two color difference components U and V are luminance component Y And YUV 4: 4: 4 format signal having the same number of samples. In addition, for RGB 4: 4: 4 format signals composed of signals of the three primary colors of red (R), green (G), and blue (B), each signal is regarded as a YUV 4: 4: 4 format signal and YUV 4: Performs the same encoding as the 4: 4 format. However, the correspondence of each signal (RGB) in RGB 4: 4: 4 format to each signal (YUV) in YUV 4: 4: 4 format is not limited (can be arbitrarily set). This association may be encoded as an index information with a high-order header so that the image decoding apparatus can recognize it. By doing in this way, the color of the decoded image decoded by the image decoding apparatus can be correctly displayed.

In the case of a YUV 4: 4: 4 format signal or an RGB 4: 4: 4 format signal, each signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV 4: 0: 0) to generate a bit stream. May be. In this way, encoding processing can be performed in parallel with each signal. At this time, information indicating which color signal each monochrome signal is may be encoded as index information in the upper header so that the image decoding apparatus can recognize it. By doing in this way, the color of the decoded image decoded by the image decoding apparatus can be correctly displayed.
The processing data unit corresponding to each frame of the video is referred to as “picture”. In the first embodiment, “picture” is described as a signal of a video frame that has been sequentially scanned (progressive scan). However, when the video signal is an interlace signal, the “picture” may be a field image signal which is a unit constituting a video frame.

  In FIG. 1, the encoding control unit 1 determines the maximum size of an encoding block that is a processing unit when the encoding process is performed, and the upper limit when the encoding block of the maximum size is hierarchically divided. By determining the number of layers, the process of determining the size of each encoded block is performed. The encoding control unit 1 also has one or more selectable encoding modes (one or more intra encoding modes having different prediction block sizes indicating prediction processing units, etc., and different prediction block sizes indicating prediction processing units). Processing for selecting a coding mode to be applied to a coding block output from the block division unit 3 from one or more intra block copy coding modes and one or more inter coding modes having different prediction block sizes. To implement. As an example of the selection method, there is a method of selecting a coding mode having the highest coding efficiency for the coding block output from the block dividing unit 3 from one or more selectable coding modes.

  In addition, when the coding mode with the highest coding efficiency is the intra coding mode, the coding control unit 1 sets the intra prediction parameters used when performing the intra prediction process on the coding block in the intra coding mode. It is determined for each prediction block that is a prediction processing unit indicated by the intra coding mode, and when the coding mode having the highest coding efficiency is the intra block copy coding mode, the intra block copy coding mode is used for the coding block. An intra block copy prediction parameter used when performing the intra block copy prediction process is determined for each prediction block which is a prediction processing unit indicated by the intra block copy coding mode, and the coding mode having the highest coding efficiency is an inter code. The inter code The inter prediction parameters used in practicing the inter prediction process for encoding blocks to implement the process of determining for each prediction block is a prediction processing unit indicated by the inter coding mode in the mode.

  Furthermore, the encoding control unit 1 performs a process of determining a prediction difference encoding parameter to be given to the transform / quantization unit 9 and the inverse quantization / inverse transform unit 10. The prediction difference coding parameter includes transform block partition information indicating transform block partition information that is a unit of orthogonal transform processing in the coded block, and a quantization parameter that specifies a quantization step size when transform coefficients are quantized. Etc. are included.

  Here, FIG. 2 is an explanatory diagram showing the conversion block size when the compression processing (conversion processing and quantization processing) of the luminance signal and the color difference signal in the YUV 4: 2: 0 format signal is performed. As shown in FIG. 2, the transform block size is determined by hierarchically dividing the encoded block into a quadtree. For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining, it is possible to determine the optimal division shape of the transform block from the viewpoint of the trade-off between the code amount and the coding error.

  For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

  When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. Further, since the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction, the divided block is further divided into two parts in the vertical direction, so that the YUV 4: 2: 0 signal The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).

  When the input signal format is a YUV 4: 4: 4 signal, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block as shown in FIG. Configure as follows. The division information of the conversion block of the luminance signal is output to the variable length encoding unit 15 as a conversion block division flag indicating whether or not to divide for each layer, for example.

When a video signal is input as an input image, the slice division unit 2 performs a process of dividing the input image into one or more partial images called “slices” according to the slice division information determined by the encoding control unit 1. The slice division unit can be finely divided to the above-described coding block unit.
Each time the block dividing unit 3 inputs the slice divided by the slice dividing unit 2, the slice dividing unit 3 divides the slice into maximum coding blocks that are coding blocks of the maximum size determined by the coding control unit 1, and Until the maximum number of hierarchies determined by the coding control unit 1 is reached, a process of dividing the maximum coding block hierarchically into each coding block is performed. That is, the block division unit 3 divides the slice into each coding block according to the division determined by the coding control unit 1, and performs a process of outputting the coding block. Each coding block is divided into one or a plurality of prediction blocks which are prediction processing units.

  If the coding mode determined by the coding control unit 1 is the intra coding mode, the changeover switch 4 outputs the coded block output from the block dividing unit 3 to the intra prediction unit 5, and the coding control unit 1 If the coding mode determined by is the intra block copy coding mode, the coding block output from the block division unit 3 is output to the intra block copy prediction unit 6, and the code determined by the coding control unit 1 is output. If the encoding mode is the inter encoding mode, a process of outputting the encoded block output from the block dividing unit 3 to the motion compensation prediction unit 7 is performed.

  When the intra control mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra prediction unit 5 performs local decoding stored in the intra memory 12. With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter determined by the encoding control unit 1 is performed to generate an intra predicted image.

  That is, for the luminance signal, the intra prediction unit 5 performs an intra prediction process (intraframe prediction process) using an intra prediction parameter of the luminance signal to generate a prediction image of the luminance signal. On the other hand, for the color difference signal, the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal. FIG. 5 shows a case where the same directionality prediction is used for a luminance signal and a color difference signal in a YUV 4: 2: 0 format signal, and FIG. 6 shows that the luminance signal and the color difference signal are the same in a YUV 4: 2: 2 format signal. The case where directionality prediction is used is shown.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  When the intra block copy encoding mode is selected by the encoding control unit 1 as the encoding mode corresponding to the encoded block output from the changeover switch 4, the intra block copy predicting unit 6 predicts the encoded block. The pixel value of each pixel in an area that has not been subjected to local decoding is assumed in a predetermined method among the maximum size encoded block (maximum encoded block) to which the block (prediction processing unit block) belongs. Perform the process. In addition, the intra block copy prediction unit 6 approximates the prediction block for each prediction block in the encoded block most closely to the prediction block from the regions that have already been encoded and locally decoded in the same picture. A process is performed for searching for a reference block that is an existing block and determining the reference block as a prediction image of the prediction block.

  When the inter coding mode is selected by the coding control unit 1 as the coding mode corresponding to the coding block output from the changeover switch 4, the motion compensated prediction unit 7 and the motion compensated prediction frame memory 14 A motion vector is searched by comparing locally decoded images of one frame or more stored in the image, and the motion vector and an inter prediction parameter such as a frame number to be determined determined by the encoding control unit 1 are used to encode the code. A process for generating an inter-predicted image by performing an inter prediction process (motion-compensated prediction process) on a block is performed.

  The subtracting unit 8 is an intra prediction image generated by the intra prediction unit 5, an intra block copy prediction image generated by the intra block copy prediction unit 6, or motion compensated prediction from the encoded block output from the block dividing unit 3. A process of subtracting the inter prediction image generated by the unit 7 and outputting a prediction difference signal indicating the difference image as a subtraction result to the transform / quantization unit 9 is performed.

The transform / quantization unit 9 refers to the transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 and performs orthogonal transform processing on the prediction difference signal output from the subtraction unit 8 (for example, DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), orthogonal transform processing such as KL transform, which is previously designed for a specific learning sequence, is performed for each transform block, and transform coefficients are calculated. Then, referring to the quantization parameter included in the prediction differential encoding parameter, the transform coefficient of the transform block unit is quantized, and the compressed data that is the transform coefficient after quantization is dequantized / inverse transform unit 10 and variable A process of outputting to the long encoding unit 15 is performed.
When the transform / quantization unit 9 quantizes the transform coefficient, the transform / quantization unit 9 performs a transform coefficient quantization process using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient. You may do it.

  FIG. 11 is an explanatory diagram showing an example of a 4 × 4 DCT quantization matrix. The numbers in the figure indicate the scaling value of the quantization step size of each transform coefficient. For example, in order to suppress the encoding bit rate, as shown in FIG. 11, by scaling the quantization step size to a larger value for a higher frequency transform coefficient, a higher frequency generated in a complex image region or the like. It is possible to perform coding without dropping information on low-frequency coefficients that greatly affect subjective quality while suppressing the amount of codes by suppressing conversion coefficients. Thus, when it is desired to control the quantization step size for each transform coefficient, a quantization matrix may be used.

  In addition, the quantization matrix can use an independent matrix for each color signal and coding mode (intra coding or inter coding) with each orthogonal transform size. It is possible to select whether to use a quantization matrix that is commonly prepared in advance by the decoding apparatus or an already encoded quantization matrix or to use a new quantization matrix. Accordingly, the transform / quantization unit 9 sets flag information indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, in a quantization matrix parameter to be encoded. .

  Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 11 is set as a quantization matrix parameter to be encoded. On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance by the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Thus, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, it is possible to select only the quantization matrix that is prepared in advance by the image encoding device and the image decoding device.

  The inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1, and transforms and transforms the transform block 9 in units of transform blocks. A local decoded prediction difference corresponding to the prediction difference signal output from the subtracting unit 8 is obtained by dequantizing the output compressed data and performing an inverse orthogonal transform process on the transform coefficient that is the compressed data after the inverse quantization. A process of calculating a signal is performed. In addition, when the transform / quantization unit 9 uses the quantization matrix to perform the quantization process, the corresponding inverse quantization can be performed by referring to the quantization matrix even during the inverse quantization process. Implement the process.

The addition unit 11 includes a local decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 10, an intra prediction image generated by the intra prediction unit 5, and an intra block copy prediction image generated by the intra block copy prediction unit 6. Alternatively, the inter prediction image generated by the motion compensation prediction unit 7 is added to perform a process of calculating a local decoded image corresponding to the encoded block output from the block dividing unit 3.
The intra memory 12 is a recording medium that stores the locally decoded image calculated by the adding unit 11.

The loop filter unit 13 performs a predetermined filter process on the local decoded image calculated by the adding unit 11 and performs a process of outputting the local decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 13 determines whether or not to perform each of the deblocking filter process, the pixel adaptive offset process, the adaptive filter process, and the histogram correction process, and uses a variable length code with the valid flag of each process as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 12 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 12 are merely examples, and the types and application order of the loop filters to be applied are not necessarily as shown in FIG. For example, the loop filter unit 13 is configured by only three types of deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, and the application order is in order of deblocking filter processing, histogram correction processing, and pixel adaptive offset processing. And so on.

In the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are used. One classification method is selected from the methods. Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating the coding distortion is calculated for each class. Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.

Therefore, in the pixel adaptive offset processing, block division information, an index indicating a class classification method for each block, and offset information for specifying an offset value of each class in block units are output to the variable length coding unit 15 as header information.
In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  In adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for the distortion that is superimposed is designed for each region (local decoded image) belonging to each class. Filter the local decoded image. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information. As the class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of each block. Further, the number of classes used in the adaptive filter processing may be set in advance as a value common to the image encoding device and the image decoding device, or may be a parameter to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

FIG. 13 is a configuration diagram illustrating a processing unit in the loop filter unit 13 that performs the histogram correction processing.
In FIG. 13, the histogram calculation unit 21 divides an input image (encoding target image) indicated by the video signal into a plurality of blocks, divides the local decoded image calculated by the addition unit 11 into a plurality of blocks, A process of calculating a histogram of pixel values is performed for each block of the decoded image. The histogram calculator 21 constitutes a histogram calculator.
The pixel value correcting unit 22 specifies a pixel on which noise accompanying coding distortion is superimposed from the histogram calculated by the histogram calculating unit 21, and performs processing for correcting the pixel value of the pixel on which the noise is superimposed. To do. The pixel value correcting unit 22 constitutes a pixel value correcting unit.

The first error calculation unit 23 of the pixel value correction unit 22 performs a process of calculating an error between the input image in units of blocks and the locally decoded image.
The second error calculation unit 24 is a local decoded image in block units on the assumption that the pixel value within the correction pixel value range including the peak pixel value in the histogram calculated by the histogram calculation unit 21 is replaced with the peak pixel value. And processing for calculating an error between the input image and the block unit.

If the error calculated by the second error calculation unit 24 is smaller than the error calculated by the first error calculation unit 23, the corrected pixel value range storage unit 25 calculates the error calculated by the first error calculation unit 23. The correction pixel value range is overwritten and stored in place of the error calculated by the second error calculation unit 24.
The iterative processing unit 26 updates the correction pixel value range until the correction pixel value range reaches a preset maximum range, and repeatedly instructs the second error calculation unit 24 to re-execute the error calculation process. Perform the process.
After the corrected pixel value range updated by the iterative processing unit 26 reaches a preset maximum range, the pixel value replacement unit 27 stores pixel values within the corrected pixel value range stored by the corrected pixel value range storage unit 25. A process of substituting for the peak pixel value is performed.

That is, in the histogram correction process, first, the histogram calculation unit 21 divides the input image indicated by the video signal into a plurality of blocks, and divides the local decoded image calculated by the addition unit 11 into a plurality of blocks. A histogram of pixel values is calculated for each block of the locally decoded image.
The pixel value correction unit 22 specifies the peak pixel value of the histogram calculated by the histogram calculation unit 21 by a predetermined method. Since noise due to encoding distortion exists in the vicinity of the peak pixel value, the pixel value is set to the peak pixel value for a pixel having a pixel value close to the peak pixel value (a pixel value within the correction pixel value range). The encoding distortion is reduced by replacing with.
Here, FIG. 14 is an explanatory diagram showing a histogram of pixel values before and after correction.
In FIG. 14, a correction width (correction pixel value range) indicating how much of the pixel value is to be corrected is represented by w, and this correction width w indicates the position of the peak pixel value or the image. Since the optimum value changes according to the characteristics, the pixel value correction unit 22 calculates.

In FIG. 1, a motion compensated prediction frame memory 14 is a recording medium that stores a locally decoded image after the filter processing of the loop filter unit 13.
The variable length encoding unit 15 and the compressed data output from the transform / quantization unit 9 and the output signal of the encoding control unit 1 (block division information in the maximum encoding block, encoding mode, prediction difference encoding parameter, A loop filter parameter, an intra prediction parameter, an intra block copy prediction parameter, an inter prediction parameter), a motion vector output from the motion compensation prediction unit 7 (when the encoding mode is an inter encoding mode), and a pixel value correction unit. A process for generating encoded data by performing variable length encoding on the optimal correction width w (correction pixel value range) calculated in Step 22 is performed.
Further, as illustrated in FIG. 15, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data. carry out. Note that the variable length coding unit 15 constitutes coding means.

However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.
The sequence level header includes an image size, a color signal format, a bit depth of a signal value of a luminance signal or a color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. , Histogram correction processing) effective flag information, quantization matrix effective flag information, and the like, which are generally common header information for each sequence unit.

The picture level header is a collection of header information set in units of pictures, such as an index of a sequence level header to be referenced, the number of reference pictures at the time of motion compensation, an entropy coding probability table initialization flag, and a quantization matrix parameter. .
The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a collection of parameters in units of slices such as flag information indicating whether or not each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process, and adaptive offset process) in the filter unit 13 is performed.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded. Supplemental information is also defined as a unique NAL unit if it exists. The picture data is defined as an access unit and indicates a unit of data access including encoded data of one picture.

  In the example of FIG. 1, a coding control unit 1, a slice division unit 2, a block division unit 3, a changeover switch 4, an intra prediction unit 5, an intra block copy prediction unit 6, and motion compensation prediction, which are components of the image coding apparatus. Unit 7, subtraction unit 8, transform / quantization unit 9, inverse quantization / inverse transform unit 10, addition unit 11, intra memory 12, loop filter unit 13, motion compensated prediction frame memory 14, and variable length coding unit 15 Are configured by dedicated hardware (components other than the intra memory 12 and the motion compensation prediction frame memory 14 are configured by, for example, a semiconductor integrated circuit on which a CPU is mounted, a one-chip microcomputer, or the like). Although the thing is assumed, the image coding apparatus may be configured by a computer.

When the image encoding device is configured by a computer, the intra memory 12 and the motion compensated prediction frame memory 14 are configured on the computer memory, and the encoding control unit 1, the slice dividing unit 2, the block dividing unit 3, and the changeover switch. 4, intra prediction unit 5, intra block copy prediction unit 6, motion compensation prediction unit 7, subtraction unit 8, transformation / quantization unit 9, inverse quantization / inverse transformation unit 10, addition unit 11, loop filter unit 13 and variable A program describing the processing contents of the long encoding unit 15 may be stored in a memory of a computer so that the CPU of the computer executes the program stored in the memory.
FIG. 16 is a flowchart showing the processing contents of the image coding apparatus according to Embodiment 1 of the present invention.

FIG. 17 is a block diagram showing an image decoding apparatus according to Embodiment 1 of the present invention.
In FIG. 17, when the variable length decoding unit 31 receives the encoded bit stream generated by the image encoding apparatus of FIG. 1, each header information such as a sequence level header, a picture level header, a slice level header, and the like is extracted from the bit stream. A process of decoding picture data is performed.
However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data. At this time, the header information includes information indicating that each of the YUV 4: 4: 4 format signal and the RGB 4: 4: 4 format signal is regarded as a monochrome image signal and is independently encoded in monochrome (YUV 4: 0: 0). In the case where it is included, the decoding process can be performed independently on the encoded bit stream of each color signal.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the sequence level header), a picture parameter header (corresponding to the picture level header), and slice data are defined as unique NAL unit types, respectively, and identification information (index) of the NAL unit type is used. It is identified by decoding. The supplemental information is also identified as a unique NAL unit if it exists. The picture data is identified as an access unit in which NAL units indicating slice data are collected.

  When the valid flag information of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding of the quantization matrix parameter, and specifies the quantization matrix. Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, a quantization matrix prepared in advance in the image encoding device and the image decoding device, or When indicating that the quantization matrix has already been decoded (not a new quantization matrix), the quantization matrix is identified with reference to the index information identifying which quantization matrix of the matrix, When the quantization matrix parameter indicates that a new quantization matrix is used, the quantization matrix is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter.

  In addition, the variable length decoding unit 31 refers to the slice level header, specifies the slice division state, decodes the slice data of each slice, specifies the encoded data of the maximum encoded block, and slice data The block division information included in the block, the encoding block that is a unit for performing decoding processing by hierarchically dividing the maximum encoding block, and specifying the compressed data, the encoding mode, Intra prediction parameters (when the encoding mode is the intra encoding mode), intra block copy prediction parameters (when the encoding mode is the intra block copy encoding mode), inter prediction parameters (the encoding mode is the inter encoding mode) ) Motion vector (if the coding mode is inter coding mode) If), carries out a process of variable length decoding optimum correction width w (corrected pixel value range) and predictive differential coding parameters. The variable length decoding unit 31 constitutes decoding means.

  The inverse quantization / inverse transform unit 32 refers to the quantization parameter and transform block division information included in the prediction difference encoding parameter variable length decoded by the variable length decoding unit 31, and the variable length decoding unit 31 performs variable length decoding. 1 is inversely quantized on a transform block basis, and inverse orthogonal transform processing is performed on transform coefficients that are compressed data after inverse quantization, and output from the inverse quantization / inverse transform unit 10 in FIG. A process of calculating the same decoded prediction difference signal as the local decoding prediction difference signal is performed.

Here, the division state of the transform block in the coding block is specified from the transform block partition information. For example, in the case of a YUV 4: 2: 0 format signal, the transform block size is determined by hierarchically dividing the encoded block into quadtrees as shown in FIG.
For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. Further, since the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction, the divided block is further divided into two parts in the vertical direction, so that the YUV 4: 2: 0 signal The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
When the input signal format is a YUV 4: 4: 4 signal, as shown in FIG. 4, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block so that the conversion block has the same size. Configure.

  In addition, when each header information variable-length decoded by the variable-length decoding unit 31 indicates that inverse quantization processing is to be performed using the quantization matrix in the slice, inverse quantization is performed using the quantization matrix. Process. Specifically, inverse quantization processing is performed using a quantization matrix specified from each header information.

  The changeover switch 33 outputs the intra-prediction parameter variable-length decoded by the variable-length decoding unit 31 to the intra-prediction unit 34 if the coding mode variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode. If the encoding mode variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode, the intra block copy prediction parameters variable length decoded by the variable length decoding unit 31 are output to the intra block copy prediction unit 35. If the coding mode variable length decoded by the variable length decoding unit 31 is the inter coding mode, the inter prediction parameter and the motion vector variable length decoded by the variable length decoding unit 31 are output to the motion compensation prediction unit 36. Perform the process.

  The intra prediction unit 34, when the coding mode related to the coding block identified from the block division information variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode, the decoding stored in the intra memory 38 With reference to the image, an intra prediction process (intraframe prediction process) using the intra prediction parameter output from the changeover switch 33 is performed to generate an intra predicted image.

  That is, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal, and generates a prediction image of the luminance signal. On the other hand, for the color difference signal, the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal.

  Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed. Specifically, an index conversion table may be prepared, and the index may be converted by referring to the conversion table. Alternatively, a conversion formula is prepared in advance, and the index is converted according to the conversion formula. You may comprise. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

  The intra block copy prediction unit 35, when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the intra block copy coding mode, In the maximum coding block to which the prediction block belongs, a process of assuming a pixel value of each pixel in a region not yet decoded by a predetermined method is performed. In addition, the intra block copy prediction unit 35 selects, for each prediction block in the encoded block, a reference block that is a block that is closest to the prediction block from among regions that have already been decoded in the same picture. A process of searching and determining the reference block as a prediction image of the prediction block is performed.

  The motion compensation prediction unit 36 is stored in the motion compensation prediction frame memory 40 when the coding mode related to the coding block specified from the block division information variable-length decoded by the variable length decoding unit 31 is the inter coding mode. The inter prediction process (motion compensation prediction process) using the motion vector output from the changeover switch 33 and the inter prediction parameter is performed while referring to the decoded image, and the process of generating the inter prediction image is performed.

The addition unit 37 includes a decoded prediction difference signal calculated by the inverse quantization / inverse conversion unit 32, an intra prediction image generated by the intra prediction unit 34, an intra block copy prediction image generated by the intra block copy prediction unit 35, Or the inter prediction image produced | generated by the motion compensation prediction part 36 is added, and the process which calculates the decoding image same as the local decoding image output from the addition part 11 of FIG. 1 is implemented.
The intra memory 38 is a recording medium that stores the decoded image calculated by the adding unit 37 as a reference image used in the intra prediction process and the intra block copy prediction process.

The loop filter unit 39 performs a predetermined filter process on the decoded image calculated by the adder unit 37 and performs a process of outputting the decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, adaptive filter processing, and histogram correction processing. Specify whether to do this in slices.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image coding apparatus is configured as shown in FIG. 12, a loop filter unit 39 is configured as shown in FIG. The Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, the loop filter unit 39 also performs deblocking filter processing, pixel adaptive offset processing, and histogram correction processing. Consists of.

In the deblocking filter processing, when there is information for changing various parameters used for selecting the filter strength applied to the block boundary from the initial value with reference to the header information variable-length decoded by the variable-length decoding unit 31, the change information Based on the above, deblocking filtering is performed. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. If the index indicating the block classification method is not an index indicating that “offset processing is not performed”, each pixel in the block is classified according to the class classification method indicated by the index. To do. Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

However, in the pixel adaptive offset processing of the loop filter unit 13 of the image encoding device, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit). When a class classification method is selected for each block and adaptive offset processing for each class is performed, the loop filter unit 39 also has a pixel adaptive offset in units of blocks having the same fixed size as the loop filter unit 13. Perform the process.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

FIG. 19 is a configuration diagram illustrating a processing unit in the loop filter unit 39 that performs the histogram correction processing.
In FIG. 19, the histogram calculation unit 41 divides the decoded image calculated by the addition unit 11 into a plurality of blocks, and performs a process of calculating a pixel value histogram for each block of the local decoded image. The histogram calculator 41 constitutes a histogram calculator.
The pixel value correction unit 42 identifies a pixel on which noise accompanying coding distortion is superimposed from the histogram calculated by the histogram calculation unit 41, and performs a process of correcting the pixel value of the pixel on which the noise is superimposed. To do.
That is, the pixel value correction unit 42 performs a process of replacing the pixel value within the correction width w (correction pixel value range) including the peak pixel value in the histogram calculated by the histogram calculation unit 41 with the peak pixel value. The pixel value correction unit 42 constitutes a pixel value correction unit.

That is, in the histogram correction process, the histogram calculation unit 41 divides the decoded image based on the block division information of the histogram correction process variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 for each block. Referring to the flag indicating whether or not to perform histogram correction that has been subjected to variable length decoding in accordance with the above, if the flag is not a value indicating that “histogram correction processing is not performed”, a histogram of pixel values is calculated in the block .
The pixel value correction unit 42 specifies the peak pixel value of the histogram calculated by the histogram calculation unit 41 by a predetermined method. Since noise due to coding distortion exists in the vicinity of the peak pixel value, a pixel having a pixel value close to the peak pixel value (a pixel value within the correction width w that has been variable-length decoded by the variable-length decoding unit 31). On the other hand, encoding distortion is reduced by replacing the pixel value with a peak pixel value.

  In FIG. 17, the motion compensation prediction frame memory 40 is a recording medium that stores the decoded image after the filter processing of the loop filter unit 39 as a reference image used in the inter prediction processing (motion compensation prediction processing).

  In the example of FIG. 17, a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, an intra prediction unit 34, an intra block copy prediction unit 35, and a motion compensation prediction unit 36 that are components of the image decoding device. Each of the adder 37, the intra memory 38, the loop filter unit 39, and the motion compensated prediction frame memory 40 is configured by dedicated hardware (components other than the intra memory 38 and the motion compensated prediction frame memory 40 are, for example, It is assumed that the CPU is mounted on a semiconductor integrated circuit or a one-chip microcomputer), but the image decoding apparatus may be configured by a computer.

When the image decoding device is configured by a computer, the intra memory 38 and the motion compensated prediction frame memory 40 are configured on the computer memory, and a variable length decoding unit 31, an inverse quantization / inverse conversion unit 32, a changeover switch 33, A program describing the processing contents of the intra prediction unit 34, the intra block copy prediction unit 35, the motion compensation prediction unit 36, the addition unit 37, and the loop filter unit 39 is stored in the memory of the computer, and the CPU of the computer stores the memory. The program stored in the program may be executed.
FIG. 20 is a flowchart showing the processing contents of the image decoding apparatus according to Embodiment 1 of the present invention.

Next, the operation will be described.
In the first embodiment, each frame image of a video is used as an input image, intra prediction from encoded neighboring pixels or motion compensation prediction between adjacent frames is performed, and an obtained prediction difference signal is obtained. An image encoding device that performs compression processing by orthogonal transform / quantization and then performs variable length encoding to generate an encoded bitstream, and an image that decodes the encoded bitstream output from the image encoding device A decoding apparatus will be described.

  The image encoding apparatus in FIG. 1 performs intra-frame and inter-frame adaptive encoding by dividing a video signal into blocks of various sizes in response to local changes in the spatial and temporal directions of the video signal. It is characterized by. In general, a video signal has a characteristic that the complexity of the signal changes locally in space and time. When viewed spatially, a small image, such as a picture with a uniform signal characteristic in a relatively wide image area such as the sky or a wall, or a picture containing a person or fine texture, on a video frame. A pattern having a complicated texture pattern in the region may be mixed. Even when viewed temporally, the change in the pattern of the sky and the wall locally in the time direction is small, but because the outline of the moving person or object moves rigidly or non-rigidly in time, the temporal change Is big.

In the encoding process, a prediction difference signal with small signal power and entropy is generated by temporal and spatial prediction to reduce the overall code amount. However, the parameters used for the prediction are set as large as possible in the image signal region. If it can be applied uniformly, the code amount of the parameter can be reduced. On the other hand, if the same prediction parameter is applied to a large image region with respect to an image signal pattern having a large temporal and spatial change, the number of prediction differential signals increases because prediction errors increase. . Therefore, in a region where the temporal and spatial changes are large, the block size for performing the prediction process by applying the same prediction parameter is reduced, the amount of parameter data used for prediction is increased, and the power and entropy of the prediction difference signal are increased. It is desirable to reduce
In the first embodiment, in order to perform coding adapted to the general characteristics of such a video signal, first, prediction processing or the like is started from a predetermined maximum block size, and the video signal region is divided hierarchically. In addition, the prediction process and the encoding process of the prediction difference are adapted for each divided area.

First, the processing contents of the image encoding device in FIG. 1 will be described.
First, the encoding control unit 1 determines the slice division state of a picture to be encoded (current picture), and the size of the maximum encoding block used for encoding the picture and the hierarchy for dividing the maximum encoding block into layers. The upper limit of the number is determined (step ST1 in FIG. 16).
As a method of determining the size of the maximum coding block, for example, the same size may be determined for all the pictures according to the resolution of the video signal of the input image, or the local motion of the video signal of the input image The size difference may be quantified as a parameter, and a small size may be determined for a picture with high motion, while a large size may be determined for a picture with little motion.

  For example, the upper limit of the number of division layers can be determined by, for example, determining the same number of layers for all pictures according to the resolution of the video signal of the input image, or when the motion of the video signal of the input image is severe There is a method in which the number of hierarchies is increased so that finer movements can be detected, and when there are few movements, the number of hierarchies is set to be suppressed. Note that the size of the maximum coding block and the upper limit of the number of hierarchies for dividing the maximum coding block into layers are coded by a sequence level header or the like. In that case, the minimum block size of the encoded block may be encoded instead of the upper limit of the number of division layers. That is, since the size of the block when the maximum encoded block is divided up to the upper limit of the number of division layers is the minimum block size of the encoded block, the size of the maximum encoded block and the size of the encoded block are determined on the image decoding device side. The upper limit of the number of divided hierarchies can be specified from the minimum block size.

  Also, the encoding control unit 1 selects an encoding mode corresponding to each encoding block divided hierarchically from one or more available encoding modes (step ST2 in FIG. 16). That is, the encoding control unit 1 divides the image area of the maximum encoding block size into encoded blocks having the encoding block size hierarchically until reaching the upper limit of the number of division layers defined above. A coding mode for each coding block is determined.

The coding modes include one or more intra coding modes (collectively referred to as “INTRA”), one or more intra block copy coding modes (collectively referred to as “ICOPY”), There are one or a plurality of inter coding modes (collectively referred to as “INTER”), and the coding control unit 1 can select from all the coding modes available in the picture or a subset thereof. The encoding mode corresponding to each encoding block is selected.
However, each coding block that is hierarchically divided by the block division unit 3 to be described later is further divided into one or a plurality of prediction blocks, which are units for performing prediction processing, and the division state of the prediction block is also coded mode. Is included as information. That is, the coding mode is an index for identifying what kind of prediction block division the intra coding mode, intra block copy coding mode, or inter coding mode is.
Since the encoding mode selection method by the encoding control unit 1 is a known technique, a detailed description thereof will be omitted. For example, an encoding process for an encoding block is performed using any available encoding mode. There is a method in which coding efficiency is verified by performing and a coding mode having the best coding efficiency is selected from among a plurality of available coding modes.

  The encoding control unit 1 determines a quantization parameter and a transform block division state used when the differential image is compressed for each encoding block, and is used when the prediction process is performed. A prediction parameter (intra prediction parameter, intra block copy prediction parameter, or inter prediction parameter) is determined. However, when the encoded block is further divided into prediction block units for performing prediction processing, a prediction parameter (intra prediction parameter, intra block copy prediction parameter, or inter prediction parameter) is selected for each prediction block.

Here, FIG. 2 is an explanatory diagram showing the conversion block size when performing compression processing (conversion processing, quantization processing) of the luminance signal and the color difference signal in the 4: 2: 0 format signal. As shown in FIG. 2, the transform block size is determined by hierarchically dividing the encoded block into a quadtree.
For example, whether or not to divide the transform block so that the evaluation value is minimized based on the amount of code when the transform block is divided and when the transform block is not divided, the evaluation scale that takes into account the coding error, etc. By determining, it is possible to determine the optimal division shape of the transform block from the viewpoint of the trade-off between the code amount and the coding error.

  For example, as shown in FIG. 2, the luminance signal is configured so that the encoding block is hierarchically divided into one or a plurality of square transform blocks. For the color difference signal, as shown in FIG. 2, when the input signal format is a YUV 4: 2: 0 signal, the encoding block is hierarchically divided into one or a plurality of square transform blocks as with the luminance signal. To be configured. In this case, the conversion block size of the color difference signal is half the vertical and horizontal sizes of the corresponding luminance signal conversion block.

When the input signal format is a YUV 4: 2: 2 signal, quadtree-like hierarchical division similar to the luminance signal is performed as shown in FIG. Further, since the shape of the divided block is a rectangle in which the number of pixels in the vertical direction is twice the number of pixels in the horizontal direction, the divided block is further divided into two parts in the vertical direction, so that the YUV 4: 2: 0 signal The color difference signal is composed of two conversion blocks having the same block size (half the vertical and horizontal sizes of the luminance signal conversion block).
When the input signal format is a YUV 4: 4: 4 signal, as shown in FIG. 4, the color difference signal conversion block is always divided in the same manner as the luminance signal conversion block, and becomes a conversion block of the same size. Configure as follows.

The encoding control unit 1 includes predictive difference encoding including transform block partition information indicating transform block partition information in a block to be encoded, a quantization parameter that defines a quantization step size when transform coefficients are quantized, and the like. The parameter is output to the transform / quantization unit 9, the inverse quantization / inverse transform unit 10, and the variable length coding unit 15.
The encoding control unit 1 outputs intra prediction parameters to the intra prediction unit 5 as necessary.
Also, the encoding control unit 1 outputs the intra block copy prediction parameters to the intra block copy prediction unit 6 as necessary.
Also, the encoding control unit 1 outputs inter prediction parameters to the motion compensation prediction unit 7 as necessary.

When a video signal is input as an input image, the slice division unit 2 divides the input image into slices that are one or more partial images according to the slice division information determined by the encoding control unit 1.
Each time each slice is input from the slice dividing unit 2, the block dividing unit 3 divides the slice into the maximum encoded block size determined by the encoding control unit 1, and further encodes the divided maximum encoded block. The coding block is hierarchically divided into coding blocks determined by the coding control unit 1, and the coding blocks are output.

Here, FIG. 21 is an explanatory diagram showing an example in which the maximum coding block is hierarchically divided into a plurality of coding blocks.
In FIG. 21, the maximum coding block is a coding block whose luminance component described as “0th layer” has a size of (L ⁰ , M ⁰ ). Starting from the maximum encoding block, the encoding block is obtained by performing hierarchical division to a predetermined depth separately defined by a quadtree structure. At depth n, the coding block is an image area of size (L ⁿ , M ⁿ ). However, L ⁿ and M ⁿ may be the same or different, but FIG. 21 shows a case of L ⁿ = M ⁿ .

Hereinafter, the coding block size determined by the coding control unit 1 is defined as the size (L ⁿ , M ⁿ ) in the luminance component of the coding block. Since quadtree partitioning is performed, (L ^{n + 1} , M ^{n + 1} ) = (L ⁿ / 2, M ⁿ / 2) always holds. Note that in a color video signal (4: 4: 4 format) in which all color components have the same number of samples, such as RGB signals, the size of all color components is (L ⁿ , M ⁿ ), but 4: 2. : When the 0 format is handled, the encoding block size of the corresponding color difference component is (L ⁿ / 2, M ⁿ / 2).

Hereinafter, the coding block of the n hierarchy expressed in B ^n, denote the encoding modes selectable by the coding block B ⁿ with m (B ^n). In the case of a color video signal composed of a plurality of color components, the encoding mode m (B ⁿ ) may be configured to use an individual mode for each color component, or common to all color components. It may be configured to use a mode. Hereinafter, unless otherwise specified, description will be made assuming that it indicates a coding mode for a luminance component of a coding block of a YUV signal and 4: 2: 0 format.

As shown in FIG. 21, the encoded block B ⁿ is divided into one or a plurality of prediction blocks representing a prediction processing unit by the block dividing unit 3. Hereinafter, a prediction block belonging to the coding block B ⁿ is ^denoted as P _i ⁿ (i is a prediction block number in the n-th layer). FIG. 21 shows an example of P ₀ ⁰ and P ₁ ⁰ . How the prediction block is divided in the coding block ^Bn is included as information in the coding mode m ( ^Bn ). All prediction blocks P _i ⁿ are subjected to prediction processing according to the encoding mode m (B ⁿ ), but for each prediction block P _i ⁿ , individual prediction parameters (intra prediction parameters, intra block copy prediction parameters, or inter prediction parameters) are used. ) Can be selected.

For example, the encoding control unit 1 generates a block division state as illustrated in FIG. 22 for the maximum encoding block, and specifies the encoding block. A rectangle surrounded by a dotted line in FIG. 22A represents each encoded block, and a block painted with diagonal lines in each encoded block represents a division state of each prediction block. FIG. 22B shows a situation in which the coding mode m (B ⁿ ) is assigned by hierarchical division in the quadtree graph for the example of FIG. Nodes surrounded by □ in FIG. 22B are nodes (encoding blocks) to which the encoding mode m (B ⁿ ) is assigned. Information of the quadtree graph is output from the encoding control unit 1 to the variable length encoding unit 15 together with the encoding mode m (B ⁿ ), and is multiplexed into the bit stream.

The changeover switch 4 is output from the block dividing unit 3 when the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA). When the coding block B ⁿ is output to the intra prediction unit 5 and the coding mode m (B ⁿ ) determined by the coding control unit 1 is the intra block copy coding mode (m (B ⁿ ) ∈ICOPY The encoding block B ⁿ output from the block division unit 3 is output to the intra block copy prediction unit 6, and the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode. In some cases (when m (B ⁿ ) εINTER), the encoded block B ⁿ output from the block dividing unit 3 is output to the motion compensation prediction unit 7.

In the intra prediction unit 5, the coding mode m (B ⁿ ) determined by the coding control unit 1 is an intra coding mode (when m (B ⁿ ) ∈INTRA), and the coding block B is changed from the changeover switch 4 to the coding block B. ^{When n} is received (step ST3 in FIG. 16), the encoding block B is determined using the intra prediction parameter determined by the encoding control unit 1 while referring to the local decoded image stored in the intra memory 12. and implementing intra prediction process for each of the prediction block _P ^{i n} in the ^n, generates an intra prediction image _{P INTRAi} ⁿ (step ST4 in FIG. 16).

However, although details will be described later, since the encoded pixel adjacent to the prediction block is used when performing the process of generating the intra prediction image, the process of generating the intra prediction image is the prediction block used for the prediction process. Must always be performed in units of transform blocks so that pixels adjacent to are already encoded. Therefore, in a coding block in which the coding mode is the intra coding mode, the block size of the selectable transform block is limited to the size of the prediction block or smaller and the transform block is smaller than the prediction block (in the prediction block). In the case where there are a plurality of transform blocks, the intra prediction process using the intra prediction parameters defined in the prediction block is performed for each transform block to generate an intra predicted image.
Incidentally, since it is necessary to image decoding apparatus of FIG. 17 to generate exactly the same intra prediction image and the intra prediction image P _INTRAi ^n, intra prediction parameters used for generating the intra prediction image P _INTRAi ^n, the encoding control unit 1 is output to the variable length encoding unit 15 and multiplexed into the bit stream. Details of processing contents of the intra prediction unit 5 will be described later.

The intra block copy prediction unit 6 is the intra block copy encoding mode when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is m (B ⁿ ) ∈ICOPY, and When the encoded block B ⁿ is received (step ST3 in FIG. 16), each prediction block P _i ⁿ in the encoded block B ⁿ is compared with the locally decoded image stored in the intra memory 12, and a block shift vector is obtained. Explore. That is, the intra block copy prediction unit 6, in the local decoded image stored in the intra-memory 12, to identify the block (reference block) in the region which is the most approximate to the prediction block P _i ^n, that Search for a block shift vector pointing to the reference block.
Intra block copy prediction unit 6, when searching for a block shift vector pointing to the reference block, the reference block whose block shift vector is pointing as the predicted image of the prediction block P _i ^n, generates an intra block copy predicted image P _ICOPYi ⁿ ( Step ST5 in FIG.

In the image decoding apparatus of FIG. 17, it is necessary to generate exactly the same intra block copy predictive image and the intra block copy predicted image P _ICOPYi ^n, intra block copy prediction is used for generating the intra block copy predicted image P _ICOPYi ⁿ The parameters are output from the encoding control unit 1 to the variable length encoding unit 15 and multiplexed into the bit stream.
Examples of the intra block copy prediction parameter include a block shift vector searched by the intra block copy prediction unit 6. Also, the block shift vector is encoded by using the block shift vector of the previous encoded prediction block or the difference value with the block shift vector of the encoded prediction block around the prediction block as a part of the intra block copy prediction parameter. You may make it do.

The motion compensated prediction unit 7 uses the encoding switch 4 from the changeover switch 4 when the encoding mode m (B ⁿ ) determined by the encoding control unit 1 is the inter encoding mode (when m (B ⁿ ) ∈INTER). When receiving B ⁿ (step ST3 in FIG. 16), each prediction block P _i ⁿ in the encoded block B ⁿ is compared with the locally decoded image after filtering stored in the motion compensated prediction frame memory 14. It searches a motion vector, by using the inter prediction parameters determined by the motion vector and the encoding control unit 1, to implement the inter-prediction processing for each of the prediction block P _i ⁿ in the encoding block B ^n, inter generating a predicted image _{P INTERi} ⁿ (step ST6 in FIG. 16).

Since the image decoding apparatus of FIG. 17 needs to generate exactly the same inter prediction image and the inter-predicted image P _INTERi ^n, inter prediction parameters used for generating the inter prediction image P _INTERi ⁿ from the coding controller 1 The data is output to the variable length encoding unit 15 and multiplexed into the bit stream.
The motion vector searched by the motion compensation prediction unit 7 is also output to the variable length encoding unit 15 and multiplexed into the bit stream.

Subtracting unit 8 receives the encoded block ^{B n} from the block dividing unit 3, from its prediction block _P ^{i n} the coded block ^{B n,} the intra prediction image _{P INTRAi} ⁿ generated by the intra prediction unit 5, the intra block copy prediction unit 6 intra block copy predicted image P _{ICOPYi n} generated ^by, or subtracts one of the inter-prediction image P _INTERi ⁿ generated by the motion compensation prediction unit 7, which is the subtraction result difference image the prediction difference signal _e ^{i n} outputs the transform and quantization unit 9 shown (step ST7 in FIG. 16).

When the transform / quantization unit 9 receives the prediction difference signal e _i ⁿ from the subtraction unit 8, the transform / quantization unit 9 refers to the transform block division information included in the prediction difference encoding parameter determined by the encoding control unit 1, and performs the prediction. orthogonal transform processing with respect to the difference signal e _i ⁿ (e.g., DCT (discrete cosine transform) or DST (discrete sine transform), the orthogonal transform for KL conversion and the base design have been made in advance to the particular learning sequence) to transform This is performed for each block, and a conversion coefficient is calculated.
Also, the transform / quantization unit 9 refers to the quantization parameter included in the prediction differential encoding parameter, quantizes the transform coefficient of the transform block unit, and reverses the compressed data that is the transform coefficient after quantization. The data is output to the quantization / inverse transform unit 10 and the variable length coding unit 15 (step ST8 in FIG. 16). At this time, the quantization process may be performed using a quantization matrix that scales the quantization step size calculated from the quantization parameter for each transform coefficient.

  As the quantization matrix, an independent matrix can be used for each color signal and coding mode (intra coding or inter coding) at each orthogonal transform size. As an initial value, an image coding device and an image are used. In the decoding apparatus, it is possible to select whether to use a previously prepared quantization matrix or an already encoded quantization matrix or to use a new quantization matrix. Accordingly, the transform / quantization unit 9 sets flag information indicating whether or not to use a new quantization matrix for each orthogonal transform size for each color signal or coding mode, in a quantization matrix parameter to be encoded. .

  Furthermore, when a new quantization matrix is used, each scaling value of the quantization matrix as shown in FIG. 11 is set as a quantization matrix parameter to be encoded. On the other hand, when a new quantization matrix is not used, as an initial value, a quantization matrix prepared in advance by the image encoding device and the image decoding device, or a quantization matrix that has already been encoded is used. Thus, an index for specifying a matrix to be used is set as a quantization matrix parameter to be encoded. However, when there is no already-encoded quantization matrix that can be referred to, only the quantization matrix prepared in advance can be selected in advance in the image encoding device and the image decoding device. Then, the transform / quantization unit 9 outputs the set quantization matrix parameter to the variable length coding unit 15.

When the inverse quantization / inverse transform unit 10 receives the compressed data from the transform / quantization unit 9, the inverse quantization / inverse transform unit 10 refers to the quantization parameter and transform block division information included in the prediction difference coding parameter determined by the coding control unit 1 Then, the compressed data is inversely quantized for each transform block. When the transform / quantization unit 9 uses a quantization matrix for the quantization process, the corresponding inverse quantization process is performed with reference to the quantization matrix even during the inverse quantization process.
Further, the inverse quantization / inverse transform unit 10 performs inverse orthogonal transform processing (for example, inverse DCT, inverse DST, inverse KL transform, etc.) on transform coefficients that are compressed data after inverse quantization for each transform block. calculates a local decoded prediction difference signal corresponding to the prediction difference signal e _i ⁿ output from the subtraction unit 8 outputs to the adder 11 (step ST9 of FIG. 16).

Addition unit 11, when the inverse quantization and inverse transform unit 10 receives the local decoded prediction difference signal, and the local decoded prediction difference signal, an intra prediction image P _{INTRAi n} generated by the intra prediction unit ^5, the intra block copy prediction part 6 intra block copy predicted image _{P ICOPYi} ⁿ generated by, or by adding one of the inter-prediction image _{P INTERi} ⁿ generated by the motion compensation prediction unit 7 calculates a local decoded image (FIG. 16 Step ST10).
The adding unit 11 outputs the locally decoded image to the loop filter unit 13 and stores the locally decoded image in the intra memory 12. This locally decoded image becomes an encoded image signal used in the subsequent intra prediction process and intra block copy prediction process.

When the loop filter unit 13 receives the local decoded image from the adder unit 11, the loop filter unit 13 performs a predetermined filter process on the local decoded image, and stores the filtered local decoded image in the motion compensated prediction frame memory 14. (Step ST11 in FIG. 16). Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 13 determines whether or not to perform each of the deblocking filter process, the pixel adaptive offset process, the adaptive filter process, and the histogram correction process, and uses a variable length code with the valid flag of each process as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 12 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 12 are merely examples, and the types and application order of the loop filters to be applied are not necessarily as shown in FIG. For example, the loop filter unit 13 is configured by only three types of deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, and the application order is in order of deblocking filter processing, histogram correction processing, and pixel adaptive offset processing. And so on.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and when the offset process is not performed for each block, it is defined as one of the class classification methods, and a plurality of class classifications prepared in advance are used. One classification method is selected from the methods. Next, each pixel in the block is classified by the selected class classification method, and an offset value for compensating the coding distortion is calculated for each class. Finally, the image quality of the locally decoded image is improved by performing a process of adding the offset value to the pixel value of the locally decoded image.

As a classifying method, a method of classifying by the size of a pixel value of a locally decoded image (referred to as a BO method), or a classification according to a situation around each pixel (whether it is an edge portion or the like) for each edge direction. There is a technique (referred to as EO technique). These methods are prepared in advance by the image encoding device and the image decoding device in advance. For example, as shown in FIG. 23, when no offset processing is performed, these methods are defined as one of the class classification methods. An index indicating which method is used for class classification is selected for each block.
Therefore, the pixel adaptive offset processing outputs the block division information, the index indicating the class classification method for each block, and the offset information for each block to the variable length encoding unit 15 as header information.

In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.
Also, in adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for superimposed distortion is designed for each region (local decoded image) belonging to each class. Then, the local decoded image is filtered. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.

  Here, as a class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of the image in units of blocks. Further, the number of classes used in the adaptive filter process may be set in advance to a common value in the image encoding device and the image decoding device, or may be one of the parameters to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

24A and 24B are flowcharts showing the contents of histogram correction processing in the image coding apparatus according to Embodiment 1 of the present invention.
Hereinafter, the flow of the histogram correction process will be described with reference to FIGS. 24A and 24B.
In the histogram correction process, first, the histogram calculation unit 21 in FIG. 13 divides the input image indicated by the video signal into a plurality of blocks and also divides the local decoded image calculated by the addition unit 11 into a plurality of blocks.
The first error calculation unit 23 of the pixel value correction unit 22 calculates an error E _min between the input image in units of blocks divided by the histogram calculation unit 21 and the locally decoded image (steps ST100 to ST101 in FIG. 24A).

As a calculation method of the error _Emin , for example, SAD (Sum of Absolute Differences) for summing absolute value differences of pixel values, SSD (Sum of Squared Differences) for summing values obtained by squaring pixel value differences, or the like may be used. Conceivable. The smaller these indexes, the more similar the input image and the local decoded image, and the smaller the distortion due to encoding.
In addition, an index representing image quality instead of error may be calculated. As an index representing image quality, PSNR (Peak Signal-to-Noise Ratio) representing the ratio of the maximum power that a signal can take and the noise that causes deterioration can be considered. When an index representing image quality is used, the larger the value, the smaller the encoding distortion.

Further, the histogram calculation unit 21 calculates a histogram of pixel values for each block of the locally decoded image (step ST102 in FIG. 24A), selects a pixel value with the maximum histogram value, and sets the pixel value to p ( Hereinafter, it is referred to as “peak pixel value”) (step ST103 in FIG. 24A).
Note that the step width of the histogram is not necessarily set to 1, and may be 1 or more. Increasing the step width for generating the histogram reduces the accuracy of the histogram, but it is possible to reduce the memory required for generating the histogram. When the histogram step width is 1 or more, the peak pixel value p may be set to the median value of the step width.

  Further, it is not always necessary to calculate the histogram using all the pixels in the block, and the histogram may be obtained using only some of the pixels. By reducing the number of pixels used for the histogram calculation, it is possible to reduce the amount of calculation required for the histogram calculation.

  In addition to directly using the obtained histogram to select the pixel value that maximizes the value, it is also conceivable to obtain the maximum value after differentiating the histogram, for example. By doing in this way, the effect which can specify only the remarkable peak with a big difference from an adjacent peak can be anticipated.

Next, the second error calculation unit 24, the correction pixel value range storage unit 25, and the iterative processing unit 26 perform a process of determining the correction width w in FIG.
First, as the initial value of the correction width w, after the correction width w _temp is set to 1, for example (step ST104 in FIG. 24A), the second error calculation unit 24 uses the pixel value ((peak) in the correction width w _temp Pixel value p-correction width w _temp ) to (pixel value of peak pixel value p + correction width w _temp ) are replaced with peak pixel value p (steps ST 105 to ST 108 in FIG. 24A). However, the replacement here is only temporary replacement to determine the correction width w, and the actual replacement (correction of the pixel value) is performed by the pixel value replacement unit 27 after the optimum correction width w is determined. Done.

ここで、（ｘ，ｙ）はブロック単位の局所復号画像における画素の座標を表し、Ｌ（ｘ，ｙ）は置換前の画素値、Ｌ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ_ｔｅｍｐが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ_ｔｅｍｐが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。
ここでは、補正幅ｗ_ｔｅｍｐをピーク画素値ｐよりも大きい側と小さい側で同一としているが、必ずしも両者を一致させる必要はなく、それぞれに対して異なる補正幅ｗ_ｔｅｍｐを使用してもよい。 Specifically, pixel value replacement is performed as follows.

Here, (x, y) represents the pixel coordinates in the local decoded image in block units, L (x, y) represents the pixel value before replacement, and L ′ (x, y) represents the pixel value after replacement. ing.
The above correction is performed on all or some of the pixels in the block. In the above equation, when p-w _temp falls below the lower limit of the pixel value, this is replaced with the lower limit of the pixel value, and when p + w _temp exceeds the upper limit of the pixel value, it is replaced with the upper limit of the pixel value. Also good.
Here, although the correction width w _temp is the same on the side larger than the peak pixel value p and the smaller side, it is not always necessary to match both, and different correction widths w _temp may be used for each.

When the pixel value in the correction width w _temp is replaced with the peak pixel value p, the second error calculation unit 24 calculates an error E between the block-unit local decoded image and the block-unit input image (FIG. 24A). Step ST109).
When the second error calculation unit 24 calculates the error E, the corrected pixel value range storage unit 25 compares the error E with the error E _min calculated by the first error calculation unit 23 (step of FIG. 24A). ST110) If the error E is smaller than the error E _min (E <E _min ), the error E _min calculated by the first error calculation unit 23 is replaced with the error E, and the correction width w _{temp is changed} to the current peak pixel. The optimum correction width w ⁱ (initial value is 0) for the value is overwritten and saved (step ST111 in FIG. 24A).
Here, i is an index indicating the peak pixel value being processed, and the initial value is 0. When the error E is larger than the error E _min , the error is increased by replacing the pixel value, and therefore the error E _min and the correction width w _temp are not updated. When an index representing image quality such as PSNR is used as an error, the inequality sign in step ST110 is reversed.

Then, repetitive processing unit 26 for the correction width _{w temp,} for example, by adding 1, to update the correction width _{w temp} (step of FIG. 24A ST 112), substituted to the peak pixel value p After the pixel value in the corrected correction width w _temp is returned to the original pixel value (step ST113 in FIG. 24A), the updated correction width w _temp is compared with the preset maximum correction width w _max (in FIG. 24A). Step ST114).
If the updated correction width w _temp is equal to or smaller than the maximum correction width w _max (w _temp ≦ w _max ), the iterative processing unit 26 instructs the second error calculation unit 24 to perform the error E calculation process again. Then, the process returns to step ST105.
On the other hand, if the updated correction width w _temp is larger than the maximum correction width w _max (w _temp > w _max ), the process proceeds to step ST115.

The number of repetitions of the processes of steps ST105 to ST114 depends on the maximum correction width w _max . The value of the maximum correction width w _max may be freely set by the image coding apparatus. However, if a large value is set, the image quality improvement effect by the histogram correction processing is enhanced, but the calculation time increases because the number of repetitions increases. Conversely, if a small value is set, the image quality improvement effect is reduced, but there is a trade-off relationship that the calculation time can be suppressed. A specific example of the maximum correction width w _max is 15 or the like.

Note that the value of w _max does not always have to be a fixed value, and may be changed according to the encoding parameter during the encoding process. For example, when the quantization parameter is large and the coding distortion is large, w _max is increased to improve the image quality improvement effect, and when the quantization parameter is small and the coding distortion is small, w _max is decreased to reduce the calculation time. It is possible to do.

  Further, the correction width w may be limited to a value such as a power of 2, for example. In this case, since only the exponent part needs to be encoded when encoding the correction width w, it is possible to reduce the code amount necessary for encoding the correction width w.

ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。 When the updated correction width w _temp becomes larger than the maximum correction width w _max (w _temp > w _max ), the pixel value replacement unit 27 optimizes that is stored by the correction pixel value range storage unit 25 as follows. The pixel values in the correction width w ⁱ ((peak pixel value p−correction width w ⁱ ) to (pixel value of peak pixel value p + correction width w ⁱ )) are replaced with the peak pixel value p (step ST115 in FIG. 24B). ST118).

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.

Next, the pixel value replacement unit 27 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculation unit 21 in ST102 ( Steps ST119 to ST120 in FIG. 24B).
The pixel value replacement unit 27 compares the index i with a preset maximum peak number P _max (step ST121 in FIG. 24B), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 21 is instructed to reselect values, and the process returns to step ST103.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.

In the first embodiment, the pixel value correction unit 22 obtains an optimum correction width w ⁱ for each peak pixel value of the histogram, and the variable length coding unit 15 obtains the optimum correction width w ⁱ for each peak pixel value. Encode.
Therefore, it is necessary to encode the correction width w ⁱ by the number of peaks P _max to be corrected (encoding all of w ⁰ , w ¹ , w ² ,..., W ^Pmax−1 ), and the maximum peak When the number P _max increases, the image quality improvement effect increases, but the code amount increases. Conversely, when the maximum peak number P _max decreases, the image quality improvement effect decreases but the code amount can be suppressed.
Naturally, the calculation time increases as the maximum number of peaks P _max increases. As a specific example of the maximum peak number P _max , 4 or the like can be considered.

For example, when the correction width w ⁱ for all the peak pixel values is 0 (when no effect is seen even if the histogram correction process is performed), the histogram correction process is not applied to the block. It is also possible to do. By doing in this way, it can avoid encoding unnecessary information and can improve encoding efficiency by reducing code amount.
Various other methods can be used to determine whether or not to apply the histogram correction processing. For example, how much encoding efficiency is achieved when encoding is performed by actually performing the histogram correction processing. It may be determined whether or not it can be improved, or may be determined using image characteristics such as texture and edges, and the result of determining whether or not the screen content.
Therefore, in the histogram correction process, first, the block division information is output as header information to the variable length encoding unit 15, and a flag indicating whether or not to perform correction for each block. An appropriate correction width w ⁱ (i = 0, 1,..., P _max −1) is output to the variable length encoding unit 15 as header information.
Whether or not to perform the histogram correction process may be determined in units of video, slices, or pictures. For example, if a flag is added to the header unit (sequence parameter set) in units of video and it can be determined in advance that the video is not screen content and histogram correction processing is not effective, this flag should be set to 0 or the like. Thus, the histogram correction process may not be performed. Of course, it is also conceivable to add a flag to the header (picture parameter set, slice parameter set) in units of pictures and slices, and switch in units of pictures and slices. By doing so, it is not necessary to encode parameters relating to the histogram correction process for a video, picture, or slice that does not have the effect of the histogram correction process, so that the code amount can be reduced.
In the histogram correction process, for example, the block is always divided into fixed-size block units such as the maximum coding block, a histogram is calculated for each block, and the correction width w ⁱ for each peak pixel value is obtained to perform the correction process. May be. In this case, the above-described block division information becomes unnecessary, and the code amount is reduced by the amount of code required for the block division information, so that the coding efficiency can be improved.

Further, the correction width w ^{i of the} block being processed is predicted using the correction width of the neighboring block that has already undergone the histogram correction processing, and only the prediction error is output to the variable length encoding unit 15 for encoding. May be. In this way, the code amount of the correction width w ⁱ can be reduced.

The processes in steps ST3 to ST10 in FIG. 16 are repeatedly performed until the processes for all the encoding blocks ^Bn divided hierarchically are completed. When the processes for all the encoding blocks ^Bn are completed, the process of step ST14 is performed. The process proceeds (steps ST12 and ST13 in FIG. 16).

The variable length encoding unit 15 uses the compressed data output from the transform / quantization unit 9 and the block division information (FIG. 22B) in the maximum encoded block output from the encoding control unit 1 as an example. (Quadrant tree information), encoding mode m (B ⁿ ), prediction differential encoding parameter, loop filter parameter, and intra prediction parameter output from the encoding control unit 1 (the encoding mode is an intra encoding mode). ), An intra block copy prediction parameter (when the encoding mode is the intra block copy encoding mode) or an inter prediction parameter (when the encoding mode is the inter encoding mode), and the motion compensated prediction unit 7 Motion vectors (when the encoding mode is the inter encoding mode) and all the peaks calculated by the pixel value correction unit 22. A correction width w ⁱ with respect to the pixel value to the variable length coding to generate a coded data indicating their encoding result (step ST14 in FIG. 16).

At this time, as a method of encoding compressed data that is a quantized orthogonal transform coefficient, the transform block is further divided into blocks of 4 × 4 pixel units (encoding sub-blocks) called Coefficient Group (CG), and CG Coding of coefficients is performed for each unit.
FIG. 25 shows the coding order (scan order) of coefficients in a 16 × 16 pixel transform block. In this way, 16 CGs in units of 4 × 4 pixels are encoded in order from the lower right CG, and each CG encodes 16 coefficients in the CG in order from the lower right coefficient.

Specifically, first, flag information indicating whether or not a significant (non-zero) coefficient exists in 16 coefficients in the CG is encoded, and then a significant (non-zero) coefficient exists in the CG. Only in this case, whether each coefficient in the CG is a significant (non-zero) coefficient is encoded in the above order, and finally, the coefficient value information is encoded in order for the significant (non-zero) coefficient. This is performed in the above order in units of CG. In this case, the encoding efficiency by entropy encoding can be increased by using a biased scan order so that significant (non-zero) coefficients are generated as continuously as possible.
Since the coefficient after the orthogonal transformation represents the lower coefficient of the low frequency component as it is closer to the upper left, including the DC component located at the upper left, generally, the closer to the upper left, the more significant as the example is shown in FIG. Since many (non-zero) coefficients are generated, as shown in FIG. 25, encoding can be efficiently performed by encoding sequentially from the lower right.

  In the above description, a 16 × 16 pixel conversion block has been described. However, in a block size other than 16 × 16 pixels, such as an 8 × 8 pixel conversion block or a 32 × 32 pixel conversion block, a code in CG (encoding subblock) unit is used. It shall be implemented. In addition, for 4 × 4 pixel and 8 × 8 pixel conversion blocks for which intra prediction is selected, processing is performed in the scan order shown in FIG. 27 instead of the scan order of FIG. 25 according to the index of the intra prediction mode. . This is because the frequency component distribution of the residual signal tends to differ depending on the direction of intra prediction.

Further, as illustrated in FIG. 15, the variable length encoding unit 15 encodes a sequence level header and a picture level header as header information of the encoded bit stream, and generates an encoded bit stream together with the picture data. However, picture data is composed of one or more slice data, and each slice data is a combination of a slice level header and the encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.
The sequence level header includes an image size, a color signal format, a bit depth of a signal value of a luminance signal or a color difference signal, and each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process) in the loop filter unit 13 in sequence units. , Histogram correction processing) effective flag information, quantization matrix effective flag information, and the like, which are generally common header information for each sequence unit.

The picture level header is a collection of header information set in units of pictures such as an index of a sequence level header to be referenced, the number of reference pictures at the time of motion compensation, an entropy encoding probability table initialization flag, and the like.
The slice level header includes position information indicating where the slice is located in the picture, an index indicating which picture level header is referred to, a slice coding type (all-intra coding, inter coding, etc.), and a loop. This is a collection of parameters in units of slices such as flag information indicating whether or not to perform each filter process (adaptive filter process, pixel adaptive offset process, deblocking filter process, histogram correction process) in the filter unit 13.

  Each header information and picture data is identified by a NAL unit. Specifically, a sequence parameter set (corresponding to the above sequence level header), a picture parameter header (corresponding to the above picture level header), and slice data are each defined as a unique NAL unit type, together with identification information (index) of the NAL unit type Encoded. If supplementary information also exists, it is defined as a unique NAL unit. The picture data is defined as an access unit and indicates a unit of data access including encoded data of one picture.

Next, the processing content of the intra estimation part 5 is demonstrated in detail.
The intra prediction unit 5, as described above, with reference to the intra prediction parameters of the prediction block P _i ^n, to implement intra prediction processing for the prediction block P _i ^n, but to generate an intra prediction image P _INTRAi ⁿ , will be described here intra process for generating an intra prediction image predicted block P _i ⁿ in the luminance signal.

Figure 28 is an explanatory diagram showing an example of the prediction block P _i ^n-selectable intra prediction modes for intra-coded blocks B ^n, and the index value of the intra prediction mode, the prediction direction vector indicated by the intra-prediction mode Show. The index value of the intra prediction mode indicates the intra prediction parameter. In addition, you may comprise so that the number of intra prediction modes may differ according to the size of the block used as a process target. Intra-prediction efficiency decreases for large-sized blocks, so the number of selectable intra-prediction directions is reduced, and for small-sized blocks, the number of selectable intra-prediction directions is increased to reduce the amount of computation. can do.

First, since the process which produces | generates an intra estimated image uses the encoded pixel adjacent to the block of a process target, as above-mentioned, it must be performed per conversion block. Here, the transform block that generates the intra predicted image is referred to as a predicted image generation block. Therefore, the intra prediction unit 5 may implement an intra-prediction image generation processing described below to the predicted image generation block generates an intra prediction image predicted block P _i ^n. The size of the predicted image generation block is assumed to be l _i ⁿ × m _i ⁿ pixels.
FIG. 29 is an explanatory diagram illustrating an example of a pixel used when generating a predicted value of a pixel in a predicted image generation block in the case of l _i ⁿ = m _i ⁿ = 4. In FIG. 29, the encoded pixels (2 × l _i ⁿ +1) and the left encoded pixels (2 × m _i ⁿ ) on the predicted image generation block are used as pixels for prediction. The number of pixels used for prediction may be more or less than that shown in FIG. In FIG. 29, pixels for one row or one column in the vicinity of the predicted image generation block are used for prediction. However, two or two or more pixels may be used for prediction.

If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs is 0 (plane (Planar) prediction) includes a coded pixels adjacent to the top of the predicted image generation block, the predicted image generation Using the encoded pixels adjacent to the left of the block, a predicted image is generated using a value interpolated according to the distance between these pixels and the prediction target pixel in the predicted image generation block as a predicted value.
When the index value of the intra prediction mode for the prediction block P _i ⁿ to which the prediction image generation block belongs is 1 (average (DC) prediction), encoded pixels adjacent to the prediction image generation block and the prediction image A predicted image is generated using the average value of the encoded pixels adjacent to the left of the generated block as the predicted value of the pixels in the predicted image generating block.

・領域Ａ（Ｐ_ｉ ^ｎの左上の画素）
ａ_０＝１／２，ａ_１＝１／４，ａ_２＝１／４
・領域Ｂ（領域Ａ以外のＰ_ｉ ^ｎの上端の画素）
ａ_０＝３／４，ａ_２＝１／４，（ａ_１＝０）
・領域Ｃ（領域Ａ以外のＰ_ｉ ^ｎの左端の画素）
ａ_０＝３／４，ａ_１＝１／４，（ａ_２＝０） Further, the region A, B, and C in FIG. 30 positioned at the upper end and the left end of the predicted image generation block is subjected to filter processing for smoothing the block boundary to obtain a final predicted image. For example, according to the following formula (1), the filter processing is performed using the following filter coefficients with the reference pixel arrangement of the filter of FIG.

· Area A _(the upper left pixel of the _P ^{i n)}
a ₀ = 1/2, a ₁ = ¼, a ₂ = ¼
- region B (the upper end of the pixel of _P ^{i n} other than the region A)
a ₀ = 3/4, a ₂ = ¼, (a ₁ = 0)
· Area C (the leftmost pixel of the _P ^{i n} other than the region A)
a ₀ = 3/4, a ₁ = ¼, (a ₂ = 0)

In equation (1), a _n (n = 0, 1, 2) is a filter coefficient applied to the reference pixel, and p _n (n = 0, 1, 2) is a reference to a filter including the pixel to be filtered p ₀ . The pixel, S ′ (p ₀ ) is the predicted value after the filtering process in the filtering target pixel p ₀ , and S (p _n ) (n = 0, 1, 2) is the filter of the reference pixel including the filtering target pixel p ₀ It represents the predicted value before processing.

  Furthermore, the block size of the predicted image generation block that performs the filtering process may be limited. In general, when the prediction value is changed by filtering only at the block edge, since the ratio of the area where the prediction value changes due to the filter processing is small in a block having a large block size, the prediction residual caused by the change in the prediction value is generated. The change of the difference signal is expressed by a very high frequency component, and the encoding efficiency tends to be deteriorated because the high frequency component is encoded. Also, by giving priority to encoding efficiency and not encoding this high frequency component, the change in the prediction residual signal at the block end cannot be restored, and the block boundary tends to be distorted. .

  On the other hand, in a block having a small block size, since the ratio of the area where the prediction value changes due to the filter processing is large, the change in the prediction residual signal caused by the change in the prediction value is as in the case of the block having a large block size. Therefore, the residual signal can be appropriately encoded, and the quality of the decoded image can be improved by increasing the continuity of the block boundary by this filter processing. Therefore, for example, in a predicted image generation block having a block size of 32 × 32 pixels or more, the above-described filter processing is not applied, and the above-described filter processing is applied only to blocks smaller than 32 × 32 pixels, thereby obtaining a conventional average value. It is possible to suppress an increase in the calculation amount while improving the prediction performance rather than the prediction.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図３２を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の輝度値（復号された輝度値）である。また、算出した予測値が輝度値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ belonging to the prediction image generation block 26 (vertical prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (2) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 32) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is the luminance value (decoded luminance value) of the encoded pixel at the coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the luminance value can take, the value is rounded so that the predicted value falls within the range.

  Note that the equation in the first line of equation (2) is MPEG-4 AVC / H. The amount of change S (−1, y) −S (−1, −1) in the vertical direction of adjacent encoded pixels with respect to S (x, −1), which is the predicted value of the vertical direction prediction in H.264. ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (2) is , MPEG-4 AVC / H. The same prediction formula as the vertical direction prediction in H.264 is shown.

ただし、座標（ｘ，ｙ）は予測画像生成ブロック内の左上画素を原点とする相対座標（図３２を参照）であり、Ｓ’（ｘ，ｙ）は座標（ｘ，ｙ）における予測値、Ｓ（ｘ，ｙ）は座標（ｘ，ｙ）における符号化済み画素の輝度値（復号された輝度値）である。また、算出した予測値が輝度値の取り得る値の範囲を超えている場合、予測値がその範囲内に収まるように値を丸めるようにする。 If the index value of the intra prediction mode for prediction block P _i ⁿ that the predicted image generation block belongs 10 (horizontal prediction), the prediction calculates the prediction value of the pixel of the predicted image generation block according to the following formula (3) Generate an image.

However, the coordinates (x, y) are relative coordinates (see FIG. 32) with the upper left pixel in the predicted image generation block as the origin, and S ′ (x, y) is the predicted value at the coordinates (x, y), S (x, y) is the luminance value (decoded luminance value) of the encoded pixel at the coordinates (x, y). Further, when the calculated predicted value exceeds the range of values that the luminance value can take, the value is rounded so that the predicted value falls within the range.

  Note that the expression on the first line of Expression (3) is MPEG-4 AVC / H. The amount of change in luminance value S (x, −1) −S (−1, −1) in the horizontal direction of an adjacent encoded pixel with respect to S (−1, y), which is the predicted value of the horizontal direction prediction in H.264 ) Is added to the value obtained by halving, and the result of filtering so that the block boundary is smoothed is used as the predicted value. The expression in the second row of Expression (3) is , MPEG-4 AVC / H. The same prediction formula as the horizontal prediction in H.264 is shown.

  However, the block size of the predicted image generation block that performs the vertical direction prediction of Expression (2) and the horizontal direction prediction of Expression (3) may be limited. In general, when the prediction value is changed by performing a filter process that adds a value proportional to the amount of change in the luminance value in the prediction direction only at the block edge, the block edge filter of the prediction image generation block described above is used in a block having a large block size. Since the ratio of the area where the predicted value changes due to processing is small, the change in the prediction residual signal caused by the change in the predicted value is represented by a very high frequency component. Therefore, the encoding efficiency tends to deteriorate. Also, by giving priority to encoding efficiency and not encoding this high frequency component, the change in the prediction residual signal at the block end cannot be restored, and the block boundary tends to be distorted.

  On the other hand, in a block with a small block size, since the ratio of the region where the prediction value changes due to the above filtering process is large, the change in the prediction residual signal caused by the change in the prediction value is large when the block size is large. The residual signal can be appropriately encoded without being represented by such high frequency components, and the quality of the decoded image can be improved by the increase in the continuity of the block boundary by this filter processing. Therefore, for example, in a prediction image generation block having a block size of 32 × 32 pixels or more, the expressions in the second row of Expression (2) and Expression (3) are always used regardless of the coordinates of the prediction target pixel (prediction). By applying the equations (2) and (3) for performing the above-described filter processing only to blocks smaller than 32 × 32 pixels, the filter edge of the image generation block is not performed. It is possible to suppress an increase in the calculation amount while improving the prediction performance compared to the direction prediction and the horizontal direction prediction.

ただし、ｋは負の実数である。 When the index value in the intra prediction mode is other than 0 (plane prediction), 1 (average value prediction), 26 (vertical direction prediction), and 10 (horizontal direction prediction), the prediction direction vector υ _p = ( Based on (dx, dy), a predicted value of a pixel in the predicted image generation block is generated.
As shown in FIG. 32, when the upper left pixel of the predicted image generation block is the origin and the relative coordinates in the predicted image generation block are set to (x, y), the position of the reference pixel used for prediction is adjacent to the following L This is the intersection of pixels.

However, k is a negative real number.

  When the reference pixel is at the integer pixel position, the integer pixel is set as the prediction value of the prediction target pixel. When the reference pixel is not at the integer pixel position, an interpolation pixel generated from the integer pixel adjacent to the reference pixel is selected. Estimated value. In the example of FIG. 29, since the reference pixel is not located at the integer pixel position, a value interpolated from two pixels adjacent to the reference pixel is set as a predicted value. Note that an interpolation pixel may be generated not only from two adjacent pixels but also from two or more adjacent pixels, and used as a predicted value. While increasing the number of pixels used in the interpolation process has the effect of improving the interpolation accuracy of the interpolated pixels, it increases the complexity of the calculation required for the interpolation process, requiring high coding performance even when the calculation load is large. In the case of an image encoding device, it is better to generate interpolation pixels from a larger number of pixels.

The processing described above, to generate a predicted pixel for all the pixels of the luminance signals of the prediction block P _i ⁿ in the predicted image generation block, and outputs an intra prediction image P _INTRAi ^n. Incidentally, the intra prediction parameters used for generating the intra prediction image P _INTRAi ⁿ (intra prediction mode) is output to the variable length coding unit 15 for multiplexing the bitstream.

  Note that the MPEG-4 AVC / H. Similarly to the smoothing process performed on the reference pixels at the time of intra prediction of the 8 × 8 pixel block in H.264, the intra prediction unit 5 predicts the reference pixels when generating the predicted image of the predicted image generation block. Even when the encoded pixels adjacent to the image generation block are configured to be the smoothed pixels, it is possible to perform the same filtering process on the predicted image as in the above example. By doing in this way, the noise of the reference pixel by the filter process to a reference pixel is removed, and prediction accuracy can be improved by performing prediction using this. Alternatively, the filtering process on the reference pixel may be performed only in the prediction other than the average value prediction, the vertical direction prediction, and the horizontal direction prediction for performing the filtering process on the predicted image. By doing in this way, it is only necessary to perform at most one filter process for each prediction mode, and an increase in the amount of calculation can be suppressed.

In the above description, the predicted image generation process of the luminance signal has been described, but the predicted image for the color difference component is generated as follows. The color difference signal of the prediction block P _i ^n, conduct intra prediction processing based on the intra prediction parameter of the color difference signal (intra prediction mode), the variable length coding unit intra prediction parameter used to generate the intra-prediction image 15 is output.

FIG. 33 is an explanatory diagram showing an example of correspondence between intra prediction parameters (index values) of color difference signals and color difference intra prediction modes. When the intra prediction parameter of the chrominance signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (when the intra prediction parameter indicates the luminance / chrominance common intra prediction mode (DM mode)), the luminance signal The same intra-frame prediction is performed to generate a prediction image of the color difference signal.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.
Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed.

FIG. 34 shows a conversion example of the intra prediction mode index in the intra prediction mode of FIG. In the conversion table of FIG. 34, when the angle of the prediction direction is θ (see FIG. 10), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to.
As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

Further, the LM mode may not be performed on the color difference signal. FIG. 34 shows a correspondence example between the intra prediction parameter (index value) of the color difference signal and the color difference intra prediction mode at this time. By not using the LM mode in this way, the dependency of the luminance signal and the color difference signal of the pixel to be predicted is eliminated, so that the prediction process of the luminance signal and the color difference signal can be parallelized, and high-speed calculation processing can be performed. Can be realized.
Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. It is good also as the prediction method similar to H.264. By not performing the filtering process in this way, it is possible to reduce the prediction process.

Next, the processing contents of the image decoding apparatus in FIG. 17 will be specifically described.
When the encoded bit stream generated by the image encoding device in FIG. 1 is input, the variable length decoding unit 31 performs a variable length decoding process on the bit stream (step ST21 in FIG. 20), and performs one or more frames. Each header information such as header information (sequence level header) and header information (picture level header) in units of a sequence composed of pictures is decoded. However, picture data is composed of one or more slice data, and each slice data is a collection of a slice level header and encoded data in the slice. The picture data may include header information indicating supplementary information in addition to the slice data.

  At this time, when the valid flag information of the quantization matrix included in the header information indicates “valid”, the variable length decoding unit 31 performs variable length decoding of the quantization matrix parameter to identify the quantization matrix. Specifically, for each color signal or encoding mode of each orthogonal transform size, the quantization matrix parameter is set as an initial value, and a quantization matrix prepared in advance by the image encoding device and the image decoding device, or When indicating that the quantization matrix is already decoded (not a new quantization matrix), refer to the index information for specifying which quantization matrix among the matrices included in the quantization matrix parameter. When the quantization matrix is specified, and the quantization matrix parameter indicates that a new quantization matrix is to be used, it is specified as a quantization matrix that uses the quantization matrix included in the quantization matrix parameter. Then, slice unit header information (slice level header) such as slice division information is decoded from slice data constituting picture unit data, and encoded data of each slice is decoded.

Further, the variable length decoding unit 31 specifies the maximum encoding block size and the upper limit of the number of division layers from the header information (step ST22 in FIG. 20). However, when the minimum block size of the encoded block is encoded instead of the upper limit of the number of division layers, the upper limit of the number of division layers is determined by decoding this. That is, the upper limit of the number of division layers is obtained when the maximum encoded block is divided to the minimum block size.
The variable length decoding unit 31 decodes the division state of the maximum encoded block as shown in FIG. 22 for each determined maximum encoded block. Based on the decoded division state, coding blocks are identified hierarchically (step ST23 in FIG. 20).

Next, the variable length decoding unit 31 decodes the encoding mode assigned to the encoding block. Based on the information included in the decoded coding mode, the coded block is further divided into one or more prediction blocks which are prediction processing units, and the prediction parameters assigned to the prediction block units are decoded (FIG. 20). Step ST24).
That is, when the encoding mode assigned to the encoding block is the intra encoding mode, the variable length decoding unit 31 is included in the encoding block, and each of one or more prediction blocks serving as a prediction processing unit When the intra prediction parameter is decoded and the coding mode assigned to the coding block is the intra block copy coding mode, one or more prediction blocks included in the coding block and serving as a prediction processing unit The intra block copy prediction parameter is decoded every time. Further, when the coding mode assigned to the coding block is the inter coding mode, the inter prediction parameter and the motion vector are included in the coding block and each of one or more prediction blocks serving as a prediction processing unit. Is decoded (step ST24 in FIG. 20). Also, the correction width w ⁱ for all peak pixel values is decoded.

  Furthermore, the variable length decoding unit 31 decodes the compressed data (transformed and transformed transform coefficients) for each transform block based on the transform block division information included in the prediction difference encoding parameter (step ST24 in FIG. 20). . At that time, similarly to the encoding process of the compressed data in the variable length encoding unit 15 of the image encoding apparatus of FIG. Accordingly, as shown in FIG. 25, 16 CGs in units of 4 × 4 pixels are decoded in order from the lower right CG, and each CG decodes 16 coefficients in the CG in order from the lower right coefficient. Will do.

  Specifically, first, flag information indicating whether or not a significant (non-zero) coefficient exists in 16 coefficients in the CG is decoded, and then the decoded flag information is significant (non-zero) in the CG. Only when it indicates that a coefficient exists, whether each coefficient in the CG is a significant (non-zero) coefficient is decoded in the above order, and finally, coefficient value information for the coefficient indicating the significant (non-zero) coefficient Are sequentially decoded. This is performed in the above order in units of CG. However, with regard to the scan order, in the case of a 4 × 4 pixel and 8 × 8 pixel conversion block for which intra prediction is selected, the scan shown in FIG. 27 is used instead of the scan order shown in FIG. Process in order.

If the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is an intra-encoding mode (when m (B ⁿ ) ∈INTRA), the changeover switch 33 is changed by the variable-length decoding unit 31. The intra-prediction parameter for each prediction block subjected to variable-length decoding is output to the intra-prediction unit 34, and the coding mode m (B ⁿ ) subjected to variable-length decoding by the variable-length decoding unit 31 is the intra-block copy coding mode. (In the case of m (B ⁿ ) εICOPY), the intra block copy prediction parameter of the prediction block unit variable length decoded by the variable length decoding unit 31 is output to the intra block copy prediction unit 35 and variable by the variable length decoding unit 31 If the length decoded coding mode m ^{(B n)} is an inter coding mode (m (the case of ^{B n)} ∈INTER), variable recovery And it outputs the inter prediction parameter and the motion vector variable length decoding prediction block to the motion compensation prediction unit 36 by the part 31.

When the encoding mode m (B ⁿ ) variable-length decoded by the variable-length decoding unit 31 is the intra-coding mode (m (B ⁿ ) εINTRA) (Step ST25 in FIG. 20), the intra prediction unit 34 The intra prediction parameter output from the selector switch 33 is received, and the intra prediction is performed with reference to the decoded image stored in the intra memory 38 in the same procedure as the intra prediction unit 5 in FIG. and implementing intra prediction process for each of the prediction block _P ^{i n} the coded block ^{B n} using a parameter to generate an intra prediction image _{P INTRAi} ⁿ (step ST26 in FIG. 20).
In addition, for the luminance signal, the intra prediction unit 34 performs an intra prediction process (intra-frame prediction process) using the intra prediction parameter for the luminance signal to generate a prediction image of the luminance signal.
On the other hand, for the color difference signal, intra prediction processing based on the intra prediction parameter of the color difference signal is performed to generate a predicted image of the color difference signal.

As shown in FIG. 33, when the intra prediction parameter of the color difference signal indicates that the same prediction mode as the intra prediction mode for the luminance signal is used (the intra prediction parameter indicates the luminance color difference common intra prediction mode (DM mode)). The same intra-frame prediction as that of the luminance signal is performed to generate a predicted image of the color difference signal.
Further, when the intra prediction parameter of the color difference signal indicates the vertical direction prediction mode or the horizontal direction prediction mode, the directionality prediction for the color difference signal is performed to generate a prediction image of the color difference signal. Further, when the intra prediction parameter of the color difference signal indicates the luminance correlation use color difference signal prediction mode (LM mode), the luminance signal and color difference of a plurality of pixels adjacent above and to the left of the prediction image generation target block. A correlation parameter indicating the correlation between the luminance signal and the color difference signal is calculated using the signal, and a predicted image of the color difference signal is generated using the correlation parameter and the luminance signal corresponding to the block of the color difference signal to be predicted.

When the input signal format is a YUV 4: 2: 2 signal and the luminance signal is a square block as shown in FIG. 7, the color difference signal has half the number of pixels in the horizontal direction compared to the luminance signal. It becomes a rectangular block. Therefore, as shown in FIG. 8, when the YUV4: 4: 4 signal is converted into a YUV4: 2: 2 signal, in order to predict the luminance signal and the color difference signal in the same direction, YUV4: 2 : On two signals, in the case of directional prediction other than vertical prediction and horizontal prediction, the prediction direction of the color difference signal is different from the prediction direction of the luminance signal. Specifically, as shown in FIG. 9, when the prediction direction vector of the luminance signal is v _L = (dx _L , dy _L ), the prediction direction vector of the color difference signal is v _C = (dx _L / 2, dy _L ). That is, as shown in FIG. 10, when the angle of the prediction direction is θ, the angle of the prediction direction of the luminance signal is θ _L , the angle of the prediction direction of the color difference signal is θ _C , and tan θ _C = 2tan θ _L It is necessary to predict in the prediction direction.

  Therefore, when the input signal format is a YUV 4: 2: 2 signal in order to correctly implement the DM mode in which the prediction in the same direction is performed with the luminance signal and the color difference signal, the intra prediction mode used for the luminance signal is used. The index is converted into an index of an intra prediction mode used for prediction of the color difference signal, and the prediction process of the color difference signal in the intra prediction mode corresponding to the converted index is performed.

In the conversion table of FIG. 34, when the angle of the prediction direction is θ (see FIG. 10), the relationship of tan θ _C = 2 tan θ _L when the direction prediction of the intra prediction mode is tan θ shown in FIG. It is an example of the table converted into the angle θ _C closest to. As described above, the conversion process may be realized by preparing an index conversion table and converting the index by referring to the conversion table, or by preparing a conversion formula and according to the conversion formula. You may comprise so that an index may be converted. With this configuration, it is possible to perform appropriate prediction of the color difference signal according to the format of the YUV 4: 2: 2 signal only by converting the index without changing the directionality prediction process itself.

In addition, when the image encoding device is configured not to perform the LM mode on the color difference signal, the image decoding device is also configured so that the encoded bitstream generated from the image encoding device can be decoded. The configuration is as follows. FIG. 34 shows a correspondence example between the intra prediction parameter (index value) of the color difference signal and the color difference intra prediction mode at this time. By not using the LM mode in this way, the dependency of the luminance signal and the color difference signal of the prediction target pixel is eliminated, so that the prediction process of the luminance signal and the color difference signal can be parallelized, and high-speed calculation processing is possible. Can be realized.
Further, in the color difference signal, the average value (DC) prediction, the vertical direction prediction, and the horizontal direction prediction are performed without performing the block boundary filtering process described in the case of the luminance signal, and the MPEG-4 AVC / H. When an image encoding apparatus is configured as a prediction method similar to H.264, the image decoding apparatus has the same configuration so that an encoded bit stream generated from the image encoding apparatus can be decoded. By not performing the filtering process in this way, it is possible to reduce the prediction process.

The intra block copy prediction unit 35, when the encoding mode m (B ⁿ ) that has been variable length decoded by the variable length decoding unit 31 is the intra block copy encoding mode (m (B ⁿ ) εICOPY) (step ST25). The intra block copy prediction parameter including the block shift vector of the prediction block unit output from the changeover switch 33 is received, and the intra block copy prediction parameter is used while referring to the decoded image stored in the intra memory 38. There was then conducted intra block copy prediction processing for each of the prediction block _P ^{i n} the coded block ^{B n,} generates an intra block copy predicted image _{P ICOPYi} ⁿ (step ST27). In the image coding apparatus, the block shift vector is obtained by calculating the difference value between the block shift vector of the previous encoded (decoded) predicted block or the block shift vector of the encoded (decoded) predicted block around the predicted block. When encoding is performed as a part of the block copy prediction parameter, the difference value included in the intra block copy prediction parameter and the block shift vector are added to calculate the block shift vector of the prediction block.

The motion compensation prediction unit 36, when the encoding mode m (B ⁿ ) that has been variable length decoded by the variable length decoding unit 31 is the inter encoding mode (m (B ⁿ ) εINTER) (step ST25 in FIG. 20). The motion vector and inter prediction parameters output from the changeover switch 33 are received and the inter prediction parameters are received while referring to the decoded image after filtering stored in the motion compensated prediction frame memory 40. by carrying out inter-prediction processing for each of the prediction block _P ^{i n} the coded block ^{B n} using a parameter to generate an inter prediction image _{P INTERi} ⁿ (step ST28 in FIG. 20).

  When receiving the compressed data and the prediction difference encoding parameter from the variable length decoding unit 31, the inverse quantization / inverse conversion unit 32 performs the prediction difference encoding in the same procedure as the inverse quantization / inverse conversion unit 10 of FIG. With reference to the quantization parameter and transform block division information included in the parameters, the compressed data is inversely quantized in transform block units. At this time, when referring to each header information variable-length decoded by the variable-length decoding unit 31, each header information indicates that the inverse quantization process is performed using the quantization matrix in the slice. Inverse quantization processing is performed using a quantization matrix.

  At this time, referring to each header information variable length decoded by the variable length decoding unit 31, for each color signal and coding mode (intra coding, intra block copy coding, inter coding) with each orthogonal transform size Specify the quantization matrix to be used. Further, the inverse quantization / inverse transform unit 32 performs inverse orthogonal transform processing on transform coefficients that are compressed data after inverse quantization in units of transform blocks, and outputs from the inverse quantization / inverse transform unit 10 in FIG. The same decoded prediction difference signal as the local decoding prediction difference signal thus obtained is calculated (step ST29 in FIG. 20).

Addition unit 37, an intra block generated by the decoded prediction difference signal calculated by the inverse quantization and inverse transform unit 32, an intra prediction image P _{INTRAi ^n,} intra block copy prediction unit 35 generated by the intra prediction unit 34 copy predicted image _{P ICOPYi} ^n, or, together with by adding one of generated by the motion compensation prediction unit 36 inter-prediction image _{P INTERi} ⁿ calculates a decoded image, and outputs the decoded image to the loop filter unit 39, The decoded image is stored in the intra memory 38 (step ST30 in FIG. 20). This decoded image becomes a decoded image signal used in the subsequent intra prediction processing and intra block copy prediction processing.

Loop filter unit 39, the process of step ST23~ST30 of all the coding blocks ^{B n} is completed (step ST31 in FIG. 20), with respect to output decoded image from the adder 37, a predetermined filter process performed Then, the decoded image after filter processing is stored in the motion compensated prediction frame memory 40 (step ST32 in FIG. 20).
Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, and a sufficient image quality improvement effect can be obtained. Absent. Therefore, in the first embodiment, histogram correction processing is performed as new loop filter processing.

The loop filter unit 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, adaptive filter processing, and histogram correction processing. Specify whether to do this in slices.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image encoding device of FIG. 1 is configured as shown in FIG. 12, the loop filter unit 39 as shown in FIG. Is configured. Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing, pixel adaptive offset processing, and histogram correction processing, the loop filter unit 39 also performs deblocking filter processing, pixel adaptive offset processing, and histogram correction processing. Consists of.

Here, in the deblocking filter processing, when there is information for referring to the header information that has been variable-length decoded by the variable-length decoding unit 31 and changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Performs a deblocking filter process based on the change information. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the block is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the block unit of variable-length decoded by the variable-length decoding unit 31 is divided into the blocks. When an index indicating a class classification method is referred to and the index is not an index indicating that “offset processing is not performed”, each pixel in the block is classified into blocks in accordance with the class classification method indicated by the index. Note that the same class classification method candidates as those for the pixel adaptive offset processing class classification method of the loop filter unit 13 are prepared in advance.

Then, the loop filter unit 39 performs processing for adding the offset to the pixel value of the decoded image with reference to the offset information that has been variable-length decoded by the variable-length decoding unit 31 that identifies the offset value of each class in block units. .
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

  In the histogram correction process, the histogram calculation unit 41 divides the decoded image based on the block division information of the histogram correction process variable-length decoded by the variable-length decoding unit 31, and variable by the variable-length decoding unit 31 for each block. With reference to a flag indicating whether or not to perform long-decoded histogram correction, if the flag is not a value indicating that “no histogram correction processing is performed”, the histogram correction processing is executed for the block.

FIG. 37 is a flowchart showing the contents of the histogram correction processing in the image decoding apparatus according to Embodiment 1 of the present invention.
Hereinafter, the flow of the histogram correction process will be described with reference to FIG.
In the histogram correction processing, first, the histogram calculation unit 41 in FIG. 19 divides the decoded image calculated by the addition unit 37 into a plurality of blocks, and calculates a histogram of pixel values for each block of the decoded image (FIG. 19). 37 steps ST200 to ST201). Further, the pixel value having the maximum histogram value is selected, and the pixel value is set to p (hereinafter referred to as “peak pixel value”) (step ST202 in FIG. 37).
Note that the step width of the histogram does not necessarily have to be 1, but the output image of the loop filter unit 39 needs to match the output image of the loop filter unit 13 in the image encoding device of FIG. The same step width as that of the loop filter unit 13 is used. When the step width of the histogram is set to 1 or more, it is conceivable that p is set to the median value of the step width or the like, but this also needs to be matched with the loop filter unit 13 for the same reason as described above.

  Further, it is not always necessary to calculate the histogram using all the pixels in the block, and the histogram may be obtained using only some of the pixels. By reducing the number of pixels used for the histogram calculation, it is possible to reduce the amount of calculation required for the histogram calculation. This also needs to be matched with the loop filter unit 13 for the same reason as described above.

  In addition to directly using the obtained histogram to select the pixel value that maximizes the value, for example, it may be possible to obtain the maximum value after differentiating the histogram, for the same reason as above. It is necessary to match with the loop filter unit 13.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。 Pixel value correction unit 42, the pixel values in the correction width w ⁱ with respect to the variable-length decoded i-th peak by variable-length decoder 31 ((peak pixel value p- correction width w ⁱ⁾ ~ (peak pixel value p + correction The pixel value of the width w ⁱ ) is replaced with the peak pixel value p (steps ST203 to ST206 in FIG. 37).

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 in the image encoding device of FIG. 1, a method similar to that for the loop filter unit 13 must be used.

Next, the pixel value correcting unit 42 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculating unit 41 in ST202 ( Steps ST207 to ST208 in FIG. 37).
The pixel value correcting unit 42 compares the index i with a preset maximum peak number P _max (step ST209 in FIG. 37), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 41 is instructed to reselect the value, and the process returns to step ST202.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.
The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

As is apparent from the above, according to the first embodiment, the histogram calculation unit 21 that calculates the histogram of the pixel values in the local decoded image obtained from the compressed data of the input image, and the histogram calculated by the histogram calculation unit 21 And the pixel value correcting unit 22 for correcting the pixel value of the pixel on which the noise is superimposed and specifying the pixel on which the noise due to the coding distortion is superimposed. This produces an effect of sufficiently reducing noise accompanying encoding distortion occurring in the locally decoded image of the screen content.
That is, the pixel value correction unit 22 of the loop filter unit 13 in the image encoding device obtains an optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, and the correction value w ⁱ within the correction width w ⁱ By replacing the pixel value with the peak pixel value p so as to remove noise from the local decoded image, as shown in FIG. 42, for screen content where each peak of the histogram is sharp and sparse, There is an effect that the image quality of the locally decoded image can be significantly improved. However, this effect is not limited to the screen content, and the same effect can be obtained as long as it is a video signal having a similar histogram characteristic.

Further, according to the first embodiment, the variable length encoding unit 15 of the image encoding device encodes and outputs the correction width w ⁱ for each peak pixel value, while the variable length decoding unit 31 of the image decoding device. However, the correction width w ⁱ for each peak pixel value is decoded, and the pixel value correction unit 42 of the loop filter unit 39 sets the pixel value within the correction width w ⁱ decoded by the variable length decoding unit 31 to the peak pixel value p. Since it is configured to remove noise from the local decoded image by replacing, as shown in FIG. 42, the image quality of the decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and the same effect can be obtained as long as it is a video signal having a similar histogram characteristic.
In addition, according to the first embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above effects can be obtained. Play.

Embodiment 2. FIG.
In the first embodiment, histogram correction processing is added to the loop filter unit 13 of the image encoding device of FIG. 1 as shown in FIG. 12, and the loop filter unit 39 of the image decoding device of FIG. On the other hand, as shown in FIG. 18, the histogram correction processing is added. However, by configuring the histogram correction processing as a part of the existing loop filter, the amount of information to be encoded can be reduced. You may make it reduce.

The image coding apparatus according to the second embodiment has the same configuration as the image coding apparatus of FIG. 1 in the first embodiment, but only the internal configuration of the loop filter unit 13 shown in FIG. Will be changed as follows.
Here, only the configuration of the modified loop filter unit 13 shown in FIG. 38 will be described.

  The loop filter unit 13 in FIG. 38 performs a predetermined filtering process on the local decoded image calculated by the adding unit 11, and performs a process of outputting the local decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable.

The loop filter unit 13 in FIG. 38 determines whether or not to perform each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and uses a variable length code with the valid flag of each processing as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 38 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 38 are merely examples, and the types and application order of the loop filters to be applied do not necessarily have to be as shown in FIG. For example, it is conceivable that the loop filter unit 13 is configured with only two types of deblocking filter processing and pixel adaptive offset processing, or that the application order is in order of pixel adaptive offset processing and deblocking filter processing.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and one of a plurality of offset application methods is selected for each block including the case where the offset process is not performed.
In the second embodiment, histogram correction processing can be selected as one method of applying this offset. The loop filter unit 13 in FIG. 38 classifies each pixel in the block according to the selected offset application method, and calculates an offset value for compensating for coding distortion for each class, or performs histogram correction. An optimum correction width W ⁱ is calculated for each peak pixel value.
Finally, processing for adding an offset value to the pixel value of the locally decoded image is performed, or the image quality of the locally decoded image is improved by correcting the histogram of the locally decoded image.

ここで、（ｘ，ｙ）はブロック単位の局所復号画像における画素の座標を表し、Ｌ（ｘ，ｙ）は置換前の画素値、Ｌ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。
ここでは、補正幅ｗ^ｉをピーク画素値ｐよりも大きい側と小さい側で同一としているが、必ずしも両者を一致させる必要はなく、それぞれに対して異なる補正幅ｗ^ｉを使用してもよい。 When the correction width is w ⁱ , the histogram correction process is performed as follows.

Here, (x, y) represents the pixel coordinates in the local decoded image in block units, L (x, y) represents the pixel value before replacement, and L ′ (x, y) represents the pixel value after replacement. ing.
The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each.

In the histogram correction process, after correcting the first peak as described above, the process moves to the next peak, and the same process is repeated until a predetermined number of peaks is reached.
In the pixel adaptive offset process, header information includes block division information, an index indicating a class classification method for each block, offset information for specifying an offset value of each class in block units, or a correction width for each peak of a histogram for histogram correction. The data is output to the variable length coding unit 15.

  In the first embodiment, it is necessary to encode information for the histogram correction process in addition to the information for the pixel adaptive offset process. However, in the second embodiment, as a part of the pixel adaptive offset process, By performing the histogram correction process, only one of the information for the offset process and the information for the histogram correction process needs to be encoded, so that the code amount in the entire loop filter can be reduced.

  In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  In adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for the distortion that is superimposed is designed for each region (local decoded image) belonging to each class. Filter the local decoded image. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information. As the class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of each block. Further, the number of classes used in the adaptive filter processing may be set in advance as a value common to the image encoding device and the image decoding device, or may be a parameter to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

The image decoding apparatus according to the second embodiment has the same configuration as that of the image decoding apparatus in FIG. 17 in the first embodiment, but only the internal configuration of the loop filter unit 39 shown in FIG. 18 is as shown in FIG. Be changed.
Here, only the configuration of the modified loop filter unit 39 shown in FIG. 39 will be described.

  The loop filter unit 39 in FIG. 39 performs a predetermined filter process on the decoded image calculated by the adder unit 37, and performs a process of outputting the decoded image after the filter process. Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable. However, the combination method needs to be the same as that of the loop filter unit 13 of the image encoding device.

The loop filter unit 39 in FIG. 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing described above, and Specify whether to do it.
At this time, when two or more filter processes are performed, for example, if the loop filter unit 13 of the image encoding device is configured as shown in FIG. 38, the loop filter unit 39 is configured as shown in FIG. The Naturally, if the loop filter unit 13 of the image encoding device is configured by deblocking filter processing and pixel adaptive offset processing, the loop filter unit 39 is also configured by deblocking filter processing and pixel adaptive offset processing.

Here, in the deblocking filter processing, with reference to the header information that has been subjected to variable length decoding by the variable length decoding unit 31, there is information for changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Based on the change information, deblocking filter processing is performed. When there is no change information, it is performed according to a predetermined method.
In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. Referring to an index indicating a block classification method, if the index is not an index indicating that “offset processing is not performed” and is not an index indicating that “histogram correction processing is performed”, the index Accordingly, each pixel in the block is classified according to the class classification method indicated by the index. As the class classification method candidates, the same class classification method candidates for the pixel adaptive offset process of the loop filter unit 13 of FIG. 38 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

  On the other hand, the index indicating the block class classification method variable-length decoded by the variable-length decoding unit 31 is not an index indicating that “offset processing is not performed” and is an index indicating that “histogram correction processing is performed”. In some cases, a histogram of pixel values in the block is calculated, and the position of the peak pixel value in the histogram is determined by a predetermined method.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。
ブロック内の全部または一部の画素に対して上記の補正を実施する。なお、上式において、ｐ−ｗ^ｉが画素値の下限を下回った場合はこれを画素値の下限に置き換え、ｐ＋ｗ^ｉが画素値の上限を上回った場合はこれを画素値の上限に置き換えてもよい。 The pixel value correction unit 42 includes pixel values ((peak pixel value p−correction width w ⁱ ) to (peak pixel value p + correction) within the correction width w ⁱ for the i-th peak variable-length decoded by the variable length decoding unit 31. Replace the pixel value of width w ⁱ ) with the peak pixel value p.

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.
The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.

Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 in the image encoding device, the same method as the loop filter unit 13 must be used.
In the histogram correction process, after correcting the first peak as described above, the process moves to the next peak, and the same process is repeated until a predetermined number of peaks is reached.

However, in the pixel adaptive offset processing of the loop filter unit 13 of the image encoding device, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit). When a class classification method is selected for each block and adaptive offset processing for each class is performed, the loop filter unit 39 also has a pixel adaptive offset in units of blocks having the same fixed size as the loop filter unit 13. Perform the process.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.

Next, in the image coding apparatus according to the second embodiment, the operation of the loop filter unit 13 having a configuration different from that of the image coding apparatus according to the first embodiment will be described. Other operations are the same as those in the first embodiment.
When the loop filter unit 13 in FIG. 38 receives the local decoded image from the adder unit 11, the loop filter unit 13 performs a predetermined filter process on the local decoded image, and the local decoded image after the filter process is subjected to the motion compensated prediction frame memory. 14 (step ST11 in FIG. 16). Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.

These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.
In the second embodiment, a case where histogram correction processing is performed as part of the pixel adaptive offset will be described.
However, the combination with the histogram correction processing is not necessarily limited to the pixel adaptive offset, and combinations with other loop filters are naturally conceivable.

The loop filter unit 13 in FIG. 38 determines whether or not to perform each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and uses a variable length code with the valid flag of each processing as header information. To the conversion unit 15. When a plurality of the above filter processes are used, each filter process is performed in order. FIG. 38 shows a configuration example of the loop filter unit 13 when a plurality of filter processes are used.
However, the types and order of the loop filters shown in FIG. 38 are merely examples, and the types and application order of the loop filters to be applied do not necessarily have to be as shown in FIG. For example, it is conceivable that the loop filter unit 13 is configured with only two types of deblocking filter processing and pixel adaptive offset processing, or that the application order is in order of pixel adaptive offset processing and deblocking filter processing.

Here, in the deblocking filter process, various parameters used for selecting the filter strength applied to the block boundary can be changed from the initial values. When changing, the parameter is output to the variable length coding unit 15 as header information.
In the pixel adaptive offset process, first, an image is divided into a plurality of blocks, and one of a plurality of offset application methods is selected for each block including the case where the offset process is not performed.
In the second embodiment, histogram correction processing can be selected as one method of applying this offset. The loop filter unit 13 in FIG. 38 classifies each pixel in the block according to the selected offset application method, and calculates an offset value for compensating for coding distortion for each class, or performs histogram correction. An optimum correction width W ⁱ is calculated for each peak pixel value.
Finally, processing for adding an offset value to the pixel value of the locally decoded image is performed, or the image quality of the locally decoded image is improved by correcting the histogram of the locally decoded image.

As an offset application method, a method of classifying by the size of a pixel value of a locally decoded image (referred to as a BO method), a situation around each pixel (whether or not an edge portion, etc.) for each edge direction, etc. There is a method of classifying (called an EO method).
In the second embodiment, unlike the first embodiment, a histogram correction process is added here. These methods are prepared in advance by the image encoding device and the image decoding device in advance. For example, as shown in FIG. 40, when the offset process is not performed, it is defined as one of the offset application methods. Among the methods, an index indicating which method is used is selected for each block.

  Also, the offset application method shown in FIG. 40 may be prepared by preparing several sets in advance and switching them. For example, a flag (herein referred to as an SC flag) indicating whether or not the input video is screen content is prepared in a header (sequence parameter set) assigned for each sequence, and two flags shown in FIG. It is possible to switch the set. In FIG. 59, when the input video is not screen content, the BO method or EO method is used, and when the input video is screen content, the BO method or histogram correction processing is used. By doing so, it is possible to reduce the number of bits of the index necessary for expressing the offset application method. Note that the SC flag does not necessarily have to be placed in the sequence parameter set. The SC flag is placed in a header (picture parameter set) given for each picture in order to switch to a picture unit, or a header (slice given to each slice in order to switch in a slice unit). It may be placed in the parameter set. Furthermore, switching to a block unit is naturally conceivable. Of course, the SC flag can be generalized to a flag that can be changed under an arbitrary condition.

FIG. 41 is a flowchart showing a procedure for selecting an offset application method to be used from among a plurality of offset application methods in FIG.
First, the loop filter unit 13 in FIG. 38 obtains an encoding cost C _off when encoding is performed without applying the pixel adaptive offset processing (step ST300 in FIG. 41).
There are various methods for calculating the encoding cost, but it is generally expressed in the form of d + λr, where d is the error from the original image and r is the amount of code required to send the header information. Is. Here, λ is a coefficient determined by the image coding apparatus according to the bit rate or the like.

Next, the loop filter unit 13 of FIG. 38 classifies each pixel in the block for each of the BO method and the EO method shown in FIG. 40, and calculates an offset value for compensating the coding distortion for each class. Since the class classification method and the offset calculation method are known techniques, description thereof is omitted here.
And the loop filter part 13 of FIG. 38 calculates | requires the encoding cost at the time of adding these offset values with respect to the pixel value of a local decoded image (step ST301-ST305 of FIG. 41).

In the second embodiment, in addition to the above, the loop filter unit 13 of FIG. 38 calculates a correction width w ⁱ for each peak pixel value of the histogram for performing histogram correction. It is conceivable that the correction width w ⁱ is calculated in the same manner as in the first embodiment.
The loop filter unit 13 in FIG. 38 obtains the coding cost when the histogram correction of the locally decoded image is performed using the correction width w ⁱ for each peak pixel value (step ST306 in FIG. 41).
The loop filter unit 13 in FIG. 38 selects a technique indicating the lowest coding cost from the plurality of offset application techniques in FIG. 40 (step ST307 in FIG. 41).

As a result of the above comparison, if no improvement in coding efficiency is observed even when pixel adaptive offset processing is applied (when C _off is minimized), index 0 is output to the variable length coding unit 15 as header information. (Steps ST308 to ST309 in FIG. 41). This means that pixel adaptive offset processing is not applied.
On the other hand, when the BO method or the EO method is selected as a method for minimizing the coding cost, an index representing the selected method is output to the variable length coding unit 15 as header information according to FIG. The offset for the class is output as header information to the variable length coding unit 15 and the process ends (steps ST310 to ST311 in FIG. 41).
Further, when histogram correction is selected as a method for minimizing the coding cost, an index representing the selected method is output to the variable length coding unit 15 as header information according to FIG. 40, and each peak of the histogram is further displayed. Is output to the variable length encoding unit 15 as header information, and the process ends (step ST312 in FIG. 41).

As is clear from FIG. 41, in the second embodiment, only one of the offset for applying the BO method or the EO method or the correction width w ⁱ for applying the histogram correction is encoded. Is done. Therefore, it is possible to reduce the code amount as compared with the first embodiment in which both need to be encoded.
However, in contrast to the first embodiment in which both the BO method or the EO method and the histogram correction can be applied, in the second embodiment, only either the BO method or the EO method or the histogram correction is performed. Since it cannot be applied, the image quality improvement effect may be inferior to that of the first embodiment.

If the number of classes classified by the BO method or EO method is the same as the number of histogram peaks to which the histogram correction processing is applied (maximum peak number P _max in the first embodiment), the offset or the correction width is encoded. A common syntax can be simplified. Of course, the number of classes and the number of peaks in the histogram may be different from each other.
Therefore, the pixel adaptive offset processing outputs block division information, an index indicating an offset application technique in units of blocks, offset information in units of blocks, or correction width w ⁱ for histogram correction to the variable length encoding unit 15 as header information. .

  In the pixel adaptive offset processing, for example, it may be always divided into fixed-size block units such as a maximum coding block, and a class classification method may be selected for each block to perform adaptive offset processing for each class. In this case, the block division information becomes unnecessary, the code amount is reduced by the amount of code required for the block division information, and the coding efficiency can be improved.

  FIG. 44 shows an example of a case where division is always performed in units of Coding Tree Unit (CTU) which is the maximum coding block in the pixel adaptive offset process, and the position where the parameter in the pixel adaptive offset process is encoded and inserted into the bitstream is shown. It is illustrated. Note that FIG. 44 is merely an example, and it is not always necessary to perform such encoding. In FIG. 44, an index (FIG. 40) indicating an offset application method is first encoded and inserted at the head of each CTU data following the slice header, and then, when using the EO method or the BO method, an offset and a histogram for each class. When performing the correction process, the correction width corresponding to each peak of the pixel value histogram is encoded and inserted. When the pixel adaptive offset process is not performed, nothing is encoded after the index.

  FIG. 45 is a flowchart showing how the variable-length encoding unit 15 encodes the pixel adaptive offset processing parameter and adds it to the bitstream. First, the variable length encoding unit 15 encodes an index corresponding to each offset application method shown in FIG. 40 (step ST400 in FIG. 45). Then, when no offset is applied (step ST401 in FIG. 45), the processing ends as it is, and when the EO method or the BO method is used, all the offsets for each class are encoded (steps ST402 and ST403 in FIG. 45). On the other hand, when the histogram correction process is performed, all correction widths corresponding to the respective peaks of the pixel value histogram are encoded (step ST404 in FIG. 45). In FIG. 45, the number of classes in the EO method or the BO method and the number of peaks of the pixel value histogram to be corrected in the histogram correction are both 4.

  Also, in adaptive filter processing, local decoded images are classified by a predetermined method, and a filter that compensates for superimposed distortion is designed for each region (local decoded image) belonging to each class. Then, the local decoded image is filtered. Then, the filter designed for each class is output to the variable length encoding unit 15 as header information.

  Here, as a class classification method, there are a simple method for spatially dividing an image at equal intervals, and a method for classifying an image according to local characteristics (dispersion, etc.) of the image in units of blocks. Further, the number of classes used in the adaptive filter process may be set in advance to a common value in the image encoding device and the image decoding device, or may be one of the parameters to be encoded. Compared to the former, the latter can set the number of classes to be used freely, so the image quality improvement effect will be improved, but on the other hand, the amount of code will be increased to encode the number of classes. To do.

Next, in the image decoding apparatus according to the second embodiment, the operation of the loop filter unit 39 having a configuration different from that of the image decoding apparatus according to the first embodiment will be described. Other operations are the same as those in the first embodiment.
In the flowchart representing the operation of the image decoding apparatus shown in FIG. 20, the loop filter unit 39 in FIG. 39 adds the processing after completing the processing of steps ST23 to ST30 for all the coding blocks ^Bn (step ST31 in FIG. 20). A predetermined filter process is performed on the decoded image output from the unit 37, and the decoded image after the filter process is stored in the motion compensated prediction frame memory 40 (step ST32 in FIG. 20).

Specifically, filter (deblocking filter) processing that reduces distortion occurring at the boundaries of transform blocks and prediction blocks, processing for adaptively adding an offset (pixel adaptive offset) for each pixel, Wiener filter, etc. Performs adaptive filter processing for adaptively switching linear filters and performing filter processing.
These loop filters are applied for the purpose of improving the image quality of the decoded image, but do not consider the special histogram shape of the screen content as shown in FIG. 42, so that a sufficient image quality improvement effect is obtained. I can't.
Therefore, in the first embodiment, the histogram correction process is performed by adding a new loop filter. In the second embodiment, the histogram correction process is performed as a part of the existing loop filter.

However, the loop filter unit 39 in FIG. 39 refers to each header information variable length decoded by the variable length decoding unit 31 for each of the above deblocking filter processing, pixel adaptive offset processing, and adaptive filter processing, and Specify whether to perform processing in slices. At this time, when two or more filter processes are performed, for example, when the loop filter unit 13 of the image encoding device is configured as shown in FIG. 38, the loop filter unit 39 is configured as shown in FIG. Made.
Here, in the deblocking filter processing, when there is information for referring to the header information that has been variable-length decoded by the variable-length decoding unit 31 and changing various parameters used for selecting the filter strength applied to the block boundary from the initial value. Performs a deblocking filter process based on the change information. When there is no change information, it is performed according to a predetermined method.

In the pixel adaptive offset processing, the decoded image is divided based on the block division information of the pixel adaptive offset processing variable-length decoded by the variable-length decoding unit 31, and the variable-length decoding unit 31 performs variable-length decoding on the block basis. Referring to an index indicating a block classification method, if the index is not an index indicating that “offset processing is not performed” and is not an index indicating that “histogram correction processing is performed”, the index Accordingly, each pixel in the block is classified according to the class classification method indicated by the index.
As the class classification method candidates, the same class classification method candidates for the pixel adaptive offset processing of the loop filter unit 13 of FIG. 38 are prepared in advance. Then, a process of adding the offset to the pixel value of the decoded image is performed with reference to the offset information specifying the offset value of each class in block units.

  On the other hand, the index indicating the block class classification method variable-length decoded by the variable-length decoding unit 31 is not an index indicating that “offset processing is not performed” and is an index indicating that “histogram correction processing is performed”. If there is, the histogram correction processing is executed for the block.

  FIG. 46 is a flowchart showing the decoding procedure of the pixel value adaptive offset processing parameter by the variable length decoding unit 31.

  The processing content of the histogram correction process in the image decoding apparatus according to the second embodiment is the same as the histogram correction process in the image decoding apparatus according to the first embodiment.

That is, the histogram calculation unit 41 divides the decoded image calculated by the addition unit 37 into a plurality of blocks, and calculates a histogram of pixel values for each block of the decoded image (steps ST200 to ST201 in FIG. 37). Also, the pixel value with the maximum histogram value is selected, and the pixel value is set as the peak pixel value p (step ST202 in FIG. 37).
Note that the step width of the histogram does not necessarily have to be 1, but the output image of the loop filter unit 39 needs to match the output image of the loop filter unit 13 in FIG. Use the same step width. When the step width of the histogram is set to 1 or more, it is conceivable that p is set to the median value of the step width or the like, but this also needs to be matched with the loop filter unit 13 for the same reason as described above.

ここで、（ｘ，ｙ）はブロック単位の復号画像における画素の座標を表し、Ｄ（ｘ，ｙ）は置換前の画素値、Ｄ’（ｘ，ｙ）は置換後の画素値を表している。 Pixel value correction unit 42, the pixel values in the correction width w ⁱ with respect to the variable-length decoded i-th peak by variable-length decoder 31 ((peak pixel value p- correction width w ⁱ⁾ ~ (peak pixel value p + correction The pixel value of the width w ⁱ ) is replaced with the peak pixel value p (steps ST203 to ST206 in FIG. 37).

Here, (x, y) represents the pixel coordinates in the decoded image in block units, D (x, y) represents the pixel value before replacement, and D ′ (x, y) represents the pixel value after replacement. Yes.

The above correction is performed on all or some of the pixels in the block. Incidentally, in the above equation, replace it if p-w ⁱ is below the lower limit of the pixel value to the lower limit of the pixel value, p + w ⁱ is replaced this if it exceeds the upper limit of the pixel value to the upper limit of the pixel value Also good.
Here, the correction width w ⁱ is the same on the side larger than the peak pixel value p and the smaller side, but it is not always necessary to match both, and a different correction width w ⁱ may be used for each. However, since it is necessary to match the output image of the loop filter unit 39 with the output image of the loop filter unit 13 of FIG. 38, the same method as the loop filter unit 13 must be used.

Next, the pixel value correcting unit 42 adds 1 to the index i indicating the peak pixel value, and deletes a portion corresponding to a pixel having a pixel value of p from the histogram calculated by the histogram calculating unit 41 in ST202 ( Steps ST207 to ST208 in FIG. 37).
The pixel value correcting unit 42 compares the index i with a preset maximum peak number P _max (step ST2091 in FIG. 37), and if the index i is smaller than the maximum peak number P _max (i <P _max ), the peak pixel. The histogram calculation unit 41 is instructed to reselect the value, and the process returns to step ST202.
On the other hand, if the index i has reached the maximum number of peaks P _max (i ≧ P _max ), the histogram correction process is terminated.

However, in the pixel adaptive offset processing of the loop filter unit 13 in FIG. 38, the block division information is not encoded, and the image is always divided into fixed-size block units (for example, the maximum encoded block unit), When the class classification method is selected and adaptive offset processing is performed for each class, the loop filter unit 39 also performs pixel adaptive offset processing in units of blocks having the same fixed size as the loop filter unit 13. carry out.
In the adaptive filter process, after classifying by the same method as the image encoding apparatus of FIG. 1 using the filter for each class variable-length decoded by the variable-length decoding unit 31, the filter process is performed based on the class classification information. I do.
The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the second embodiment, as in the first embodiment, the loop filter unit 13 of the image encoding device obtains the optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, Since the configuration is such that noise is removed from the locally decoded image by correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  Furthermore, in the second embodiment, the above-described histogram correction is executed as a part of an existing loop filter (for example, pixel adaptive offset), and one method is selected and applied by the image coding apparatus (for example, 40, only one of the methods shown in FIG. 40 is selected and applied), and header information to be encoded with respect to the loop filter can be reduced as compared with the first embodiment. There is an effect that the amount of codes can be reduced.

In the second embodiment, similarly to the first embodiment, the variable length coding unit 15 of the image coding apparatus codes and outputs the correction width w ⁱ of each peak necessary for the histogram correction. Then, the image decoding apparatus is configured to perform variable length decoding on the correction width w ⁱ of each peak and to perform a histogram correction of the decoded image in the loop filter unit 39 to remove noise from the decoded image, as shown in FIG. There is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the second embodiment, the image decoding apparatus and the image decoding method capable of correctly decoding the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above effects can be obtained. Play.

Embodiment 3 FIG.
In Embodiment 1 described above, the image encoding apparatus always calculates and encodes a new histogram correction parameter (such as a correction width for each peak of the pixel value histogram) for each block. In the third embodiment, a case will be described in which the amount of information to be encoded can be reduced by configuring so that parameters of other blocks calculated in the past can be used.

  The image coding apparatus according to Embodiment 3 has basically the same configuration and operation as the image coding apparatus according to Embodiment 1 shown in FIG. However, only the operations related to the histogram correction processing of the loop filter unit 13 and the variable length coding unit 15 shown in FIG. 12 are different. Hereinafter, first, an operation related to the histogram correction processing of the loop filter unit 13 will be described.

47A and 47B are flowcharts showing the operation of the loop filter unit 13 in the third embodiment, and the operation of the loop filter unit 13 will be described based on this. First, the loop filter unit 13 obtains the encoding cost C _OFF when the histogram correction processing is not performed (step ST600 in FIG. 47A), and then the pixel value histogram is obtained by the same procedure as in the first embodiment (FIG. 24). An optimum correction width for each peak is obtained, and an encoding cost C _NEW for encoding this parameter is obtained (step ST601 in FIG. 47A).

  Here, it is assumed that block division as shown in FIG. 48 is performed by the histogram correction processing, and the lower right block is a block during the histogram correction processing. At this time, it is assumed that the histogram correction processing has already been completed for the surrounding blocks 1, 2, and 3. In the third embodiment, a newly calculated parameter may be used in the block whose histogram is being corrected, or any of the parameters of blocks 1, 2, and 3 that have already been processed may be used. . Note that FIG. 48 merely shows examples of blocks that can be referred to, and various other variations such as referring to just above and to the left of the block being processed can be considered.

FIG. 49 shows a parameter diversion method. For example, it is assumed that the histogram correction processing is performed as shown in the upper part of FIG. 49 in the histogram correction processed block 1 shown in FIG. In the upper part of FIG. 49, p ⁰ , p ¹ , p ² , and p ³ represent peak positions that have been subjected to correction processing, and the values of the histogram are p ⁰ > p ¹ > p ² > p ³ . Further, the correction width for each peak is set to w ⁰ , w ¹ , w ² , and w ³ . At this time, in the block during histogram correction, a histogram of pixel values is obtained as shown in the lower part of FIG. 49, and the correction width w ⁰ is determined with reference to the histogram corrected block 1 and the peak p ⁰ ′ at which the histogram value is maximum is obtained. Apply to the above to correct the peak. Subsequently, correction is performed with a correction width of w ¹ for p ¹ ′ having the histogram value next to p ⁰ ′. Thereafter, this is repeated, and correction of p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ is performed using the parameters of the block 1 that has been subjected to the histogram correction processing.

Further, as a method of diverting different parameters, it is conceivable to always perform correction on the same peak position as shown in FIG. In FIG. 49, only the correction widths w ⁰ , w ¹ , w ² , and w ^{3 of} the histogram-corrected blocks are referenced, and the peak positions p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ are being corrected. Was newly calculated. On the other hand, in the diversion method shown in FIG. 50, the corrected block is also referred to for the peak positions p ⁰ , p ¹ , p ² , and p ³ in addition to the correction width, regardless of the histogram shape of the block being corrected, The correction of the widths w ⁰ , w ¹ , w ² , and w ³ is always applied to p ⁰ , p ¹ , p ² , and p ³ .

An advantage of the parameter diversion method shown in FIG. 49 is that the amount of parameters to be held can be minimized. Only the correction widths w ⁰ , w ¹ , w ² , and w ³ in each block need to be retained in order to use the parameters. However, a histogram of pixel values must be calculated in order to obtain the peak position in the block being corrected. On the other hand, in the parameter diversion method shown in FIG. 50, since the peak position is also diverted in addition to the correction width, it is not necessary to calculate a histogram in the block being corrected. However, since it is necessary to hold not only the correction width but also the peak position, the memory size for holding the parameter becomes larger than the method shown in FIG.

  Now, as shown in FIG. 48, if the blocks that can be referred to are upper left, right above, and left, the histogram correction processing parameters in these blocks are diverted, newly calculated parameters are used, or histogram correction processing is not performed. Can be expressed by the index shown in FIG. That is, when the histogram correction process is not performed, index 0 is encoded. When a newly calculated parameter is used, index 1 is encoded and then the calculated histogram correction process parameter is encoded. On the other hand, when the parameters of the surrounding blocks are used, only the index indicating the block position needs to be encoded, and there is no need to separately encode the parameters. Therefore, it is possible to reduce the amount of codes compared to the case of newly encoding parameters. Note that FIG. 51 is merely an example of an index, and an index different from that in FIG. 51 may be used.

When the parameters of the surrounding blocks are diverted in this way, the encoding cost C when the histogram correction processing parameters of the upper left, right above, and left blocks are diverted in consideration that only the index needs to be encoded. _TL, calculating _C T, the _{C L,} respectively (ST 604 from step ST602 of FIG. 47A).

Then, the respective coding costs are compared, and when C _OFF is the smallest, index 0 is output to the variable length coding unit 15 based on FIG. 51 and the process ends (steps ST605 and ST606 in FIG. 47B). On the other hand, when C _NEW is the smallest, index 1 which means that a parameter is newly encoded is output to the variable length encoding unit 15 (steps ST607 and ST608 in FIG. 47B), and subsequently the histogram correction processing parameters are set. It outputs to the variable-length encoding part 15 and complete | finishes (step ST609 of FIG. 47B).

If any one of C _TL , C _T , and C _L is the minimum cost, only the index indicating the position of the corresponding block is output to the variable length coding unit 15 and the process ends (step ST610 in FIG. 47B). ST614).

  FIG. 52 is a flowchart showing the operation of the variable length coding unit 15. The operation of the variable length coding unit 15 in the histogram correction processing according to the third embodiment will be described with reference to FIG. The variable length encoding unit 15 receives an index representing the parameter encoding method shown in FIG. 51 for the histogram correction processing from the loop filter unit 13, and first encodes this (step ST700 in FIG. 52).

  Subsequently, when the parameter is newly encoded (index is 1 in FIG. 51) (step ST701), the parameter for the histogram correction processing (correction width for each peak) received from the loop filter unit 13 after the index is encoded. And finishes (step ST702 in FIG. 52). On the other hand, when a parameter is not newly encoded (in FIG. 51, the index is other than 1), only the index is encoded and the process is terminated.

  The image decoding apparatus according to the third embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, only the operations relating to the histogram correction processing of the loop filter unit 39 and the variable length decoding unit 31 shown in FIG. 18 are different. Hereinafter, first, an operation related to the histogram correction processing of the variable length decoding unit 31 will be described.

  FIG. 53 is a flowchart showing the operation of the variable length decoding unit 31 relating to the histogram correction process. First, the variable length decoding unit 31 decodes an index indicating a parameter encoding method for the histogram correction processing of the block from the bit stream (step ST800 in FIG. 52). When the decoded index is a value meaning that a parameter is newly encoded (index 1 in FIG. 51), the histogram correction processing parameter is further decoded (steps ST801 and ST802 in FIG. 52).

  The processing content of the histogram correction processing in the image decoding device according to the third embodiment is basically the same as the histogram correction processing in the image decoding device according to the first embodiment, but the operation when the parameters of other blocks are diverted. Is different.

  That is, when the index of FIG. 51 obtained by referring to the variable length decoding unit 31 is 0 or 1, the operation is the same as the histogram correction processing in the image decoding apparatus according to the first embodiment. On the other hand, when the index is any of 2, 3, and 4, the block corresponding to the index among the blocks that have been subjected to the histogram correction processing shown in FIG. 48 is referred to, and parameters relating to the histogram correction processing are used.

FIG. 49 shows a parameter diversion method. For example, it is assumed that the histogram correction processing is performed as shown in the upper part of FIG. 49 in the histogram correction processed block 1 shown in FIG. In the upper part of FIG. 49, p ⁰ , p ¹ , p ² , and p ³ represent peak positions that have been subjected to correction processing, and the values of the histogram are p ⁰ > p ¹ > p ² > p ³ . Further, the correction width for each peak is set to w ⁰ , w ¹ , w ² , and w ³ . At this time, in the block during histogram correction, a histogram of pixel values is obtained as shown in the lower part of FIG. 49, and the correction width w ⁰ is determined with reference to the histogram corrected block 1 and the peak p ⁰ ′ at which the histogram value is maximum is obtained. Apply to the above to correct the peak. Subsequently, correction is performed with a correction width of w ¹ for p ¹ ′ having the histogram value next to p ⁰ ′. Thereafter, this is repeated, and correction of p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ is performed using the parameters of the block 1 that has been subjected to the histogram correction processing.

Further, as a method of diverting different parameters, it is conceivable to always perform correction on the same peak position as shown in FIG. In FIG. 49, only the correction widths w ⁰ , w ¹ , w ² , and w ^{3 of} the histogram-corrected blocks are referenced, and the peak positions p ⁰ ′, p ¹ ′, p ² ′, and p ³ ′ are being corrected. Was newly calculated. On the other hand, in the diversion method shown in FIG. 50, the corrected block is also referred to for the peak positions p ⁰ , p ¹ , p ² , and p ³ in addition to the correction width, regardless of the histogram shape of the block being corrected, The correction of the widths w ⁰ , w ¹ , w ² , and w ³ is always applied to p ⁰ , p ¹ , p ² , and p ³ .

An advantage of the parameter diversion method shown in FIG. 49 is that the amount of parameters to be held can be minimized. Only the correction widths w ⁰ , w ¹ , w ² , and w ³ in each block need to be retained in order to use the parameters. However, a histogram of pixel values must be calculated in order to obtain the peak position in the block being corrected. On the other hand, in the parameter diversion method shown in FIG. 50, since the peak position is also diverted in addition to the correction width, it is not necessary to calculate a histogram in the block being corrected. However, since it is necessary to hold not only the correction width but also the peak position, the memory size for holding the parameter becomes larger than the method shown in FIG.

  Since the output image of the loop filter unit 39 must match the output image of the loop filter unit 13 in FIG. 38, the parameter diversion method must be the same as that of the loop filter unit 13.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the third embodiment, as in the first embodiment, the loop filter unit 13 of the image encoding device obtains the optimum correction width w ⁱ for each peak pixel value of the histogram for each block of the locally decoded image, Since the configuration is such that noise is removed from the locally decoded image by correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved for screen contents in which each peak of the histogram is sharp and sparse. There is an effect that can be realized. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  Furthermore, in the third embodiment, since the parameters relating to the above histogram correction can be referred to from the blocks that have already been subjected to the histogram correction processing, the header information that is always encoded as compared with the case where the above parameters are encoded. Can be reduced as compared with the first embodiment, and the amount of code can be reduced.

In the third embodiment, similarly to the first embodiment, the variable length coding unit 15 of the image coding apparatus performs the correction width w ^{i of} each peak necessary for the histogram correction or the block to be referred to. The image decoding apparatus encodes and outputs the index, and the image decoding apparatus performs variable length decoding on the correction width w ^{i of} each peak or the index of the block to be referred to, and performs a histogram correction of the decoded image in the loop filter unit 39 to generate noise from the decoded image Since it is configured to be removed, as shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the third embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

  Further, the third embodiment can be easily combined with the second embodiment. For example, the index table shown in FIG. 40 can be replaced with one using two types of indexes as shown in FIG.

  FIG. 55 illustrates an operation related to the offset processing of the variable length coding unit 15 when the third embodiment is combined with the second embodiment. First, the variable length encoding unit 15 encodes index 1 shown in FIG. 54 (step ST900 in FIG. 55). That is, 0 is performed when no offset processing (including histogram correction processing) is performed in the block, 1 is performed when offset processing is performed using a newly encoded parameter, and offset processing is performed using the parameters of the processed block. When 2 is implemented, 2 is encoded.

  Subsequently, when the offset process is not performed in the block, the process ends (step ST901 in FIG. 55), and when it is performed, index 2 is encoded (step ST902 in FIG. 55). That is, when a parameter is newly encoded (index 1 is 1), a value indicating an offset application method is encoded, and when a parameter of a processed block is used, a value indicating a block position is encoded.

  Subsequently, when the parameters of the processed block are diverted, the processing ends. When the parameters are newly encoded, the parameters are encoded (steps ST903 and ST904 in FIG. 55).

  FIG. 56 illustrates an operation related to the offset process of the variable length decoding unit 31 when the third embodiment is combined with the second embodiment. First, the variable length decoding unit 31 decodes index 1 shown in FIG. 54 (step ST1000 in FIG. 56).

  Subsequently, the decoded index 1 is referred to, and if the offset process is not performed in the block, the process ends (step ST1001 in FIG. 56), and if performed, the index 2 is decoded (step ST1002 in FIG. 56).

  Subsequently, the decoded index 2 is referred to, and when the parameter of the processed block is diverted, the process ends. When the parameter is newly encoded, the parameter is decoded (steps ST1003 and ST1004 in FIG. 56).

Embodiment 4 FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable-length encoded and added to the bitstream. In Embodiment 4, however, A case will be described in which the amount of information to be encoded can be reduced by encoding the number of peaks for which noise removal is performed instead of the correction width.

  The image coding apparatus according to the fourth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter necessary for the histogram correction processing, not the correction width of each peak of the histogram but the number of peaks for which noise removal is performed is encoded.

Loop filter unit 13 in FIG. 1, and all w correction width for each peak pixel value histogram as shown in FIG. 57, determines the number N _P of the peak of implementing the correction. FIG. 57 shows an example of N _P = 4. p ⁰ , p ¹ , p ² , p ³ , p ⁴ , and p ⁵ represent the height of each peak, and p ⁰ > p ¹ > p ² > p ³ > p ⁴ > p ⁵ . That is, the width w is corrected in order from the peak having the largest value.

In this case, the loop filter unit 13, in the same manner as that for determining the optimum value of the correction width for each peak in the first embodiment, so that the image quality improving effect and the coding efficiency is maximized while changing the N _P seek such _{N P.} For the correction width w, a constant such as 15 may be set in advance, or an optimal value may be obtained for each block on which the histogram correction processing is performed.

The encoding outputs an optimum value N _P of the number of peaks found in the variable length coding unit 15. When the correction width w is changed for each block, the correction width w is also encoded.

The image decoding apparatus according to the fourth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes the number N _P of the peak of implementing noise reduction as a parameter necessary for histogram correction process, the loop filter unit 39 with reference to this implementing histogram correction process. Further, when the correction width w is encoded, this is also decoded by the variable length decoding unit 31 and is referenced by the loop filter unit 39 to perform the histogram correction processing.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the fourth embodiment, the loop filter unit 13 of the image encoding device sets the optimum number N _P (or correction width w) of the histogram peak for performing noise removal for each block of the locally decoded image. Since the configuration is such that noise is removed from the locally decoded image by obtaining and correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved with respect to screen content in which each peak of the histogram is sharp and sparse. The effect which can implement | achieve improvement is produced. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

In the fourth embodiment, to encode one type of number N _P of the peak, or two of the number N _P and the width w only as a parameter necessary for histogram correction process, as compared to the first embodiment codes The number of parameters to be reduced is reduced, header information to be encoded can be reduced, and the amount of code can be reduced.

In the fourth embodiment, the variable length coding unit 15 of the image coding apparatus codes and outputs the number of peaks N _P (or correction width w) in addition to the above-described histogram correction. The image decoding apparatus is configured to perform variable-length decoding on the number of peaks N _P (or in addition to the correction width w) and to perform noise correction on the decoded image in the loop filter unit 39 to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the fourth embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

In addition, the fourth embodiment can be easily combined with other embodiments. For example, in addition to obtaining and encoding the optimum correction width for each peak of the histogram as in the first to third embodiments, the number of peaks to be corrected is also encoded (N _P and w ⁱ (Encoding all of i = 0 to _{N P} −1) can further enhance the image quality improvement effect. However, it should be noted that the amount of code increases as the header information to be encoded increases.

  Note that the combination with the other embodiments described above may be adaptively changed in units of video, or in units of pictures, slices, or blocks. By doing so, the image quality improvement effect and the encoding efficiency are further improved. At this time, a flag indicating a combination may be added to the header (sequence parameter set, picture parameter set, slice parameter set, etc.) according to the change unit, and may be encoded by the image encoding device.

Embodiment 5. FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable-length encoded and added to the bitstream. In Embodiment 5, however, A case will be described in which the calculation amount in the decoding process can be reduced by encoding the peak position where noise removal is performed instead of the correction width.

  The image coding apparatus according to the fifth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter necessary for the histogram correction processing, the peak position where noise removal is performed is used instead of the correction width of each peak of the histogram.

As shown in FIG. 58, the loop filter unit 13 in FIG. 1 sets w as the correction width for each peak of the pixel value histogram, and determines peak positions v ⁰ , v ¹ , v ² , and v ³ to be corrected. . In FIG. 57, the number of peaks to be corrected is four.

At this time, the loop filter unit 13 performs the correction value v ⁱ (i = 0, 1, 2, in the example of FIG. 57) in the same manner as in the case of obtaining the optimum value of the correction width for each peak in the first embodiment. While changing 3), v ⁱ that maximizes the image quality improvement effect and coding efficiency is obtained. For the correction width w, a constant such as 15 may be set in advance, or an optimal value may be obtained for each block on which the histogram correction processing is performed.

Then, the obtained optimum value v ⁱ of the peak position is output to the variable length encoding unit 15 and encoded. When the correction width w is changed for each block, the correction width w is also encoded.

The image decoding apparatus according to the fifth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes the position v ⁱ of the peak of implementing noise reduction as a parameter necessary for histogram correction process, the loop filter unit 39 with reference to this implementing histogram correction process. Further, when the correction width w is encoded, this is also decoded by the variable length decoding unit 31 and is referenced by the loop filter unit 39 to perform the histogram correction processing.

Since the peak position v ^{i to} be corrected is known by the loop filter unit 39, it is not actually necessary to obtain a histogram by the loop filter unit 39. That is, in the fifth embodiment, it is possible to reduce the calculation amount in the loop filter unit 39 as compared with the first embodiment.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

In the fifth embodiment, the loop filter unit 13 of the image encoding device sets the optimum histogram peak position v ⁱ (or correction width w) for performing noise removal for each block of the locally decoded image. Since the configuration is such that noise is removed from the locally decoded image by obtaining and correcting the histogram, as shown in FIG. 42, the image quality of the locally decoded image is greatly improved with respect to screen content in which each peak of the histogram is sharp and sparse. The effect which can implement | achieve improvement is produced. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

In the fifth embodiment, since the peak position v ⁱ (or the correction width w) is encoded as a parameter necessary for the histogram correction process, it is necessary to perform histogram calculation by the loop filter unit 39 at the time of decoding. As a result, the amount of calculation for the decoding process can be reduced as compared with the first embodiment.

Further, in the fifth embodiment, the variable length coding unit 15 of the image encoding apparatus, the position of the peak required in the above histogram correction v ^{i (or} in addition to the correction width w) and outputs the encoded The image decoding apparatus is configured to perform variable-length decoding on the peak position v ⁱ (or correction width w in addition to this) and to perform histogram correction of the decoded image in the loop filter unit 39 to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the fifth embodiment, the image decoding apparatus and the image decoding method that can correctly decode the encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

In addition, the fifth embodiment can be easily combined with other embodiments. For example, in addition to obtaining and encoding the optimum correction width for each peak of the histogram as in the first to third embodiments, the number of peaks to be corrected as in the fourth embodiment is also encoded. (Encoding all of N _P , v ⁱ , and w ⁱ (i = 0 to _{N P} −1)) can further enhance the image quality improvement effect. However, it should be noted that the amount of code increases as the header information to be encoded increases.

  Note that the combination with the other embodiments described above may be adaptively changed in units of video, or in units of pictures, slices, or blocks. By doing so, the image quality improvement effect and the encoding efficiency are further improved. At this time, a flag indicating a combination may be added to the header (sequence parameter set, picture parameter set, slice parameter set, etc.) according to the change unit, and may be encoded by the image encoding device.

Embodiment 6 FIG.
In Embodiment 1 described above, an optimum correction width is obtained for each peak of the pixel value histogram in the image encoding device, and these are variable length encoded and added to the bitstream. In Embodiment 6, however, A case will be described in which the code amount related to the encoding of the parameters necessary for the histogram correction process can be reduced by encoding only the flag indicating whether or not to perform the histogram correction for each block.

  The image coding apparatus according to the sixth embodiment has basically the same configuration and operation as the image coding apparatus according to the first embodiment shown in FIG. However, as a parameter for histogram correction processing, only a flag indicating whether or not to perform histogram correction on the block is encoded.

  The loop filter unit 13 in FIG. 1 corrects a predetermined number of peaks with a predetermined correction width, and obtains an encoding cost at that time. Note that the number of peaks may always be a fixed value, or may be changed in an arbitrary unit such as a slice or a block. As for the correction width, the same fixed value may always be used for all peaks, or a different value may be used for each peak.

  If the obtained encoding cost is smaller than the encoding cost when the histogram correction processing is not performed, for example, 1 is substituted into a flag indicating whether or not the histogram correction processing is performed in the block. The data is output to the long encoding unit 15. In addition, when the obtained coding cost is larger than the coding cost when the histogram correction processing is not performed, it is variable by substituting, for example, 0 for a flag indicating whether or not the histogram correction processing is performed in the block. The data is output to the long encoding unit 15.

  The image decoding apparatus according to the sixth embodiment has basically the same configuration and operation as the image decoding apparatus according to the first embodiment shown in FIG. However, the variable length decoding unit 31 decodes a flag indicating whether or not the histogram correction processing is performed in the block as a parameter necessary for the histogram correction processing, and the loop filter unit 39 refers to this to perform the histogram correction processing. To implement.

  That is, the loop filter unit 39 refers to the flag decoded by the variable length decoding unit 31, and when this is 1, the loop filter unit 39 corrects the predetermined number of peaks with a predetermined correction width. The number of peaks may always be a fixed value, or may be changed in arbitrary units such as slices and blocks, but the output image of the loop filter unit 39 is used as the loop filter unit 13 in the image encoding device of FIG. Therefore, it is necessary to make the same as the loop filter unit 13. As for the correction width, the same fixed value may be always used for all the peaks, or a different value may be used for each peak, which is also the same as the loop filter unit 13. There is a need.

  When the flag decoded by the variable length decoding unit 31 is 0, the loop filter unit 39 does not perform the histogram correction process on the block.

  The decoded image after the filter processing by the loop filter unit 39 becomes a reference image for motion compensation prediction and also becomes a reproduced image.

  In the sixth embodiment, the loop filter unit 13 of the image encoding device removes noise from the local decoded image by correcting the histogram with reference to a predetermined number of peaks and correction width for each block of the local decoded image. Since it is configured, as shown in FIG. 42, there is an effect that a significant improvement in the image quality of the locally decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In the sixth embodiment, since only the flag indicating whether or not the histogram correction processing is to be performed is encoded in the block, it is necessary for encoding parameters related to the histogram correction processing as compared with the first embodiment. There is an effect that the amount of codes can be reduced.

  In the sixth embodiment, the variable length encoding unit 15 of the image encoding device encodes and outputs a flag indicating whether or not the above-described histogram correction is performed in the block, and the image decoding device Since the above-mentioned flag is variable-length decoded and the flag is a value indicating that the histogram correction processing is performed, the loop filter unit 39 performs histogram correction of the decoded image to remove noise from the decoded image. As shown in FIG. 42, there is an effect that a significant improvement in the image quality of the decoded image can be realized for screen content in which each peak of the histogram is sharp and sparse. However, this effect is not limited to the screen content, and naturally the same effect can be obtained as long as the video signal has the same histogram feature.

  In addition, according to the sixth embodiment, an effect of obtaining an image decoding apparatus and an image decoding method capable of correctly decoding an encoded bitstream generated by the image encoding apparatus and the image encoding method having the above-described effects can be obtained. Play.

Embodiment 7 FIG.
In the above embodiment, the encoding apparatus and decoding apparatus provided with the loop filter that reduces the encoding noise have been described.
In the seventh embodiment, an example using the encoding device and the decoding device will be described.

  FIG. 60 is an explanatory diagram illustrating an example of a system configuration according to the seventh embodiment. In FIG. 60, control devices (101A to 101C) correspond to, for example, a sequencer in the FA system or a control device in the plant system. The terminal devices (102A to 102C) correspond to, for example, GOT (Graphic Operation Terminal) in the FA system and OPS (Operator Station) in the plant system. The maintenance staff terminal 103 is, for example, a terminal such as a smartphone or a tablet used by a maintenance staff in the FA system or the plant system to access from the site.

  In the system as shown in FIG. 60, for example, there is a request to record and leave information on a verification screen in which the control device (101A to 101C) has been verified from the terminal device (102A to 102C). Alternatively, there is a request that a maintenance staff at the maintenance site wants to perform work based on information from the terminal devices (102A to 102C) using the maintenance staff terminal 103. Furthermore, there is a request that it is desired to multiplex graphic text such as a comment on a moving image photographed by the maintenance staff terminal 103 and transmit it to the terminal devices (102A to 102C).

  If the data amount is not so large, such as still image data or text data, VNC (Virtual Network Computing) is a protocol for remotely operating a remote computer on the network as a conventional technology. It was possible to send and receive data using a communication protocol. However, in the case where moving image data is included in the information on the verification screen as described above, information from the terminal devices (102A to 102C), or when processing data such as a moving image in which graphic text is multiplexed, There was a problem that a communication protocol such as VNC could not cope.

  For this reason, when processing data including moving image information as described above, an encoding / decoding technique with higher compression efficiency is required. Therefore, in the seventh embodiment, a case will be described in which data processing including moving image information as described above is performed using the encoding / decoding technique described in the above embodiment.

  For example, when recording large-capacity information on a verification screen in which the control device (101A to 101C) has been verified from the terminal device (102A to 102C), the terminal device (102A to 102C) is described in the above embodiment. Using the encoding method, the verification screen information is encoded and stored in a storage device (not shown).

  Further, when the maintenance staff at the maintenance site uses the maintenance staff terminal 103 to perform work based on a large amount of information including moving images from the terminal devices (102A to 102C), the terminal devices (102A to 102C). Uses the encoding method described in the above embodiment, encodes information to be displayed on the maintenance staff terminal 103, and transmits the encoded information to the maintenance staff terminal 103 via the Internet. The maintenance staff terminal 103 that has received the encoded information from the terminal devices (102A to 102C) decodes and displays it using the decoding method described in the above embodiment.

  In addition, when a graphic text such as a comment is multiplexed on a moving image captured by the maintenance staff terminal 103 and transmitted to the terminal devices (102A to 102C), the maintenance staff terminal 103 uses the encoding method described in the above embodiment. The information obtained by multiplexing the graphic text such as the comment on the captured moving image is encoded and transmitted to the terminal devices (102A to 102C) via the Internet. The terminal devices (102A to 102C) that have received the encoded information from the maintenance staff terminal 103 decode and display them using the decoding method described in the above embodiment. Here, since it is a large amount of data including a moving image, it is effective to use the encoding / decoding method described in the above embodiment. Therefore, the H.265 technique described in Non-Patent Document 1, which is the base of the encoding / decoding technique described in the seventh embodiment, is effective.

Next, the configuration and operation of each device will be described.
FIG. 61 is an example showing the internal configuration of the terminal apparatus 102A.
The terminal device 102A includes a receiving unit 201, an arithmetic processing unit 202, a display processing unit 203, an encoding unit 204, a data storage unit 205, a recording unit 206, a request processing unit 207, a distribution unit 208, and a transmission. Unit 209, browsing processing unit 210, and decryption unit 211.
The receiving unit 201 receives data output from the control device via the control network.
The receiving unit 201 analyzes the received data (its header part).

  The terminal device 102A according to the seventh embodiment receives three types of data, that is, data output from the control devices (101A to 101C), data output from the maintenance worker terminal 103, and request instruction output from the maintenance worker terminal 103. However, other data may be received and the corresponding processing may be performed.

  As a result of analyzing the received data, the receiving unit 201 (1) passes the received data to the arithmetic processing unit 202 when the data is uncoded data output from the control devices (101A to 101C). 2) When the encoded data is output from the maintenance staff terminal 103, the received data is transferred to the recording unit 206. (3) When the request instruction is output from the maintenance staff terminal 103, the data is received by the request processing unit 207. Pass the data.

The arithmetic processing unit 202 receives data from the receiving unit 201 and performs the original operation of the terminal device 102A. For example, the program may be a program that displays data output from the control device 101A on a graph and issues an alarm when a preset value is exceeded.
In addition, the arithmetic processing unit 202 displays a diagram simulating the control devices (101A to 101C) to be monitored on the screen, and a functional system in which correlations of processing functions of the control devices (101A to 101C) are arranged. It may be a program for displaying a figure.

  Further, when the data received from the control devices (101A to 101C) is a moving image, an image in which a character string, a graph, or a symbol is superimposed on the moving image may be created.

  The arithmetic processing unit 202 passes the result of performing arithmetic processing on the data received from the receiving unit 201 to the display processing unit 203.

The recording unit 206 receives data from the receiving unit 201 when the data received by the receiving unit 201 is encoded data output from the maintenance staff terminal 103.
The recording unit 206 adds unique information of the data to the data received from the receiving unit 201 and passes the data to the data storage unit 205. As the data unique information, for example, information indicating from which maintenance personnel terminal the data that has been sent and information on the date and time that the data has been sent may be used. In addition, information indicating from which maintenance staff terminal the data received from the receiving unit 201 is sent, information indicating the date and time when the data is sent, and the like may be confirmed by analyzing the header portion of the data. .

The request processing unit 207 receives data from the receiving unit 201 when the data received by the receiving unit 201 is a request from the maintenance staff terminal 103.
The data received from the receiving unit 201 is, for example, unique information that can uniquely identify which maintenance personnel terminal the request data is from. For example, it may be specified by the IP (Internet Protocol) address of the maintenance staff terminal or control device from which the information is output, and the time at which the information is output. Also good.
Further, it may be real-time information currently output to an output device (not shown, for example, a display) of the terminal apparatus 102A. The real-time information output to the output device of the terminal device 102A may be the entire screen or a part of the screen. In the case of a part of the screen, the area may be designated or automatically determined.

  The distribution unit 208 receives a condition for specifying data from the request processing unit 207, acquires data that matches the condition from the data storage unit 205, and passes the acquired data to the transmission unit 209.

The transmission unit 209 outputs the data passed from the distribution unit 208 to the control network.
Data that the transmission unit 209 outputs to the control network may be received by anyone by broadcast transmission. Further, it may be transmitted only to the maintenance staff terminal that has issued the request process.

The browsing processing unit 210 is used when the data stored in the data storage unit 205 is to be output to the display device of the terminal apparatus 102A.
The browsing processing unit 210 selects data stored in the data storage unit 205 using, for example, a user interface (not shown, for example, a keyboard or a touch panel) for operating the terminal device 102A.
The browsing processing unit 210 reads the selected data from the data storage unit 205 and outputs it to the decryption unit 211.

The decoding unit 211 has a function equivalent to that of the decoding device described in the above embodiment.
The browsing processing unit 210 receives the data decoded by the decoding unit 211 and passes it to the display processing unit 203.

The display processing unit 203 displays the data received from the browsing processing unit 210 on a display device (not shown, for example, a display).
The display processing unit 203 displays the data received from the arithmetic processing unit 202 on the display device.
For example, a liquid crystal screen or a monitor may be used as the display device.

  The display processing unit 203 encodes data used for display on the display device by the encoding unit 204.

  When the display processing unit 203 receives the data from the arithmetic processing unit 202 and displays the data on the display device, the display processing unit 203 outputs the data received from the arithmetic processing unit 202 to the encoding unit 204 and encodes the data.

The encoding unit 204 has a function equivalent to that of the encoding device described in the above embodiment.
The display processing unit 203 stores the data encoded by the encoding unit 204 in the data storage unit 205.

  In the seventh embodiment, all data stored in the data storage unit 205 is encoded data. However, by adding information indicating whether or not the data is encoded, Data that has been encoded and data that has not been encoded may be mixed.

  In addition, when mixing the encoded data and the non-encoded data in the data storage unit 205, when transmitting the non-encoded data, the data is encoded from the distribution unit 208 or the transmission unit 209. The encoding unit 204 is called so that the encoded data can be transmitted.

A plurality of terminal devices may be provided. Further, the encoding unit 204 and the decoding unit 211 operate via the display processing unit 203 and the browsing processing unit 210, but may operate after the display processing unit 203 or after the browsing processing unit 210. Good.
FIG. 62 is an example showing the internal configuration of the maintenance staff terminal 103.

  The maintenance engineer terminal 103 in FIG. 62 will be described by taking an example in which the maintenance staff terminal 103 communicates with the terminal device 102A.

  The maintenance staff terminal 103 includes a reception unit 301, a decoding unit 211, a display processing unit 302, a data acquisition unit 303, a transmission unit 304, and an encoding unit 204.

The receiving unit 301 receives data output from the terminal device 102A via the control network.
The receiving unit 301 passes the received data to the decoding unit 211.
In addition, when the receiving unit 301 receives data other than the data output from the terminal devices 102A to 102C, the receiving unit 301 passes the data to another processing unit (not shown) that performs processing according to the data.

  The decoding unit 211 decodes the data received from the receiving unit 301 and outputs the data to the receiving unit 301.

  The display processing unit 302 displays the data output from the receiving unit 301 on a display device (not shown, for example, a display).

The data acquisition unit 303 is a sensor or camera (for still images or moving images).
The data acquisition unit 303 outputs measurement data from the sensor and still images or moving images from the camera to the transmission unit 304.

The transmission unit 304 outputs the data output from the data acquisition 303 to the encoding unit 204.
The encoding unit 204 has a function equivalent to that of the encoding device described in the above embodiment.
The transmission unit 304 outputs the data encoded by the encoding unit 204 to the control network.
The other party to which the transmission unit 304 transmits the encoded data may be set with which terminal device the user interface (not shown) of the maintenance staff terminal 103 is used for, or a destination fixed in advance. Alternatively, it may be set to broadcast without designating the other party.

  A plurality of maintenance staff terminals may be provided. Also, the encoding unit 204 and the decoding unit 211 operate via the transmission unit 304 and the reception unit 301, but may operate before the transmission unit 304 or after the reception unit 301.

FIG. 63 is an explanatory diagram showing another system configuration example according to the seventh embodiment. In FIG. 63, information devices (104A to 104C) are information devices having various information in telematics, for example, and these information devices are mounted on a vehicle. For example, the data management center 105 collects information from a vehicle equipped with an information device in telematics. The information devices (104A to 104C) and the data management center 105 are connected by a network, which may be wired or wireless.
In the system as shown in FIG. 63, for example, information on the vehicle on which the information devices (104A to 104C) are mounted, information on traffic jams and accidents using moving images or still images of cameras mounted on the vehicle, etc. There is a request to consolidate to the management center. Alternatively, there is a request (feedback) that a vehicle equipped with an information device wants to receive from the data management server a traffic jam, a past situation in which an accident has occurred, etc. from the data management center.

  If the data amount is not so large, such as still image data or text data, VNC (Virtual Network Computing) is a protocol for remotely operating a remote computer on the network as a conventional technology. It was possible to send and receive data using a communication protocol. However, when moving image data is included in the information from the information devices (104A to 104C) as described above, or when data processing such as moving images in which graphic text is multiplexed is performed, communication such as VNC is performed. There was a problem that the protocol could not cope.

  For this reason, when processing data including moving image information as described above, an encoding / decoding technique with higher compression efficiency is required. Therefore, in the seventh embodiment, a case will be described in which data processing including moving image information as described above is performed using the encoding / decoding technique described in the above embodiment.

  For example, when graphic texts such as road guidance and traffic information are multiplexed on moving images captured by the information devices (104A to 104C) and collected in the data management center 105, and transmitted to the information devices (104A to 104C), The data management center 105 uses the encoding method described in the above embodiment to encode information obtained by multiplexing graphic text such as road guidance and traffic information on the collected moving images, and then transmits information via the Internet. Transmit to the device (104A-104C).

  The data management center 105 that has received the encoded information from the information devices (104A to 104C) performs processing such as data analysis, classification, and organization by decoding using the decoding method described in the above embodiment. .

  Since the information used here is large-capacity data including moving images, it is effective to use an encoding / decoding method. However, when graphic text is further multiplexed on moving images, the image quality at the boundary portion is reduced. Since the degradation occurs, the H.265 technique described in Non-Patent Document 1, which is the base of the encoding / decoding technique described in the seventh embodiment, is effective.

FIG. 64 is an example showing the internal configuration of the data management center 105.
The data management center 105 includes a reception unit 201, a recording unit 206, a data storage unit 205, a data generation unit 501, a decoding unit 211, a distribution unit 208, a transmission unit 209, and an encoding unit 204.

The receiving part 201 receives the data which the information terminal mounted in a vehicle outputs via a network. The network may be wired or wireless.
The receiving unit 201 adds data unique information to the received data, and outputs the data to the recording unit 206.
The unique information may be a unique ID or an IP address that can uniquely identify the information terminal that has sent the data. Further, as the unique information, the time when the data is sent or the serial number of the data sent from the information terminal may be used.

The recording unit 206 outputs the data output from the receiving unit 301 to the data storage unit 205.
In the seventh embodiment, the data received by the receiving unit 201 has been described using a configuration in which only unique information is added and stored in the data storage unit 205. However, data that is easy to be distributed to an information terminal is stored in advance as data. May be processed or rearranged.

The data generation unit 501 refers to the data stored in the data storage unit 205 and generates data for distribution to an information terminal (mounted on the vehicle).
The data generation unit 501 outputs the data read from the data storage unit 205 to the decryption unit 211.
The decoding unit 211 has a function equivalent to that of the decoding device described in the above embodiment.
The decryption unit 211 decrypts the data received from the data generation unit 501 and outputs the data to the data generation unit 501.
The data generation unit 501 generates distribution data using the decoded data, and outputs the distribution data to the encoding unit 204.

The transmission unit 209 outputs the distribution data passed from the distribution unit 208 to the encoding unit 204.
The encoding unit 204 has a function equivalent to that of the encoding device described in the above embodiment.
The encoding unit 204 encodes the distribution data passed from the transmission unit 209 and outputs the encoded data to the transmission unit 209.
The transmission unit 209 transmits the encoded distribution data. As a transmission method, the other party may be designated, or transmission may be performed by broadcast.

  There may be a plurality of data management centers. Further, the encoding unit 204 operates via the transmission unit 209, but may operate before the transmission unit 209.

FIG. 65 is an example showing the internal configuration of the information device 104A.
The information device 104A includes a reception unit 201, a decoding unit 211, a display processing unit 302, a data acquisition unit 303, a transmission unit 209, and an encoding unit 204.
The receiving unit 201 receives data output from the data management center 105 via the network.

The receiving unit 201 outputs the received data to the decoding unit 211.
When receiving data other than data output from the data management center 105, the receiving unit 201 passes the data to another processing unit (not shown) that performs processing according to the data.

  The decoding unit 211 decodes the data received from the receiving unit 301 and outputs the data to the receiving unit 301.

  The display processing unit 302 displays the data passed from the receiving unit 201 on a display device (not shown, for example, a display).

The data acquisition unit 303 is a sensor or camera (for still images or moving images).
The data acquisition unit 303 outputs measurement data from the sensor and still images or moving images from the camera to the transmission unit 209.

The transmission unit 209 outputs the data output from the data acquisition unit 303 to the encoding unit 204.
The encoding unit 204 has a function equivalent to that of the encoding device described in the above embodiment.
The transmission unit 209 transmits the data encoded by the encoding unit 204 to the data management center 105.
The transmission unit 209 may be configured to set which data management center is a user interface (not shown) of the information apparatus 700 as a partner to which the encoded data is transmitted, or a destination that is fixed in advance. Alternatively, it may be set to broadcast without designating the other party.

There may be a plurality of information devices. Further, the encoding unit 204 and the decoding unit 211 operate via the transmission unit 304, the reception unit 301, and the like, but may operate before the transmission unit 304 or after the reception unit 301.
Further, in the seventh embodiment, the case where the data sent from the control device (101A to 101C) to the terminal device (102A to 102C) is not encoded has been described. However, the control device (101A to 101C) The encoding device described in the embodiment may be provided, the encoded data may be sent, and the terminal devices (102A to 102C) may receive the encoded data.

  As described above, according to the seventh embodiment, by using the encoding unit 204 and the decoding unit 211, encoding noise can be reduced and compression efficiency can be increased (635492 prediction efficiency can be increased). Data) including moving picture information can be processed.

  Also, by using the encoding unit 204 and the decoding unit 211, it is not necessary to use a protocol for remotely operating a remote computer such as a VNC, so that security can be improved.

  In the present invention, within the scope of the invention, any combination of the embodiments, or any modification of any component in each embodiment, or omission of any component in each embodiment is possible. .

  DESCRIPTION OF SYMBOLS 1 Encoding control part, 2 slice division part, 3 block division part, 4 changeover switch, 5 intra prediction part, 6 intra block copy prediction part, 7 motion compensation prediction part, 8 subtraction part, 9 conversion and quantization part, 10 Inverse quantization / inverse transform unit, 11 addition unit, 12 intra memory, 13 loop filter unit, 14 motion compensated prediction frame memory, 15 variable length coding unit (encoding unit), 21 histogram calculation unit (histogram calculation unit) , 22 Pixel value correction unit (pixel value correction means), 23 First error calculation unit, 24 Second error calculation unit, 25 Correction pixel value range storage unit, 26 Repeat processing unit, 27 Pixel value replacement unit, 31 Variable Long decoding unit (decoding means), 32 inverse quantization / inverse conversion unit, 33 changeover switch, 34 intra prediction unit, 35 intra block copy prediction unit, 3 Motion compensation prediction unit, 37 addition unit, 38 intra memory, 39 loop filter unit, 40 motion compensation prediction frame memory, 41 histogram calculation unit (histogram calculation unit), 42 pixel value correction unit (pixel value correction unit), 101A to 101C control device, 102A to 102C terminal device, 103 maintenance staff terminal, 104A to 104C information device, 105 data management center, 201 receiving unit, 202 arithmetic processing unit, 203 display processing unit, 204 encoding unit, 205 data storage unit, 206 recording unit, 207 request processing unit, 208 distribution unit, 209 transmission unit, 210 browsing processing unit, 211 decoding unit, 301 reception unit, 302 display processing unit, 303 data acquisition unit, 304 transmission unit, 501 data generation unit.

Claims (3)

A receiver for receiving data;
An arithmetic processing unit for processing data received by the receiving unit;
A histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image is calculated with respect to the data processed by the arithmetic processing unit, and noise associated with encoding distortion is superimposed from the calculated histogram. An encoding unit that identifies and encodes a pixel value of a pixel on which the noise is superimposed;
A data storage unit for storing the data encoded by the encoding unit;
With
The encoding unit includes:
In the pixel value within the correction pixel value range including the peak pixel value in the histogram,
A first error between the encoding target image and the locally decoded image;
Calculating a second error between the encoding target image and a locally decoded image having a pixel value within the corrected pixel value range as the peak pixel value;
If the second error is smaller than the first error, the terminal device replaces a pixel value within the correction pixel value range with the peak pixel value. A data acquisition unit for acquiring data;
A pixel in which a histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image is calculated with respect to data from a data acquisition unit, and noise associated with encoding distortion is superimposed from the calculated histogram An encoding unit that corrects and encodes a pixel value of a pixel on which the noise is superimposed,
A transmission unit for transmitting encoded data encoded by the encoding unit;
With
The encoding unit includes:
In the pixel value within the correction pixel value range including the peak pixel value in the histogram,
A first error between the encoding target image and the locally decoded image;
Calculating a second error between the encoding target image and a locally decoded image having a pixel value within the corrected pixel value range as the peak pixel value;
If the second error is smaller than the first error, the terminal device replaces a pixel value within the correction pixel value range with the peak pixel value. A data storage unit for storing the encoded data;
A histogram of pixel values in a decoded image obtained from compressed data of an encoding target image is calculated with respect to data stored in the data storage unit, and noise associated with encoding distortion is superimposed from the calculated histogram. A decoding unit that identifies and corrects the pixel value of the pixel on which the noise is superimposed;
A data generation unit that generates transmission data based on the decoded data decoded by the decoding unit;
A histogram of pixel values in a locally decoded image obtained from compressed data of an encoding target image is calculated with respect to transmission data generated by the data generation unit, and noise accompanying encoding distortion is superimposed from the calculated histogram An encoding unit that identifies the pixel that is performing and corrects and encodes the pixel value of the pixel on which the noise is superimposed;
A transmission unit for transmitting encoded data encoded by the encoding unit;
With
The encoding unit includes:
In the pixel value within the correction pixel value range including the peak pixel value in the histogram,
A first error between the encoding target image and the locally decoded image;
Calculating a second error between the encoding target image and a locally decoded image having a pixel value within the corrected pixel value range as the peak pixel value;
If the second error is smaller than the first error, the pixel value in the correction pixel value range is replaced with the peak pixel value.

Images