JP2006032999A - Image decoding device and image decoding method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image decoding device capable of performing filtering adapted to the characteristics of an image, and acquiring a decoded image having properly reduced noises while preventing the blur due to the filter. <P>SOLUTION: A noise reducing filter unit 112 decodes image frames encoded per block to reduce noises generated in the decoded images. A block classifying unit 109 classifies the blocks in accordance with the state of an edge extracted from the decoded image by an edge extracting unit 108. A quantization parameter operating unit 110 divides the image frames into predetermined process units, and executes a predetermined operation for the quantization parameter of encoded data of the blocks in the predetermined process units to calculate the state of the quantization parameter in each process unit. A filter control unit 111 controls filtering by the noise reducing filter unit 112 in conformity with a classification result for each block and the state of the quantization parameter in each predetermined process unit of the image frame to which the block belongs to. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to an image decoding apparatus and an image decoding method, and more specifically, to reduce block distortion and mosquito noise generated in a decoded image when decoding encoded image data used in image communication, image recording, and the like. The present invention relates to an image decoding apparatus and an image decoding method.

  In recent years, with the widespread use of broadband communication means and large-capacity recording media, the use of moving image data has become active. Since the amount of moving image data is enormous, the bandwidth of communication means and the capacity of recording media are increasing, but in order to use moving image data, compression of the amount of data by encoding is necessary. It is indispensable.

  As a moving picture encoding method, international standards such as MPEG-2 and MPEG-4 are known. These use the statistical properties of an image to reduce the amount of information by removing redundancy contained in the image. Specifically, it is a hybrid coding scheme that combines motion compensated prediction in which an image frame is divided into blocks that are basic processing units and coding is performed in units of blocks, and discrete cosine transform (DCT).

  On the other hand, in the decoded image obtained by decoding the encoded image using the encoding method as described above, block distortion and mosquito noise are generated as encoding noise due to compression, thereby degrading the image quality. Therefore, post-filters that reduce these are implemented on the image decoding apparatus side to improve image quality.

  Also, JPEG, which is a still image coding method, performs DCT on a block basis of an image frame in the same manner as the above moving image coding method. Therefore, block distortion and mosquito noise are generated as coding noise. On the other hand, JPEG2000 with improved compression efficiency uses discrete wavelet transform (DWT) instead of DCT, but ringing noise similar to mosquito noise occurs due to aliasing components of quantization noise generated in this transform. To do. Also, in JPEG2000, when tile division is performed and DWT is performed in the same manner as JPEG block division, a boundary between tiles is seen, and image quality degradation similar to block distortion occurs. Therefore, even in the case of a still image, the image decoding apparatus side implements a post filter for reducing these to improve the image quality.

  In such an image decoding apparatus, it is a problem to reduce only block distortion and mosquito noise while preventing deterioration such as image quality blur due to a post filter.

FIG. 12 is a block diagram illustrating a configuration example of an image decoding device according to the related art.
Hereinafter, with reference to FIG. 12, a conventional image decoding apparatus using motion compensation prediction and two-dimensional orthogonal transform will be described. In FIG. 12, 1201 is a variable length decoding unit, 1202 is an inverse quantization unit, 1203 is an inverse orthogonal transform unit, 1204 is an adder, 1205 is a frame memory unit, 1206 is a motion compensation prediction unit, 1207 is a switch, and 1208 is a filter. A control unit 1209 indicates a noise reduction filter unit, and an image decoding apparatus described here is configured by these components.

  An encoded bit stream that is encoded information is input to the variable length decoding unit 1201. The encoded bit stream includes header information such as an orthogonal transform coefficient and picture type (I picture / P picture / B picture) of an image generated by a moving image encoding apparatus, and an encoding mode indicating an encoding state, that is, an encoding Information on the presence / absence of encoding, motion vectors, and intra-frame (intra) encoding / inter-frame prediction (inter) encoding, which are prediction encoding modes, are variable-length encoded and multiplexed.

  The variable length decoding unit 1201 obtains coding mode information such as orthogonal transform coefficients, motion vectors, and intra coding / inter coding obtained by decoding and quantizing the coded bit stream. The inverse quantization unit 1202 performs inverse quantization on the quantized orthogonal transform coefficients to obtain orthogonal transform coefficients, and the inverse orthogonal transform unit 1203 performs two-dimensional inverse orthogonal transform to obtain a difference image.

  When the predictive coding mode 1210 is inter coding, the difference image is stored in the frame memory unit 1205 by the adder 1204 so that the output of the switch 1207 becomes the output of the motion compensation prediction unit 1206. Based on the decoded image, the motion compensated prediction value from the motion compensation prediction unit 1206 is added to accumulate the decoded image of the current frame in the frame memory unit 1205.

  When the block is intra-coded, the switch 1207 is disconnected from the motion compensation prediction unit 1206 and nothing is added to the output from the inverse orthogonal transform unit 1203. As a result, the output of the inverse orthogonal transform unit 1203 Is stored as a decoded image in the frame memory unit 1205 as it is.

  The filter control unit 1208 adaptively controls the filter according to the size of the quantization scale 1211. The noise reduction filter unit 1209 reduces block distortion and mosquito noise by switching the filter coefficient and the number of taps according to the control information 1212 from the filter control unit 1208 for the decoded image of the current frame output from the adder 1204. A post filter is adaptively applied to output a decoded image with reduced noise.

  In addition, it reduces the block distortion that occurs when a large quantization step is used to reduce the amount of code for an image with a large amount of motion, and improves the display image quality by preventing the outline of the image from being blurred. An intended image decoding apparatus has been proposed (see, for example, Patent Document 1).

According to the technique described in Patent Document 1, since the filter control unit 1208 controls the filter based on the quantization parameter, the low-pass filter is not applied to the portion with high quantization accuracy, and the block distortion does not exist. Has no low-pass filter. On the other hand, since block distortion exists in a portion where the quantization accuracy is low, a low-pass filter acts. Also, the occurrence of block distortion and mosquito noise is predicted from the quantization parameter, and an image enhancement is performed by applying a contour enhancement filter to a portion where no noise is generated.
JP 2003-18600 A

  However, in the technique described in Patent Document 1, only the magnitude of the quantization parameter is used for controlling the filter. Therefore, since filtering is performed without discriminating edges, textures, and flat portions having different image characteristics, there is a problem that image blur due to a post filter or noise cannot be reduced appropriately.

  In addition, since various kinds of information such as a prediction coding mode (intra coding and non-intra coding) and a picture type (I, P, B picture) of a block on which a filter is applied are not used, it is fine-tuned according to the characteristics of an image. Cannot perform proper filtering.

  Furthermore, although the image quality varies greatly depending on the ratio of the total code amount of each image frame, the DCT coefficient code amount, the motion vector code amount, and other code amounts, such information is used for filter control. Absent. Therefore, appropriate noise reduction cannot be expected.

  In addition, the technique described in Patent Literature 1 sets a local filter block in an image frame, and weights the number of pixels to the quantization parameter value of each pixel included therein, thereby averaging the filter block average quantization The value is calculated. However, in this method, since the filter block is local, the average quantization value changes in units of filter block, so that the image after filtering becomes uncomfortable. In addition, although the quantization parameter is obtained for each macroblock, the average obtained by weighting the number of pixels is obtained as described above, so the process for calculating the average quantization value, which is a guideline for filter control, is obtained. There is a disadvantage that the amount is large.

  The present invention has been made in view of the above circumstances, and when decoding encoded image data, filtering suitable for the characteristics of the image can be performed, and a decoded image in which block distortion and mosquito noise are appropriately reduced. Is to provide an image decoding apparatus and an image decoding method.

In order to solve the above-described problems, the present invention is constituted by the following technical means.
In the image decoding apparatus, the first technical means decodes an image frame encoded for each block, reduces noise generated in the decoded image, and edge extraction extracts an edge from the decoded image. Means, a block classification means for classifying blocks according to the edge state extracted by the edge extraction means, and an image frame is divided into predetermined processing units, and for the quantization parameter of the encoded data of the block in the processing units Quantization parameter calculation means for calculating a quantization parameter state in each processing unit by performing a predetermined calculation, and a quantization result in the predetermined processing unit of the classification result for each block and the image frame to which the block belongs Filter control means for controlling filtering by the noise reduction filter means according to the state of the parameter It is obtained by further comprising a and.

  According to a second technical means, in the first technical means, the noise reduction filter means includes block distortion reduction means for reducing block distortion of the decoded image, and mosquito noise reduction means for reducing mosquito noise in the decoded image. It is characterized by having.

  According to a third technical means, in the first or second technical means, the edge extracting means determines a differential processing means for differentiating the decoded image, and determines an edge portion of the decoded image based on an output of the differential processing means. And an edge determination means.

  According to a fourth technical means, in the third technical means, the edge determination means compares the output of the differentiation processing means with a predetermined threshold value, and determines an edge and a non-edge for each pixel. It is what.

  According to a fifth technical means, in any one of the first to fourth technical means, the block classification means has a counting means for counting pixels determined to be edges for each block, and the number of edge pixels is a predetermined number. A block is classified as an edge block when it is equal to or greater than a threshold value, and a block is classified as a non-edge block when it is smaller than a predetermined threshold value.

  According to a sixth technical means, in any one of the first to fifth technical means, the block classification unit is configured to perform edge-pixel to non-edge pixel and non-edge pixel to edge pixel for blocks classified as edge blocks. When the number of change points is counted and the number of change points is equal to or greater than a predetermined threshold, the block is classified as a texture block, and when it is smaller than the predetermined threshold, the block is classified as a significant edge block. It is a feature.

  A seventh technical means is any one of the first to sixth technical means, wherein the quantization parameter calculation means uses a unit obtained by dividing an image frame in a band shape composed of a plurality of blocks as a processing unit. The average value of the quantization scale is calculated by adding the quantization scale for each block, and this is used as the state of the quantization parameter in each processing unit.

  According to an eighth technical means, in any one of the first to sixth technical means, the quantization parameter calculating means uses a unit obtained by dividing an image frame in a band shape composed of a plurality of blocks as a processing unit. In each block predictive coding mode, add the quantization scale for each block and calculate the average value of the quantization scale separately for intra-frame coding and non-frame coding. This is characterized in that the quantization parameter states of intra-frame coding and non-frame coding are set.

  According to a ninth technical means, in any one of the first to eighth technical means, the processing unit is an encoding unit composed of two or more macroblocks capable of independently decoding an image in an image frame, Alternatively, it is a video packet or a group of blocks.

  A tenth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and this block is controlled to be filtered with the second filter strength.

  An eleventh technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. When the filter strength is determined and the classification result for each block is a non-edge block, only the block boundary pixels of this block are filtered with the first filter strength, and the block classification result is the edge block. In the case of, the second filter strength is determined such that filtering stronger than the first filter strength is performed, and control is performed so that all pixels in the block are filtered with the second filter strength. It is characterized by that.

  A twelfth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. When the filter strength is determined, and the classification result for each block is a non-edge block, only the block boundary pixels of this block are filtered with the first filter strength, and the block classification result is the texture block. The second filter strength is determined such that filtering stronger than the first filter strength is performed, and all pixels in the block are filtered with the second filter strength, If the block classification result is a significant edge block, filtering that is stronger than the second filter strength is performed. Determining a third filter strength, the block all the pixels in which the filtering by the third filter strength is characterized by controlling so as implemented.

  In a thirteenth technical means according to any one of the first to ninth technical means, the filter control means is configured to determine a first value based on an average value of a quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, the filtering is performed on the block with the second filter strength, and the predictive coding mode of the block is an intraframe code. In the case of encoding, the first filter strength or the second filter strength is set so that the filter strength is stronger than that of non-frame coding. Filtering by changing the strength is obtained by and controls as implemented.

  A fourteenth technical means according to any one of the first to ninth technical means, wherein the filter control means is based on an average value of a quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, If the second filter strength is determined such that filtering stronger than the first filter strength is performed, the block is filtered with the second filter strength, and the picture type is bi-predictive coding Is the first filter strength or the previous filter strength to be stronger than the intraframe coding and interframe forward prediction coding. Filtering by changing the second filter strength is obtained by and controls as implemented.

  A fifteenth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the second filter strength is determined based on the total code amount of each frame. One filter strength or the second filter strength is changed and controlled to be filtered. That.

  A sixteenth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the second filter strength is determined based on the total code amount of each frame. One filter strength and the second filter strength are changed and controlled to be filtered. .

  A seventeenth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the DCT coefficient code amount and motion vector in each frame are determined. Based on the code amount and the ratio of the other code amounts, the first filter strength or the second filter strength is changed to Rutaringu is obtained by and controls as implemented.

  The eighteenth technical means is the first technical means according to any one of the first to ninth technical means, wherein the filter control means is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the DCT coefficient code amount and motion vector in each frame are determined. The first filter strength and the second filter strength are changed on the basis of the code amount and the ratio of the other code amounts to change the filter strength. Taringu is obtained by and controls as implemented.

  According to a nineteenth technical means, in the image decoding method, a noise reduction filter step for decoding an image frame encoded for each block and reducing noise generated in the decoded image, and edge extraction for extracting an edge from the decoded image A block classification step for classifying blocks according to the edge state extracted in the edge extraction step, and dividing the image frame into predetermined processing units, and for the quantization parameter of the encoded data of the block in the processing unit A quantization parameter calculation step for calculating a quantization parameter state in each processing unit by performing a predetermined calculation, and a quantization result in the predetermined processing unit of the classification result for each block and the image frame to which the block belongs Filtering by the noise reduction filter step according to the parameter state Is obtained is characterized by comprising a filter control step of controlling.

  In a twentieth technical means according to the nineteenth technical means, the noise reduction filter step comprises: a block distortion reduction step for reducing block distortion of the decoded image; and a mosquito noise reduction step for reducing mosquito noise in the decoded image. It is characterized by having.

  In a twenty-first technical means, in the nineteenth or twentieth technical means, the edge extracting step determines a differential processing step for performing differential processing on the decoded image, and determines an edge portion of the decoded image based on an output of the differential processing step. And an edge determination step.

  According to a twenty-second technical means, in the twenty-first technical means, the edge determining step compares an output of the differentiation processing step with a predetermined threshold value and determines an edge and a non-edge for each pixel. It is what.

  According to a twenty-third technical means, in any one of the nineteenth to twenty-second technical means, the block classification step includes a counting step for counting pixels determined to be edges for each block, and the number of edge pixels is a predetermined number. A block is classified as an edge block when it is equal to or greater than a threshold value, and a block is classified as a non-edge block when it is smaller than a predetermined threshold value.

  In a twenty-fourth technical means according to any one of the nineteenth to twenty-third technical means, in the block classification step, for the blocks classified as edge blocks, edge pixels to non-edge pixels and non-edge pixels to edge pixels When the number of change points is counted and the number of change points is equal to or greater than a predetermined threshold, the block is classified as a texture block, and when it is smaller than the predetermined threshold, the block is classified as a significant edge block. It is a feature.

  In a twenty-fifth technical means according to any one of the nineteenth to twenty-fourth technical means, the quantization parameter calculation step uses a unit obtained by dividing an image frame into strips composed of a plurality of blocks as a processing unit. The average value of the quantization scale is calculated by adding the quantization scale for each block, and this is used as the state of the quantization parameter in each processing unit.

  According to a twenty-sixth technical means, in the technical means according to any one of the nineteenth to twenty-fourth technical means, the quantization parameter calculating step uses a unit obtained by dividing an image frame into strips composed of a plurality of blocks as a processing unit. In each block predictive coding mode, add the quantization scale for each block and calculate the average value of the quantization scale separately for intra-frame coding and non-frame coding. This is characterized in that the quantization parameter states of intra-frame coding and non-frame coding are set.

  According to a twenty-seventh technical means, in any one of the nineteenth to twenty-sixth technical means, the processing unit is an encoding unit composed of two or more macroblocks capable of independently decoding an image in an image frame, Alternatively, it is a video packet or a group of blocks.

  In a twenty-eighth technical means according to any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and this block is controlled to be filtered with the second filter strength.

  According to a twenty-ninth technical means, in the technical means according to any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. When the filter strength is determined and the classification result for each block is a non-edge block, only the block boundary pixels of this block are filtered with the first filter strength, and the block classification result is the edge block. In the case of, the second filter strength is determined such that filtering stronger than the first filter strength is performed, and control is performed so that all pixels in the block are filtered with the second filter strength. It is characterized by that.

  A thirtieth technical means is the method according to any one of the nineteenth to twenty-seventh technical means, wherein the filter control step is based on a magnitude of an average value of a quantization scale calculated for each of the predetermined processing units. When the filter strength is determined, and the classification result for each block is a non-edge block, only the block boundary pixels of this block are filtered with the first filter strength, and the block classification result is the texture block. The second filter strength is determined such that filtering stronger than the first filter strength is performed, and all pixels in the block are filtered with the second filter strength, If the block classification result is a significant edge block, filtering that is stronger than the second filter strength is performed. Determine a so that a third filter strength, the block all the pixels in which the filtering by the third filter strength is characterized by controlling so as implemented.

  According to a thirty-first technical means, in any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, the filtering is performed on the block with the second filter strength, and the predictive coding mode of the block is an intraframe code. In the case of conversion, the first filter strength or the second filter strength is set so that the filter strength is stronger than that of non-intraframe coding. Filtering by changing the filter strength is obtained by and controls as implemented.

  According to a thirty-second technical means, in any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, If the second filter strength is determined such that filtering stronger than the first filter strength is performed, the block is filtered with the second filter strength, and the picture type is bi-predictive coding The first filter strength to be stronger than intra-frame coding and inter-frame forward prediction coding, There is obtained by and controls as filtered by changing the second filter strength is performed.

  In a thirty-third technical means, in any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the second filter strength is determined based on the total code amount of each frame. One filter strength or the second filter strength is changed to control so that filtering is performed. It is intended.

  In a thirty-fourth technical means according to any one of the nineteenth to twenty-seventh technical means, the filter control step is based on a magnitude of an average value of a quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, A second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the second filter strength is determined based on the total code amount of each frame. The control is performed so that filtering is performed by changing one filter strength and the second filter strength. Than it is.

  In a thirty-fifth technical means according to any one of the nineteenth to twenty-seventh technical means, the filter control step is based on a magnitude of an average value of a quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the DCT coefficient code amount and motion vector in each frame are determined. The first filter strength or the second filter strength is changed based on the code amount and the ratio of the other code amounts. Is obtained by control means controls so filtering is carried out.

  In a thirty-sixth technical means according to any one of the nineteenth to twenty-seventh technical means, the filter control step is based on the average value of the quantization scale calculated for each of the predetermined processing units. Filter strength is determined, and if the classification result for each block is a non-edge block, the block is filtered with a first filter strength. If the classification result of the block is an edge block, The second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength, and the DCT coefficient code amount and motion vector in each frame are determined. The first filter strength and the second filter strength are changed based on the code amount and the ratio of the other code amounts. Is obtained by control means controls so filtering is performed Te.

  According to the present invention, when decoding the encoded image data, based on the classification result by the edge state for each block, the state of the quantization parameter calculated in a predetermined processing unit, and various types of encoding information, Since the filter is adaptively controlled, filtering suitable for the characteristics of the image can be performed, and a decoded image in which block distortion and mosquito noise are appropriately reduced can be obtained while preventing image blur due to the filter.

  FIG. 1 is a block diagram of a configuration example of an image decoding apparatus according to an embodiment of the present invention. In FIG. 1, 101 is a variable length decoding unit, 102 is an inverse quantization unit, 103 is an inverse orthogonal transform unit, 104 is an adder, 105 is a frame memory unit, 106 is a motion compensation prediction unit, 107 is a switch, and 108 is an edge. An extraction unit, 109 is a block classification unit, 110 is a quantization parameter calculation unit, 111 is a filter control unit, and 112 is a noise reduction filter unit. In FIG. 1, reference numerals 113, 114, 115, 116, 117, and 118 denote transmission paths of the predictive coding mode, block classification result, quantization scale, quantization scale average value, picture type, and filter parameter, respectively. However, in the following description, these reference numbers are described as simply transmitted information itself (for example, 113 is a predictive coding mode).

  The image decoding apparatus according to the present invention includes at least noise reduction filter means, edge extraction means, block classification means, quantization parameter calculation means, and filter control means described later. In the image decoding processing according to the present invention, when the noise of the decoded image is reduced, not only the size of the quantization parameter is used for the filter control, but the classification result based on the edge state for each block, in a predetermined processing unit. The filter is adaptively controlled based on the calculated quantization parameter state and various types of encoding information. For example, filtering is performed by discriminating edges, textures, and flat portions having different image characteristics.

  Hereinafter, an image decoding apparatus according to an embodiment of the present invention includes an edge extraction unit 108 configured by an edge extraction unit, a block classification unit 109 configured by a block classification unit, and a quantization configured by a quantization parameter calculation unit. In addition to the parameter calculation unit 110, the filter control unit 111 configured by the filter control unit, and the noise reduction filter unit 112 configured by the noise reduction filter unit, the variable length decoding unit 101, the inverse quantization unit 102, and the inverse orthogonal transform unit Each unit will be described focusing on an example in which 103, an adder 104, a frame memory unit 105, a motion compensation prediction unit 106, and a switch 107 are used as its constituent elements. The description of the image decoding method according to the present invention is substituted by a processing procedure in the image decoding apparatus described below.

<Variable length decoding>
The variable length decoding unit 101 is configured by means for variable length decoding the input encoded bitstream. The variable length decoding unit 101 includes information on quantized DCT coefficients obtained by performing discrete cosine transform (DCT) on an image by an image coding apparatus and quantizing the DCT coefficients, and intra coding / inter coding information. Predictive coding mode, quantization scale, which is a parameter indicating the size of quantization, quantization matrix, motion vector, encoding information, etc. For example, an encoded bit stream according to the MPEG-2 standard (ISO / IEC 13818-2) which is an international standard is input. Then, by performing variable-length decoding here, DCT coefficients quantized for each block, intra coding / inter coding information (predictive coding mode), quantization scale, motion vector, presence / absence of coding execution Encoding information such as the above information is obtained.

<Inverse quantization>
The inverse quantization unit 102 is connected to the variable length decoding unit 101, and includes means for inversely quantizing the quantized orthogonal transform coefficient. In the inverse quantization unit 102, the quantized DCT coefficient obtained by the variable length decoding unit 101 is inversely quantized according to the quantization scale and the quantization matrix to obtain the inversely quantized DCT coefficient.

<Inverse orthogonal transform>
The inverse orthogonal transform unit 103 is connected to the inverse quantization unit 102 and includes means for obtaining a difference image by performing two-dimensional inverse orthogonal transform on the orthogonal transform coefficient. The inverse orthogonal transform unit 103 performs a two-dimensional inverse DCT on the inversely quantized DCT coefficient to obtain a difference image.

<Addition>
The adder 104 is connected to the inverse orthogonal transform unit 103 and the switch 107, and is configured by means for adding the difference image obtained by the inverse orthogonal transform unit 103 and the motion compensated prediction value, and obtains a decoded image by addition. . In the adder 104, when the block is inter-coded, the motion compensated prediction value calculated based on the motion vector of the block from the decoded image of the previous frame stored in the frame memory unit 105, or the block is intra-coded. , The switch 107 switches the output so that the motion compensation prediction value is zero, and the difference image obtained by the inverse orthogonal transform unit 103 is added to obtain a decoded image.

<Decoded image storage>
The frame memory unit 105 is connected to the adder 104 and may be configured by means for accumulating the decoded image obtained by the adder 104.

<Motion compensation prediction>
The motion compensation prediction unit 106 is connected to the variable length decoding unit 101 and the frame memory unit 105, and includes means for outputting a motion compensation prediction value. The motion compensation prediction unit 106 obtains a motion compensation prediction value based on the decoded image of the previous frame in the frame memory unit 105 and the motion vector that is information from the variable length decoding unit 101.

<Switching>
The switch 107 is connected to the variable length decoding unit 101 and the motion compensation prediction unit 106, and is configured by means for switching the output of the motion compensation prediction value. The switch 107 switches the input to the adder 104 according to the information of the predictive coding mode 113 decoded by the variable length decoding unit 101. The output from the adder 104 is output to the frame memory unit 105 and also to the noise reduction filter unit 112.

<Edge extraction>
The edge extraction unit 108 is connected to the frame memory unit 105 and includes edge extraction means for extracting an edge portion of the image from the decoded image. The edge extraction means preferably includes differentiation processing means for differentiating the decoded image and edge determination means for determining the edge portion of the decoded image based on the output of the differentiation processing means. Further, it is preferable that the edge determination means is a means for comparing the output of the differentiation processing means with a predetermined threshold value and determining an edge and a non-edge for each pixel.

  2 is a diagram for explaining a configuration example of the edge extraction unit in the image decoding apparatus of FIG. 1, and FIG. 3 is a diagram for explaining differentiation processing in the differentiation processing unit in the edge extraction unit of FIG. is there. 2 and 3, 201 is a differentiation processing unit composed of differentiation processing means, 202 is an edge determination unit composed of edge determination means, and 301 is a luminance value of a decoded image in a 3 × 3 pixel window. ing.

  The edge extraction unit 108 may be configured by a differential processing unit 201 connected to the frame memory unit 105 and an edge determination unit 202 connected to the differential processing unit 201. The differential processing unit 201 performs differential processing on the decoded image input from the frame memory unit 105. The edge determination unit 202 determines an edge pixel based on the differentiation processing result by the differentiation processing unit 201.

Here, the operation of the edge extraction unit configured as described above will be described.
The differential processing unit 201 performs differential processing on the decoded image stored in the frame memory unit 105 using a Sobel operator to create a differential image. The differential value Gi, j obtained by this operator is expressed by the following expression (1) when the luminance values of the decoded image in the 3 × 3 pixel window in FIG.

Gi, j = | A + 2B + CG-2H-I |
+ | A + 2D + G-C-2F-I | (1)

  The edge determination unit 202 obtains a binary edge image by performing threshold processing with a predetermined threshold THg as shown in the following equation (2) in order to extract an edge from the differential image. In the following equation (2), Gi, j represents a differential image, and Ei, j represents an edge image. A pixel where Ei, j = 1 is defined as an edge pixel.

When Gi, j ≧ THg, Ei, j = 1
When Gi, j <THg, Ei, j = 0 (2)

<Block classification>
The block classification unit 109 is connected to the edge extraction unit 108 and includes block classification means for determining the attribute of the edge image and classifying the block. The block classification means may be any means that classifies blocks according to the edge state extracted by the edge extraction means.

  As another preferred mode, the block classification unit may include a counting unit that counts pixels determined to be edges for each block. The block classifying unit classifies the block as an edge block when the number of edge pixels is equal to or greater than a predetermined threshold value, and classifies the block as a non-edge block when the number is smaller than the predetermined threshold value.

  As another preferred mode, the block classification means counts the change points of edge pixels to non-edge pixels and non-edge pixels to edge pixels for blocks classified as edge blocks, and the number of change points is a predetermined threshold. If it is greater than or equal to the value, the block is classified as a texture block, and if it is less than a predetermined threshold, the block is classified as a significant edge block. This form may be employed after the above-described classification of edge blocks and non-edge blocks.

  FIG. 4 is a diagram for explaining significant edge blocks and texture blocks classified by the block classification unit in the image decoding apparatus of FIG. 1. In FIG. 4A, 401 indicates an example of an edge block, and in FIG. 4B, 402 indicates another example of an edge block. With reference to FIG. 4, the preferable form of the block classification | category part 109 mentioned above is demonstrated.

  The block classification unit 109 classifies the blocks of the decoded image into three types of blocks of significant edge, texture, and non-edge using the edge image. In this process, the edge image is first examined for each block, and those having fewer edge pixels than the predetermined threshold THf are classified as non-edge blocks. Next, the number of change points from the edge pixel to the non-edge pixel and from the non-edge pixel to the edge pixel is counted in the horizontal direction and the vertical direction with respect to the edge block including the edge pixel equal to or greater than the predetermined threshold THf. Then, the number of change points in the vertical and horizontal directions is summed, and the value is set as the number of edge change points of the edge block. Here, if the number of edge change points is small, it can be determined that the edge pixels are concentrated, that is, it has a clear edge. On the other hand, if the number of edge change points is large, it can be determined that the texture has a fine random pattern.

  Therefore, edge blocks can be classified into significant edge blocks and texture blocks using this feature. That is, when the number of edge change points in each edge block is Vk, l, the threshold value THv is used to classify the block into a texture block and a significant edge block as shown in the following equation (3).

When Vk, l ≧ THv, texture block When Vk, l <THv, significant edge block (3)

  In the edge blocks exemplified by the edge blocks 401 and 402, the pixel 403 indicated by ● is an edge pixel, and the pixel 404 indicated by ○ is a non-edge pixel. Here, the number of edge change points of the edge block 401 is 32, and the number of edge change points of the edge block 402 is 71. If the threshold value THv is 60, the edge block 401 can be classified as a significant edge block, and the edge block 402 can be classified as a texture block. The number of edge change points may be calculated only in the vertical direction or only in the horizontal direction.

  As described above, the block classification unit 109 outputs information indicating any one of the three types of significant edge blocks, texture blocks, and non-edge blocks for each block as the block classification result 114. This makes it possible to control noise reduction processing suitable for mosquito noise and block distortion. That is, since mosquito noise is generated inside the edge block (significant edge block, texture block), it is possible to control filtering on all pixels in the block by using this classification result. On the other hand, since no mosquito noise is generated in the non-edge block, only the block boundary needs to be filtered, and filtering control suitable for block distortion reduction can be performed using this classification result. Further, from the viewpoint of reducing the processing amount, the block classification result may be classified into two, that is, an edge block and a non-edge block without obtaining an edge change point.

<Quantization parameter calculation>
The quantization parameter calculation unit 110 is connected to the variable length decoding unit 101 and includes means for calculating the state of the quantization parameter from the quantization scale. This means is not limited to this and may be any of the following quantization parameter calculation means. That is, the quantization parameter calculation means divides the image frame into predetermined processing units, performs predetermined calculation on the quantization parameter of the encoded data of the block in the processing unit, and performs quantization in each processing unit. Calculate the parameter state.

  As another preferred embodiment, a unit obtained by dividing an image frame into strips composed of a plurality of blocks may be used as a processing unit. Then, the quantization parameter calculation means adds the quantization scale for each block within the processing unit to calculate the average value of the quantization scale, and sets this as the state of the quantization parameter in each processing unit.

  As another preferred mode in which a unit obtained by dividing an image frame into strips composed of a plurality of blocks is used as a processing unit, the quantization parameter calculation means performs quantization for each block for each prediction coding mode of the block within the processing unit. It is preferable to calculate the average value of the quantization scale separately for intra-frame (intra) coding and non-frame (non-intra) coding by adding the scales. Then, the quantization parameter calculating means sets the calculated parameters as the quantization parameter states of intra-frame (intra) encoding and non-intra-frame (non-intra) encoding in each processing unit.

  Further, as a preferable form relating to the processing unit, it is preferable to use a coding unit composed of two or more macroblocks that can independently decode an image in an image frame, and a slice, a video packet, or a group of blocks (GOB). In this form, it is possible to use in combination with a form in which an image frame is divided into strips composed of a plurality of blocks and used as a processing unit.

  5 is a diagram for explaining a configuration example of a quantization parameter calculation unit in the image decoding apparatus of FIG. 1, and FIG. 6 is a diagram for explaining division of processing units in the quantization parameter calculation unit of FIG. FIG. 7 is a diagram for explaining division of processing units in the quantization parameter calculation unit of FIG. 5, particularly division of processing units relating to MPEG-4 video packets.

  With reference to FIG. 5 thru | or FIG. 7, the preferable form of the quantization parameter calculating part 110 mentioned above is demonstrated. In FIG. 5, reference numeral 501 denotes a processing unit division unit, 502 denotes a quantization scale addition unit, 503 denotes a quantization scale average value calculation unit, and the quantization parameter calculation unit 110 is configured by these components. explain. 6 and 7, reference numerals 601 and 701 denote examples of image frames.

  The quantization parameter calculation unit 110 includes a processing unit division unit 501, a quantization scale addition unit 502, and a quantization scale average value calculation unit 503.

  The processing unit dividing unit 501 is connected to the variable length decoding unit 101 and divides an image frame into processing units in order to calculate the quantization parameter state. The quantization scale addition unit 502 is connected to the processing unit division unit 501 and the variable length decoding unit 101, and adds the quantization scale according to the block predictive coding mode 113 within the processing unit. The quantization scale average value calculation unit 503 is connected to the quantization scale addition unit 502 and calculates the average value of the quantization scale within the processing unit.

  The processing unit division unit 501 is a part that divides an image frame into units for obtaining a quantization scale state that is a guideline representing image quality, and it is important that the image frame is not global and is not too local. That is, if this processing unit is the entire image frame, an image that is not uniform throughout the entire frame, for example, an image in which one part is flat and has a small amount of information but another part has a fine texture and a large amount of information. On the other hand, it is impossible to grasp an accurate quantization scale state for each part of an image. On the other hand, if it is too local, for example, for each macroblock (MB), the filter strength determined by the filter control unit 111 fluctuates in units of macroblocks, so that when filtering is performed, the image quality fluctuates in units of macroblocks. This is not preferable because there is a possibility. Therefore, a processing unit configured by dividing an image frame in a horizontal direction and including a plurality of blocks is desirable.

  In this configuration example, as shown in FIG. 6, a slice obtained by dividing an image frame 601 by a macroblock band having a line width of 16 lines is used as a processing unit (for example, slice 602, slice 603, slice 606 in FIG. 6). However, since the slice 604 composed of one macroblock is too local, in this case, the one merged with the next slice 605 is handled as one processing unit. That is, the processing unit is composed of two or more macroblocks.

  Further, this processing unit is set to the video packet 702, video packet 703, and video packet 704 in the MPEG-4 image frame (VOP) 701 shown in FIG. It may be continuous. What is important here is that this processing unit is an encoding unit composed of two or more macroblocks that can independently decode an image by an image frame, that is, a size of a slice, a video packet, a group of blocks (GOB), or the like. It is to be. In general, encoding is performed using the same encoding condition in these encoding units, so that the image quality is uniform. Therefore, since it is desirable that the noise reduction filter does not fluctuate frequently within this encoding unit, the state of the quantization scale is obtained in this processing unit.

  The quantization scale addition unit 502 adds the quantization scale value for each prediction encoding mode 113 that is an output from the variable length decoding unit 101 for each processing unit divided by the processing unit division unit 501. That is, according to the predictive coding mode 113 in each processing unit, the values of the respective quantization scales are added to the intra-coded macroblock and the non-intracoded macroblock. In addition, the number of intra-coded macro blocks and the number of non-intra-coded macro blocks are also obtained.

  The quantization scale average value calculation unit 503 obtains the quantization scale average value of the intra-coded macroblock and the non-intra-coded macroblock from the addition result of the quantization scale value obtained by the quantization scale adding unit 502. calculate. The value is output as the quantization scale average value 116.

  Further, from the viewpoint of reducing the processing amount, the average value of all quantization scales may be calculated within the processing unit regardless of the predictive coding mode 113. In addition, ITU-T recommendation H.264. H.264, H.C. 263, H.M. The quantization step size in H.261 or the like can be handled in the same way as the quantization scale in this configuration example.

<Filter control>
The filter control unit 111 is connected to the variable length decoding unit 101, the block classification unit 109, and the quantization parameter calculation unit 110, and includes filter control means for controlling the operation of the filter. The filter control means controls filtering by the noise reduction filter means, that is, filtering in the noise reduction filter unit 112, according to the classification result for each block and the state of the quantization parameter in a predetermined processing unit of the image frame to which the block belongs.

  The filter control means preferably controls the noise reduction filter unit 112 with the first and second two filter intensities or the three filter intensities including the third. The first filter strength may be determined based on the average value of the quantization scale calculated by the filter control unit for each predetermined processing unit. Regarding the relationship between the second filter strength and the first filter strength and the control method, it is preferable to employ any one of various modes (I to VII) described below or a combination thereof.

[I. Control by non-edge block / edge block a]
If the classification result for each block is a non-edge block, the filter control means controls the block so that filtering is performed with the first filter strength. When the block classification result is an edge block, a second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength. To control.

[II. Control by non-edge block / edge block b]
When the classification result for each block is a non-edge block, the filter control means performs control so that only the block boundary pixel of this block is filtered with the first filter strength. In addition, when the block classification result is an edge block, the second filter strength is determined such that filtering stronger than the first filter strength is performed, and all pixels in this block are filtered with the second filter strength. Is controlled to be implemented.

[III. Control by non-edge block / texture block / significant edge block]
When the classification result for each block is a non-edge block, the filter control means performs control so that only the block boundary pixel of this block is filtered with the first filter strength. If the block classification result is a texture block, the second filter strength is determined such that filtering stronger than the first filter strength is performed, and all pixels in this block are filtered with the second filter strength. Is controlled to be implemented. In addition, if the block classification result is a significant edge block, a third filter strength is determined such that filtering higher than the second filter strength is performed, and all pixels in this block have the third filter strength. Control so that filtering is performed.

[IV. Control by non-edge block / edge block / intra / non-intra]
If the classification result for each block is a non-edge block, the filter control means controls the block so that filtering is performed with the first filter strength. When the block classification result is an edge block, a second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength. To control. Further, when the block predictive coding mode is intra-frame (intra) coding, the first filter strength or the second filter strength is set so that the filter strength is stronger than non-frame (non-intra) coding. Change and control so that filtering is performed.

[V. Control by non-edge block / edge block / B / I / P picture]
If the classification result for each block is a non-edge block, the filter control means controls the block so that filtering is performed with the first filter strength. When the block classification result is an edge block, a second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength. To control. Furthermore, when the picture type is bidirectional predictive coding (B picture), the first filter strength is set to be stronger than in intraframe coding (I picture) and interframe forward predictive coding (P picture). Control is performed so that filtering is performed by changing the filter strength or the second filter strength.

[VI. Control by non-edge block / edge block / total code amount]
If the classification result for each block is a non-edge block, the filter control means controls the block so that filtering is performed with the first filter strength. When the block classification result is an edge block, a second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength. To control. Further, control is performed such that filtering is performed by changing the first filter strength or the second filter strength based on the total code amount of each frame. Further, the filter control means may change both the first filter strength and the second filter strength instead of changing either of the filter strengths.

[VII. Control by non-edge block / edge block / code amount such as DCT coefficient]
If the classification result for each block is a non-edge block, the filter control means controls the block so that filtering is performed with the first filter strength. When the block classification result is an edge block, a second filter strength is determined such that filtering stronger than the first filter strength is performed, and filtering is performed on the block with the second filter strength. To control. Furthermore, based on the ratio of the DCT coefficient code amount, motion vector code amount, and other code amount in each frame, control is performed so that filtering is performed by changing the first filter strength or the second filter strength. . Further, the filter control means may change both the first filter strength and the second filter strength instead of changing either of the filter strengths.

  Hereinafter, although the example which combined the form of I-VII mentioned above is given and demonstrated, the detail about each form is understandable if the description is diverted. The filter control unit 111 in this embodiment determines the filter strength based on the block classification result 114, the quantization scale average value 116, the predictive coding mode 113, and the picture type 117, and controls filtering for each block.

  8 is a flowchart for explaining the flow of processing for determining the filter strength for each block by the filter control unit in the image decoding apparatus of FIG. 1, and FIG. 9 is a quantization determined in the processing of FIG. It is the figure which showed the relationship between a scale average value and filter strength. In the description with reference to FIG. 8, if there is no particular description, it means processing in units of blocks. In FIG. 9, reference numerals 901 and 902 denote examples of filter strength lines.

  The filter control unit 111 first determines whether the block to be filtered is non-intra coding (step S1). The determination process here is determined based on the predictive coding mode 113. If non-intra coding, the process proceeds to step S2, and if not, the process proceeds to step S3.

  In step S2, the filter strength for non-intra coding is determined. In the relationship between the quantization scale average value 116 and the filter strength shown in FIG. 9, it can be seen that the filter strength increases as the quantization scale average value increases. Based on this, the filter strength for the quantization scale average value 116 is obtained from the filter strength line 901 for non-intra coding.

  In step S3, the filter strength in the case of intra coding is determined. In this case, the filter strength is obtained based on the intra-encoded filter strength line 902 shown in FIG. This is because, in the case of intra coding, generally higher compression is applied and image quality deteriorates than in non-intra coding, so that a strong filter is executed even with the same quantization scale average value as in non-intra coding. Keep it. Note that the filter intensity line 901 and the filter intensity line 902 may be curved instead of a straight line, and may have an appropriate offset.

  Subsequent to steps S2 and S3, the filter control unit 111 determines whether or not the block to be filtered is a non-edge block (step S4). The determination process here is based on the block classification result 114. If it is a non-edge block, the process proceeds to step S8, and if not, the process proceeds to step S5.

  Step S5 is a process for determining whether or not the block to be filtered is a texture block. The determination process here is determined based on the block classification result 114. If it is a texture block, the process proceeds to step S6, and if not, the process proceeds to step S7.

  In step S6, the filter strength for the texture block is determined. Here, a predetermined value α (α> 0) is added to the filter strength determined in step S2 or step S3 to obtain a new filter strength. By doing in this way, filtering stronger than a non-edge block is implemented. This is because mosquito noise may occur more in the texture portion than in the non-edge portion, and the noise is effectively reduced.

  In step S7, the filter strength for the significant edge block is determined. Here, a predetermined value β (β> 0) having a relationship of β> α is added to the filter strength determined in step S2 or step S3 to obtain a new filter strength. By doing so, filtering stronger than non-edge blocks and texture blocks is performed. This is because a lot of mosquito noise may occur in a portion having a clear edge, and the noise is more effectively reduced.

  Step S8 is processing for determining whether or not the picture type of the block to be filtered is a B picture. The determination process here is based on the picture type 117. If it is a B picture, the process proceeds to step S9. If not, the process proceeds to step S10.

  In step S9, the filter strength when the filter target block belongs to the B picture is determined. Here, a predetermined value γ (γ> 0) is added to the filter strength determined by the processing up to step S8 to obtain a new filter strength. By doing so, stronger filtering is performed on a B picture, which is generally highly compressed and deteriorated in image quality, compared to an I picture and a P picture, and noise is effectively reduced.

  In step S10, the filter strength and the block classification result obtained in the processes up to step S9 are output to the noise reduction filter unit 112 as the filter parameters 118, and the process ends.

<Noise reduction filter>
The noise reduction filter unit 112 is connected to the adder 104 and the above-described filter control unit 111, and includes a unit that filters the decoded image to reduce noise based on the control by the filter control unit 111. This means may be any noise reduction filter means that decodes an image frame encoded for each block and reduces noise generated in the decoded image, and is controlled by the above-described filter control means.

  The noise reduction filter means preferably includes block distortion reduction means for reducing block distortion of a decoded image and mosquito noise reduction means for reducing mosquito noise of the decoded image.

  10 and 11 are diagrams for explaining the operation of the noise reduction filter unit in the image decoding apparatus of FIG. 1, and FIG. 10 is a diagram for explaining pixels to be filtered in edge blocks and non-edge blocks. FIG. 11 is a diagram for explaining an example of the noise reduction filter. 10A shows an example of an edge block (texture block and significant edge block), FIG. 10B shows an example of a non-edge block, and 1101 shows an example of a noise reduction filter in FIG. An example of the non-linear filter is shown. Further, in FIG. 10, in order to indicate the pixels that are not the filtering target pixels in the edge block and the non-edge block, the filtering target pixel is indicated by ●, and the non-filtering target pixel is indicated by ○.

  The noise reduction filter unit 112 performs filtering for reducing noise on the decoded image output from the adder 104 based on the filter parameter 118 determined by the filter control unit 111.

  The noise reduction filter is executed only with the filter intensity of the filter parameter 118 only for the pixel to be filtered as exemplified by ● in FIG. A non-filtering target pixel is not subjected to filtering, and the value is output as it is. The specific operation is as follows.

  As illustrated in FIG. 10A, when the filter parameter 118 is an edge block (texture block and significant edge block), the filter strength of the filter parameter 118 is applied to all pixels in the block in order to reduce mosquito noise. Perform filtering according to On the other hand, as illustrated in FIG. 10B, when the filter parameter 118 is a non-edge block, filtering is performed on the boundary pixels in the block according to the filter strength of the filter parameter 118 in order to reduce block distortion. . Therefore, since filtering suitable for each noise is performed for each edge block and non-edge block, both mosquito noise and block distortion noise can be reduced while preventing unnecessary filtering.

  In this configuration example, a nonlinear filter is used as the noise reduction filter. As shown in FIG. 11, a pixel to be filtered is set as a pixel of interest Xi, j, a 3 × 3 pixel window is set around the pixel, and an output pixel Yi, j is obtained by the following equation (4). Replace with Xi, j. Here, ε is the filter strength of the filter parameter 118.

  Through the processing as described above, adaptive filtering of the decoded image can be performed based on the filter parameter 118. The decoded image is appropriately reduced in noise depending on the edge state and the like, and this is used as the output of the noise reduction filter unit 112.

  In the present embodiment, a nonlinear filter such as Expression (4) is used, but various conventionally proposed methods such as a median filter and a smoothing filter can be used. Therefore, there is no particular limitation here.

  The embodiments of the present invention have been described above. However, the image decoding apparatus and the image decoding method according to the present invention are not limited to the above-described embodiments, and are within the scope not departing from the gist of the present invention. Of course, various changes can be made.

It is a block diagram of the example of 1 structure of the image decoding apparatus which concerns on one Embodiment of this invention. It is a figure for demonstrating the structural example of the edge extraction part in the image decoding apparatus of FIG. It is a figure for demonstrating the differentiation process in the differentiation process part in the edge extraction part of FIG. It is a figure for demonstrating the significant edge block and texture block classified by the block classification | category part in the image decoding apparatus of FIG. It is a figure for demonstrating the structural example of the quantization parameter calculating part in the image decoding apparatus of FIG. It is a figure for demonstrating the division | segmentation of the processing unit in the quantization parameter calculating part of FIG. FIG. 6 is a diagram for explaining division of a processing unit in the quantization parameter calculation unit of FIG. 5, particularly division of a processing unit related to an MPEG-4 video packet. It is a flowchart for demonstrating the flow of the process which determines the filter strength for every block by the filter control part in the image decoding apparatus of FIG. It is the figure which showed the relationship between the quantization scale average value and filter strength which are determined in the process of FIG. It is a figure for demonstrating operation | movement of the noise reduction filter part in the image decoding apparatus of FIG. 1, and is a figure for demonstrating the pixel used as the object of filtering in an edge block and a non-edge block. It is a figure for demonstrating the operation | movement of the noise reduction filter part in the image decoding apparatus of FIG. 1, and is a figure for demonstrating the example of a noise reduction filter. It is a block diagram which shows the structural example of the image decoding apparatus by a prior art.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 ... Variable length decoding part, 102 ... Inverse quantization part, 103 ... Inverse orthogonal transformation part, 104 ... Adder, 105 ... Frame memory part, 106 ... Motion compensation prediction part, 107 ... Switch, 108 ... Edge extraction part, 109 ... Block classification unit, 110 ... Quantization parameter calculation unit, 111 ... Filter control unit, 112 ... Noise reduction filter unit, 113 ... Predictive coding mode, 114 ... Block classification result, 115 ... Quantization scale, 116 ... Quantization scale Average value, 117... Picture type, 118... Filter parameter, 201 .. differentiation processing unit, 202... Edge determination unit, 501 .. processing unit division unit, 502 .. quantization scale addition unit, 503.

Claims (36)

  A noise reduction filter unit that decodes an image frame encoded for each block and reduces noise generated in the decoded image, an edge extraction unit that extracts an edge from the decoded image, and an edge extracted by the edge extraction unit Block classification means for classifying blocks according to the state, and dividing the image frame into predetermined processing units, performing predetermined operations on the quantization parameters of the encoded data of the blocks in the processing units, and each processing unit A quantization parameter calculation means for calculating a quantization parameter state in the filtering unit, and filtering by the noise reduction filter means according to a classification result for each block and a quantization parameter state in a predetermined processing unit of the image frame to which the block belongs. An image decoding apparatus comprising: a filter control means for controlling   2. The image according to claim 1, wherein the noise reduction filter means includes block distortion reduction means for reducing block distortion of the decoded image and mosquito noise reduction means for reducing mosquito noise of the decoded image. Decoding device.   3. The edge extraction means comprises: differentiation processing means for differentiating the decoded image; and edge determination means for determining an edge portion of the decoded image based on an output of the differentiation processing means. The image decoding apparatus described in 1.   4. The image decoding apparatus according to claim 3, wherein the edge determination unit compares the output of the differentiation processing unit with a predetermined threshold value and determines an edge and a non-edge for each pixel.   The block classification means includes a counting means for counting pixels determined to be edges for each block, and when the number of edge pixels is equal to or greater than a predetermined threshold, the block is classified as an edge block, and has a predetermined threshold. The image decoding device according to claim 1, wherein when the value is smaller than the value, the block is classified as a non-edge block.   The block classification means counts the change point of the edge pixel from the edge pixel to the non-edge pixel and the non-edge pixel to the edge pixel with respect to the block classified as the edge block. 6. The image decoding device according to claim 1, wherein the block is classified as a texture block, and if the block is smaller than a predetermined threshold, the block is classified as a significant edge block.   The quantization parameter calculation means calculates a mean value of the quantization scale by adding a quantization scale for each block within a processing unit obtained by dividing an image frame into strips composed of a plurality of blocks. The image decoding apparatus according to any one of claims 1 to 6, wherein this is set as a state of a quantization parameter in each processing unit.   The quantization parameter calculation means uses a unit obtained by dividing an image frame in a band composed of a plurality of blocks as a processing unit, and adds a quantization scale for each block for each prediction coding mode of the block within the processing unit. The average value of the quantization scale is calculated separately for intra-frame coding and non-frame coding, and this is set as the state of quantization parameters for intra-frame coding and non-frame coding in each processing unit. The image decoding apparatus according to claim 1, wherein the image decoding apparatus is characterized in that   9. The processing unit is a coding unit composed of two or more macroblocks that can independently decode an image in an image frame, and is a slice, a video packet, or a group of blocks. The image decoding device according to any one of the above.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The image decoding apparatus according to claim 1, wherein the image decoding apparatus determines and controls the block so that filtering is performed with the second filter strength.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, Only the block boundary pixels of this block are filtered with the first filter strength, and when the block classification result is an edge block, the filtering that is stronger than the first filter strength is performed. The image decoding according to any one of claims 1 to 9, wherein a second filter strength is determined and control is performed so that all pixels in the block are filtered with the second filter strength. apparatus.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, Only the block boundary pixels of this block are filtered with the first filter strength, and when the block classification result is a texture block, the filtering that is stronger than the first filter strength is performed. A second filter strength is determined so that all pixels in this block are filtered with the second filter strength, and if the block classification result is a significant edge block, it is stronger than the second filter strength. Determine the third filter strength at which filtering will be performed, and all pixels in this block will The image decoding apparatus according to any one of claims 1 to 9 filtered by the filter strength and controls as implemented.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is filtered with the second filter strength, and if the prediction coding mode of the block is intraframe coding, the filter strength is higher than that of non-frame coding. Control the filtering to be performed by changing the first filter strength or the second filter strength. The image decoding apparatus according to any one of claims 1 to 9, symptom.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is filtered with the second filter strength, and if the picture type is bi-directional predictive coding, the filter strength is stronger than intra-frame coding and inter-frame forward predictive coding. As described above, filtering is performed by changing the first filter strength or the second filter strength. The image decoding apparatus according to any one of claims 1 to 9, characterized in that Gosuru.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. And filtering is performed by changing the first filter strength or the second filter strength based on the total code amount of each frame. The image decoding device according to claim 1, wherein the image decoding device is controlled so as to perform the control.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. And the filtering is performed by changing the first filter strength and the second filter strength based on the total code amount of each frame. The image decoding device according to claim 1, wherein the image decoding device is controlled so as to perform the control.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is subjected to filtering with the second filter strength, and the first filter strength based on the ratio of the DCT coefficient code amount, the motion vector code amount, and the other code amount in each frame, Alternatively, control is performed such that filtering is performed by changing the second filter strength. The image decoding apparatus according to any one of Itaru 9.   The filter control means determines the first filter strength based on the average value of the quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is subjected to filtering with the second filter strength, and the first filter strength based on the ratio of the DCT coefficient code amount, the motion vector code amount, and the other code amount in each frame, And controlling to change the second filter strength so that filtering is performed. The image decoding apparatus according to any one of 9.   A noise reduction filter step for decoding an image frame encoded for each block and reducing noise generated in the decoded image; an edge extraction step for extracting an edge from the decoded image; and an edge extracted in the edge extraction step A block classification step for classifying the blocks according to the state, and dividing the image frame into predetermined processing units, and performing a predetermined operation on the quantization parameter of the encoded data of the block in the processing units, A quantization parameter calculation step for calculating a quantization parameter state in step (b), filtering by the noise reduction filter step according to a classification result for each block and a quantization parameter state in a predetermined processing unit of the image frame to which the block belongs Including a filter control step to control Image decoding method comprising Rukoto.   The image according to claim 19, wherein the noise reduction filter step includes a block distortion reduction step of reducing block distortion of the decoded image, and a mosquito noise reduction step of reducing mosquito noise of the decoded image. Decryption method.   21. The edge extraction step includes a differentiation processing step for differentiating the decoded image, and an edge determination step for determining an edge portion of the decoded image based on an output of the differentiation processing step. The image decoding method described in 1.   The image decoding method according to claim 21, wherein the edge determination step compares an output of the differentiation processing step with a predetermined threshold value and determines an edge and a non-edge for each pixel.   The block classification step includes a counting step of counting pixels determined to be edges for each block, and when the number of edge pixels is equal to or greater than a predetermined threshold, the block is classified as an edge block, and a predetermined threshold is set. The image decoding method according to any one of claims 19 to 22, wherein if the value is smaller than the value, the block is classified as a non-edge block.   The block classification step counts change points of edge pixels to non-edge pixels and non-edge pixels to edge pixels for blocks classified as edge blocks, and when the number of change points is equal to or greater than a predetermined threshold, The image decoding method according to any one of claims 19 to 23, wherein a block is classified as a texture block, and if the block is smaller than a predetermined threshold, the block is classified as a significant edge block.   In the quantization parameter calculation step, a processing unit is obtained by dividing an image frame into strips composed of a plurality of blocks, and an average value of the quantization scale is calculated by adding the quantization scale for each block within the processing unit. The image decoding method according to any one of claims 19 to 24, wherein the state is a quantization parameter state in each processing unit.   In the quantization parameter calculation step, an image frame is divided into strips composed of a plurality of blocks as a processing unit, and a quantization scale for each block is added for each prediction encoding mode of the block within the processing unit. The average value of the quantization scale is calculated separately for intra-frame coding and non-frame coding, and this is set as the state of quantization parameters for intra-frame coding and non-frame coding in each processing unit. 25. The image decoding method according to any one of claims 19 to 24, wherein:   27. The processing unit is a coding unit composed of two or more macroblocks capable of independently decoding an image in an image frame, and is a slice, a video packet, or a group of blocks. The image decoding method according to any one of the above.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The image decoding method according to any one of claims 19 to 27, wherein the determination is performed and the block is controlled so as to be filtered with the second filter strength.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, Only the block boundary pixels of this block are filtered with the first filter strength, and when the block classification result is an edge block, the filtering that is stronger than the first filter strength is performed. 28. Image decoding according to any one of claims 19 to 27, characterized in that a second filter strength is determined and all pixels in the block are controlled to be filtered with the second filter strength. Method.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, Only the block boundary pixels of this block are filtered with the first filter strength, and when the block classification result is a texture block, the filtering that is stronger than the first filter strength is performed. A second filter strength is determined so that all pixels in this block are filtered with the second filter strength, and if the block classification result is a significant edge block, it is stronger than the second filter strength. Determine the third filter strength at which filtering will be performed and all pixels in this block will Image decoding method according to any one of claims 19 to 27 and controls so that filtering is performed in three of the filter strength.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is filtered with the second filter strength, and if the prediction coding mode of the block is intraframe coding, the filter strength is higher than that of non-frame coding. Change the first filter strength or the second filter strength to control the filtering. Image decoding method according to any one of claims 19 to 27, characterized in.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is filtered with the second filter strength, and if the picture type is bi-directional predictive coding, the filter strength is stronger than intra-frame coding and inter-frame forward predictive coding. As described above, filtering is performed by changing the first filter strength or the second filter strength. Image decoding method according to any one of claims 19 to 27 and controlling the.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. And filtering is performed by changing the first filter strength or the second filter strength based on the total code amount of each frame. The image decoding method according to any one of claims 19 to 27, wherein control is performed so that   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. And the filtering is performed by changing the first filter strength and the second filter strength based on the total code amount of each frame. The image decoding method according to any one of claims 19 to 27, wherein control is performed so that   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is subjected to filtering with the second filter strength, and the first filter strength based on the ratio of the DCT coefficient code amount, the motion vector code amount, and the other code amount in each frame, Alternatively, control is performed so that filtering is performed by changing the second filter strength. Image decoding method according to any one of 19 to 27.   The filter control step determines a first filter strength based on an average value of a quantization scale calculated for each predetermined processing unit, and when the classification result for each block is a non-edge block, This block is filtered with the first filter strength, and when the block classification result is an edge block, the second filter strength is set such that filtering stronger than the first filter strength is performed. The block is subjected to filtering with the second filter strength, and the first filter strength based on the ratio of the DCT coefficient code amount, the motion vector code amount, and the other code amount in each frame, And controlling to change the second filter strength so that filtering is performed. Image decoding method according to any one of 9 to 27.

Images