JP2013046255A - Moving image encoder, moving image decoder, moving image encoding method, moving image decoding method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image encoder,a moving image decoder, a moving image encoding method, a moving image decoding method, and a program capable of realizing improvement of a subjective image quality accurately.SOLUTION: A moving image encoder AA calculates an NLM filter coefficient m1 acquired from an original image a and an NLM filter coefficient m2 acquired from a local decoded image k. Degradation measure from the original image a when using the NLM filter coefficient m1 and degradation measure from the original image a when using the NLM filter coefficient m2 are compared, and on the basis of the comparison result, it is determined that filter processing is performed with either the NLM filter coefficient m1 or m2.

Description

  The present invention relates to a video encoding device, a video decoding device, a video encoding method, a video decoding method, and a program.

  Non-Patent Documents 1 to 3 describe a technique for improving image quality by performing filter processing on a coded image in video compression coding processing.

  In particular, Non-Patent Document 1 discloses a standard for video compression encoding. According to this standard, it is possible to apply a filter process that reduces compression degradation that occurs at block boundaries due to compression coding.

  On the other hand, Non-Patent Document 2 discloses a method of adaptively updating a filter for restoring coding degradation for each frame. According to this method, one type of filter can be calculated for the entire screen so that the square error from the original image is minimized for the entire screen.

  Similarly to Non-Patent Document 1, Non-Patent Document 3 includes a Non-Local Means filter (image restoration processing filter shown in Non-Patent Document 4) in order to reduce the compression deterioration that occurs at the block boundary. Hereinafter, a technique for performing filter processing based on “NLM filter”) is shown. According to this method, the subjective quality can be improved.

Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, "Text of ISO / IEC 14496-10 Advanced Video Coding," July 2004. D.-H. Kim, H.-Y. Oh, O. Urhan, S. Erturk and T.-G. Chang, "Optimal Post-Process / In-Loop Filtering for Improved Video Compression Performance," IEEE Trans. Consumer Electronics, Vol. 53, No. 4, 2007. M. Mtsumura, Y. Bandoh, S. Takamura, H. Jozawa, "In-loop filter based on non-local means filter," Joint Collaborative Team on Video Coding (JCT-VC), JCTVC-E206, March, 2011. A. Buades, B. Coll, and J. M. Morel, "A Non Local Algorithm for Image Denoising," Proc. IEEE Int. Conf. On Computer Vision and Pattern Recognition, vol. 2, pp. 60-65, June 2005. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," In Proceedings of the Sixth International Conference on Computer Vision, page 839-846, 1998. S. Smith and J. Brady, "Susan-a new approach to low level image processing," International Journal of Computer Vision, 23 (1): 45-78, 1997.

  In the standard shown in Non-Patent Document 1, filtering is performed only on block boundaries. For this reason, the image quality improvement effect by the filter processing is not sufficient.

  In contrast to the standard shown in Non-Patent Document 1, the technique shown in Non-Patent Document 2 performs an adaptive filter process on the entire screen. However, this adaptive filter is for minimizing the square error with the original image for the entire screen, and the edge component, which is a visually important picture, cannot be reliably retained. .

  In the method shown in Non-Patent Document 3, the image restoration processing filter shown in Non-Patent Document 4 is adopted for the filter processing at the time of decoding. However, in the method shown in Non-Patent Document 4, since the filter coefficient is determined according to the similarity to the pattern near the pixel to be filtered, the pattern near the pixel to be filtered is greatly increased by compression coding. If it is deteriorated, an inappropriate filter coefficient is calculated, and appropriate image restoration cannot be performed. As a characteristic case in which a local picture greatly deteriorates due to compression deterioration, for example, there is an edge picture. For this reason, in the methods shown in Non-Patent Documents 3 and 4, there may be cases where sufficient image quality cannot be obtained with respect to edge components that are visually important patterns, and an improvement in subjective image quality is achieved accurately. There were cases where it was not possible.

  Therefore, the present invention has been made in view of the above-described problems, and is a moving image encoding device, a moving image decoding device, a moving image encoding method, a moving image decoding method, and a moving image decoding method that can accurately realize improvement in subjective image quality, and The purpose is to provide a program.

  The present invention proposes the following matters in order to solve the above problems.

  (1) The present invention allows a moving image coding apparatus (for example, the moving image coding apparatus AA in FIG. 1) that allows adaptive filter processing within a coding processing loop and controls filter processing for each filter processing unit block. ) And a filter coefficient (for example, an NLM filter obtained from the original image a in FIG. 2) based on local pattern similarity with respect to the input image (for example, corresponding to the original image a in FIG. 2) A first filter coefficient calculation means (for example, corresponding to the first filter coefficient calculation unit 51 in FIG. 2) and a local decoded image (for example, corresponding to the local decoded image k in FIG. 2). In contrast, a second filter coefficient calculation that calculates a filter coefficient (e.g., corresponding to the NLM filter coefficient m2 obtained from the local decoded image k in FIG. 2) based on the local pattern similarity. A first filter coefficient application unit that applies the filter coefficient calculated by the first filter coefficient calculation unit to the local decoded image (for example, corresponding to the second filter coefficient calculation unit 52 in FIG. 2) (E.g., corresponding to the first filter coefficient application unit 61 in FIG. 2) and second filter coefficient application means for applying the filter coefficient calculated by the second filter coefficient calculation means to the local decoded image ( For example, the first filter is applied to each of the second filter coefficient application unit 62 of FIG. 2 and a predetermined unit region (for example, a unit region composed of N × N pixels described later). Which one of the filter coefficient calculated by the coefficient calculating means and the filter coefficient calculated by the second filter coefficient calculating means is used to perform the filter processing on the local decoded image Filter coefficient suitability judging means for judging (e.g., corresponding to the filter coefficient suitability judging section 63 in FIG. 2), the filter coefficient calculated by the first filter coefficient calculating means, and the second filter coefficient calculating means. Filter processing means (for example, FIG. 2) that performs filter processing on the locally decoded image using the filter coefficients that have been determined to be used for each unit region by the filter coefficient suitability determination means. 2) and appropriateness information encoding means for encoding the determination result (for example, equivalent to appropriateness information r described later) by the filter coefficient appropriateness determining means (for example, the appropriateness information encoding part of FIG. 2). Among the filter coefficients calculated by the first filter coefficient calculation means, and used by the filter coefficient suitability determination means. Filter coefficient encoding means (for example, corresponding to the filter coefficient encoding unit 65 of FIG. 2), and the filter coefficient suitability determining means for each unit region. Determining the degradation scale from the input image of the image obtained by the first filter coefficient applying means and the degradation scale from the input image of the image obtained by the second filter coefficient applying means; and for each unit region In addition, based on the degradation measure, the local decoded image is used by using either the filter coefficient calculated by the first filter coefficient calculating unit or the filter coefficient calculated by the second filter coefficient calculating unit. Has been proposed to determine whether or not to perform a filtering process.

  According to this invention, the first filter coefficient calculation means, the second filter, and the like are allowed in the moving picture coding apparatus that allows the adaptive filter processing in the coding processing loop and controls the filter processing for each filter processing unit block. Coefficient calculation means, first filter coefficient application means, second filter coefficient application means, filter coefficient suitability determination means, filter processing means, suitability information encoding means, and filter coefficient encoding means are provided. The first filter coefficient calculation means calculates the filter coefficient based on the local pattern similarity with respect to the input image, and the second filter coefficient calculation means calculates the local decoded image with respect to the local decoded image. The filter coefficient was calculated based on the similarity of the patterns. The first filter coefficient applying means applies the filter coefficient calculated by the first filter coefficient calculating means to the local decoded image, and the second filter coefficient applying means applies the second decoded coefficient to the local decoded image. The filter coefficient calculated by the filter coefficient calculation means is applied. Further, the filter coefficient suitability determining means, for each predetermined unit area, either the filter coefficient calculated by the first filter coefficient calculating means or the filter coefficient calculated by the second filter coefficient calculating means. Is used to determine whether to perform filter processing on the local decoded image. Specifically, first, for each unit region, a degradation scale from the input image of the image obtained by the first filter coefficient application means and a degradation scale from the input image of the image obtained by the second filter coefficient application means. And we decided to ask. Next, for each unit region, based on the obtained degradation measure, either the filter coefficient calculated by the first filter coefficient calculation means or the filter coefficient calculated by the second filter coefficient calculation means It was decided to determine whether to perform filtering on the locally decoded image. In addition, the filter processing unit determines whether the filter coefficient is calculated by the first filter coefficient calculation unit and the filter coefficient calculated by the second filter coefficient calculation unit by the filter processing unit for each unit region. Using what is determined to be used, the filtering process is performed on the locally decoded image. Further, the judgment result by the filter coefficient suitability judging means is coded by the suitability information coding means. In addition, the filter coefficient encoding means encodes the filter coefficients calculated by the first filter coefficient calculation means that are determined to be used by the filter coefficient suitability determination means.

  For this reason, since the moving picture encoding apparatus of (1) can perform a filter process using the filter based on the local similarity of a picture, it can aim at the improvement of subjective image quality.

  Further, the moving picture coding apparatus of (1) calculates the filter coefficient obtained from the input image by the first filter coefficient calculation means, and obtained from the locally decoded image by the second filter coefficient calculation means. Filter coefficients can be calculated. Then, by the filter coefficient suitability determining means, the degradation scale from the input image when the filter coefficient obtained from the input image is used, and the degradation scale from the input image when the filter coefficient obtained from the local decoded image is used. And which filter coefficient is used for the filtering process according to the comparison result. For this reason, of the filter coefficients obtained from the input image and the filter coefficients obtained from the locally decoded image, for example, the one with less deterioration from the input image when used for the filter process is used for the filter process. It can be judged. In addition, for example, it is basically determined that the filter coefficient obtained from the local decoded image is used for the filter processing, and the filter coefficient obtained from the local decoded image is more used by using the filter coefficient obtained from the input image. When the deterioration from the input image can be suppressed to a predetermined level or more as compared to the case of using the filter, it can be determined that the filter coefficient obtained from the input image is used for the filter processing. According to the above, sufficient image quality can be obtained even in cases where local patterns such as edge components, which are visually important patterns, are greatly deteriorated, so that the above-described improvement in subjective image quality can be realized accurately. .

  Also, the moving image encoding apparatus of (1) encodes only the filter coefficients used for the filter process among the filter coefficients obtained from the input image by the filter coefficient encoding means. Here, the filter coefficient obtained from the locally decoded image can be calculated also in the moving picture decoding apparatus that decodes the encoded data generated by the moving picture encoding apparatus. For this reason, it is possible to reduce the amount of information to be transmitted while ensuring the realization of filter processing in the video decoding device.

  (2) The present invention relates to the moving image encoding apparatus according to (1), wherein the deterioration scale from the input image of the image obtained by the first filter coefficient applying unit is a degree of deterioration of the image with respect to the input image. The deterioration scale from the input image of the image obtained by the second filter coefficient applying means is a numerical value that increases as the degree of deterioration of the input image with respect to the input image increases. The filter coefficient suitability determining means is a difference obtained by subtracting the deterioration scale for the image obtained by the first filter coefficient applying means from the deterioration scale for the image obtained by the second filter coefficient applying means. For unit regions whose values are greater than or equal to a predetermined threshold (e.g., corresponding to a predetermined threshold TH described later) It is determined that the filter coefficient calculated by the first filter coefficient calculation unit is used, and for the unit region whose difference value is less than the threshold value, the filter coefficient calculated by the second filter coefficient calculation unit is used. A moving picture coding apparatus characterized in that it is determined to be used has been proposed.

  According to the present invention, in the moving picture coding apparatus of (1), the degradation scale from the input image of the image obtained by the first filter coefficient applying means is increased as the degree of degradation of the image with respect to the input image increases. The degradation scale from the input image of the image obtained by the second filter coefficient applying unit expressed by a large numerical value is represented by a large numerical value as the degree of degradation of the image with respect to the input image increases. Further, a difference value obtained by subtracting the deterioration scale for the image obtained by the first filter coefficient applying means from the deterioration scale for the image obtained by the second filter coefficient applying means by the filter coefficient suitability determining means is determined in advance. It is determined that the filter coefficient calculated by the first filter coefficient calculation unit is used for the unit area that is equal to or greater than the threshold value, and the second filter is used for the unit area whose difference value is less than the threshold value. It was decided to use the filter coefficient calculated by the coefficient calculating means.

  For this reason, in the moving image encoding apparatus of (2), the deterioration from the input image is more effective when the filter coefficient obtained from the input image is used than when the filter coefficient obtained from the locally decoded image is used. When the level can be suppressed to a level determined by the threshold, it can be determined that the filter coefficient calculated by the first filter coefficient calculation unit is used. Therefore, the same effects as those described above can be achieved.

  (3) In the moving image encoding apparatus according to (1) or (2), the filter coefficient suitability determining unit is a measure of deterioration of the image obtained by the first filter coefficient applying unit from the input image. Using the Mean Square Error or Structural Similarity of the input image, the image obtained by the first filter coefficient applying means, and the deterioration of the image obtained by the second filter coefficient applying means from the input image. As a measure, the input image, the image obtained by the second filter coefficient applying means, the Mean Square Error and the Structural Simality of the image obtained by the first filter coefficient applying means, from the input image. As a measure of deterioration It proposes a moving picture encoding apparatus characterized by the use of those used.

  According to the present invention, in the moving image encoding apparatus according to (1) or (2), the input coefficient is used as a degradation measure from the input image of the image obtained by the first filter coefficient applying means by the filter coefficient suitability determining means. And an image obtained by the first filter coefficient applying means and Mean Square Error or Structural Similarity are used. Further, the Mean Square Error of the input image and the image obtained by the second filter coefficient applying unit is used as a degradation measure from the input image of the image obtained by the second filter coefficient applying unit by the filter coefficient suitability determining unit. Of the structural similarity, the one used as a measure of deterioration of the image obtained by the first filter coefficient applying means from the input image is used.

  For this reason, the moving picture encoding apparatus of (3) obtains a locally decoded image when Mean Square Error is used as a degradation measure from the input image when the filter coefficient obtained from the input image is used. Mean Square Error can be used as a measure of deterioration from the input image when the obtained filter coefficients are used. In addition, when Structural Similarity is used as a degradation measure from the input image when the filter coefficient obtained from the input image is used, the filter image obtained from the local decoded image is used. As a degradation scale, Structural Similarity can be used. According to the above, an effect similar to the effect described above can be achieved.

  (4) In the moving image encoding device according to any one of (1) to (3), the adequacy information encoding unit expresses a determination result by the filter coefficient appropriateness determination unit as flag information. For example, a moving picture coding apparatus characterized by coding the flag information using arithmetic coding or variable length coding). is suggesting.

  According to the present invention, in the moving picture encoding apparatus according to any one of (1) to (3), the determination result by the filter coefficient suitability judging means is expressed as flag information by the suitability information encoding means, and the flag information Is coded using arithmetic coding or variable length coding.

  For this reason, the moving image encoding apparatus of (4) uses either the filter coefficient obtained from the input image or the filter coefficient obtained from the locally decoded image while suppressing an increase in the amount of information to be transmitted. Thus, it is possible to indicate for each unit area whether the filtering process has been performed.

  (5) In the moving image encoding apparatus according to (4), the propriety information encoding unit expresses the flag information using two-dimensional information or a multi-level layer structure. An encoding device is proposed.

  According to the present invention, in the moving picture encoding apparatus of (4), the flag information is expressed using the two-dimensional information or the multistage layer structure by the suitability information encoding means.

  For this reason, the moving picture encoding apparatus of (5) can express the above-mentioned flag information using two-dimensional information or a multistage layer structure, and can produce the same effect as the above-described effect.

  (6) In the moving image encoding device according to any one of (1) to (5), the filter coefficient encoding unit is configured to calculate L × L for each of N × N pixels constituting the unit region. Filter coefficient information orthogonal transform means for orthogonally transforming each of the filter coefficients as a four-dimensional signal of N × N × L × L, and quantizing the result of orthogonal transform by the filter coefficient information orthogonal transform means, Proposed is a moving picture coding apparatus comprising transform coefficient coding means for coding by coding or variable length coding.

  According to the present invention, in any one of the moving picture encoding apparatuses according to (1) to (5), the filter coefficient encoding unit includes the filter coefficient information orthogonal transform unit and the transform coefficient encoding unit. The filter coefficient information orthogonal transform means orthogonally transforms L × L filter coefficients for N × N pixels constituting the unit area as a four-dimensional signal of N × N × L × L. In addition, the transform coefficient coding unit quantizes the result of the orthogonal transform by the filter coefficient information orthogonal transform unit, and encodes the result by arithmetic coding or variable length coding.

  For this reason, the moving picture coding apparatus of (6) greatly increases energy as the inter-pixel correlation of each of the N × N pixels constituting the unit region increases by the orthogonal transform by the file coefficient information orthogonal transform unit. Can be compressed. Therefore, the amount of encoded data can be further reduced by quantizing and encoding the result of orthogonal transform.

  (7) The present invention is characterized in that a filter based on Non Local Means is used for the filter processing to be controlled for each filter processing unit block in the moving image encoding device according to any one of (1) to (6). A video encoding device is proposed.

  According to the present invention, in the moving picture encoding apparatus according to any one of (1) to (6), a filter based on Non Local Means is used for the filter processing controlled for each filter processing unit block.

  For this reason, the moving picture encoding apparatus of (7) can perform a filter process using the filter based on Non Local Means, and can show the same effect as the effect mentioned above.

  (8) The present invention allows adaptive filter processing in a decoding processing loop, controls the filter processing for each filter processing unit block, and selects any one of the moving image encoding devices (1) to (7) (for example, A video decoding device (for example, the video decoding device BB in FIG. 4) for decoding the encoded data generated in the video encoding device AA in FIG. Filter information decoding that decodes the encoded data generated by the means and the filter coefficient encoding means (e.g., equivalent to the encoded data c relating to suitability information in FIG. 5 and the encoded data relating to the filter coefficient in FIG. 5). For the means (for example, equivalent to the filter coefficient decoding unit 131 and the suitability information decoding unit 133 in FIG. 5) and the decoded image (for example, equivalent to the decoded image E in FIG. 5), the local A decoded image filter coefficient calculation unit (for example, corresponding to the filter coefficient calculation unit 132 in FIG. 5) that calculates filter coefficients based on the similarity of patterns, and a filter coefficient decoded by the filter information decoding unit (for example, FIG. 5). Corresponding to the NLM filter coefficient m1 obtained from the original image a) and the filter coefficient calculated by the decoded image filter coefficient calculation means (for example, equivalent to the NLM filter coefficient F obtained from the decoded image E in FIG. 5). And a decoded image filter processing means (for example, FIG. 5) that performs a filtering process on the decoded image using the one corresponding to the determination result for each unit area decoded by the filter information decoding means. The video decoding device is provided with a filter processing unit 134).

  According to the present invention, adaptive filter processing is allowed in the decoding processing loop, the filter processing is controlled for each filter processing unit block, and is generated in any one of (1) to (7). The moving picture decoding apparatus that decodes the encoded data is provided with filter information decoding means, decoded image filter coefficient calculation means, and decoded image filter processing means. The encoded information generated by the suitability information encoding unit and the filter coefficient encoding unit is decoded by the filter information decoding unit. Also, the decoded image filter coefficient calculation means calculates the filter coefficient based on the local pattern similarity with respect to the decoded image. Further, among the filter coefficients decoded by the filter information decoding means by the decoded image filter processing means and the filter coefficients calculated by the decoded image filter coefficient calculation means, each unit area decoded by the filter information decoding means A filter process is performed on the decoded image using an image according to the determination result.

  For this reason, since the moving picture decoding apparatus of (8) can perform a filter process using the filter coefficient based on the local similarity of a picture, it can aim at the improvement of subjective image quality.

  The moving picture decoding apparatus according to (8) obtains the filter coefficient obtained from the input image used for the filter processing in the moving picture coding apparatus by the filter information decoding means, and the decoded image filter coefficient calculating means Filter coefficients obtained from the decoded image can be calculated. Then, the decoded image filter processing means performs a filter process using the filter coefficient obtained from the input image for the unit area in which the filter coefficient obtained from the input image is used in the moving image coding apparatus. be able to. On the other hand, it is possible to perform a filtering process on the unit region in which the filter coefficient obtained from the locally decoded image is used in the moving image coding apparatus, using the filter coefficient obtained from the decoded image. Therefore, sufficient image quality can be obtained even in the case where a local pattern such as an edge component, which is a visually important pattern, is greatly deteriorated, so that the above-described improvement in subjective image quality can be realized accurately.

  The moving picture decoding apparatus according to (8) calculates the filter coefficient obtained from the locally decoded image calculated by the moving picture encoding apparatus as the filter coefficient obtained from the decoded image, and the moving picture encoding apparatus The filter coefficient obtained from the input image calculated in step 1 can be obtained by decoding the encoded data transmitted from the moving image encoding device. For this reason, since it is not necessary to transmit the filter coefficient obtained by the local decoded image, the amount of information transmitted can be reduced while ensuring the realization of decoding by the video decoding device.

  (9) In the moving image decoding apparatus according to (8), the filter information decoding unit includes encoded data generated by the appropriateness information encoding unit (for example, encoded data c relating to appropriateness information in FIG. 5). Is calculated using arithmetic coding or variable length coding, and a determination result (for example, corresponding to the suitability information r in FIG. 5) obtained by the filter coefficient suitability judging means expressed as flag information is obtained. Has been proposed.

  According to the present invention, in the moving picture decoding apparatus according to (8), the filter information decoding unit decodes the encoded data generated by the suitability information encoding unit using arithmetic encoding or variable length encoding. The determination result by the filter coefficient suitability determination means expressed as flag information is obtained.

  For this reason, the moving picture decoding apparatus according to (9) uses either the filter coefficient obtained from the input image or the filter coefficient obtained from the local decoded image while suppressing an increase in the amount of transmitted information. Thus, it can be determined for each unit area whether the filtering process has been performed in the moving image encoding apparatus.

  (10) The present invention proposes a moving picture decoding apparatus characterized in that, in the moving picture decoding apparatus of (9), the flag information is expressed using two-dimensional information or a multistage layer structure.

  According to the present invention, in the moving picture decoding apparatus of (9), the flag information is expressed using two-dimensional information or a multistage layer structure.

  For this reason, the moving picture decoding apparatus of (10) can judge the above-mentioned flag information using two-dimensional information or a multistage layer structure, and can produce the same effect as the above-described effect.

  (11) In the moving image decoding device according to any one of (8) to (10), the present invention provides that the filter information decoding unit performs arithmetic decoding or variable length decoding on the encoded data generated by the filter coefficient encoding unit. The unit region is configured by performing inverse quantization on the transform coefficient decoding means for decoding by the above-described method and inversely quantizing the result of decoding by the transform coefficient decoding means and performing a four-dimensional signal of N × N × L × L. Proposed is a moving picture decoding apparatus comprising: filter coefficient information inverse orthogonal transform means for obtaining L × L filter coefficients for each of N × N pixels.

  According to this invention, in any one of the moving picture decoding apparatuses according to (8) to (10), the filter information decoding means is provided with transform coefficient decoding means and filter coefficient information inverse orthogonal transform means. The transformed coefficient decoding means decodes the encoded data generated by the filter coefficient encoding means by arithmetic decoding or variable length decoding. Further, the filter coefficient information inverse orthogonal transform means performs inverse quantization on the decoding result by the transform coefficient decoding means, and inverse orthogonal transforms as a four-dimensional signal of N × N × L × L to form a unit region. It was decided to obtain L × L filter coefficients for each of N × N pixels.

  For this reason, the moving picture decoding apparatus according to (11) can restore the filter coefficient obtained from the input image used for the filter processing in the moving picture coding apparatus from the encoded data, and has the same effect as described above. The effect of can be produced.

  (12) The present invention is a moving image encoding method that allows adaptive filter processing in an encoding processing loop and controls filter processing for each filter processing unit block, and includes an input image (for example, the original image in FIG. 2). a first step (for example, corresponding to the NLM filter coefficient m1 obtained from the original image a in FIG. 2) based on local pattern similarity 2) and a locally decoded image (e.g., corresponding to the local decoded image k in FIG. 2) based on the local pattern similarity. A second step (for example, processing by the second filter coefficient calculation unit 52 in FIG. 2) for calculating a coefficient (for example, equivalent to the NLM filter coefficient m2 obtained from the local decoded image k in FIG. 2). A third step of applying the filter coefficient calculated in the first step to the locally decoded image (e.g., corresponding to the processing by the first filter coefficient application unit 61 in FIG. 2), A fourth step of applying the filter coefficient calculated in the second step to the locally decoded image (for example, corresponding to the processing by the second filter coefficient application unit 62 in FIG. 2), a predetermined unit region ( For example, each of the filter coefficients calculated in the first step and the filter coefficient calculated in the second step is used for each unit area (corresponding to a unit area composed of N × N pixels described later). A fifth step (for example, corresponding to the processing by the filter coefficient suitability determination unit 63 in FIG. 2) for determining whether or not to perform filter processing on the locally decoded image; Among the filter coefficients calculated in step and the filter coefficients calculated in the second step, the one determined to be used for each unit region in the fifth step is used for the local decoded image. A sixth step for performing filter processing (for example, corresponding to processing by the filter processing unit 64 in FIG. 2) and a determination result in the fifth step (for example, corresponding to appropriateness information r described later) are encoded. Among the filter coefficients calculated in the first step (for example, corresponding to the processing by the suitability information encoding unit 66 in FIG. 2) and the first step, the one determined to be used in the fifth step is encoded. 8 (for example, equivalent to the processing by the filter coefficient encoding unit 65 in FIG. 2), and in the fifth step, the unit area For each of the unit regions, a degradation scale from the input image of the image obtained in the third step and a degradation scale from the input image of the image obtained in the fourth step are obtained. Which of the filter coefficient calculated in the first step and the filter coefficient calculated in the second step is used to perform the filtering process on the local decoded image based on the degradation scale A video encoding method characterized by judging is proposed.

  According to the present invention, firstly, the filter coefficient is calculated based on the local pattern similarity with respect to the input image, and the filter coefficient based on the local pattern similarity with respect to the local decoded image. Is calculated. Secondly, the filter coefficient calculated using the input image is applied to the local decoded image, and the filter coefficient calculated using the local decoded image is applied to the local decoded image. Third, for each predetermined unit area, the filter processing is performed on the local decoded image using either the filter coefficient calculated using the input image or the filter coefficient calculated using the local decoded image. Determine whether to do it. Specifically, for each unit region, first, a degradation scale from the input image of the image obtained by applying the filter coefficient calculated using the input image, and application of the filter coefficient calculated using the local decoded image. A deterioration scale from the input image of the obtained image is obtained. Next, for each unit region, the local decoded image is calculated using either the filter coefficient calculated using the input image or the filter coefficient calculated using the local decoded image based on the obtained degradation scale. To determine whether to perform filtering. Fourth, the filter processing is performed on the local decoded image using the filter coefficient calculated using the input image and the filter coefficient calculated using the local decoded image that has been determined to be used as described above. Do. Fifth, encode the determination result used above. Sixth, among the filter coefficients calculated using the input image, those determined to be used are encoded. According to the above, an effect similar to the effect described above can be achieved.

  (13) The present invention permits adaptive filter processing in a decoding processing loop, controls filter processing for each filter processing unit block, and decodes encoded data generated in the moving image encoding method of (12). The encoded data generated in the seventh step and the eighth step (for example, the encoded data c relating to the suitability information in FIG. 5 and the encoded data relating to the filter coefficient in FIG. 5). And a ninth step (for example, corresponding to the processing by the filter coefficient decoding unit 131 and the suitability information decoding unit 133 in FIG. 5), and a decoded image (for example, corresponding to the decoded image E in FIG. 5). The tenth step of calculating the filter coefficient based on the local pattern similarity (for example, the processing by the filter coefficient calculation unit 132 in FIG. 5). ), The filter coefficient decoded in the ninth step (for example, the NLM filter coefficient m1 obtained from the original image a in FIG. 5), and the filter coefficient calculated in the tenth step (for example, Among the NLM filter coefficients F obtained by the decoded image E in FIG. 5), and the one corresponding to the determination result for each unit area decoded in the ninth step is used for the decoded image. On the other hand, it proposes a moving picture decoding method characterized by comprising an eleventh step (for example, corresponding to the processing by the filter processing unit 134 in FIG. 5) for performing a filtering process.

  According to the present invention, first, the encoded data generated by the moving image encoding method of (12) is decoded, and the filter coefficient is calculated based on the local similarity of the pattern to the decoded image. Next, out of the filter coefficient obtained by decoding the encoded data and the calculated filter coefficient, the one according to the determination result for each unit region obtained by decoding the encoded data Is used to filter the decoded image. According to the above, an effect similar to the effect described above can be achieved.

  (14) The present invention is a program for causing a computer to execute a moving image encoding method that allows adaptive filter processing in an encoding processing loop and controls filter processing for each filter processing unit block. For an image (for example, corresponding to the original image a in FIG. 2), filter coefficients (for example, corresponding to the NLM filter coefficient m1 obtained from the original image a in FIG. 2) based on the local similarity of the pattern. For the first step (for example, equivalent to the processing by the first filter coefficient calculation unit 51 in FIG. 2) and the local decoded image (for example, equivalent to the local decoded image k in FIG. 2), A second step of calculating a filter coefficient (for example, equivalent to the NLM filter coefficient m2 obtained from the local decoded image k in FIG. 2) based on the similarity of various pictures 2 and the third step of applying the filter coefficient calculated in the first step to the local decoded image (for example, the first filter coefficient calculation unit 52 in FIG. 2). And a fourth step of applying the filter coefficient calculated in the second step to the locally decoded image (for example, by the second filter coefficient application unit 62 of FIG. 2). The filter coefficient calculated in the first step for each predetermined unit region (for example, a unit region composed of N × N pixels described later), and the second step. The fifth step of determining which of the filter coefficients calculated in step (3) is used to perform the filter processing on the locally decoded image (for example, the filter coefficient suitable in FIG. Among the filter coefficients calculated in the first step and the filter coefficients calculated in the second step, and used for each unit region in the fifth step. Using the determination, a sixth step (for example, corresponding to the processing by the filter processing unit 64 in FIG. 2) that performs filtering on the locally decoded image, and a determination result in the fifth step (for example, Among the filter coefficients calculated in the seventh step (equivalent to the processing by the suitability information encoding unit 66 in FIG. 2) and the filter coefficient calculated in the first step are encoded. An eighth step (for example, corresponding to the processing by the filter coefficient encoding unit 65 in FIG. 2) for encoding what is determined to be used in step 5; In the fifth step, for each unit region, the degradation scale from the input image of the image obtained in the third step and the input of the image obtained in the fourth step And a filter coefficient calculated in the first step and a filter coefficient calculated in the second step based on the deterioration scale for each unit region. A program for determining whether to perform filter processing on the locally decoded image using the above is proposed.

  According to the present invention, by causing a computer to execute a program, first, a filter coefficient is calculated based on a local pattern similarity with respect to an input image, and a local coefficient with respect to a local decoded image is calculated. A filter coefficient is calculated based on the similarity of the patterns. Secondly, the filter coefficient calculated using the input image is applied to the local decoded image, and the filter coefficient calculated using the local decoded image is applied to the local decoded image. Third, for each predetermined unit area, the filter processing is performed on the local decoded image using either the filter coefficient calculated using the input image or the filter coefficient calculated using the local decoded image. Determine whether to do it. Specifically, for each unit region, first, a degradation scale from the input image of the image obtained by applying the filter coefficient calculated using the input image, and application of the filter coefficient calculated using the local decoded image. A deterioration scale from the input image of the obtained image is obtained. Next, for each unit region, the local decoded image is calculated using either the filter coefficient calculated using the input image or the filter coefficient calculated using the local decoded image based on the obtained degradation scale. To determine whether to perform filtering. Fourth, the filter processing is performed on the local decoded image using the filter coefficient calculated using the input image and the filter coefficient calculated using the local decoded image that has been determined to be used as described above. Do. Fifth, encode the determination result used above. Sixth, among the filter coefficients calculated using the input image, those determined to be used are encoded. According to the above, an effect similar to the effect described above can be achieved.

  (15) The present invention allows the adaptive filter processing in the decoding processing loop, controls the filter processing for each filter processing unit block, and generates encoded data generated by causing the computer to execute the program of (14). Is a program for causing a computer to execute the moving picture decoding method, and the encoded data generated in the seventh step and the eighth step (for example, the encoded data c relating to suitability information in FIG. 5) And a ninth step (for example, corresponding to processing by the filter coefficient decoding unit 131 and the suitability information decoding unit 133 in FIG. 5), and a decoded image (corresponding to the encoded data related to the filter coefficient in FIG. 5) For example, for the decoded image E in FIG. 5), a filter coefficient is calculated based on the local pattern similarity. 0 step (for example, equivalent to the processing by the filter coefficient calculation unit 132 in FIG. 5) and the filter coefficient decoded in the ninth step (for example, equivalent to the NLM filter coefficient m1 obtained from the original image a in FIG. 5) ) And the filter coefficients calculated in the tenth step (e.g., corresponding to the NLM filter coefficient F obtained from the decoded image E in FIG. 5), for each unit region decoded in the ninth step For causing the computer to execute an eleventh step (e.g., corresponding to the processing by the filter processing unit 134 in FIG. 5) of performing a filtering process on the decoded image using the one according to the determination result of Propose a program.

  According to the present invention, by causing the computer to execute the program, first, the encoded data generated by causing the computer to execute the program of (14) is decoded, and the local pattern similarity to the decoded image. The filter coefficient is calculated based on the characteristics. Next, out of the filter coefficient obtained by decoding the encoded data and the calculated filter coefficient, the one according to the determination result for each unit region obtained by decoding the encoded data Is used to filter the decoded image. According to the above, an effect similar to the effect described above can be achieved.

  According to the present invention, it is possible to accurately improve the subjective image quality. Further, it is possible to reduce the amount of information to be transmitted while ensuring the realization of decoding by the moving image decoding apparatus.

It is a block diagram which shows the structure of the moving image encoder which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the NLM filter coefficient calculation part with which the said moving image encoder is provided, and an NLM filter process part. It is a figure for demonstrating the calculation method of the NLM filter coefficient by the said NLM filter coefficient calculation part. It is a block diagram which shows the structure of the moving image decoding apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the NLM filter process part with which the said moving image decoding apparatus is provided.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements and the like, and various variations including combinations with other existing constituent elements are possible. Accordingly, the description of the following embodiments does not limit the contents of the invention described in the claims.

[Configuration and Operation of Video Encoding Device AA]
FIG. 1 is a block diagram showing a configuration of a moving picture coding apparatus AA according to an embodiment of the present invention. The moving image encoding device AA includes a prediction value generation unit 1, a DCT / quantization unit 2, an entropy encoding unit 3, an inverse DCT / inverse quantization unit 4, an NLM filter coefficient calculation unit 5, an NLM filter processing unit 6, and A local memory 7 is provided, and encoding is performed for each encoding control unit (macroblock; MB).

  The predicted value generation unit 1 receives an original image a as an input image and a filtered local decoded image e described later that is output from the local memory 7. The predicted value generation unit 1 generates a predicted value using a prediction method such as intra prediction or inter prediction. Then, the prediction value generated using the prediction method that is expected to have the highest coding performance is output as the prediction value f.

  The DCT / quantization unit 2 inputs a difference between the original image a and the predicted value f. The DCT / quantization unit 2 performs DCT processing and quantization processing on the input signal, and outputs the result as a quantized DCT coefficient (residual signal) g.

  The entropy encoding unit 3 receives the quantized DCT coefficient (residual signal) g as an input. The entropy encoding unit 3 performs entropy encoding on the input signal and outputs it as encoded data b.

  The inverse DCT / inverse quantization unit 4 receives the quantized DCT coefficient (residual signal) g as an input. The inverse DCT / inverse quantization unit 4 performs an inverse quantization process and an inverse DCT process on the input signal and outputs it as a pixel signal h subjected to the inverse DCT.

  The local memory 7 receives a filtered local decoded image e, which will be described later, output from the NLM filter processing unit 6. The local memory 7 accumulates the filtered local decoded image e and outputs it to the predicted value generation unit 1 as appropriate.

  The NLM filter coefficient calculation unit 5 receives an original image a and a local decoded image k as a locally decoded image. The NLM filter coefficient calculation unit 5 calculates the NLM filter coefficient for all pixels of the original image a and calculates the NLM filter coefficient m1 obtained from the original image a, as will be described in detail later with reference to FIG. And NLM filter coefficients for all the pixels of the local decoded image k are calculated and output as NLM filter coefficients m2 obtained from the local decoded image k. Note that the local decoded image k is a sum of the predicted value f and the inverse DCT pixel signal h.

  The NLM filter processing unit 6 receives the original image a, the local decoded image k, the NLM filter coefficient m1 obtained from the original image a, and the NLM filter coefficient m2 obtained from the local decoded image k. The NLM filter processing unit 6, which will be described in detail later with reference to FIG. 2, determines for each unit region which of the NLM filter coefficients m 1 and m 2 should be used, and determines the NLM filter coefficient according to the determination result. Using this, the local decoded image k is filtered. Then, the filtered local decoded image is output as a filtered local decoded image e, and encoded data c related to suitability information described later and encoded data d related to filter coefficients described later are output.

  FIG. 2 is a block diagram showing the configuration of the NLM filter coefficient calculation unit 5 and the NLM filter processing unit 6.

[Configuration and Operation of NLM Filter Coefficient Calculation Unit 5]
The NLM filter coefficient calculation unit 5 includes a first filter coefficient calculation unit 51 and a second filter coefficient calculation unit 52.

  The first filter coefficient calculation unit 51 receives the original image a. The first filter coefficient calculation unit 51 calculates an NLM filter coefficient for each pixel constituting the original image a and outputs it as an NLM filter coefficient m1 obtained from the original image a. The NLM filter coefficient is a type of filter coefficient based on local pattern similarity. A method for calculating the NLM filter coefficient m1 obtained from the original image a will be described below with reference to FIG.

  In the NLM filter, filter processing is performed on L × L pixels (L is an arbitrary integer satisfying L ≧ 2) in the vicinity of the pixel i to be filtered. For each of the filter processing target pixel i and the filter coefficient calculation pixel j, the similarity between the neighboring M × M pixel areas (M is an arbitrary integer satisfying 1 ≦ M ≦ L) is evaluated. As the similarity is higher, a larger filter coefficient is set. Specifically, when the input original image a is expressed as in the following expression (1), the filter coefficient w (i, j) in the filter coefficient calculation pixel j for the filter processing target pixel i is as follows: It can be obtained from equation (2).

In Expression (2), N _i represents a region of M × M pixels in the vicinity of the filter target pixel i, and N _j represents M × M pixels in the vicinity of the filter coefficient calculation pixel j. An area shall be indicated. Also, v (N _i ) represents the lightness at the gray level for N _i , and v (N _j ) represents the lightness at the gray level for N _j . Further, h represents the strength of the filter. Also, z (i) can be obtained by the following equation (3).

  Returning to FIG. 2, the second filter coefficient calculation unit 52 receives the local decoded image k as an input. The second filter coefficient calculation unit 52 calculates an NLM filter coefficient for each pixel constituting the local decoded image k by a method similar to the method described above with reference to FIG. The obtained NLM filter coefficient m2 is output.

[Configuration and Operation of NLM Filter Processing Unit 6]
The NLM filter processing unit 6 includes a first filter coefficient application unit 61, a second filter coefficient application unit 62, a filter coefficient suitability determination unit 63, a filter processing unit 64, a filter coefficient coding unit 65, and a suitability information coding unit. 66.

  The first filter coefficient application unit 61 receives the local decoded image k and the NLM filter coefficient m1 obtained from the original image a as inputs. The first filter coefficient application unit 61 applies the NLM filter coefficient m1 obtained from the original image a to the local decoded image k to obtain a pixel value NL [v] ( i) is calculated and output as an application result p of the NLM filter coefficient m1 obtained from the original image a.

  The second filter coefficient application unit 62 receives the local decoded image k and the NLM filter coefficient m2 obtained from the local decoded image k as inputs. The second filter coefficient application unit 62 applies the NLM filter coefficient m2 obtained from the local decoded image k to the local decoded image k, and outputs the pixel value NL [v shown in the above equation (4). ] (I) is calculated and output as an application result q of the NLM filter coefficient m2 obtained from the local decoded image k.

  The filter coefficient suitability determination unit 63 inputs the original image a, the application result p of the NLM filter coefficient m1 obtained from the original image a, and the application result q of the NLM filter coefficient m2 obtained from the local decoded image k. And The filter coefficient suitability determination unit 63 determines which of the NLM filter coefficient m1 and the NLM filter coefficient m2 is used for each unit region composed of N × N pixels (N is an arbitrary integer satisfying N ≧ 1). Judgment is made and the judgment result is output as suitability information r.

  Specifically, first, for each unit region composed of N × N pixels, a degradation scale from the original image a when the NLM filter coefficient m1 is applied is obtained using the application result p, and the NLM filter coefficient A degradation scale from the original image a when m2 is applied is obtained using the application result q. Note that the degradation scale from the original image a when the NLM filter coefficient m1 is applied is expressed by a larger numerical value as the degree of degradation with respect to the original image a when the NLM filter coefficient m1 is applied becomes larger. . Further, the degradation scale from the original image a when the NLM filter coefficient m2 is applied is represented by a larger numerical value as the degree of degradation with respect to the original image a when the NLM filter coefficient m2 is applied increases. .

  Next, for each unit region composed of N × N pixels, a difference value obtained by subtracting the degradation scale when the NLM filter coefficient m1 is applied from the degradation scale when the NLM filter coefficient m2 is applied is determined in advance. The threshold value TH is compared. Then, it is determined that the NLM filter coefficient m1 obtained from the original image a is used for the unit region whose difference value is equal to or greater than the threshold value TH. On the other hand, it is determined that the NLM filter coefficient m2 obtained from the local decoded image k is used for the unit area whose difference value is less than the threshold value TH.

  The filter coefficient suitability determination unit 63 uses, as the above-described two deterioration measures, for example, one that can evaluate the deterioration degree from the original image and the encoded image, such as MSE (Mean Square Error) and SSIM (Structural SIMularity). And

  The filter processing unit 64 receives the local decoded image k, the NLM filter coefficient m1 obtained from the original image a, the NLM filter coefficient m2 obtained from the local decoded image k, and the suitability information r. The filter processing unit 64 uses the NLM filter coefficient m1 and the NLM filter coefficient m2 that are determined to be used for each unit area by the filter coefficient suitability determination unit 63 based on the suitability information r, and uses the local decoded image. NLM filter processing is performed on k and output as a filtered local decoded image e.

  The filter coefficient encoding unit 65 receives the NLM filter coefficient m1 obtained from the original image a and the suitability information r. The filter coefficient encoding unit 65 identifies the unit area determined by the filter coefficient suitability determining unit 63 to use the NLM filter coefficient m1 based on the suitability information r, and determines the NLM filter coefficient m1 in each identified unit area. It encodes and outputs as the encoding data d regarding a filter coefficient.

  Here, there are L × L NLM filter coefficients m1 in each unit region for each N × N pixels. Therefore, in the encoding of the NLM filter coefficient m1 in each unit area described above, first, the NLM filter coefficient m1 in each unit area is converted into a frequency domain by orthogonal transform as a four-dimensional signal of N × N × L × L. Then, a one-dimensional signal is generated in accordance with the scan order from the low frequency component to the high frequency component. Next, the one-dimensional signal is quantized, encoded by arithmetic encoding or variable length encoding, and output as encoded data d related to the filter coefficient. In the quantization, it is allowed that the quantization width differs for each frequency component.

  The suitability information encoding unit 66 receives the suitability information r. This suitability information encoding unit 66 expresses and expresses suitability information r as 1-bit flag information that is two-dimensional information, and outputs it as encoded data c relating to suitability information.

  When the above-described suitability information r is expressed as 1-bit flag information that is two-dimensional information, a unit region to which the NLM filter coefficient m1 is applied is represented by “0”, and a unit region to which the NLM filter coefficient m2 is applied is represented. It is represented by “1”. However, the unit region to which the NLM filter coefficient m1 is applied may be represented by “1”, and the unit region to which the NLM filter coefficient m2 is applied may be represented by “0”.

  In the encoding of the above-described suitability information r expressed as flag information, if the frame size of the original image a is W × H, (W / N) × (H / N) flag information per frame Encode by arithmetic coding or variable length coding. As arithmetic coding, for example, CABAC can be used, and as variable length coding, for example, run length coding can be used. Further, (W / N) × (H / N) pieces of flag information can be expressed as a two-dimensional vector or a hierarchical structure by quadtree representation.

[Configuration and Operation of Video Decoding Device BB]
FIG. 4 is a block diagram showing a configuration of the video decoding device BB according to one embodiment of the present invention. The video decoding device BB includes an entropy decoding unit 110, a predicted value generation unit 120, an NLM filter processing unit 130, and a memory 140, and has been decoded by decoding the encoded data b generated by the video encoding device AA Image A is generated.

  The entropy decoding unit 110 receives the encoded data b. The entropy decoding unit 110 analyzes the content described in the encoded data b according to the encoded data structure, performs entropy decoding, and outputs prediction information B and residual signal C obtained as a result of entropy decoding.

  The prediction value generation unit 120 receives the prediction information B and the decoded image A output from the memory 140 as inputs. The prediction value generation unit 120 determines a prediction method based on the prediction information B, and generates and outputs a prediction value D using the decoded image A according to the determined method.

  The NLM filter processing unit 130 receives as input the encoded data c related to suitability information, the encoded data d related to the filter coefficient, and the decoded image E. Here, the decoded image E is obtained by adding the residual signal C and the predicted value D. Although details will be described later with reference to FIG. 5, the NLM filter processing unit 130 determines the filter coefficient suitability determination unit 63 of the moving image coding apparatus AA for each unit region out of the NLM filter coefficient m1 and the NLM filter coefficient m2. Then, the decoded image E is subjected to filter processing and output as a decoded image A.

  The memory 140 receives the decoded image A output from the NLM filter processing unit 130 as input. The memory 140 accumulates the decoded image A and outputs it to the predicted value generation unit 120 as appropriate.

  FIG. 5 is a block diagram illustrating a configuration of the NLM filter processing unit 130.

[Configuration and Operation of NLM Filter Processing Unit 130]
The NLM filter processing unit 130 includes a filter coefficient decoding unit 131, a filter coefficient calculation unit 132, a suitability information decoding unit 133, and a filter processing unit 134.

  The filter coefficient decoding unit 131 receives the encoded data d related to the filter coefficient. The filter coefficient decoding unit 131 performs arithmetic decoding or variable length decoding on the encoded data d related to the filter coefficient in accordance with the encoding method used in the moving image encoding apparatus AA, and performs inverse quantization and inverse orthogonal transform. N × N × L × L filter coefficient information is obtained for each unit area, and the NLM filter coefficient m1 obtained from the original image a is obtained. Then, the NLM filter coefficient m1 obtained from the original image a is output.

  The filter coefficient calculation unit 132 receives the decoded image E as an input. The filter coefficient calculation unit 132 calculates NLM filter coefficients for all the pixels of the decoded image E by a method similar to the method described above with reference to FIG. 3 and obtains the NLM filter coefficients obtained from the decoded image E. Output as F.

  The suitability information decoding unit 133 receives the encoded data c related to suitability information. The suitability information decoding unit 133 performs arithmetic decoding or variable length decoding on the coded data c related to suitability information according to the coding method used in the moving picture coding apparatus AA to obtain the above suitability information r. Then, the suitability information r is output.

  The filter processing unit 134 receives the decoded image E, the filter coefficient F obtained from the decoded image E, the NLM filter coefficient m1 obtained from the original image a, and the suitability information r. First, based on the suitability information r, the filter processing unit 134 includes a unit region in which the NLM filter coefficient m1 is used in the filter processing unit 64, a unit region in which the NLM filter coefficient m2 is used in the filter processing unit 64, and Is determined. Next, in the decoded image E, a unit region using the NLM filter coefficient m1 is subjected to NLM filter processing using the NLM filter coefficient m1, and a unit region using the NLM filter coefficient m2 is decoded image. NLM filter processing is performed using the NLM filter coefficient F obtained by E, and a decoded image A is output.

  The moving image encoding apparatus AA described above calculates NLM filter coefficients m1 and m2, and performs filter processing using these. For this reason, filter processing can be performed using an NLM filter, so that subjective image quality can be improved.

  In addition, the moving image encoding apparatus AA applies to a unit region in which a difference value obtained by subtracting a degradation scale when the NLM filter coefficient m1 is applied from a degradation scale when the NLM filter coefficient m2 is applied is greater than or equal to a threshold value TH. Determines that the NLM filter coefficient m1 obtained from the original image a is used. On the other hand, it is determined that the NLM filter coefficient m2 obtained from the local decoded image k is used for the unit area whose difference value is less than the threshold value TH. For this reason, basically, the filtering process is performed using the NLM filter coefficient m2, but the degradation using the NLM filter coefficient m1 is less likely to be caused by the threshold than the case where the NLM filter coefficient m2 is used. When the level can be suppressed to a level determined by TH, the NLM filter coefficient m1 is used. Accordingly, sufficient image quality can be obtained even in the case where a local pattern such as an edge component, which is a visually important pattern, is greatly deteriorated, so that the above-described improvement in subjective image quality can be realized accurately.

  In addition, the moving image encoding apparatus AA encodes only the NLM filter coefficient m1 calculated by the method described above with reference to FIG. It is transmitted to the video decoding device BB. Here, the NLM filter coefficient m2 can be calculated as the NLM filter coefficient F by the video decoding device BB. For this reason, it is possible to reduce the amount of information to be transmitted while ensuring the realization of filter processing in the video decoding device BB.

  Also, in the moving image encoding device AA, the appropriateness information encoding unit 66 represents the appropriateness information r as 1-bit flag information that is two-dimensional information. Therefore, it is possible to indicate, for each unit region, which of the NLM filter coefficients m1 and m2 is used for the filtering process while suppressing an increase in the amount of information transmitted to the video decoding device BB.

  In addition, the moving image coding apparatus AA performs orthogonal transform of the NLM filter coefficient m1 by the filter coefficient coding unit 65. Here, as the inter-pixel correlation of the N × N pixels constituting the unit region increases, the energy can be significantly compressed. For this reason, the amount of information transmitted to the video decoding device BB can be further reduced by quantizing and encoding the result of the orthogonal transformation.

  In addition, the above moving picture decoding apparatus BB performs a filtering process using the NLM filter coefficient m1 for the unit area in which the NLM filter coefficient m1 is used in the moving picture encoding apparatus AA. On the other hand, the NLM filter coefficient F corresponding to the NLM filter coefficient m2 is calculated for the unit area in which the NLM filter coefficient m2 is used in the moving picture coding apparatus AA, and the filter processing is performed using the NLM filter coefficient F. I do. Therefore, sufficient image quality can be obtained even in the case where a local pattern such as an edge component, which is a visually important pattern, is greatly deteriorated, so that the above-described improvement in subjective image quality can be realized accurately.

  The processing of the moving image encoding device AA and the processing of the moving image decoding device BB of the present invention is stored in a computer-readable recording medium, and the program recorded on the recording medium is stored in the moving image encoding device AA or moving image. The present invention can be realized by causing the image decoding device BB to read and execute the image decoding device BB.

  In addition, the above-described program is transferred from the video encoding device AA or the video decoding device BB storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. May be transmitted. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  Further, the above-described program may be for realizing a part of the above-described function. Furthermore, what can implement | achieve the above-mentioned function in combination with the program already recorded on the moving image encoder AA and the moving image decoder BB, what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design that does not depart from the gist of the present invention.

  For example, in the above-described embodiment, an NLM filter is used as a filter based on local pattern similarity. However, the present invention is not limited to this, and a filter having properties similar to the NLM filter, such as Total Variation, A Bilatal Filter disclosed in Patent Document 5, a SUSAN Filter illustrated in Non-Patent Document 6 described above, or the like may be used.

  Further, in the above-described embodiment, the suitability information encoding unit 66 expresses flag information of suitability information indicating whether to use the NLM filter coefficient m1 or the NLM filter coefficient m2 for each unit region using two-dimensional information. However, the present invention is not limited to this, and may be expressed using, for example, a multistage layer structure.

AA: Moving picture encoding device 1: Prediction value generation unit 2 ... DCT / quantization unit 3 ... Entropy encoding unit 4 ... Inverse DCT / inverse quantization unit 5 ... NLM Filter coefficient calculation unit 51... First filter coefficient calculation unit 52... Second filter coefficient calculation unit 6... NLM filter processing unit 61. 2 filter coefficient application unit 63 ... filter coefficient suitability determination unit 64 ... filter processing unit 65 ... filter coefficient encoding unit 66 ... suitability information encoding unit 7 ... local memory BB ... Moving picture decoding apparatus 110 ... entropy decoding unit 120 ... predicted value generation unit 130 ... NLM filter processing unit 131 ... filter coefficient decoding unit 132 ... filter coefficient calculation unit 133 ... suitability information decoding Part 134: Filter processing unit 140: Memory

Claims (15)

A moving image encoding device that allows adaptive filter processing in an encoding processing loop and controls filter processing for each filter processing unit block,
First filter coefficient calculating means for calculating a filter coefficient based on local pattern similarity with respect to an input image;
A second filter coefficient calculating means for calculating a filter coefficient based on the local pattern similarity with respect to a local decoded image;
First filter coefficient applying means for applying the filter coefficient calculated by the first filter coefficient calculating means to the local decoded image;
Second filter coefficient application means for applying the filter coefficient calculated by the second filter coefficient calculation means to the local decoded image;
For each predetermined unit region, the local decoded image is used by using either the filter coefficient calculated by the first filter coefficient calculating unit or the filter coefficient calculated by the second filter coefficient calculating unit. Filter coefficient suitability determining means for determining whether to perform filtering on
Of the filter coefficient calculated by the first filter coefficient calculating means and the filter coefficient calculated by the second filter coefficient calculating means, the filter coefficient suitability determining means determines to use each unit region. Filter processing means for performing filter processing on the locally decoded image using
Suitability information encoding means for encoding the determination result by the filter coefficient suitability determination means;
A filter coefficient encoding unit that encodes a filter coefficient calculated by the first filter coefficient calculation unit and determined to be used by the filter coefficient suitability determination unit;
The filter coefficient suitability determining means includes
For each unit region, a degradation scale from the input image of the image obtained by the first filter coefficient application means, a degradation scale from the input image of the image obtained by the second filter coefficient application means, As well as
For each unit region, using either the filter coefficient calculated by the first filter coefficient calculation unit or the filter coefficient calculated by the second filter coefficient calculation unit based on the degradation scale A moving picture coding apparatus that judges whether to perform a filtering process on the locally decoded picture. The moving image encoding device according to claim 1,
The deterioration scale from the input image of the image obtained by the first filter coefficient applying unit is expressed by a large numerical value as the degree of deterioration of the image with respect to the input image increases.
The degradation scale from the input image of the image obtained by the second filter coefficient applying unit is expressed by a large numerical value as the degree of degradation of the image with respect to the input image increases.
The filter coefficient suitability determining means includes
A unit in which a difference value obtained by subtracting the deterioration scale for the image obtained by the first filter coefficient applying means from the deterioration scale for the image obtained by the second filter coefficient applying means is equal to or greater than a predetermined threshold value. For the region, it is determined that the filter coefficient calculated by the first filter coefficient calculation unit is used,
A moving picture encoding apparatus, wherein it is determined that a filter coefficient calculated by the second filter coefficient calculation means is used for a unit region whose difference value is less than the threshold value. The moving picture encoding apparatus according to claim 1 or 2,
The filter coefficient suitability determining means includes
As a degradation scale from the input image of the image obtained by the first filter coefficient applying means, Mean Square Error or Structural Similarity of the input image and the image obtained by the first filter coefficient applying means is used. ,
The Mean Square Error and the Structural Similarity of the input image and the image obtained by the second filter coefficient application unit as a degradation measure from the input image of the image obtained by the second filter coefficient application unit. A moving picture coding apparatus using the image obtained by the first filter coefficient applying means as a measure of deterioration from the input image. In the moving image encoder according to any one of claims 1 to 3,
The suitability information encoding unit expresses a determination result by the filter coefficient suitability determination unit as flag information, and encodes the flag information using arithmetic encoding or variable length encoding. Encoding device. In the moving image encoding device according to claim 4,
The moving picture coding apparatus characterized in that the suitability information coding means represents the flag information using two-dimensional information or a multistage layer structure. In the moving image encoder according to any one of claims 1 to 5,
The filter coefficient encoding means includes
Filter coefficient information orthogonal transform means for orthogonally transforming L × L filter coefficients for each of N × N pixels constituting the unit area as a four-dimensional signal of N × N × L × L;
A moving picture coding apparatus comprising: transform coefficient coding means for quantizing and coding the result of the orthogonal transform by the filter coefficient information orthogonal transform means by arithmetic coding or variable length coding . In the moving image encoder according to any one of claims 1 to 6,
A moving image encoding apparatus, wherein a filter based on Non Local Means is used for filter processing to be controlled for each filter processing unit block. 8. The encoded data generated in the moving picture encoding apparatus according to claim 1 is decoded by permitting adaptive filter processing in a decoding processing loop and controlling the filter processing for each filter processing unit block. A moving picture decoding device,
Filter information decoding means for decoding encoded data generated by the suitability information encoding means and the filter coefficient encoding means;
A decoded image filter coefficient calculating means for calculating a filter coefficient based on the local pattern similarity with respect to the decoded image;
Of the filter coefficients decoded by the filter information decoding means and the filter coefficients calculated by the decoded image filter coefficient calculation means, depending on the determination result for each unit area decoded by the filter information decoding means And a decoded image filter processing means for performing a filtering process on the decoded image using a video image decoding apparatus. The moving picture decoding apparatus according to claim 8, wherein
The filter information decoding means decodes the encoded data generated by the suitability information encoding means using arithmetic coding or variable length coding, and the filter coefficient suitability judging means expressed as flag information. A moving picture decoding apparatus characterized by obtaining a determination result. The moving picture decoding apparatus according to claim 9, wherein
The moving picture decoding apparatus, wherein the flag information is expressed using two-dimensional information or a multistage layer structure. The video decoding device according to any one of claims 8 to 10,
Filter information decoding means
Transform coefficient decoding means for decoding encoded data generated by the filter coefficient encoding means by arithmetic decoding or variable length decoding;
The result of decoding by the transform coefficient decoding means is inversely quantized and inverse orthogonal transformed as an N × N × L × L four-dimensional signal to obtain L for each of N × N pixels constituting the unit region. And a filter coefficient information inverse orthogonal transform means for obtaining L filter coefficients. A moving image encoding method that allows adaptive filter processing within an encoding processing loop and controls filter processing for each filter processing unit block,
A first step of calculating filter coefficients based on local pattern similarity with respect to an input image;
A second step of calculating a filter coefficient based on the local pattern similarity with respect to a locally decoded image;
A third step of applying the filter coefficient calculated in the first step to the locally decoded image;
A fourth step of applying the filter coefficient calculated in the second step to the locally decoded image;
For each predetermined unit area, which of the filter coefficient calculated in the first step and the filter coefficient calculated in the second step is used to perform the filter processing on the locally decoded image A fifth step of determining
Of the filter coefficients calculated in the first step and the filter coefficients calculated in the second step, the local decoding is performed using the filter coefficient determined to be used for each unit region in the fifth step. A sixth step of filtering the image;
A seventh step of encoding the determination result in the fifth step;
An eighth step of encoding the filter coefficient calculated in the first step that is determined to be used in the fifth step, and
In the fifth step,
For each unit region, obtain a degradation scale from the input image of the image obtained in the third step and a degradation scale from the input image of the image obtained in the fourth step;
For each unit region, a filter is applied to the locally decoded image using either the filter coefficient calculated in the first step or the filter coefficient calculated in the second step based on the degradation scale. A moving picture coding method characterized by determining whether to perform processing. 13. A moving picture decoding method for decoding coded data generated in the moving picture coding method according to claim 12, wherein adaptive filtering is allowed in a decoding processing loop, and the filtering is controlled for each filter processing unit block. Because
A ninth step of decoding the encoded data generated in the seventh step and the eighth step;
A tenth step of calculating a filter coefficient based on the local pattern similarity with respect to a decoded image;
Of the filter coefficients decoded in the ninth step and the filter coefficients calculated in the tenth step, the one corresponding to the determination result for each unit region decoded in the ninth step is used. And an eleventh step of performing a filtering process on the decoded image. A program for causing a computer to execute a moving image encoding method that allows adaptive filter processing within an encoding processing loop and controls filter processing for each filter processing unit block,
A first step of calculating filter coefficients based on local pattern similarity with respect to an input image;
A second step of calculating a filter coefficient based on the local pattern similarity with respect to a locally decoded image;
A third step of applying the filter coefficient calculated in the first step to the locally decoded image;
A fourth step of applying the filter coefficient calculated in the second step to the locally decoded image;
For each predetermined unit area, which of the filter coefficient calculated in the first step and the filter coefficient calculated in the second step is used to perform the filter processing on the locally decoded image A fifth step of determining
Of the filter coefficients calculated in the first step and the filter coefficients calculated in the second step, the local decoding is performed using the filter coefficient determined to be used for each unit region in the fifth step. A sixth step of filtering the image;
A seventh step of encoding the determination result in the fifth step;
An eighth step of encoding the filter coefficient calculated in the first step that is determined to be used in the fifth step;
In the fifth step,
For each unit region, obtain a degradation scale from the input image of the image obtained in the third step and a degradation scale from the input image of the image obtained in the fourth step;
For each unit region, a filter is applied to the locally decoded image using either the filter coefficient calculated in the first step or the filter coefficient calculated in the second step based on the degradation scale. A program for determining whether to perform processing. 15. A moving image that allows adaptive filter processing in a decoding processing loop, controls filter processing for each filter processing unit block, and decodes encoded data generated by causing a computer to execute the program according to claim 14. A program for causing a computer to execute a decryption method,
A ninth step of decoding the encoded data generated in the seventh step and the eighth step;
A tenth step of calculating a filter coefficient based on the local pattern similarity with respect to a decoded image;
Of the filter coefficients decoded in the ninth step and the filter coefficients calculated in the tenth step, the one corresponding to the determination result for each unit region decoded in the ninth step is used. A program for causing a computer to execute an eleventh step of performing a filtering process on the decoded image.

Images