JP5868157B2 - Image processing method / device, video encoding method / device, video decoding method / device, and program thereof - Google Patents

Description

  The present invention relates to an image processing technique for reducing noise during image capturing and deterioration of a deteriorated image when an image is encoded.

Each word used in this specification is defined as follows.
“Search shape”: a set of search points around the pixel to be subjected to template matching, and a shape formed by the set.
“Template shape”: A pixel group used when calculating the similarity between the pixel and each search point in template matching, and a shape formed by the pixel group. The pixel group uses the same shape around the pixel and around each search point, and compares pixel values at the same relative positional relationship.

  In the field of image processing, various noise removal filters have been proposed as a technique for reducing noise at the time of image capturing and degradation of degraded images. Among them, it is known that a noise removal filter based on the non-local means method (see Non-Patent Document 1) exhibits a high noise removal effect. The noise removal filter based on the non-local means method is hereinafter referred to as an NLM filter.

FIG. 15 is an explanatory diagram of an NLM filter. In FIG. 15, one square square is a search point, and an aggregate of these search points is a search shape. P _o is a denoising target pixel, and P _s is a pixel at a search point at the search destination. T _o and T _s is the template shape, the shape of the comparison source template shape T _o and the search target template shape T _s are the same.

The NLM filter, comparison source and the search target of the template shape T _o, by comparing the corresponding pixels to each other in T _s, and calculates the template similarity. The template similarity is generally calculated using SSD (Sum of Square Difference) or SAD (Sum of Absolute Difference).

The NLM filter calculates a pixel value after denoising for each pixel as follows. Below, the example which used SSD for calculation of a template similarity is demonstrated.
(1) The variable SW of total weight values is initialized to 0, and the variable SP of total pixel values is initialized to 0.
(2) The following processing is repeated for each search point in the search shape.
(2-1) The SSD is calculated as the template similarity.
(2-2) Weight value W = exp (−SSD / denoising coefficient)
(2-3) Weight value total SW = weight value total SW + weight value W
(2-4) Pixel value total SP = pixel value total SP + weight value W × (pixel value of search point)
(3) When the processing of (2) is completed for all search points in the search shape, the pixel value after denoising of the denoising target pixel is obtained by the following equation.

(Pixel value after denoising) = pixel value total SP / weight value total SW
In addition, “HM”, which is a model (Test Model) of “High Efficiency Video Coding” of the next generation video coding standard system currently being internationally standardized in “MPEG” and “VCEG”, is in-loop. The filter is equipped with three types of filters for the purpose of removing coding distortion (see Non-Patent Document 2). For the purpose of increasing the coding efficiency of this HM in-loop filter, NLM The introduction of a filter has been proposed (see Non-Patent Documents 3 and 4).

[Configuration of NLM Filter Processing Unit for Video Coding (Conventional Method)]
FIG. 16 is a diagram illustrating a configuration example of an NLM filter processing unit according to a conventional method in a video encoding device. The NLM filter processing unit 500 includes an optimal denoising coefficient calculation unit 501, a denoising coefficient encoding unit 502, and an NLM filter execution unit 503.

  The optimum denoising coefficient calculation unit 501 receives the encoding target image and the decoded image to be denoised, calculates a denoising coefficient that minimizes the RD cost between the encoding target image and the decoded image after filtering, and outputs the denoising coefficient. The coefficient is sent to the denoising coefficient encoding unit 502 and the NLM filter execution unit 503. The denoising coefficient encoding unit 502 encodes the denoising coefficient and outputs an in-loop filter overhead. The NLM filter execution unit 503 applies a filter using a predetermined template shape and outputs a decoded image after filtering.

[NLM filter processing procedure for video coding (conventional method)]
FIG. 17 is a process flowchart of the NLM filter processing unit 500 in the conventional video encoding apparatus shown in FIG. The input in this processing procedure is an encoding target image and a decoded image, and the output is a decoded image after filtering and an in-loop filter overhead. The function is to calculate / encode an optimal denoising coefficient and apply an NLM filter to the decoded image.
Step S501: Read an encoding target image and a decoded image.
Step S502: An optimum denoising coefficient is calculated.
Step S503: The denoising coefficient is encoded and the in-loop filter overhead is output.
Step S504: An NLM filter is applied to each pixel of the decoded image, and the decoded image after filtering is output.

[Configuration of NLM Filter Processing Unit in Video Decoding (Conventional Method)]
FIG. 18 is a diagram illustrating a configuration example of an NLM filter processing unit according to a conventional method in a video decoding device. The NLM filter processing unit 600 includes a denoising coefficient decoding unit 601 and an NLM filter execution unit 602.

  The denoising coefficient decoding unit 601 decodes the denoising coefficient from the in-loop filter overhead. The NLM filter execution unit 602 has the same function as the NLM filter execution unit 503 described with reference to FIG. 16, applies a filter to a decoded image to be denoised using a predetermined template shape, and performs decoding after filtering. Output an image.

[NLM filter processing procedure in video decoding (conventional method)]
FIG. 19 is a process flowchart of the NLM filter processing unit 600 in the conventional video decoding apparatus shown in FIG. The input in this processing procedure is the decoded image and the in-loop filter overhead, and the output is the decoded image after filtering. The function is to decode the denoising coefficient and apply an NLM filter to the decoded image.
Step S601: Read decoded image and in-loop filter overhead.
Step S602: Denoising coefficients are decoded.
Step S603: An NLM filter is applied to each pixel of the decoded image, and the decoded image after filtering is output.

A. Buades, B. Coll, and JM Morel, "A non-local algorithm for image denoising", Proc. IEEE Int. Conf. On Computer Vision and Pattern Recognition, vol. 2, pp. 60-65, June, 2005 . Benjamin Bross, Woo-Jin Han, Jens-Rainer Ohm, Gary J. Sullivan, Thomas Wiegand, "WD4: Working Draft 4 of High-Efficiency Video Coding (JCTVC-F803_d2)," ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 6th Meeting: Torino, IT, 14-22 July, 2011. Masaaki Matsumura, Yukihiro Bandoh, Seishi Takamura, Hirohisa Jozawa, "In-loop filter based on non-local means filter (JCTVC-E206)", ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 5th Meeting: Geneva, CH, 16-23 March, 2011. Masaaki Matsumura, Yukihiro Bandoh, Seishi Takamura, Hirohisa Jozawa, "Modifications of in-loop filter based on non-local means filter (JCTVC-F047)", ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG11 6th Meeting: Torino, IT, 14-22 July, 2011.

  As described above, if the NLM filter is introduced in “HM” of “High Efficiency Video Coding” of the next generation video coding standard system, improvement of coding efficiency can be expected.

  However, since the NLM filter performs template matching using the template shape at each search point of the search shape, it is necessary to refer to a wide range of pixels around the target pixel. When a function for referring to a wide range of pixels around the target pixel is implemented in hardware, more buffer memory is required, which may increase the circuit scale. Therefore, in the in-loop filter Sample Adaptive Offset (hereinafter referred to as SAO), various contrivances have been made, such as introducing a function that does not refer to pixels beyond the boundary of the Large Coding Unit (hereinafter referred to as LCU).

However, when the template shape of the NLM filter is reduced, there is a problem that the encoding efficiency is lowered as is apparent from FIG. FIG. 20 is a diagram illustrating a comparison of encoding efficiency of NLM filters according to template shapes. The comparison target software is HM4.0, and the experimental conditions are as follows.
・ Experimental conditions Encoding target sequence: MPEG standard test sequence
-Class B (1920 × 1080 5 types)
-Class C (832 × 480 4 types)
-Class D (416 x 240 4 types)
-Class E (1280 × 720 3 types)
Number of frames: First 30 frames of sequence Search shape: 5 × 5 block Template shape: 3 × 3 block (Blk: 9 pixels)
Diagonal cross (Crs: 5 pixels)
Single pixel (Pel: 1 pixel)
GOP structure: Low delay P
Quantization parameters: 22, 27, 32, 37
Coding efficiency calculation method: BD-rate (* The lower the value, the higher the efficiency)
According to the experimental results shown in FIG. 20, it can be confirmed that the coding efficiency is lowered according to the size of the template shape. For this reason, it is desirable to keep the shape of the template shape as large as possible.

  An object of the present invention is to solve the above problems and maintain the number of pixels constituting a template shape even when pixels constituting a template of an NLM filter exceed a specified boundary.

  In order to solve the above-described problem, the present invention provides a case where the pixels constituting the template shape refer to the designated boundary (top / bottom / left / right, LCU, slice, etc.) beyond the boundary of the pixel constituting the template shape. It is characterized in that it has a mechanism for ensuring the number of pixels constituting the template shape by folding back to the target pixel and rearranging it around the target pixel (denoise target pixel or search point).

  That is, in image processing for removing noise of a denoising target pixel by a weighted sum of a pixel value at a search point and a weight corresponding to the template similarity by template matching, a boundary of a specified processing target image with respect to a predetermined template shape When the number of pixels constituting the template shape exceeds the line, a template shape having a shape rearranged so that the number of pixels exceeding the boundary line fits inside the boundary line is set, and a template matching search point or Perform template matching using a template with a different set template shape or the predetermined template shape depending on whether the denoising target pixel is near the boundary line or not, and remove noise of the denoising target pixel To do. In the template shape setting, the pixels constituting the template shape are rearranged so that the shapes are equal around the search point and the denoising target pixel.

  The present invention may be applied only when the template shape at the search point exceeds the specified boundary line (see, for example, FIGS. 1 and 2), or the template shape at the denoising target pixel exceeds the specified boundary line. You may apply only to the case (for example, refer FIG. 3, FIG. 4). It may be applied to both cases.

  1 to 4 show examples of rearrangement of template shapes. For example, pixels constituting the template shape Ts at the search point, such as a 3 × 3 block template as shown in FIG. 1A or an X-shaped template having a 5-pixel configuration as shown in FIG. 1B and 2B, the template shapes Ts and To are arranged within the boundary line, and the search point pixel Ps and the denoising are obtained as shown in FIGS. The number of pixels of the template is maintained by rearranging the target pixels Po so as to have the same shape around each pixel.

  Similarly, when the denoising target pixel Po is located around the designated boundary, all of the pixels constituting the template shape To around the denoising target pixel and the template shape Ts around the search point have the designated boundary. Set the rearranged template shape so that it does not exceed. Examples of rearrangement according to this example are shown in FIGS. Similarly, it can be easily imagined that the number of pixels can be maintained by rearrangement even when the search point exceeds the specified boundary. However, in this example, the search point exceeding the specified boundary is not calculated.

  According to the present invention, in the loop filter processing in video encoding, when the pixels constituting the template shape of the NLM filter refer beyond a specified boundary (top, bottom, left and right, LCU, slice, etc.), the template shape A mechanism that maintains the number of pixels that make up the template shape by rearranging the search points and the denoising target pixels so that the shapes are equal to each other so that the pixels that make up the template are within the boundary line Thus, a structure that fits in a predetermined buffer memory can be taken. Therefore, the enlargement of the hardware circuit scale can be prevented.

It is a figure which shows the example of rearrangement of a template shape. It is a figure which shows the example of rearrangement of a template shape. It is a figure which shows the example of rearrangement of a template shape. It is a figure which shows the example of rearrangement of a template shape. It is a figure which shows the structural example of the video coding apparatus with which this invention is applied. It is a process flowchart of a video coding apparatus. It is a figure which shows the structural example of the NLM filter process part in video coding. It is a process flowchart of the NLM filter process part in video coding. It is a figure which shows the structural example of the video decoding apparatus with which this invention is applied. It is a process flowchart of a video decoding apparatus. It is a figure which shows the structural example of the NLM filter process part in video decoding. It is a process flowchart of the NLM filter process part in video decoding. FIG. 2 is a diagram illustrating a configuration example of a system in the case where a video encoding device is realized using a computer and a software program. FIG. 25 is a diagram illustrating a configuration example of a system in the case where a video decoding device is realized using a computer and a software program. It is explanatory drawing of an NLM filter. It is a figure which shows the structural example of the conventional NLM filter process part (encoding). It is a process flowchart of the conventional NLM filter process part (encoding). It is a figure which shows the structural example of the conventional NLM filter process part (decoding). It is a process flowchart of the conventional NLM filter process part (decoding). It is explanatory drawing of an NLM filter.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

[Configuration of video encoding device]
FIG. 5 is a diagram illustrating a configuration example of a video encoding device to which the present invention is applied. The present invention is applied to, for example, a video encoding apparatus as shown in FIG. 5, but this is an example, and the present invention is not necessarily limited to the apparatus configuration as shown in FIG. In the present embodiment, an example in which an NLM filter is applied between an SAO and an Adaptive Loop Filter (hereinafter, ALF) is shown. In the present embodiment, in particular, the NLM filter processing unit 1083 is a part different from the prior art, and the other parts are the same as the configuration of a conventional general video encoding apparatus.

  The video encoding apparatus in FIG. 5 receives an encoding target sequence, divides each frame into blocks, encodes each block, and outputs the bit stream as an encoded stream.

  The encoding target block is input to the subtractor 112. The subtractor 112 obtains a difference between the encoding target block and the prediction block of the intra prediction unit 101 or the inter prediction unit 102, and uses it as a prediction residual signal. The transform unit 103 performs orthogonal transform (DCT) or the like on the prediction residual signal and outputs a transform coefficient. The quantization unit 104 quantizes the transform coefficient and outputs the quantized transform coefficient.

  The inverse quantization unit 105 performs inverse quantization on the post-quantization transform coefficient that is the output of the quantization unit 104, and the inverse transform unit 106 performs inverse transform of the transform by the transform unit 103. The value after inverse transformation is added and synthesized with the prediction block by the adder 113 and stored in the decoded image storage unit 107. The image in the decoded image storage unit 107 is used in the intra prediction unit 101.

  When the above processing is completed for all the blocks of the frame, an in-loop filter processing unit 108 applies an image processing filter for removing the coding distortion of the decoded image, and the decoded image after filtering is stored in the frame buffer 109. Is done. Here, the in-loop filter processing unit 108 includes a deblocking filter processing unit (hereinafter referred to as DF processing unit) 1081, an SAO processing unit 1082, an NLM filter processing unit 1083, and an ALF processing unit 1084. Execute the image processing filter. The image in the frame buffer 109 is used by the inter prediction unit 102.

  The prediction information of the intra prediction unit 101 and the inter prediction unit 102, the quantized transform coefficient, and the overhead of the in-loop filter processing unit 108 are input to the entropy encoding unit 110, and the entropy encoding unit 110 encodes them. Output the encoded stream. The encoded information storage unit 111 is a storage unit that stores various values such as various encoded block sizes and prediction information that can be referred to on the decoding device side, coefficient values after quantization, in-loop filter overhead, and the like. The information in the encoded information storage unit 111 is cited by various processing units in the video encoding device.

[Video encoding processing procedure]
FIG. 6 is a processing flowchart of the video encoding device shown in FIG. The input in this processing procedure is a sequence to be encoded, and the output is an encoded stream. The function is to convert the encoding target sequence into an encoded stream.
Step S101: Read the encoding target sequence for each encoding target image.
Step S102: Read the encoding target image for each block.
Step S103: The difference between the read encoding target block and the prediction block is taken to generate a difference block.
Step S104: Apply orthogonal transform to the difference block.
Step S105: Quantize the coefficient value after orthogonal transformation.
Step S106: The quantized coefficient value is inversely quantized.
Step S107: Inverse orthogonal transform is performed on the coefficient value after inverse quantization.
Step S108: The prediction residual after the inverse orthogonal transform and the prediction block in Step S103 are added and synthesized and stored in the decoded image storage unit 107.
Step S109: It is determined whether all blocks have been processed. If there is an unprocessed block, the process returns to step S102, and the same process is repeated for the next block.
Step S110: When all the blocks have been processed, an in-loop filter (DF, SAO, NLM filter, ALF) is applied to the decoded image and stored in the frame buffer 109.
Step S111: Entropy-encode the prediction information, the quantized coefficient value, and the in-loop filter overhead.
Step S112: It is determined whether or not all frames have been processed. If there is an unprocessed frame, the process returns to step S101 and the process is repeated in the same manner for the next frame.
Step S113: Output the encoded stream.

[Configuration of NLM Filter Processing Unit in Video Encoding Device (This Embodiment)]
FIG. 7 is a diagram illustrating a configuration example of the NLM filter processing unit in the embodiment of the present invention. In the present embodiment, the NLM filter processing unit 1083 shown in FIG. 5 is configured as shown in FIG.

  The optimum denoising coefficient calculation unit 201 inputs the encoding target image and the decoded image to be denoised, calculates a denoising coefficient that minimizes the RD cost between the encoding target image and the decoded image after filtering, and outputs the denoising coefficient. The coefficient is sent to the denoising coefficient encoding unit 202 and the NLM filter execution unit 204. The denoising coefficient encoding unit 202 encodes the denoising coefficient and outputs an in-loop filter overhead.

  The template shape setting unit 203 configures the template shape so that the predetermined template shape at each search point of the NLM filter application pixel does not exceed the boundary in accordance with the predetermined boundary information read from the encoding information storage unit 111 in FIG. Folding / rearrangement inside the boundary of the pixel to be performed is performed. The NLM filter execution unit 204 applies the NLM filter using the template shape set by the template shape setting unit 203, and outputs a decoded image after filtering.

[NLM Filter Processing Procedure in Video Coding (this embodiment)]
FIG. 8 is a processing flowchart of the NLM filter processing unit 1083 of this embodiment shown in FIG. The input in this processing procedure is an encoding target image, a decoded image, and boundary information, and the output is a decoded image after filtering and an in-loop filter overhead. The function is to calculate / encode an optimal denoising coefficient and apply an NLM filter to the decoded image.
Step S201: Read an encoding target image and a decoded image.
Step S202: An optimum denoising coefficient is calculated.
Step S203: Denoise coefficients are encoded and in-loop filter overhead is output.
Step S204: Boundary information is read from the encoding information storage unit 111, and a template shape at each search point of the NLM filter application pixel is derived.
Step S205: An NLM filter is applied to each pixel of the decoded image using the template shape derived in step S204, and the decoded image after filtering is output.

[Configuration of video decoding device]
FIG. 9 is a diagram illustrating a configuration example of a decoding device to which the present invention is applied. The present invention is applied to, for example, a video decoding apparatus as shown in FIG. 9, but this is an example, and the present invention is not necessarily limited to the apparatus configuration as shown in FIG. In this embodiment, an example in which an NLM filter is applied between SAO and ALF is shown. In the present embodiment, the NLM filter processing unit 3073 is particularly different from the prior art, and the other parts are the same as those of a conventional general video decoding apparatus.

  The video decoding apparatus in FIG. 9 receives an encoded stream. From the input encoded stream, the entropy decoding unit 301 decodes prediction information, quantized transform coefficients, and in-loop filter overhead for each block. The decoded quantized transform coefficient is subjected to inverse quantization by the inverse quantization unit 302, and inverse orthogonal transform (IDCT) is performed by the inverse transform unit 303. The output signal of the inverse transform unit 303 is added and synthesized with the prediction block of the intra prediction unit 304 or the inter prediction unit 305 and stored in the decoded image storage unit 306. The image in the decoded image storage unit 306 is used in the intra prediction unit 304.

  When the above processing is completed for all blocks of the frame, an in-loop filter processing unit 307 performs an image processing filter for removing the coding distortion of the decoded image, and the decoded image after filtering is stored in the frame buffer 308. Is done. Here, the in-loop filter processing unit 307 includes a DF processing unit 3071, a SAO processing unit 3072, an NLM filter processing unit 3073, and an ALF processing unit 3074, and executes predetermined image processing filters. The image in the frame buffer 308 is used in the inter prediction unit 305. The encoded information storage unit 309 is a storage unit that stores various values such as various block sizes, prediction information, quantized coefficient values, in-loop filter overhead, and the like. Information in the encoded information storage unit 309 is a video decoding device. Quoted in various processing sections.

[Video decoding procedure]
FIG. 10 is a processing flowchart of the video decoding apparatus shown in FIG. The input in this processing procedure is an encoded stream, and the output is a decoded sequence. The function is to convert the input encoded stream into a decoded sequence.
Step S301: Read an encoded stream.
Step S302: Entropy decoding is performed on the encoded stream, and information such as prediction information, coefficient values after quantization, and in-loop filter overhead is decoded.
Step S303: Inversely quantize the quantized coefficient value.
Step S304: Inverse orthogonal transform is performed on the coefficient value after inverse quantization.
Step S305: The prediction residual obtained by the inverse orthogonal transform and the prediction block are added and synthesized and stored in the decoded image storage unit 306.
Step S306: An in-loop filter (DF, SAO, NLM filter, ALF) is applied to the decoded image, and the decoded image is stored in the frame buffer 308.
Step S307: Output a decoding sequence.

[Configuration of NLM Filter Processing Unit in Video Decoding Device (This Embodiment)]
FIG. 11 is a diagram illustrating a configuration example of the NLM filter processing unit in the video decoding device according to the embodiment of the present invention. In the embodiment of the present invention, the NLM filter processing unit 3073 shown in FIG. 9 is configured as shown in FIG.

  The denoising coefficient decoding unit 401 decodes the denoising coefficient from the in-loop filter overhead. The template shape setting unit 402 configures the template shape so that the predetermined template shape at each search point of the NLM filter application pixel does not exceed the boundary in accordance with the predetermined boundary information read from the encoding information storage unit 309 in FIG. Folding / rearrangement inside the boundary of the pixel to be performed is performed. The NLM filter execution unit 403 applies the NLM filter using the template shape set by the template shape setting unit 402, and outputs a decoded image after filtering.

[NLM Filter Processing Procedure in Video Decoding (this embodiment)]
FIG. 12 is a process flowchart of the NLM filter processing unit 3073 in the embodiment of the present invention shown in FIG. The input in this processing procedure is a decoded image, in-loop filter overhead and boundary information, and the output is a decoded image after filtering. The function is to decode the denoising coefficient and apply an NLM filter to the decoded image.
Step S401: Read decoded image and in-loop filter overhead.
Step S402: Denoising coefficients are decoded.
Step S403: Boundary information is read from the encoding information storage unit 309, and a template shape at each search point of the NLM filter application pixel is derived.
Step S404: An NLM filter is applied to each pixel of the decoded image using the template shape derived in step S403, and a filtered decoded image is output.

  With the above processing, the pixels constituting the NLM filter template are not referred to beyond the specified boundary (top, bottom, left and right edges, LCU, slice, etc.), and the structure can be stored in a predetermined buffer memory. It is possible to prevent an increase in the hardware circuit scale.

[Configuration example using software program]
The video encoding and video decoding processes described above can also be realized by a computer and a software program, which can be provided by recording the program on a computer-readable recording medium or via a network. Is possible.

  FIG. 13 shows an example of a hardware configuration when the video encoding apparatus is configured by a computer and a software program. This system includes a CPU 50 that executes a program, a memory 51 such as a RAM that stores programs and data accessed by the CPU 50, and a video signal input unit 52 that inputs a video signal to be encoded from a camera or the like (disk device). A program storage device 53 in which a video encoding program 54, which is a software program for causing the CPU 50 to execute a process for encoding an input video by this method, and the CPU 50, An encoded data output unit 55 for outputting encoded data generated by executing the video encoding program 54 loaded in the memory 51 via, for example, a network (a storage unit for storing encoded data by a disk device or the like) However, it may be connected by a bus.

  FIG. 14 shows an example of a hardware configuration when the video decoding apparatus is configured by a computer and a software program. This system includes a CPU 60 that executes a program, a memory 61 such as a RAM that stores programs and data accessed by the CPU 60, and encoded data input that inputs encoded data encoded by the video encoding apparatus according to the present method. A program in which a unit 62 (which may be a storage unit that stores multiplexed encoded data by a disk device or the like) and a video decoding program 64 that is a software program that causes the CPU 60 to execute processing for decoding encoded data by the present method A storage device 63 and a decoded video output unit 65 that outputs a decoded video obtained by decoding the encoded data by the CPU 60 executing the video decoding program 64 loaded in the memory 61 to a playback device or the like. , Connected by a bus.

  The embodiments of the present invention have been described above with reference to the drawings. However, the above embodiments are merely examples of the present invention, and it is obvious that the present invention is not limited to the above embodiments. is there. Accordingly, additions, omissions, substitutions, and other modifications of the components may be made without departing from the spirit and technical scope of the present invention.

1083, 3073 NLM filter processing unit 201 Optimal denoising coefficient calculating unit 202 Denoising coefficient encoding unit 203, 402 Template shape setting unit 204, 403 NLM filter executing unit 401 Denoising coefficient decoding unit

Claims (9)

An image processing method for removing noise of a denoising target pixel by a weighted sum of a pixel value at a search point and a weight according to template similarity by template matching,
When the pixels that make up the template shape exceed the boundary line of the specified image to be processed for a given template shape, the pixels are relocated so that the number of pixels that exceed the boundary line fit within the boundary line. Setting a template shape having different shapes,
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. And a process for removing noise from the target pixel. An image processing device that removes noise of a denoising target pixel by a weighted sum of a pixel value at a search point and a weight according to template similarity by template matching,
When the pixels that make up the template shape exceed the boundary line of the specified image to be processed for a given template shape, the pixels are relocated so that the number of pixels that exceed the boundary line fit within the boundary line. Means for setting a template shape having different shapes;
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. An image processing apparatus comprising: means for removing noise of a target pixel. As a loop filter, using an image processing filter that removes noise of a denoising target pixel by a weighted sum of a pixel value at a search point and a weight corresponding to the template similarity by template matching, noise for a decoded image of a predictively encoded image A video encoding method for performing processing for removing
When the pixels constituting the template shape exceed the boundary line of the decoded image with respect to the predetermined template shape of the image processing filter, the pixels corresponding to the number of pixels exceeding the boundary line are reconstituted so as to be within the boundary line. A process of setting a template shape having an arranged shape;
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. And a process of removing noise from the target pixel. As a loop filter, using an image processing filter that removes noise of a denoising target pixel by a weighted sum of a pixel value at a search point and a weight corresponding to the template similarity by template matching, noise for a decoded image of a predictively encoded image A video encoding device that performs processing for removing
When the pixels constituting the template shape exceed the boundary line of the decoded image with respect to the predetermined template shape of the image processing filter, the pixels corresponding to the number of pixels exceeding the boundary line are reconstituted so as to be within the boundary line. Means for setting a template shape having an arranged shape;
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. And a means for removing noise of the target pixel. As a loop filter, an image processing filter that removes the noise of the denoising target pixel by a weighted sum of the pixel value at the search point and the weight according to the template similarity by template matching is used to reduce the noise on the decoded image of the predicted decoded image. A video decoding method for performing removal processing,
When the pixels constituting the template shape exceed the boundary line of the decoded image with respect to the predetermined template shape of the image processing filter, the pixels corresponding to the number of pixels exceeding the boundary line are reconstituted so as to be within the boundary line. A process of setting a template shape having an arranged shape;
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. And a process of removing noise from the target pixel. As a loop filter, an image processing filter that removes the noise of the denoising target pixel by a weighted sum of the pixel value at the search point and the weight according to the template similarity by template matching is used to reduce the noise on the decoded image of the predicted decoded image. A video decoding device that performs processing to be removed,
When the pixels constituting the template shape exceed the boundary line of the decoded image with respect to the predetermined template shape of the image processing filter, the pixels corresponding to the number of pixels exceeding the boundary line are reconstituted so as to be within the boundary line. Means for setting a template shape having an arranged shape;
Template matching is performed using a template having a different template shape or the predetermined template shape, depending on whether the search point or denoising target pixel for template matching is near the boundary line or not. And a means for removing noise of the target pixel.   An image processing program for causing a computer to execute the image processing method according to claim 1.   A video encoding program for causing a computer to execute the video encoding method according to claim 3.   A video decoding program for causing a computer to execute the video decoding method according to claim 5.

Images