JP5715533B2 - Moving image quality restoration apparatus, moving image quality restoration method, and program - Google Patents

Description

  The present invention relates to a moving image quality restoration apparatus, a moving image quality restoration method, and a program.

  Conventionally, H.M. In the video compression coding technique represented by H.264 (for example, see Non-Patent Document 1), high compression performance is realized in which an error from an original image is reduced with a small code amount. However, in a video image highly compressed by this technique, a large image quality deterioration occurs particularly in textures with high frequency components due to processing such as quantization in the compression encoding process.

  Therefore, a Non-Local Means (NLM) filter method (see, for example, Non-Patent Document 2) has been proposed. In this NLM filter method, it is possible to restore the image quality of a texture in which image quality degradation has occurred by assigning a large weight to pixels (values) of a similar texture in the vicinity region in the screen.

  The NLM filter process in the NLM filter method will be described below using FIG.

  In the NLM filter process, a filter process for L × L pixels (L is an arbitrary integer satisfying L ≧ 2) in the vicinity of the filter processing target pixel i is performed using the NLM filter. The NLM filter includes a region of M × M pixels (M is an arbitrary integer satisfying 1 ≦ M ≦ L) in the vicinity of each of the filtering target pixel i and the filter coefficient calculation pixel j. The similarity is evaluated, and the higher the similarity is, the larger the filter coefficient is set. Specifically, when an input image is expressed as the following expression (1), the filter coefficient w (i, j) in the filter coefficient calculation pixel j for the filter processing target pixel i is expressed by the following expression ( 2).

In Expression (2), N _i represents a region of M × M pixels in the vicinity of the filter target pixel i, and N _j represents M × M pixels in the vicinity of the filter coefficient calculation pixel j. An area shall be indicated. Also, v (N _i ) represents the lightness at the gray level for N _i , and v (N _j ) represents the lightness at the gray level for N _j . Further, h represents the strength of the filter. Also, z (i) can be obtained by the following equation (3).

  In the NLM filter processing, the filter coefficient w (i, j) obtained as described above is applied to the degraded image. The pixel value NL [v] (i) obtained by this NLM filter processing can be expressed as the following equation (4).

Joint Video Team (JVT) of ISO / IEC MPEG and ITU-T VCEG, "Text of ISO / IEC 14496-10 Advanced Video Coding," July 2004. A. Buades, B. Coll, and J. M. Morel, "A Non Local Algorithm for Image Denoising," Proc. IEEE Int. Conf. On Computer Vision and Pattern Recognition, vol. 2, pp. 60-65, June 2005. ISO / IEC MPEG and ITU-T VCEG, "Text of ISO / IEC 13818-2 Generic coding of moving pictures and associated audio information: Video," 2000. C. Tomasi and R. Manduchi, "Bilateral filtering for gray and color images," In Proceedings of the Sixth International Conference on Computer Vision, page 839-846, 1998. S. Smith and J. Brady, "Susan-a new approach to low level image processing," International Journal of Computer Vision, 23 (1): 45-78, 1997.

  In the NLM filter method disclosed in Non-Patent Document 2, the image quality is restored by performing filter processing using only the texture of the neighboring region in the processing frame. For this reason, the restored image quality strongly depends on the degree of degradation of the image quality of the processing frame. Therefore, when the NLM filter method shown in Non-Patent Document 2 is simply applied to a compressed video to which a different quantization parameter is given for each picture type, a difference in image quality for each frame becomes large, which is unnatural. It was a serious video.

  Accordingly, the present invention has been made in view of the above-described problems, and a moving image quality restoration apparatus, a moving image quality restoration method, and a program that can restore a texture with high accuracy by reducing a difference in image quality for each frame. The purpose is to provide.

  The present invention proposes the following matters in order to solve the above problems.

  (1) The present invention is a moving image quality restoration device (for example, equivalent to the moving image quality restoration device 1 of FIG. 1) that improves the quality of moving images by filtering based on local pattern similarity. If each frame up to a specific frame of the processing frame and each frame up to a predetermined frame after the processing frame are defined as neighboring frames based on the processing frame, the filtering target in the processing frame for each neighboring frame An area similar to an area in the vicinity of a pixel (for example, an area of L × L pixels in the vicinity of the pixel to be filtered i in FIG. 3) is set as a reference area (for example, an area corresponding to the reference area in FIG. 3). The reference area determining means to be obtained (for example, equivalent to the reference area determining unit 10 in FIG. 1), the area in the vicinity of the filtering target pixel, and the reference area determining means Based on the reference region, filter coefficient calculation means (for example, equivalent to the filter coefficient calculation unit 20 in FIG. 1) for calculating a filter coefficient (for example, corresponding to an NLM filter coefficient described later) based on local pattern similarity. ) And filter processing application means (for example, the filter processing application unit 30 in FIG. 1) that performs filter processing on the region in the vicinity of the filter processing target pixel using the filter coefficient calculated by the filter coefficient calculation means. In other words, a moving image quality restoration apparatus has been proposed.

  According to the present invention, if each frame before a specific frame of the processing frame and each frame after the predetermined frame of the processing frame are neighboring frames, the moving image is obtained by the filtering process based on the local similarity of the pattern. Is provided with a reference area determining means, a filter coefficient calculating means, and a filter processing applying means. Then, the reference region determining means obtains, as the reference region, a region similar to the region near the filter processing target pixel in the processing frame for each neighboring frame. Further, the filter coefficient calculating means calculates a filter coefficient based on the local pattern similarity based on the area near the filter processing target pixel and the reference area obtained by the reference area determining means; did. In addition, the filter processing application unit performs the filter process on the region near the pixel to be filtered using the filter coefficient calculated by the filter coefficient calculation unit.

  Therefore, the moving image quality restoration apparatus (1) can restore the picture quality by performing the filter processing using the texture of the neighboring frame. Therefore, by using the texture of a neighboring frame that has a lower degree of image quality degradation than the processing frame for the filtering process, the difference in image quality for each frame can be reduced and the texture can be restored with high accuracy.

  (2) The present invention relates to the moving image quality restoration apparatus of (1), wherein each frame before the specific frame of the processing frame is m frames before the processing frame (m satisfies 0 ≦ m ≦ 15). Each frame up to a predetermined frame after the processing frame is n frames after the processing frame (n is 1 ≦ n ≦ 15 when m = 0). A moving picture image quality restoration device, which is an arbitrary integer satisfying 0 <m ≦ 15 in the case of 0 <m ≦ 15, and an arbitrary integer satisfying 0 ≦ n ≦ 15). ing.

  According to the present invention, in the moving image quality restoration apparatus of (1), each frame up to m frames before the processing frame and each frame up to n frames after the processing frame can be used as neighboring frames. The effect similar to the effect mentioned above can be show | played.

  (3) In the moving image quality restoration apparatus according to (1) or (2), the present invention provides a reference list (for example, equivalent to a reference list a to be described later), wherein the reference area determining means is obtained from a video compression coded stream. And a motion image quality characterized in that, based on a motion vector (e.g., corresponding to a motion vector b to be described later), an area indicated by the motion vector is obtained as the reference area with respect to an area in the vicinity of the filter processing target pixel. A restoration device is proposed.

  According to the present invention, in the moving image quality restoration apparatus of (1) or (2), the reference region determination unit performs filtering processing as a reference region based on the reference list and the motion vector obtained from the video compression encoded stream. The region indicated by the motion vector is obtained with respect to the region in the vicinity of the target pixel.

  For this reason, the moving image quality restoration apparatus (3) obtains a reference area using a reference list and a motion vector obtained from a video compression encoded stream. Therefore, a region similar to the region near the pixel to be filtered in the processing frame can be appropriately obtained as the reference region for each neighboring frame, and the same effect as described above can be achieved.

  (4) In the moving image image quality restoration device according to (1) or (2), the reference area determination unit performs motion vector estimation by motion estimation based on block matching between the processing frame and the neighboring frame. And a moving image quality restoration apparatus is proposed in which a region indicated by the motion vector is obtained as the reference region with respect to a region in the vicinity of the pixel to be filtered.

  According to the present invention, in the moving image image quality restoration device of (1) or (2), the reference region determining means obtains a motion vector by motion estimation based on block matching between a processing frame and a neighboring frame, and refers to it. As the region, the region indicated by the motion vector is obtained with respect to the region near the pixel to be filtered.

  For this reason, the moving image quality restoration apparatus of (4) obtains a reference region using a motion vector obtained by motion estimation based on block matching between a processing frame and a neighboring frame. Therefore, a region similar to the region near the pixel to be filtered in the processing frame can be appropriately obtained as the reference region for each neighboring frame, and the same effect as described above can be achieved.

(5) In the moving image image quality restoration device according to any one of (1) to (4), the filter coefficient calculation unit may perform a quantization step width (for example, described later) in the processing frame for the filter coefficient. be carried out as corresponding to the quantization step width w _c) in the processing frame, the quantization step width in the vicinity of the frame (e.g., a corresponds to the quantization step width w _n) in neighboring frames later, weighted according to the ratio of Has proposed a moving image quality restoration apparatus characterized by the above.

  According to this invention, in the moving image image quality restoration device according to any one of (1) to (4), the filter coefficient calculation means performs the quantization step width in the processing frame and the quantization step width in the neighboring frame for the filter coefficient. And weighting according to the ratio.

  For this reason, the moving image quality restoration apparatus of (5) weights the filter coefficient according to the ratio between the quantization step width in the processing frame and the quantization step width in the neighboring frame. Therefore, as the degree of similarity between the processing frame and the neighboring frame increases, the influence of the filter coefficient in the neighboring frame on the filter coefficient used for the filtering process can be increased. Therefore, the difference in image quality for each frame can be further reduced, and the texture can be restored with higher accuracy.

  (6) In the moving image image quality restoration device according to any one of (1) to (5), the filter coefficient calculation unit may obtain the region near the pixel to be filtered and the reference region determination unit. And calculating the filter coefficient based on the local similarity of the pattern based on a region in the vicinity of the pixel to be filtered in the processing frame. Has proposed a moving image quality restoration apparatus characterized by the above.

  According to the present invention, in the moving image image quality restoration device according to any one of (1) to (5), the calculation of the filter coefficient by the filter coefficient calculation unit is performed by the region in the vicinity of the pixel to be filtered and the reference region determination unit. Not only based on the obtained reference area, but also based on the vicinity of the filter target pixel in the processing frame.

  For this reason, the moving image quality restoration apparatus of (6) also uses the filter coefficient calculated based on the region near the pixel to be filtered in the processing frame as the filter coefficient used for the filtering process. For this reason, compared with the case where only the filter coefficient calculated based on the area in the vicinity of the pixel to be filtered and the reference area obtained by the reference area determining unit is used as the filter coefficient used for the filtering process, for each frame. The texture can be restored with higher accuracy by further reducing the difference in image quality.

  (7) The present invention is a moving image image quality restoration method for improving the image quality of moving images by filtering based on local pattern similarity, up to a specific frame of the processing frame before the processing frame. Assuming that each frame and each frame after the predetermined frame of the processing frame are neighboring frames, for each neighboring frame, an area in the vicinity of the pixel to be filtered in the processing frame (for example, the pixel to be filtered in FIG. 3). A first step (for example, a reference area determination unit in FIG. 1) that obtains an area similar to an L × L pixel area in the vicinity of i as a reference area (for example, the reference area in FIG. 3) 10), a region in the vicinity of the pixel to be filtered, and the reference region obtained in the first step. A second step (for example, equivalent to processing by the filter coefficient calculation unit 20 in FIG. 1) for calculating a filter coefficient (for example, corresponding to an NLM filter coefficient described later) based on the characteristics, and a region in the vicinity of the filter processing target pixel On the other hand, a third step (for example, corresponding to the processing by the filter processing application unit 30 in FIG. 1) for performing the filter processing using the filter coefficient calculated in the second step is provided. A moving image quality restoration method is proposed.

  According to the present invention, assuming that each frame before a specific frame of the processing frame and each frame after the predetermined frame of the processing frame are neighboring frames, first, for each neighboring frame, the filtering target pixel in the processing frame An area similar to the area in the vicinity of is determined as a reference area. Next, the filter coefficient based on the local similarity of the pattern is calculated based on the region in the vicinity of the filter processing target pixel and the reference region. Next, the filtering process is performed on the area near the filtering target pixel using the calculated filter coefficient. According to the above, an effect similar to the effect described above can be achieved.

  (8) The present invention is a program for causing a computer to execute a moving image image quality restoration method for improving the image quality of a moving image by a filtering process based on local pattern similarity, and using a processing frame as a reference, If each frame up to a specific frame of the processing frame and each frame up to a predetermined frame of the processing frame are neighboring frames, for each neighboring frame, an area in the vicinity of the pixel to be filtered in the processing frame ( For example, a first step of obtaining a region similar to a region of L × L pixels in the vicinity of the pixel to be filtered i in FIG. 3 as a reference region (for example, corresponding to the reference region in FIG. 3) ( For example, the processing is equivalent to the processing by the reference region determination unit 10 in FIG. And a second step (for example, by the filter coefficient calculation unit 20 in FIG. 1) that calculates a filter coefficient (for example, equivalent to an NLM filter coefficient described later) based on the local pattern similarity based on the reference region. And a third step (for example, the filter processing application unit in FIG. 1) that performs filter processing on the region in the vicinity of the pixel to be filtered using the filter coefficient calculated in the second step. Is equivalent to the processing by No. 30).

  According to the present invention, assuming that each frame before a specific frame of the processing frame and each frame after the predetermined frame of the processing frame are neighboring frames, by causing the computer to execute the program, first, for each neighboring frame In addition, an area similar to the area near the pixel to be filtered in the processing frame is obtained as the reference area. Next, the filter coefficient based on the local similarity of the pattern is calculated based on the region in the vicinity of the filter processing target pixel and the reference region. Next, the filtering process is performed on the area near the filtering target pixel using the calculated filter coefficient. According to the above, an effect similar to the effect described above can be achieved.

  According to the present invention, the texture can be restored with high accuracy by reducing the difference in image quality for each frame.

It is a block diagram which shows the structure of the moving image quality restoration apparatus which concerns on 1st Embodiment of this invention. It is a figure for demonstrating operation | movement of the said moving image image quality restoration apparatus. It is a figure for demonstrating the reference area calculated | required with the said moving image image quality restoration apparatus. It is a block diagram which shows the structure of the moving image image quality restoration apparatus which concerns on 2nd Embodiment of this invention. It is a figure for demonstrating operation | movement of the said moving image image quality restoration apparatus. It is a block diagram which shows the structure of the moving image image quality restoration apparatus which concerns on 3rd Embodiment of this invention. It is a figure for demonstrating operation | movement of the said moving image image quality restoration apparatus. It is a figure for demonstrating the NLM filter process in a NLM filter system.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings. Note that the constituent elements in the following embodiments can be appropriately replaced with existing constituent elements, and various variations including combinations with other existing constituent elements are possible. Accordingly, the description of the following embodiments does not limit the contents of the invention described in the claims.

<First Embodiment>
[Configuration of Moving Image Quality Restoration Apparatus 1]
FIG. 1 is a block diagram showing a configuration of a moving image quality restoration apparatus 1 according to the first embodiment of the present invention. The moving image quality restoration apparatus 1 is an H.264 standard. Restoration processing is performed on a degraded image obtained in a video decoding device (not shown) by decoding a bitstream compression-encoded according to the H.264 standard. The moving image quality restoration apparatus 1 includes a reference area determination unit 10, a filter coefficient calculation unit 20, and a filter processing application unit 30.

[Operation of Moving Image Quality Restoration Apparatus 1]
The reference area determination unit 10 is configured so that the H.264 A reference list a and a motion vector (MV) b obtained from a bitstream compression-encoded according to the H.264 standard are input. The reference area determination unit 10 performs the first reference area determination process shown in FIG. 2 for all the pixels constituting the processing frame.

  In the first reference area determination process, based on the reference list a and the motion vector b, the area indicated by the motion vector b with respect to the area of L × L pixels in the vicinity of the filter target pixel i is shown in FIG. As shown, it is acquired as a reference area for each neighboring frame. Specifically, with respect to a region of L × L pixels in the vicinity of the pixel to be processed i, the reference list a and the motion vector b in the prediction block including at least a part of this region are used, and the above-mentioned in each neighboring frame. A reference area of at least a part of the area is acquired. According to this, a region similar to a region of L × L pixels in the vicinity of the filter processing target pixel i is acquired for each neighboring frame as the reference region. Then, the obtained reference area in each neighboring frame is output as a reference area c.

  Note that the neighborhood frame refers to each frame up to m frames before the processing frame (m is an arbitrary integer satisfying 0 ≦ m ≦ 15) with respect to the processing frame including the filter processing target pixel i, and this processing. After n frames (n is an arbitrary integer that satisfies 1 ≦ n ≦ 15 when m = 0, and any integer that satisfies 0 ≦ n ≦ 15 when 0 <m ≦ 15) Each frame up to. The value of m and the value of n may be predetermined fixed values or variable values.

  Further, when the reference area is acquired using the reference list a and the motion vector b as described above, the L × L pixel area in the vicinity of the filtering target pixel i in the processing frame and the reference in each acquired neighboring frame A region is similar. Here, “similar” means that the evaluation values related to the distribution of pixel values are similar, and the evaluation values related to the distribution of pixel values can be obtained from the right side of the above equation (2). As the evaluation value related to the distribution of pixel values increases, the degree of similarity between the L × L pixel area in the vicinity of the pixel i to be filtered and the reference area increases.

  Returning to FIG. 1, the filter coefficient calculation unit 20 receives the reference region c and the pixel value d of the deteriorated image obtained in the above-described moving image decoding apparatus. The filter coefficient calculation unit 20 performs the first filter coefficient calculation process shown in FIG. 2 for all the pixels constituting the processing frame.

  In the first filter coefficient calculation process, the NLM filter coefficient e is calculated not only using the pixels of the processing frame as described above with reference to FIG. 8 but also using the pixels of the neighboring frame. Specifically, first, as described above with reference to FIG. 8, the NLM filter coefficient is obtained based on the region of L × L pixels in the vicinity of the filter target pixel i in the processing frame. According to this, the NLM filter coefficient is obtained using only the pixels of the processing frame. Next, according to the above equation (2), for each reference region in each neighboring frame included in the reference region c, the reference region and the region of L × L pixels in the vicinity of the filter processing target pixel i are Based on this, NLM filter coefficients are obtained. According to this, the NLM filter coefficient is obtained for each neighboring frame using the pixel of the processing frame and the pixel of the neighboring frame. Next, normalization is performed on the NLM filter coefficient obtained using only the processing frame and the NLM filter coefficient obtained for each neighboring frame, and the result is output as the NLM filter coefficient e. The NLM filter coefficient is a kind of filter coefficient based on local pattern similarity.

  Returning to FIG. 1, the filter processing application unit 30 receives the reference region c, the pixel value d of the deteriorated image, and the NLM filter coefficient e as inputs. The filter processing application unit 30 performs the filter processing application processing shown in FIG. 2 on all the pixels constituting the processing frame.

  In the filter process application process, a filter process is performed on a region of L × L pixels in the vicinity of the filter target pixel i using the NLM filter coefficient e of the pixel in the reference region c of the pixel value d of the deteriorated image. Thus, the filtered pixel value f of the pixel to be filtered i is output.

  When the above-described filter processing application process is performed on all the pixels constituting the processing frame, the pixel value f is output for each pixel constituting the processing frame. These output pixel values f are collected and output as a restored image.

  According to the above moving image quality restoration apparatus 1, the following effects can be obtained.

  The moving image quality restoration apparatus 1 calculates the NLM filter coefficient e using not only the pixels of the processing frame but also the pixels of the neighboring frame. For this reason, it is possible to restore the image quality by performing the filter processing using not only the texture of the region of L × L pixels in the vicinity of the filter processing target pixel i in the processing frame but also the texture in the vicinity frame. Therefore, it is possible to reduce the difference in image quality for each frame and restore the texture with high accuracy.

  The moving image quality restoration apparatus 1 uses a reference list a and a motion vector b obtained from a compression-encoded bit stream. It is included in the bit stream compressed and encoded by the H.264 standard. For this reason, it is possible to restore the texture with high accuracy without adding information to the existing bit stream, and it is possible to improve the quality of the degraded image obtained in the video decoding device.

  Note that high-precision restoration of the texture can be realized remarkably in a periodic texture such as a striped pattern.

  In the present embodiment, the moving image quality restoration apparatus 1 is an H.264 video. H.264 standard is assumed, but the present invention is not limited to this, and H.264 such as MPEG-2 shown in Non-Patent Document 3 described above. The present invention can also be applied to a compression technique using motion compensation prediction similar to the H.264 standard.

Second Embodiment
[Configuration of Moving Image Quality Restoration Apparatus 1A]
FIG. 4 is a block diagram showing a configuration of a moving image quality restoration apparatus 1A according to the second embodiment of the present invention. The moving image quality restoration apparatus 1A is different from the moving image quality restoration apparatus 1 according to the first embodiment of the present invention shown in FIG. 1 in that a reference area determination unit 10A is provided instead of the reference area determination unit 10. In the moving image quality restoration apparatus 1A, the same components as those in the moving image quality restoration apparatus 1 are denoted by the same reference numerals and description thereof is omitted.

[Operation of Moving Image Quality Restoration Apparatus 1A]
The reference area determination unit 10A receives the deteriorated image g. This reference area determination unit 10A performs the second reference area determination process shown in FIG. 5 for all the pixels constituting the processing frame.

  In the second reference region determination processing, first, motion estimation based on block matching between the processing frame and the neighboring frame is performed on the region of L × L pixels in the deteriorated image g near the pixel to be filtered i. Go and find the motion vector. Next, an area indicated by the obtained motion vector with respect to the area of L × L pixels in the vicinity of the filtering target pixel i is acquired as a reference area for each neighboring frame. Then, the obtained reference area in each neighboring frame is output as a reference area c.

  According to the above moving image quality restoration apparatus 1A, the same effects as those described above that can be achieved by the moving image quality restoration apparatus 1 can be obtained.

<Third Embodiment>
[Configuration of Moving Image Quality Restoration Apparatus 1B]
FIG. 6 is a block diagram showing a configuration of a moving image quality restoration apparatus 1B according to the third embodiment of the present invention. The moving image quality restoration apparatus 1B is different from the moving image quality restoration apparatus 1 according to the first embodiment of the present invention shown in FIG. 1 in that a filter coefficient calculation unit 20A is provided instead of the filter coefficient calculation unit 20. In the moving image quality restoration apparatus 1B, the same components as those in the moving image quality restoration apparatus 1 are denoted by the same reference numerals and description thereof is omitted.

[Operation of Moving Image Quality Restoration Apparatus 1B]

  The filter coefficient calculation unit 20A receives the reference area c, the pixel value d of the deteriorated image, and the quantization parameter information (QP) h in each frame. The filter coefficient calculation unit 20A performs the second filter coefficient calculation process shown in FIG. 7 for all the pixels constituting the processing frame.

  In the second filter coefficient calculation process, similar to the first filter coefficient calculation process shown in FIG. 2, the NLM filter coefficient e is calculated not only using the pixels of the processing frame, but also the pixels of the neighboring frames. Is also used. Specifically, first, as described above with reference to FIG. 8, the NLM filter coefficient is obtained based on the region of L × L pixels in the vicinity of the filter target pixel i in the processing frame. According to this, the NLM filter coefficient is obtained using only the pixels of the processing frame. Next, according to the above equation (2), for each reference region in each neighboring frame included in the reference region c, the reference region and the region of L × L pixels in the vicinity of the filter processing target pixel i are Based on this, NLM filter coefficients are obtained. According to this, the NLM filter coefficient is obtained for each neighboring frame using the pixel of the processing frame and the pixel of the neighboring frame. Next, the weight is controlled using the quantization parameter information h in each neighboring frame for the NLM filter coefficient obtained for each neighboring frame. Next, normalization is performed on the NLM filter coefficient obtained using only the processing frame and the NLM filter coefficient for each neighboring frame subjected to weight control, and the result is output as the NLM filter coefficient e.

In the weight control using the quantization parameter information h described above, for example, a quantization step width obtained from the quantization parameter information h can be used. When this quantization step width is used, _assuming that the quantization step width in the processing frame is w _c and the quantization step width in the neighboring frame is wn, W _c / w _n for the NLM filter coefficient in this neighboring frame Double the weight.

  According to the above-described moving image quality restoration apparatus 1B, in addition to the above-described effects that the moving image quality restoration apparatus 1 can achieve, the following effects can be obtained.

The moving image quality restoration apparatus 1B attaches a weight of W _c / w _n times to the NLM filter coefficient in each neighboring frame. For this reason, as the similarity between the processing frame and the neighboring frame increases, the influence of the NLM filter coefficient in the neighboring frame on the NLM filter coefficient e can be increased. Accordingly, the difference in image quality for each frame can be further reduced, and the texture can be restored with higher accuracy.

  The processing of the moving image quality restoration apparatus 1, 1A, 1B of the present invention is stored in a computer-readable recording medium, and the program recorded on the recording medium is read by the moving image quality restoration apparatus 1, 1A, 1B. The present invention can be realized by executing.

  The above-described program is transmitted from the moving image quality restoration apparatus 1, 1A, 1B storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. May be. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line.

  Further, the above-described program may be for realizing a part of the above-described function. Furthermore, what can implement | achieve the above-mentioned function in combination with the program already recorded on the moving image image quality restoration apparatus 1, 1A, 1B, what is called a difference file (difference program) may be sufficient.

  The embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to this embodiment, and includes a design that does not depart from the gist of the present invention.

  For example, in the above-described embodiment, an NLM filter is used as a filter based on local pattern similarity. However, the present invention is not limited to this, and a filter having properties similar to the NLM filter, such as Total Variation, A Bilatal Filter disclosed in Patent Document 4, a SUSAN Filter illustrated in Non-Patent Document 5 described above, or the like may be used.

  In the above-described embodiment, the example in which the second filter coefficient calculation process is combined with the first reference area determination process shown in FIG. 7 is shown. However, the present invention is not limited to this, and for example, the second filter coefficient calculation process shown in FIG. You may combine with 2 reference area determination processing.

  Further, in the above-described embodiment, m defining the neighborhood frame is an arbitrary integer satisfying 0 ≦ m ≦ 15, and n defining the neighborhood frame is 1 ≦ n ≦ when m = 0. 15 and any integer satisfying 0 ≦ n ≦ 15 when 0 <m ≦ 15. However, the present invention is not limited to this, and m and n are each an arbitrary integer. May be.

DESCRIPTION OF SYMBOLS 1, 1A, 1B ... Moving image image quality restoration apparatus 10, 10A ... Reference area determination part 20, 20A ... Filter coefficient calculation part 30 ... Filter process application part

Claims (7)

A moving image quality restoration device that improves the quality of moving images by filtering based on local pattern similarity,
With the processing frame as a reference, each frame up to a specific frame of the processing frame and each frame up to a predetermined frame after the processing frame are set as neighboring frames.
For each of the neighboring frames, a reference area determining means for obtaining, as a reference area, an area similar to the area near the pixel to be filtered in the processing frame;
A filter coefficient calculation unit that calculates a filter coefficient based on local pattern similarity based on a region in the vicinity of the filtering target pixel and the reference region obtained by the reference region determination unit;
Filter processing applying means for performing filter processing on a region in the vicinity of the filter processing target pixel using the filter coefficient calculated by the filter coefficient calculating means ,
The moving picture image quality restoration device, wherein the filter coefficient calculation means weights the filter coefficient according to a ratio between a quantization step width in the processing frame and a quantization step width in the neighboring frame. . Each frame before the specific frame of the processing frame is each frame up to m frames before the processing frame (m is an arbitrary integer satisfying 0 ≦ m ≦ 15),
Each frame up to a predetermined frame after the processing frame is n frames after the processing frame (n is an arbitrary integer satisfying 1 ≦ n ≦ 15 when m = 0, and 0 <m ≦ 15 The moving image quality restoration apparatus according to claim 1, wherein each frame is up to an arbitrary integer satisfying 0 ≦ n ≦ 15.   The reference area determining means obtains, as the reference area, an area indicated by the motion vector with respect to an area in the vicinity of the filtering target pixel based on a reference list and a motion vector obtained from a video compression encoded stream. The moving image quality restoration apparatus according to claim 1 or 2, characterized in that   The reference region determining means obtains a motion vector by motion estimation based on block matching between the processing frame and the neighboring frame, and a region indicated by the motion vector with respect to a region near the filtering target pixel. The moving image quality restoration device according to claim 1, wherein the moving image quality restoration device is obtained as the reference region. The filter coefficient calculation means includes
While calculating the filter coefficient based on the region in the vicinity of the filtering target pixel and the reference region obtained by the reference region determining means,
Based on the region near the filter processing target pixel in the processing frame, the moving image quality according to any one of claims 1 to 4, and calculates the filter coefficient based on the similarity of the local pattern Restore device. A moving image quality restoration method for improving the quality of moving images by filtering based on local pattern similarity,
With the processing frame as a reference, each frame up to a specific frame of the processing frame and each frame up to a predetermined frame after the processing frame are set as neighboring frames.
A first step of obtaining, for each of the neighboring frames, a region similar to a region in the vicinity of the pixel to be filtered in the processing frame as a reference region;
A second step of calculating a filter coefficient based on a local pattern similarity based on a region in the vicinity of the pixel to be filtered and the reference region obtained in the first step;
A third step of performing a filtering process on the region in the vicinity of the filtering target pixel using the filter coefficient calculated in the second step ,
In the second step, the filter coefficient is weighted according to a ratio between a quantization step width in the processing frame and a quantization step width in the neighboring frame, with respect to the filter coefficient. . A program for causing a computer to execute a moving image quality restoration method for improving the quality of moving images by filtering based on local pattern similarity,
With the processing frame as a reference, each frame up to a specific frame of the processing frame and each frame up to a predetermined frame after the processing frame are set as neighboring frames.
A first step of obtaining, for each of the neighboring frames, a region similar to a region in the vicinity of the pixel to be filtered in the processing frame as a reference region;
A second step of calculating a filter coefficient based on a local pattern similarity based on a region in the vicinity of the pixel to be filtered and the reference region obtained in the first step;
Causing the computer to execute a third step of performing a filtering process using the filter coefficient calculated in the second step on a region in the vicinity of the filtering target pixel ,
In the second step, a program for weighting the filter coefficient according to a ratio between a quantization step width in the processing frame and a quantization step width in the neighboring frame .

Images