JP5757565B2 - Noise level estimation device, noise level estimation method and program - Google Patents

Description

  The present invention relates to a digital image processing technique, and more particularly to a noise level estimation device, a noise level estimation method, and a program for estimating a noise level of a digital image from a digital image.

  In recent years, digital cameras are rapidly spreading as one of image acquisition apparatuses. In order to obtain high quality image quality, various digital image processing is performed on digital images taken with a digital camera. Among many digital image processing (hereinafter simply referred to as “image processing”), it is widely known that noise reduction processing is one of very important image processing because it greatly affects the image quality. Yes.

  By the way, in many noise reduction processes, for example, as disclosed in Non-Patent Document 1 and Non-Patent Document 2, it is assumed that the noise level is known. When performing the reduction process, it is necessary to set the noise level of the image in addition to the digital image (hereinafter also simply referred to as “image”) as an input.

  In general, in noise reduction processing, if the effect of noise reduction is increased, noise reduction is performed, but there is also a problem that the edge of the image is deteriorated at the same time. Therefore, it is necessary to adjust the noise reduction effect according to the noise level. For this reason, in many noise reduction processes, it is necessary to estimate the noise level. For this reason, a noise level estimation technique for estimating a noise level from a digital image is strongly desired.

  As an existing noise level estimation technique, for example, Patent Document 2 discloses a method for estimating a noise level of a video sequence using correlation between frames with respect to the video sequence.

  Also, in Patent Document 1, a color difference image is generated for a color digital image (hereinafter, also simply referred to as “color image”), divided for each band, and an edge portion is removed. An image processing method for estimating a noise level of the color image by calculating a standard deviation is disclosed.

  Further, Non-Patent Document 3 proposes a method of applying a filter to a digital image that is the same as a base of discrete cosine transform (DCT transform) and estimating a noise level from its statistical properties.

JP 2005-277503 A JP 2000-69487 A

Co-authored by J. Portilla, V. Strela, M. Wainwright, EP Simoncelli, “Image Denising Using Scale Mixture Off Gaussian Inn Image Denoising using Scale Mixture of Gaussians in the Wavelet Domain ", IEEE Transactions on Image Processing, Vol. 12, No. 12, p.1338-1351, 2003 Co-authored by K. Dabov, A. Foi, V. Katkovnik, and K. Egiazarian, “Image Denoising by Sparse 3D Transform Domain Collaborative Filtering (Image denoising by sparse 3Dtransform-domain collaborative filtering), IEEE Transactions on Image Processing, Vol. 16, No. 8, p.2080-2095, 2007 Co-authored by D. Zoran and Y. Weiss, “Scale Invariance and Noise in Natural Images”, International conference on computer vision ),2009

  However, the image processing method disclosed in Patent Document 1 targets a color image having a plurality of color channels (hereinafter also simply referred to as “multi-channel images”), and is a single target such as a grayscale image. Since an image having a color channel (hereinafter also simply referred to as “single channel image”) is not a processing target, the image processing method disclosed in Patent Document 1 cannot be applied to a single channel image.

  In other words, the image processing method disclosed in Patent Document 1 cannot estimate the noise level of a single channel image such as a grayscale image.

  The present invention has been made under the circumstances described above, and an object of the present invention is to obtain a noise level of an image from a single image including noise regardless of whether it is a multi-channel image or a single-channel image. It is an object to provide a noise level estimation device, a noise level estimation method, and a program capable of estimating the noise level easily and accurately.

  The present invention relates to a noise level estimation device that estimates the noise level of an image from a single image including noise, and the object of the present invention is to generate an image patch set from the image; A noise level estimation unit for estimating the noise level based on the image patch set generated by the image patch set generation unit, the noise level estimation unit based on the image patch set to the image patch set A variance-covariance matrix calculation unit that calculates a corresponding covariance matrix, a minimum eigenvalue calculation unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculation unit, and the minimum eigenvalue calculation A noise level calculation unit that calculates the noise level based on the minimum eigenvalue calculated by the unit, or from the image. An image patch set generation unit for generating a noise set, and a noise level estimation unit for estimating the noise level based on the image patch set generated by the image patch set generation unit, the noise level estimation unit An image patch average correction unit that generates an average corrected image patch set based on the image patch set; and a minimum singular value of the average corrected image patch set generated by the image patch average correction unit. This is effectively achieved by including a singular value calculation unit and a noise level calculation unit that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculation unit.

Further, the object of the present invention is a noise level estimation device for estimating the noise level of an image including noise from a single image, the high frequency component extraction unit extracting a high frequency component image from the image, and the high frequency A high frequency component image patch set generation unit that generates a high frequency component image patch set from the high frequency component image extracted by the component extraction unit; and a noise level estimation unit that estimates the noise level based on the high frequency component image patch set. The noise level estimating unit calculates a variance-covariance matrix corresponding to the high-frequency component image patch set based on the high-frequency component image patch set, and a variance-covariance matrix calculation unit, A minimum eigenvalue calculation unit that calculates a minimum eigenvalue of the calculated variance-covariance matrix, and a minimum eigenvalue calculated by the minimum eigenvalue calculation unit Based on, for calculating the noise level, by Rukoto a noise level calculation section, or a noise level estimation device for estimating a noise level of the image from a single image containing noise, from the image An image patch set generation unit that generates an image patch set, a high frequency component image patch extraction unit that extracts a high frequency component image patch set from the image patch set generated by the image patch set generation unit, and the high frequency component image patch set A noise level estimator for estimating the noise level , and the noise level estimator calculates a variance covariance matrix corresponding to the high-frequency component image patch set based on the high-frequency component image patch set. The variance-covariance matrix calculator and the minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculator are calculated. To, the minimum eigenvalue calculation unit, based on the minimum eigenvalue calculated in the minimum eigenvalue calculation unit, for calculating the noise level, by Rukoto a noise level calculation section, or from a single image containing noise A noise level estimation device for estimating a noise level of the image, the high frequency component extraction unit extracting a high frequency component image from the image, and a high frequency component image patch from the high frequency component image extracted by the high frequency component extraction unit A high-frequency component image patch set generation unit that generates a set; a noise level estimation unit that estimates the noise level based on the high-frequency component image patch set;
The noise level estimation unit is generated by the high frequency component image patch average correction unit and the high frequency component image patch average correction unit that generate an average corrected high frequency component image patch set based on the high frequency component image patch set. A minimum singular value calculating unit that calculates a minimum singular value of the average corrected high-frequency component image patch set, and a noise level that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculating unit. by providing a calculator, or a noise level estimation device for estimating a noise level of one of the image from an image including noise, and generates an image patch set from the image, and the image patch set generation unit A high frequency component image patch for extracting a high frequency component image patch set from the image patch set generated by the image patch set generation unit; And Ji extractor, based on the high-frequency component image patch set, to estimate the noise level, a noise level estimation unit, the noise level estimation unit, based on the high-frequency component image patch set, the average corrected high-frequency component A high-frequency component image patch average correction unit that generates an image patch set; and a minimum singular value calculation unit that calculates a minimum singular value of the average corrected high-frequency component image patch set generated by the high-frequency component image patch average correction unit; This is achieved effectively by including a noise level calculation unit that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculation unit.

ただし、

は画像パッチ

の分散を表し、Ｍは画像パッチ

の画素数を表し、

は画像パッチ

のｋ番目の画素の画素値を表すことにより、或いは、ノイズを含む一枚の画像から当該画像のノイズレベルを推定するノイズレベル推定装置であって、前記画像から画像パッチ集合を生成する、画像パッチ集合生成部と、前記画像パッチ集合生成部で生成された画像パッチ集合から、複数の画像パッチを選択し、選択した複数の画像パッチを選択画像パッチ集合とする、画像パッチ選択部と、前記画像パッチ選択部で得られた選択画像パッチ集合に基づき、前記ノイズレベルを推定する、ノイズレベル推定部とを備え、前記画像パッチ選択部は、前記画像パッチ集合に属する全ての画像パッチに対して、所定の指標算出方法に基づき、画像パッチの指標を算出する、指標算出部と、前記画像パッチ集合の中から、前記指標算出部で算出された指標の中で一番小さい指標から昇順に、又は、前記指標算出部で算出された指標の中で一番大きい指標から降順に、所定個数の指標に対応する画像パッチを選択し、選択した画像パッチを選択画像パッチ集合とする、上位指標に基づく画像パッチ選択部とを備え、前記指標算出部が、画像パッチ集合に属する全ての画像パッチに対して、次の数式に基づき、画像パッチの分散を画像パッチ毎に算出し、算出した画像パッチの分散を画像パッチの指標として出力し、

ただし、

は画像パッチ

の分散を表し、Ｍは画像パッチ

の画素数を表し、

は画像パッチ

のｋ番目の画素の画素値を表すことによって効果的に達成される。
Further, the object of the present invention is a noise level estimation device for estimating the noise level of an image from a single image including noise, an image patch set generation unit for generating an image patch set from the image, An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit and uses the selected plurality of image patches as a selected image patch set, and is obtained by the image patch selection unit. A noise level estimator for estimating the noise level based on the selected selected image patch set, wherein the image patch selector is configured to calculate a predetermined index for all image patches belonging to the image patch set. Based on the index calculation unit for calculating the index of the image patch and the index calculation unit for all image patches belonging to the image patch set. An image patch selection unit based on an index threshold, which selects an image patch by performing an image patch selection process based on the index threshold based on the index and a predetermined threshold, and sets the selected image patch as a selected image patch set; In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is less than or equal to the predetermined threshold, an image patch corresponding to the index is selected, and the index is An image patch corresponding to the index is not selected when the threshold is larger than the predetermined threshold, and the index calculation unit includes an x differentiation calculation unit, a y differentiation calculation unit, a differentiation matrix generation unit, and a maximum eigenvalue. A calculation unit, and the index calculation unit, for all image patches belonging to the image patch set, for each image patch, the x differentiation calculation unit The y derivative calculating unit calculates the y derivative of the image patch for each image patch, and the derivative matrix generating unit is configured to calculate the x derivative of the image patch calculated by the x derivative calculating unit, and the y Using the y derivative of the image patch calculated by the derivative calculating unit, a differential matrix of the image patch is generated for each image patch, and the maximum eigenvalue calculating unit calculates the maximum eigenvalue of the differential matrix of the image patch for each image patch. Then, by outputting the calculated maximum eigenvalue as an index of the image patch, or a noise level estimation device for estimating the noise level of the image from a single image including noise, the image patch set is obtained from the image. A plurality of image patches are selected from the image patch set generation unit to be generated and the image patch set generated by the image patch set generation unit, and the selected plurality of image patches are selected. An image patch selection unit, and a noise level estimation unit that estimates the noise level based on the selected image patch set obtained by the image patch selection unit. Calculating an index of an image patch for all image patches belonging to the image patch set based on a predetermined index calculation method, and calculating the index calculation unit from the image patch set The image patches corresponding to a predetermined number of indexes are selected and selected in ascending order from the smallest index among the calculated indices, or in descending order from the largest index among the indices calculated by the index calculation unit. And an image patch selection unit based on a higher index, wherein the index calculation unit includes an x differentiation calculation unit, a y differentiation calculation unit, a differentiation matrix generation unit, and a maximum A value calculation unit, wherein the index calculation unit calculates the x derivative of the image patch for each image patch for all image patches belonging to the image patch set, and The y differentiation calculation unit calculates the y differentiation of the image patch for each image patch, and the differentiation matrix generation unit calculates the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation calculation unit. The image patch differential matrix is generated for each image patch using the y derivative of the image patch, and the maximum eigenvalue calculation unit calculates the maximum eigenvalue of the image patch differential matrix for each image patch, A noise level estimation device for estimating a noise level of an image by outputting an eigenvalue as an index of an image patch, or from a single image including noise, wherein the image patch is obtained from the image. An image patch set generation unit that generates a combination, and a plurality of image patches selected from the image patch set generated by the image patch set generation unit, and the selected plurality of image patches as a selected image patch set A patch selection unit, and a noise level estimation unit that estimates the noise level based on the selected image patch set obtained by the image patch selection unit, wherein the image patch selection unit includes all of the image patch sets An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for the image patch, and an index calculation unit that calculates all image patches belonging to the image patch set. Based on the index and a predetermined threshold, image patch selection processing based on the index threshold is performed to select an image patch, and the selected image patch is selected as a selected image patch collection. An image patch selection unit based on an index threshold, and in the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and the index is less than or equal to the predetermined threshold The image patch corresponding to the index is selected, and when the index is larger than the predetermined threshold, the image patch corresponding to the index is not selected, and the index calculation unit includes an x differentiation calculation unit and , A y differential calculation unit, an xy differential matrix generation unit, and a maximum singular value calculation unit, and in the index calculation unit, for all image patches belonging to the image patch set, the x differential calculation unit is: The x derivative of the image patch is calculated for each image patch, the y derivative calculating unit calculates the y derivative of the image patch for each image patch, and the xy differentiation matrix generating unit is calculated by the x derivative calculating unit. The Using the x differentiation of the image patch and the y differentiation of the image patch calculated by the y differentiation calculation unit, an xy differentiation matrix of the image patch is generated for each image patch, and the maximum singular value calculation unit includes the image patch The maximum singular value of the xy differentiation matrix is calculated for each image patch, and the calculated maximum singular value is output as an index of the image patch, or the noise level of the image is estimated from one image including noise. A noise level estimation device that selects an image patch set from an image patch set generation unit that generates an image patch set from the image and an image patch set generated by the image patch set generation unit. Based on the selected image patch set obtained by the image patch selection unit and the image patch selection unit, a plurality of image patches as a selected image patch set, the noise level is set. A noise level estimation unit, wherein the image patch selection unit calculates an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set. And from the set of image patches, in ascending order from the smallest index calculated by the index calculation unit, or from the largest index calculated by the index calculation unit An image patch selection unit based on a higher index that selects image patches corresponding to a predetermined number of indexes in descending order and uses the selected image patches as a set of selected image patches, and the index calculation unit includes an x differentiation calculation unit A y differential calculation unit, an xy differential matrix generation unit, and a maximum singular value calculation unit. In the index calculation unit, for all image patches belonging to the image patch set, the x differential calculation unit includes: ,image The x derivative of the patch is calculated for each image patch, the y derivative calculating unit calculates the y derivative of the image patch for each image patch, and the xy differentiation matrix generating unit is calculated by the x derivative calculating unit. Using the x differentiation of the image patch and the y differentiation of the image patch calculated by the y differentiation calculation unit, an xy differentiation matrix of the image patch is generated for each image patch, and the maximum singular value calculation unit includes the image patch. The maximum singular value of the xy differentiation matrix is calculated for each image patch, and the calculated maximum singular value is output as an index of the image patch, or the noise level of the image is estimated from one image including noise. A noise level estimation device that generates an image patch set from the image, and a plurality of images from the image patch set generation unit and the image patch set generated by the image patch set generation unit A noise is estimated based on an image patch selection unit and a selected image patch set obtained by the image patch selection unit. A level estimation unit, and the image patch selection unit calculates an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set, and An image patch is selected and selected by performing an image patch selection process based on the index threshold value based on the index calculated by the index calculation unit and a predetermined threshold value for all image patches belonging to the image patch set. An image patch selection unit based on an index threshold that uses an image patch as a selected image patch set, and in the image patch selection process based on the index threshold, the index The predetermined threshold is compared, and if the index is less than or equal to the predetermined threshold, an image patch corresponding to the index is selected, and if the index is greater than the predetermined threshold, the image corresponding to the index A patch is not selected, and the index calculation unit calculates the variance of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and calculates the calculated image patch Is output as an index of the image patch,

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

A noise level estimation device for estimating a noise level of an image by representing a pixel value of a k-th pixel of the image or from a single image including noise, and generating an image patch set from the image A patch set generation unit, an image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set; and A noise level estimation unit for estimating the noise level based on the selected image patch set obtained by the image patch selection unit, the image patch selection unit for all image patches belonging to the image patch set An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method, and is calculated by the index calculation unit from the set of image patches Image patches corresponding to a predetermined number of indices are selected in ascending order from the smallest index among the indices, or in descending order from the largest index among the indices calculated by the index calculation unit, and the selected image An image patch selection unit based on a higher index that uses a patch as a selected image patch set, and the index calculation unit distributes image patches based on the following formula for all image patches belonging to the image patch set: Is calculated for each image patch, and the calculated variance of the image patch is output as an index of the image patch.

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

Is effectively achieved by representing the pixel value of the k th pixel of the.

  Further, the object of the present invention is to provide a variance covariance matrix calculation unit, wherein the noise level estimation unit calculates a variance covariance matrix corresponding to the selected image patch set based on the selected image patch set; A minimum eigenvalue calculation unit that calculates a minimum eigenvalue of a covariance matrix calculated by a covariance matrix calculation unit, and a noise level calculation that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculation unit Or the noise level estimation unit generates an average corrected selected image patch set based on the selected image patch set, and the selected image patch average correction unit. The minimum singular value calculation unit that calculates the minimum singular value of the average corrected selected image patch set generated in Step 1, and the minimum calculated by the minimum singular value calculation unit Based on different values, it calculates the noise level, effectively achieved by providing a noise level calculation section.

  Another object of the present invention is a noise level estimation device that estimates the noise level of an image by repeating noise level estimation from a single image including noise, the image patch set generation unit, an index A calculation unit; an image patch selection unit based on an index threshold; a noise level estimation unit; a threshold conversion unit; and a noise level estimation repetition end determination unit, wherein the image patch set generation unit generates an image patch set from the image The index calculation unit calculates an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set, and the image patch selection unit based on the index threshold value , For all image patches belonging to the set of image patches, the index threshold based on the index calculated by the index calculation unit and the threshold value from the threshold value conversion unit An image patch is selected based on the image patch, the selected image patch is set as a selected image patch set, and the noise level estimation unit is configured to select the noise patch based on the image patch set or the selected image patch set. Estimating the level, and outputting the estimated noise level to the noise level estimation repetition end determination unit, the threshold conversion unit converts the noise level from the noise level estimation repetition end determination unit to the threshold and converts When the threshold value is output to the image patch selection unit based on the index threshold value, and the noise level estimation repetition end determination unit determines the noise level estimation repetition end based on a predetermined determination condition, and does not determine the noise level estimation repetition end And outputting the noise level from the noise level estimation unit to the threshold value conversion unit, When it is determined that Izureberu estimation repeat end, the said noise level from the noise level estimation unit, the noise level estimation device effectively achieved by a noise level estimated.

  Another object of the present invention is to provide a noise level estimation device for estimating the noise level of a single image including noise, including a high frequency component extraction unit, a high frequency component image patch set generation unit, a low frequency A component extraction unit, a low frequency component image patch set generation unit, a low frequency component image patch selection unit, a corresponding high frequency component image patch extraction unit, and a noise level estimation unit, wherein the high frequency component extraction unit includes the image On the other hand, a high frequency component extraction process is performed to extract a high frequency component image of the image, and the high frequency component image patch set generation unit performs a high frequency component image patch set generation process on the high frequency component image. To generate a high-frequency component image patch set, and the low-frequency component extraction unit performs low-frequency component extraction processing on the image, thereby reducing the low-frequency component of the image. A low-frequency component image patch set generation unit performs a low-frequency component image patch set generation process on the low-frequency component image to generate a low-frequency component image patch set, The low-frequency component image patch selection unit selects a plurality of low-frequency component image patches from the set of low-frequency component image patches based on a predetermined image patch selection method, and selects the selected plurality of low-frequency component image patches. By selecting the corresponding low-frequency component image patch set, and the corresponding high-frequency component image patch extracting unit performs corresponding high-frequency component image patch extraction processing on all the low-frequency component image patches belonging to the selected low-frequency component image patch set. High frequency component image patches corresponding to all these low frequency component image patches are extracted as corresponding high frequency component image patches, and these extracted correspondences The frequency component image patch is set as a corresponding high frequency component image patch set, and the noise level estimation unit estimates the noise level based on the corresponding high frequency component image patch set. By processing to extract a high-frequency component image patch having the same pixel position as the image patch as a corresponding high-frequency component image patch, or the noise level estimator based on the corresponding high-frequency component image patch set, the corresponding high-frequency component image patch A variance-covariance matrix calculator that calculates a variance-covariance matrix corresponding to the component image patch set, and a minimum eigenvalue calculator that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculator The noise level is calculated based on the minimum eigenvalue calculated by the minimum eigenvalue calculation unit. Or a noise level estimation unit that generates an average corrected corresponding high-frequency component image patch set based on the corresponding high-frequency component image patch set; and a corresponding high-frequency component image patch average correction unit; A minimum singular value calculation unit that calculates a minimum singular value of the average corrected corresponding high frequency component image patch set generated by the corresponding high frequency component image patch average correction unit, and a minimum singular value calculated by the minimum singular value calculation unit. This is achieved effectively by including a noise level calculation unit that calculates the noise level based on the value.

  According to the noise level estimation apparatus according to the present invention, it is possible to easily and accurately estimate the noise level of an image from a single image containing noise.

  The inventors of the present invention, with respect to 100 digital images added with different levels of noise, noise level estimation results obtained by estimating the noise level using the conventional noise estimation method described in Non-Patent Document 3, The effect of the present invention was verified by comparing the noise level estimation results obtained by estimating the noise level using the present invention.

  Table 1 shows the average value of the noise level estimated by the conventional noise estimation method and the average value of the noise level estimated by the present invention for 100 digital images.

  As can be seen from Table 1, the present invention is more accurate than the conventional noise estimation method described in Non-Patent Document 3, particularly when the noise level is large, specifically, when the noise level is 15 or more. It was confirmed that the level could be estimated.

It is a block block diagram which shows 1st Embodiment (noise level estimation apparatus 1) of the noise level estimation apparatus which concerns on this invention. It is a schematic diagram for demonstrating the image patch set production | generation process performed in the image patch set production | generation part of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 1st Embodiment of the noise level estimation part 40 of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 2nd Embodiment of the noise level estimation part 40 of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 2nd Embodiment (noise level estimation apparatus 2) of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 3rd Embodiment (noise level estimation apparatus 3) of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 4th Embodiment (noise level estimation apparatus 4) of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 1st Embodiment of the image patch selection part 30 of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 2nd Embodiment of the image patch selection part 30 of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 5th Embodiment (noise level estimation apparatus 5) of the noise level estimation apparatus which concerns on this invention. It is a flowchart which shows the processing flow of the noise level estimation apparatus 5 of this invention shown in FIG. It is a block block diagram which shows one Embodiment of the parameter | index calculation part 310 of the noise level estimation apparatus which concerns on this invention. It is a block block diagram which shows 6th Embodiment (noise level estimation apparatus 6) of the noise level estimation apparatus which concerns on this invention.

  The present invention relates to a noise level estimation device, a noise level, and a noise level estimation device, which can easily and accurately estimate the noise level of a single digital image including noise regardless of whether it is a multi-channel image or a single-channel image. The present invention relates to a level estimation method and a program.

  Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a block diagram showing a first embodiment of a noise level estimation apparatus according to the present invention (hereinafter also simply referred to as “noise level estimation apparatus 1”). Hereinafter, the noise level estimation apparatus 1 will be described in detail with reference to FIG.

  As shown in FIG. 1, the noise level estimation device 1 estimates the noise level of an image from a single digital still image including noise (hereinafter, also simply referred to as “input image” or “image”). An image patch set generation unit 10 that generates an image patch set from an input image, and a noise level estimation that estimates a noise level of the input image based on the image patch set generated by the image patch set generation unit 10 Part 40.

  As described above, if the image is a digital still image, the noise level estimation technique according to the present invention can be applied to easily and accurately estimate the noise level of the image regardless of whether the image is a multi-channel image or a single channel image. it can.

  In addition, the noise level estimation technique according to the present invention can be applied to a frame of a moving image, not limited to a digital still image, and the noise level of the frame can be estimated easily and accurately. It can be said that it can also be applied to noise level estimation.

  Hereinafter, for convenience of explanation, the digital image input to the noise level estimation apparatus of the present invention is, for example, a single channel image such as a gray scale image (that is, an image having a single color channel). Assuming

  Further, when the noise level estimation technique according to the present invention is applied to a multi-channel image (a color image having a plurality of color channels), for each color channel, a single channel image as described below is used as the input image. What is necessary is just to repeat the process performed with the noise level estimation apparatus which concerns on this invention.

  As shown in FIG. 1, in the noise level estimation apparatus 1, first, the image patch set generation unit 10 generates an image patch set of the input image from the input image.

  Specifically, the image patch set generation unit 10 generates an image patch set by performing an image patch set generation process on the input image in order to obtain an image patch set from the input image. As shown in FIG. 2, the image patch set generation process is a process of generating a plurality of image patches while shifting the pixel position with respect to the input image.

  In the present invention, in the image patch set generation processing, the shift amount of the pixel position can be arbitrarily set. As a specific example of the shift amount, there is one pixel, that is, a plurality of pixel positions are shifted while shifting by one pixel. Generate an image patch.

  In the present invention, a plurality of image patches obtained by the image patch set generation processing are set as one set (that is, an image patch set). The image patch can be converted into a column vector by arranging the pixel values of all the pixels belonging to the image patch in a line. That is, one image patch corresponds to one column vector.

  Hereinafter, as a specific example of the image patch, a rectangular area of m pixels × n pixels in the image is set as an image patch (see FIG. 2). For convenience of explanation, in the following description, the image patch is a rectangular area (m pixels × n pixels). However, in the present invention, the image patch is not limited to the rectangular area, and an arbitrary shape (for example, (Polygon) can be used as an image patch.

  Next, in the noise level estimation apparatus 1, as shown in FIG. 1, the noise level estimation unit 40 estimates the noise level of the input image based on the image patch set generated by the image patch set generation unit 10.

  Thus, according to the noise level estimation apparatus 1 of the present invention, the noise level of the image can be estimated from the input image.

  Next, an embodiment of the noise level estimation unit 40 of the noise level estimation apparatus according to the present invention will be described in detail with reference to FIGS. The noise level estimation unit 40 estimates the noise level of the input image based on the image patch set generated by the image patch set generation unit 10.

  FIG. 3 is a block diagram showing a first embodiment of the noise level estimation unit 40 of the noise level estimation apparatus according to the present invention. FIG. 4 is a block diagram showing a second embodiment of the noise level estimation unit 40 of the noise level estimation apparatus according to the present invention.

  As shown in the first embodiment of the noise level estimation unit 40 in FIG. 3, the noise level estimation unit 40 estimates the noise level based on the minimum eigenvalue of the variance-covariance matrix corresponding to the image patch set. The variance covariance matrix calculation unit 410, the minimum eigenvalue calculation unit 420, and the noise level calculation unit 430 are configured.

  As shown in FIG. 3, in the noise level estimation unit 40, first, the variance / covariance matrix calculation unit 410 calculates a variance / covariance matrix corresponding to the image patch set based on the image patch set. Next, the minimum eigenvalue calculation unit 420 calculates the minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculation unit 410. Then, the noise level calculation unit 430 calculates the noise level of the input image based on the minimum eigenvalue calculated by the minimum eigenvalue calculation unit 420.

  Here, the processing performed by the variance-covariance matrix calculation unit 410, the minimum eigenvalue calculation unit 420, and the noise level calculation unit 430 will be specifically described with reference to mathematical expressions.

は、下記数１によって表すことができる。以下、この行列

を単に「画像パッチ集合」とも言う。 As described above, one image patch corresponds to one column vector. Therefore, the image patch set can be set as a set of column vectors, and a matrix of column vectors arranged to mean the image patch set.

Can be represented by Equation 1 below. Hereafter, this matrix

Is also simply referred to as “image patch set”.

ここで、

は入力画像におけるｉ番目の画像パッチに対応する列ベクトルを表す。以下、この列ベクトル

を単に「ｉ番目の画像パッチ」とも言う。また、Ｎは画像パッチ集合の要素数を表す。つまり、Ｎは入力画像から得られた複数の画像パッチの個数である。

here,

Represents a column vector corresponding to the i-th image patch in the input image. Here is the column vector

Is also simply referred to as the “i th image patch”. N represents the number of elements in the image patch set. That is, N is the number of a plurality of image patches obtained from the input image.

（即ち、各画像パッチに対し、画像パッチの平均を補正して得られた、画像パッチ集合）は、下記数２によって表すことができる。以下、この行列

を単に「平均補正済み画像パッチ集合」とも言う。 Next, the matrix obtained by subtracting the average of the image patches from each image patch

(That is, a set of image patches obtained by correcting the average of image patches for each image patch) can be expressed by the following equation (2). Hereafter, this matrix

Is simply referred to as “average corrected image patch set”.

ここで、

は画像パッチの平均であり、下記数３に基づいて算出される。

here,

Is the average of the image patches, and is calculated based on the following equation (3).

そして、画像パッチ集合

に対応する分散共分散行列

は、下記数４に基づいて算出される。

And the image patch set

Variance-covariance matrix corresponding to

Is calculated based on Equation 4 below.

分散共分散行列算出部４１０が、上記数１、数２、数３及び数４に基づき、画像パッチ集合

から画像パッチ集合

に対応する分散共分散行列

を算出するための分散共分散行列算出処理を行うことにより、画像パッチ集合

に対応する分散共分散行列

を算出する。

The variance-covariance matrix calculation unit 410 calculates the image patch set based on Equation 1, Equation 2, Equation 3, and Equation 4 above.

Image patch set from

Variance-covariance matrix corresponding to

Image patch set by performing a variance-covariance matrix calculation process to calculate

Variance-covariance matrix corresponding to

Is calculated.

に対応する分散共分散行列

の最小固有値

を算出する。 Next, the minimum eigenvalue calculation unit 420 performs a minimum eigenvalue calculation process, whereby an image patch set is obtained.

Variance-covariance matrix corresponding to

Minimum eigenvalue of

Is calculated.

を算出する。 Finally, the noise level calculation unit 430 performs the noise level calculation process based on the following formula 5 to obtain the noise level of the input image.

Is calculated.

ここで、

は入力画像のノイズレベルであり、即ち、入力画像のノイズの標準偏差に対応する値でもある。

here,

Is the noise level of the input image, that is, a value corresponding to the standard deviation of the noise of the input image.

  The first embodiment of the noise level estimator 40 of the noise level estimator according to the present invention has been described above. Hereinafter, the second embodiment of the noise level estimator 40 of the noise level estimator according to the present invention will be described. Will be described.

  In the first embodiment of the noise level estimation unit 40 of the present invention, the noise level is estimated based on the minimum eigenvalue of the variance-covariance matrix corresponding to the image patch set, whereas the noise level of the present invention is The second embodiment of the estimation unit 40 is characterized in that the noise level is estimated based on the minimum singular value of the average corrected image patch set.

  Here, the principle of noise level estimation according to the second embodiment of the noise level estimation unit 40 of the present invention will be described.

の間に、

の関係が成立する。本発明では、この関係に着目し、入力画像のノイズレベルを推定する際に、画像パッチ集合に対応する分散共分散行列の最小固有値を算出し、算出した最小固有値に基づき、入力画像のノイズレベルを算出する（即ち、本発明のノイズレベル推定部４０の第１実施形態）かわりに、平均補正済み画像パッチ集合の最小特異値を算出し、算出した最小特異値に基づき、入力画像のノイズレベルを算出する（即ち、本発明のノイズレベル推定部４０の第２実施形態）ようにしても良い。 In general, matrix

Between,

The relationship is established. In the present invention, paying attention to this relationship, when estimating the noise level of the input image, the minimum eigenvalue of the variance-covariance matrix corresponding to the image patch set is calculated, and the noise level of the input image is calculated based on the calculated minimum eigenvalue. (Ie, the first embodiment of the noise level estimator 40 of the present invention), the minimum singular value of the average corrected image patch set is calculated, and the noise level of the input image is calculated based on the calculated minimum singular value. (That is, the second embodiment of the noise level estimation unit 40 of the present invention) may be calculated.

  As shown in the second embodiment of the noise level estimation unit 40 of FIG. 4, the noise level estimation unit 40 estimates the noise level based on the minimum singular value of the average corrected image patch set, The image patch average correction unit 411, the minimum singular value calculation unit 421, and the noise level calculation unit 431 are included.

  As shown in FIG. 4, in the noise level estimation unit 40, first, the image patch average correction unit 411 generates an average corrected image patch set based on the image patch set. Next, the minimum singular value calculation unit 421 calculates the minimum singular value of the average corrected image patch set generated by the image patch average correction unit 411. Then, the noise level calculation unit 431 calculates the noise level of the input image based on the minimum singular value calculated by the minimum singular value calculation unit 421.

  Here, the processing performed by the image patch average correction unit 411, the minimum singular value calculation unit 421, and the noise level calculation unit 431 will be specifically described with reference to mathematical expressions.

から平均補正済み画像パッチ集合

を生成するための平均補正済み画像パッチ集合生成処理を行うことにより、平均補正済み画像パッチ集合

を算出する。 First, the image patch average correction unit 411 performs the image patch set based on the above formulas 1, 2, and 3.

Average corrected image patch set

The average corrected image patch set is generated by performing the average corrected image patch set generation process.

Is calculated.

の最小特異値

を算出する。 Next, the minimum singular value calculation unit 421 performs the minimum singular value calculation process, thereby performing the average corrected image patch set.

Minimum singular value of

Is calculated.

を算出する。 Finally, the noise level calculation unit 431 performs a noise level calculation process based on the following Equation 6 to obtain a noise level of the input image.

Is calculated.

ここで、

は入力画像のノイズレベルであり、即ち、入力画像のノイズの標準偏差に対応する値でもある。

here,

Is the noise level of the input image, that is, a value corresponding to the standard deviation of the noise of the input image.

  FIG. 5 is a block diagram showing a second embodiment of the noise level estimation device according to the present invention (hereinafter also simply referred to as “noise level estimation device 2”). Hereinafter, the noise level estimation apparatus 2 will be described in detail with reference to FIG.

  As shown in FIG. 5, the noise level estimation device 2 is a device for estimating a noise level of an image from an input image, and includes a high frequency component extraction unit 20 that extracts a high frequency component image from the input image, and a high frequency component. Based on the high-frequency component image patch set generation unit 11 that generates a high-frequency component image patch set from the high-frequency component image extracted by the extraction unit 20, and based on the high-frequency component image patch set generated by the high-frequency component image patch set generation unit 11 And a noise level estimation unit 40 for estimating the noise level of the image.

  As shown in FIG. 5, in the noise level estimation device 2, first, the high frequency component extraction unit 20 performs a high frequency component extraction process on the input image, thereby extracting a high frequency component image of the input image.

  Here, as a specific example of the high-frequency component extraction processing, a high-frequency component image of the input image may be extracted by applying a high-pass filter to the input image.

  As another specific example of the high-frequency component extraction processing, first, a low-frequency component image of the input image is generated by applying a low-pass filter (for example, a low-pass filter such as a Gaussian filter or a bilateral filter) to the input image. Then, the high frequency component image of the input image may be extracted by subtracting the generated low frequency component image from the input image.

  That is, the “high frequency component extraction process” referred to in the present invention is a process for extracting a high frequency component image of an input image, and is not limited to the two specific examples described above. If the process can be extracted, the process can be referred to as the “high frequency component extraction process” of the present invention.

  Next, in the noise level estimation device 2, as shown in FIG. 5, the high frequency component image patch set generation unit 11 generates a high frequency component image patch set from the high frequency component images extracted by the high frequency component extraction unit 20.

  Specifically, the high-frequency component image patch set generation unit 11 performs high-frequency component image patch set generation processing on the high-frequency component image to obtain a high-frequency component image patch set from the high-frequency component image. Create a patch set.

  The high-frequency component image patch set generation process is basically the same as the above-described image patch set generation process, and thus detailed description thereof is omitted.

  However, the image patch set generation process is a process of generating a plurality of image patches while shifting the pixel position with respect to the input image, whereas the high frequency component image patch set generation process is performed on the high frequency component image. This is a process for generating a plurality of high-frequency component image patches while shifting the pixel position.

  In the present invention, a plurality of high-frequency component image patches obtained by the high-frequency component image patch set generation processing are set as one set (that is, a high-frequency component image patch set).

  Next, in the noise level estimation device 2, as shown in FIG. 5, the noise level estimation unit 40 determines the noise level of the input image based on the high frequency component image patch set generated by the high frequency component image patch set generation unit 11. presume.

  Here, the processing performed in the noise level estimation unit 40 of the noise level estimation device 2 is basically the same as the processing performed in the noise level estimation unit 40 of the noise level estimation device 1 described above. Detailed description is omitted.

  However, the processing performed by the noise level estimation unit 40 of the noise level estimation device 1 is performed by the noise level estimation unit 40 of the noise level estimation device 2 whereas the processing performed on the image patch set is performed. The process is a process performed on the high-frequency component image patch set.

  That is, the first embodiment (see FIG. 3) and the second embodiment (see FIG. 4) of the noise level estimation unit 40 described above can be applied to the noise level estimation unit 40 of the noise level estimation device 2. However, in the first embodiment and the second embodiment of the noise level estimation unit 40, an image patch set is input, whereas in the noise level estimation unit 40 of the noise level estimation device 2, a high frequency component image patch set is input. It is said.

  Thus, according to the noise level estimation apparatus 2 of the present invention, the noise level of the image can be estimated from the input image.

  FIG. 6 is a block diagram showing a third embodiment of the noise level estimation apparatus according to the present invention (hereinafter also simply referred to as “noise level estimation apparatus 3”). Hereinafter, the noise level estimation apparatus 3 will be described in detail with reference to FIG.

  As shown in FIG. 6, the noise level estimation device 3 is a device for estimating the noise level of an image from an input image, and an image patch set generation unit 10 that generates an image patch set from the input image, A high-frequency component image patch extraction unit 21 that extracts a high-frequency component image patch set from the image patch set generated by the patch set generation unit 10 and an input based on the high-frequency component image patch set extracted by the high-frequency component image patch extraction unit 21 And a noise level estimation unit 40 for estimating the noise level of the image.

  As shown in FIG. 6, in the noise level estimation device 3, first, the image patch set generation unit 10 generates an image patch set of the input image from the input image.

  Here, the processing performed in the image patch set generation unit 10 of the noise level estimation device 3 is the same as the processing performed in the image patch set generation unit 10 of the noise level estimation device 1 described above. Description is omitted.

  Next, in the noise level estimation device 3, as shown in FIG. 6, the high frequency component image patch extraction unit 21 applies high frequency components to all image patches belonging to the image patch set generated by the image patch set generation unit 10. By performing image patch extraction processing, high-frequency component image patches of all these image patches are extracted, and these extracted high-frequency component image patches are set as a high-frequency component image patch set.

  Here, a specific example of the high frequency component image patch extraction process will be described. For example, in the high-frequency component image patch extraction process, the image patch is first approximated with a linear function of the image patch as the low-frequency component image patch of the image patch, and then the low-frequency component is extracted from the image patch. By subtracting the image patch, the high frequency component image patch of the image patch is extracted.

  That is, the “high-frequency component image patch extraction process” referred to in the present invention is a process for extracting a high-frequency component image patch set from an image patch set, and the present invention is not limited to the specific example described above. Any process that can extract a high-frequency component image patch set from a patch set can be referred to as the “high-frequency component image patch extraction process” of the present invention.

  Next, in the noise level estimation device 3, as shown in FIG. 6, the noise level estimation unit 40 estimates the noise level of the input image based on the high frequency component image patch set extracted by the high frequency component image patch extraction unit 21. To do.

  Here, since the process performed in the noise level estimation unit 40 of the noise level estimation device 3 is the same as the process performed in the noise level estimation unit 40 of the noise level estimation device 2 described above, a detailed description thereof will be given. Omitted.

  Thus, according to the noise level estimation apparatus 3 of the present invention, the noise level of the image can be estimated from the input image.

  FIG. 7 is a block diagram showing a fourth embodiment of the noise level estimation apparatus according to the present invention (hereinafter also simply referred to as “noise level estimation apparatus 4”). Hereinafter, the noise level estimation apparatus 4 will be described in detail with reference to FIG.

  As shown in FIG. 7, the noise level estimation device 4 is a device for estimating the noise level of an image from an input image, and includes an image patch set generation unit 10 that generates an image patch set from the input image, An image patch selection unit 30 that selects a plurality of image patches from the image patch set generated by the patch set generation unit 10 and uses the selected plurality of image patches as a selected image patch set; The noise level estimation unit 40 estimates the noise level of the input image based on the set of selected image patches.

  As shown in FIG. 7, in the noise level estimation device 4, first, the image patch set generation unit 10 generates an image patch set of the input image from the input image.

  Here, the processing performed in the image patch set generation unit 10 of the noise level estimation device 4 is the same as the processing performed in the image patch set generation unit 10 of the noise level estimation device 1 described above. Description is omitted.

  Next, in the noise level estimation apparatus 4, as shown in FIG. 7, the image patch selection unit 30 first applies a predetermined image patch selection method from the image patch set generated by the image patch set generation unit 10. Based on this, a plurality of image patches are selected, and then the selected plurality of image patches are set as a selected image patch set.

  Next, in the noise level estimation device 4, as shown in FIG. 7, the noise level estimation unit 40 estimates the noise level of the input image based on the selected image patch set obtained by the image patch selection unit 30.

  Here, the processing performed in the noise level estimation unit 40 of the noise level estimation device 4 is basically the same as the processing performed in the noise level estimation unit 40 of the noise level estimation device 1 described above. Detailed description is omitted.

  However, the processing performed by the noise level estimation unit 40 of the noise level estimation device 1 is performed by the noise level estimation unit 40 of the noise level estimation device 4 whereas the processing performed on the image patch set. The process is a process performed on the selected image patch set.

  That is, the first embodiment (see FIG. 3) and the second embodiment (see FIG. 4) of the noise level estimation unit 40 described above can be applied to the noise level estimation unit 40 of the noise level estimation device 4. However, in the first and second embodiments of the noise level estimation unit 40, the image patch set is input, whereas in the noise level estimation unit 40 of the noise level estimation device 4, the selected image patch set is input. Yes.

  Thus, according to the noise level estimation apparatus 4 of the present invention, the noise level of the image can be estimated from the input image.

  Here, an embodiment of the image patch selection unit 30 of the noise level estimation apparatus according to the present invention will be described in detail with reference to FIGS. 8 and 9. The image patch selection unit 30 generates a selected image patch set based on the image patch set generated by the image patch set generation unit 10.

  FIG. 8 is a block diagram showing a first embodiment of the image patch selection unit 30 of the noise level estimation apparatus according to the present invention. FIG. 9 is a block diagram showing a second embodiment of the image patch selection unit 30 of the noise level estimation apparatus according to the present invention.

  As shown in the first embodiment of the image patch selection unit 30 in FIG. 8, the image patch selection unit 30 includes an index calculation unit 310 and an image patch selection unit 320 based on an index threshold.

  In the first embodiment of the image patch selection unit 30, as shown in FIG. 8, first, the index calculation unit 310 performs the processing for all image patches belonging to the image patch set generated by the image patch set generation unit 10. Based on a predetermined index calculation method, the index of these image patches is calculated. Here, since one index is calculated based on one image patch, this index is hereinafter also referred to as “image patch index”.

  Next, in the first embodiment of the image patch selection unit 30, the image patch selection unit 320 based on the index threshold is calculated by the index calculation unit 310 from the image patch set generated by the image patch set generation unit 10. An image patch is selected based on the selected index and a predetermined threshold, and the selected image patch is set as a selected image patch set. The predetermined threshold is a value set in advance.

  Specifically, the image patch selection unit 320 based on the index threshold selects an image patch by performing image patch selection processing based on the index threshold for all image patches belonging to the image patch set, and selects the selected image. The patch is a selected image patch set.

  Here, in the image patch selection process based on the index threshold, the index of the image patch is compared with a predetermined threshold, and when the index is equal to or less than the predetermined threshold, the image patch corresponding to the index is selected, When the index is larger than a predetermined threshold, the image patch corresponding to the index is not selected.

  The first embodiment of the image patch selection unit 30 of the noise level estimation apparatus according to the present invention has been described above. Hereinafter, the second embodiment of the image patch selection unit 30 of the noise level estimation apparatus according to the present invention will be described. Will be described.

  As shown in the second embodiment of the image patch selection unit 30 in FIG. 9, the image patch selection unit 30 includes an index calculation unit 310 and an image patch selection unit 321 based on a higher index.

  In the second embodiment of the image patch selection unit 30, as shown in FIG. 9, first, the index calculation unit 310 applies to all image patches belonging to the image patch set generated by the image patch set generation unit 10. Based on a predetermined index calculation method, the index of these image patches is calculated.

  Next, in the second embodiment of the image patch selection unit 30, the image patch selection unit 321 based on the higher index corresponds to a predetermined number of higher indexes from the image patch set generated by the image patch set generation unit 10. An image patch to be selected is selected, and the selected image patch is set as a selected image patch set.

  Here, the predetermined number is, for example, a number corresponding to what percentage (for example, 25%) of the total number of all image patches belonging to the image patch set generated by the image patch set generation unit 10. Of course, in the present invention, the predetermined number is not limited to a percentage corresponding to what percentage of the total number of all image patches, and the predetermined number may be determined by another method.

  In addition, the “image patch corresponding to a predetermined number of higher indices” refers to the index calculated in ascending order from the smallest index calculated by the index calculation unit 310 or the index calculated by the index calculation unit 310. Means image patches corresponding to a predetermined number of indices in descending order from the largest index.

  That is, the image patch selection unit 321 based on the upper index is calculated by the index calculation unit 310 in ascending order from the smallest index among the indexes calculated by the index calculation unit 310 from the image patch set. Image patches corresponding to a predetermined number of indices are selected in descending order from the largest index among the indices, and the selected image patches are set as a selected image patch set.

  FIG. 10 is a block diagram showing a fifth embodiment of the noise level estimation device according to the present invention (hereinafter also simply referred to as “noise level estimation device 5”). FIG. 11 is a flowchart showing the processing flow of the noise level estimation apparatus 5 of the present invention shown in FIG. Hereinafter, the noise level estimation device 5 will be described in detail with reference to FIGS. 10 and 11.

  As shown in FIG. 10, the noise level estimation device 5 is a device for estimating the noise level of an image from an input image, and is based on an image patch set generation unit 12, an index calculation unit 310, and an index threshold. The image patch selection unit 320, the noise level estimation unit 41, the noise level estimation repetition end determination unit 50, and the threshold conversion unit 60 are configured.

  The biggest feature of the noise level estimation device 5 is that the noise level is estimated by repeating the noise level estimation process. Specifically, the noise level estimated by the noise level estimation unit 41 is converted into an image patch. The threshold value necessary for selection is converted, and then a selected image patch set is generated based on the converted threshold value, the image patch set, and the index. Further, based on the generated selected image patch set, a noise level estimation unit 41, that is, the noise level estimation repetition end determination unit 50 while converting a threshold necessary for selecting an image patch based on the noise level estimated by the noise level estimation unit 41. The noise level of the input image is repeatedly estimated until it is determined that the noise level estimation repetition has been completed.

  Here, the flow of processing performed by the noise level estimation device 5 in order to estimate the noise level of the image from the input image will be described with reference to FIGS. 10 and 11.

  First, in the noise level estimation device 5, the image patch set generation unit 12 generates an image patch set of the input image from the input image and sets the number k of repetitions of the noise level estimation process to 0 (FIG. 11). (See step S100).

  Here, the processing excluding the processing of setting the number of repetitions k to 0 performed by the image patch set generation unit 12 of the noise level estimation device 5 is performed by the image patch set generation unit 10 of the noise level estimation device 1 described above. Therefore, detailed description thereof is omitted.

  Next, in the noise level estimation device 5, the index calculation unit 310 applies these image patches to all image patches belonging to the image patch set generated by the image patch set generation unit 12 based on a predetermined index calculation method. (See step S110 in FIG. 11).

  Here, the process performed by the index calculation unit 310 of the noise level estimation device 5 is the same as the process performed by the index calculation unit 310 in the first embodiment (see FIG. 8) of the image patch selection unit 30 described above. Therefore, detailed description thereof is omitted.

を推定する（図１１のステップＳ１２０を参照）。ノイズレベル推定部４１で推定されたノイズレベル

は、ノイズレベル推定繰り返し終了判定部５０に入力される。ここで、繰り返し回数ｋの値が０であるため、画像パッチ集合に基づいて推定されたノイズレベル

となる。 Next, in the noise level estimation device 5, the noise level estimation unit 41 is based on the image patch set obtained by the image patch set generation unit 12, and the noise level

Is estimated (see step S120 in FIG. 11). Noise level estimated by the noise level estimation unit 41

Is input to the noise level estimation repetition end determination unit 50. Here, since the value of the number of repetitions k is 0, the noise level estimated based on the image patch set

It becomes.

  Note that the processing performed by the noise level estimation unit 41 of the noise level estimation device 5 is the same as the processing performed by the noise level estimation unit 40 of the noise level estimation device 1 described above, and thus detailed description thereof is omitted. To do.

に基づき、ノイズレベル推定繰り返し終了に関する第１の判定条件、即ち、ｋ＞０を満たすか否かを判定する（図１１のステップＳ１３０を参照）。 Next, in the noise level estimation device 5, the noise level estimation repetition end determination unit 50 receives the noise level input from the noise level estimation unit 41.

Based on the above, it is determined whether or not the first determination condition regarding the end of repeated noise level estimation, that is, k> 0 is satisfied (see step S130 of FIG. 11).

であるため、ｋの値は０となり、第１の判定条件（ｋ＞０）を満たさない。よって、ノイズレベル推定繰り返し終了判定部５０が、ノイズレベル推定繰り返し終了と判定せず、ノイズレベル

を閾値変換部６０に出力するようにする。 At this time, the noise level input from the noise level estimation unit 41 to the noise level estimation repetition end determination unit 50 is:

Therefore, the value of k is 0, which does not satisfy the first determination condition (k> 0). Therefore, the noise level estimation repetition end determination unit 50 does not determine that the noise level estimation repetition end has ended, and the noise level

Is output to the threshold converter 60.

を閾値τに変換し、変換した閾値τを指標閾値に基づく画像パッチ選択部３２０に出力する（図１１のステップＳ１５０を参照）。 Next, in the noise level estimation device 5, the threshold level conversion unit 60 performs the threshold level conversion process, whereby the noise level from the noise level estimation repetition end determination unit 50 is detected.

Is converted into a threshold value τ, and the converted threshold value τ is output to the image patch selection unit 320 based on the index threshold value (see step S150 in FIG. 11).

を利用することにより、ノイズレベル

を閾値τに変換する。ここで言う「単調増加関数」とは、ｘ＜ｙであれば、ｆ(ｘ)＜ｆ(ｙ)を満足する関数である。 Specifically, in the threshold conversion process, a monotonically increasing function

By using the noise level

Is converted to a threshold value τ. The “monotonically increasing function” here is a function that satisfies f (x) <f (y) if x <y.

である。つまり、下記数７に基づき、閾値変換処理を行うことにより、ノイズレベル

を閾値τに変換する。 As an example of a specific monotonically increasing function, for example,

It is. That is, the noise level is obtained by performing the threshold value conversion process based on the following formula 7.

Is converted to a threshold value τ.

ただし、γは実験的に設定する所定のパラメータである。γの一例としては、例えば、３である。

However, γ is a predetermined parameter set experimentally. An example of γ is 3, for example.

  Next, in the noise level estimation device 5, the image patch selection unit 320 based on the index threshold value includes the index calculated by the index calculation unit 310 from the image patch set generated by the image patch set generation unit 12, and Based on the threshold τ converted by the threshold converter 60, an image patch is selected, and the selected image patch is set as a selected image patch set (see step S160 in FIG. 11).

  Here, the processing performed in the image patch selection unit 320 based on the index threshold of the noise level estimation device 5 is the image patch selection based on the index threshold in the first embodiment (see FIG. 8) of the image patch selection unit 30 described above. Since the processing is the same as that performed by the unit 320, detailed description thereof is omitted. Incidentally, the threshold τ converted by the threshold converter 60 is used as a predetermined threshold.

  Next, in the noise level estimation device 5, the value of the number of repetitions k is increased by 1 (see step S170 in FIG. 11).

を推定する（図１１のステップＳ１８０を参照）。ノイズレベル推定部４１で推定されたノイズレベル

は、ノイズレベル推定繰り返し終了判定部５０に入力される。 Next, in the noise level estimation device 5, the noise level estimation unit 41 determines the noise level based on the selected image patch set obtained by the image patch selection unit 320 based on the index threshold.

Is estimated (see step S180 in FIG. 11). Noise level estimated by the noise level estimation unit 41

Is input to the noise level estimation repetition end determination unit 50.

  At this time, the processing performed in the noise level estimation unit 41 of the noise level estimation device 5 (that is, the processing in step S180) is the same as the processing performed in the noise level estimation unit 40 of the noise level estimation device 4 described above. Since they are the same, detailed description thereof is omitted.

に基づき、ノイズレベル推定繰り返し終了に関する第２の判定条件、即ち、

が十分小さくなっていること（その一例として、

ここで、所定の値は、例えば、０.０１である。）を満たすか否かを判定する（図１１のステップＳ１４０を参照）。 Next, in the noise level estimation device 5, the noise level estimation repetition end determination unit 50 receives the noise level input from the noise level estimation unit 41.

Based on the second determination condition regarding the end of the noise level estimation repetition, that is,

Is sufficiently small (for example,

Here, the predetermined value is, for example, 0.01. ) Is satisfied (see step S140 in FIG. 11).

を閾値変換部６０に出力する。 In step S140, when the noise level estimation repetition end determination unit 50 determines that the second determination condition is not satisfied, that is, when it is not determined that the noise level estimation repetition ends, the noise level is determined.

Is output to the threshold converter 60.

を閾値τに変換し、変換した閾値τを指標閾値に基づく画像パッチ選択部３２０に出力する（図１１のステップＳ１５０を参照）。なお、図１１のステップＳ１５０での処理は既述したため、その説明は省略する。 In the noise level estimation device 5, the threshold level conversion unit 60 performs the threshold value conversion process, whereby the noise level from the noise level estimation repetition end determination unit 50 is detected.

Is converted into a threshold value τ, and the converted threshold value τ is output to the image patch selection unit 320 based on the index threshold value (see step S150 in FIG. 11). Since the processing in step S150 in FIG. 11 has already been described, the description thereof is omitted.

  Next, in the noise level estimation device 5, the image patch selection unit 320 based on the index threshold value includes the index calculated by the index calculation unit 310 from the image patch set generated by the image patch set generation unit 12, and Based on the threshold τ converted by the threshold converter 60, an image patch is selected, and the selected image patch is set as a selected image patch set (see step S160 in FIG. 11). Since the processing in step S160 in FIG. 11 has already been described, the description thereof is omitted.

  Next, in the noise level estimation device 5, the value of the number of repetitions k is increased by 1 (see step S170 in FIG. 11).

を推定する（図１１のステップＳ１８０を参照）。ノイズレベル推定部４１で推定されたノイズレベル

は、ノイズレベル推定繰り返し終了判定部５０に入力される。 Next, in the noise level estimation device 5, the noise level estimation unit 41 determines the noise level based on the selected image patch set obtained by the image patch selection unit 320 based on the index threshold.

Is estimated (see step S180 in FIG. 11). Noise level estimated by the noise level estimation unit 41

Is input to the noise level estimation repetition end determination unit 50.

に基づき、ノイズレベル推定繰り返し終了に関する第２の判定条件を満たすか否かを判定する（図１１のステップＳ１４０を参照）。 Then, the noise level estimation repetition end determination unit 50 receives the noise level input from the noise level estimation unit 41.

Based on the above, it is determined whether or not the second determination condition regarding the end of repeated noise level estimation is satisfied (see step S140 in FIG. 11).

  As described above, until the noise level estimation repetition end determination unit 50 determines that the second determination condition is satisfied, the noise level estimation apparatus 5 performs the processing in step S140, the processing in step S150, and the step S160. The process in step S170 and the process in step S180 are repeated.

を、ノイズレベル推定装置５が推定したノイズレベルとして外部に出力する（図１１のステップＳ１９０を参照）。これで、ノイズレベル推定装置５での処理が終了する。 On the other hand, when the noise level estimation repetition end determination unit 50 determines in step S140 that the second determination condition is satisfied, that is, when it is determined that the noise level estimation repetition end has been completed,

Is output to the outside as the noise level estimated by the noise level estimation device 5 (see step S190 in FIG. 11). This completes the processing in the noise level estimation device 5.

を、入力画像のノイズレベルとして出力する。 That is, in the noise level estimation device 5 of the present invention, when the first determination condition is satisfied and the second determination condition is also satisfied (that is, the first determination condition and the second determination condition). Noise level when it is determined that the noise level estimation repetition has been completed and the noise level estimation repetition has been completed.

Are output as the noise level of the input image.

  Here, in the noise level estimation device of the present invention, the reason for estimating the noise level of the input image based on the selected image patch set, which is composed of only the selected image patches, is selected from the image patch set. explain.

  When estimating the noise level of an image including noise using the noise level estimation device of the present invention (the noise level estimation device 1, the noise level estimation device 2, and the noise level estimation device 3 described above) that does not select an image patch. If the image patch set generated from the image including noise is an image patch set including only a flat region not including an image texture or an edge, the noise level of the image can be accurately estimated.

  By the way, because it is difficult to separate the signal energy of noise from the signal energy of a noise-free image, the image patch set generated from the image including noise includes an image including the signal energy of images such as textures and edges. If it is a patch set, when estimating the noise level of an image including noise using the noise level estimation apparatus of the present invention that does not select an image patch, compared to the true noise level of the image including noise, A large noise level may be estimated.

  Therefore, in order to eliminate such a possibility, the inventors of the present invention select a noise level estimation device (the noise level estimation device 4 described above) that selects an image patch used for noise level estimation from the image patch set. Invented the noise level estimation device 5 and the noise level estimation device 6) described later.

  That is, in the present invention, when selecting an image patch from the set of image patches, only an image patch in a flat region is selected. Therefore, as shown in an embodiment of the index calculation unit 310 of the present invention to be described later, an image patch is selected. The image patch is selected by using the maximum eigenvalue of the differential matrix, the maximum singular value of the xy differential matrix of the image patch, or the variance of the image patch as an index of the image patch.

  The image patch selected as the image patch index using the maximum eigenvalue of the differentiation matrix of the image patch, the maximum singular value of the xy differentiation matrix of the image patch, or the distribution of the image patch as an index of the image patch is selected as described above In the case where the noise level of an image is estimated based on a selected image patch set composed of only the selected image patches, the noise level of the image can be accurately estimated.

  Thus, when estimating the noise level of an image including noise using the noise level estimation apparatus of the present invention, it is necessary to select an image patch from an image patch set generated from the image including noise.

  However, a threshold value suitable for selecting only flat area image patches depends on the noise level included in the image patch. Therefore, it may be difficult to set an appropriate threshold value in advance.

  Therefore, in view of such a situation, the inventors of the present invention select only a flat area image patch with an appropriate threshold, and based on the estimated noise level, the flat area image patch. The technical idea of performing the noise level of the image by repetitive processing while resetting (converting) the threshold for selecting only the image is applied to the noise level estimation device 5 described above.

  According to the noise level estimation apparatus 5 of the present invention, the noise level can be accurately estimated even for an image patch set including image signal energy such as texture and edge.

  Next, an embodiment of the index calculation unit 310 of the noise level estimation apparatus according to the present invention will be described. The index calculation unit 310 calculates the index of these image patches for all image patches belonging to the image patch set generated by the image patch set generation unit 10 based on a predetermined index calculation method.

  FIG. 12 is a block configuration diagram showing an embodiment of the index calculation unit 310 of the noise level estimation apparatus according to the present invention.

  As shown in FIG. 12, the index calculation unit 310 calculates, for each image patch, the maximum eigenvalue of a differentiation matrix formed by the differentiation of the image patch as an index of the image patch. A unit 311, a y differentiation calculation unit 312, a differentiation matrix generation unit 313, and a maximum eigenvalue calculation unit 314 are configured.

  As shown in FIG. 12, in the index calculation unit 310, first, for all image patches belonging to the image patch set, the x differentiation calculation unit 311 calculates the x differentiation of the image patch for each image patch, and the y differentiation. The calculation unit 312 calculates the y derivative of the image patch for each image patch.

  Next, in the index calculation unit 310, the differentiation matrix generation unit 313 uses the x differentiation of the image patch calculated by the x differentiation calculation unit 311 and the y differentiation of the image patch calculated by the y differentiation calculation unit 312. Based on the following equations 8 and 9, a differential matrix of image patches is generated for each image patch.

ただし、

は入力画像におけるｉ番目の画像パッチの微分行列である。

はｘ微分算出部３１１で算出された、入力画像におけるｉ番目の画像パッチのｘ微分に対応する列ベクトルを表す。また、

はｙ微分算出部３１２で算出された、入力画像におけるｉ番目の画像パッチのｙ微分に対応する列ベクトルを表す。さらに、以下、行列

を単に、「入力画像におけるｉ番目の画像パッチのｘｙ微分行列」とも言う。

However,

Is the differential matrix of the i-th image patch in the input image.

Represents a column vector corresponding to the x derivative of the i-th image patch in the input image, calculated by the x derivative calculator 311. Also,

Represents a column vector corresponding to the y derivative of the i-th image patch in the input image, calculated by the y derivative calculator 312. In addition, the matrix

Is also simply referred to as the “xy differential matrix of the i-th image patch in the input image”.

  Then, in the index calculation unit 310, the maximum eigenvalue calculation unit 314 calculates the maximum eigenvalue of the differentiation matrix of the image patch for each image patch, and outputs the calculated maximum eigenvalue as an index of the image patch.

  As described above, according to the index calculation unit 310 of the noise level estimation apparatus according to the present invention illustrated in FIG. 12, the index of all the image patches belonging to the image patch set (that is, the image patch of the image patch). The maximum eigenvalue of the differentiation matrix) can be calculated.

の関係が成立するので、本発明の指標算出部３１０では、画像パッチの微分行列の最大固有値を画像パッチの指標として算出するかわりに、画像パッチのｘｙ微分行列の最大特異値を画像パッチの指標として算出するようにしても良い。 As mentioned above,

Therefore, instead of calculating the maximum eigenvalue of the differentiation matrix of the image patch as the index of the image patch, the index calculation unit 310 of the present invention uses the maximum singular value of the xy differentiation matrix of the image patch as the index of the image patch. It may be calculated as:

を画像パッチ毎に生成し、そして、画像パッチのｘｙ微分行列の最大特異値を画像パッチ毎に算出し、算出した最大特異値を画像パッチの指標として出力する。 In that case, the index calculation unit 310 uses the x differentiation of the image patch calculated by the x differentiation calculation unit 311 and the y differentiation of the image patch calculated by the y differentiation calculation unit 312, based on the above equation (9). Xy differentiation matrix of image patch

Is generated for each image patch, the maximum singular value of the xy differentiation matrix of the image patch is calculated for each image patch, and the calculated maximum singular value is output as an index of the image patch.

  That is, instead of the differential matrix generation unit 313, the index calculation unit 310 in FIG. 12 performs the x differentiation of the image patch calculated by the x differentiation calculation unit 311 and the y of the image patch calculated by the y differentiation calculation unit 312. What is necessary is just to provide the xy differentiation matrix production | generation part which produces | generates the xy differentiation matrix of an image patch for every image patch based on said Formula 9 using differentiation.

  Also, instead of the maximum eigenvalue calculation unit 314, the index calculation unit 310 in FIG. 12 calculates the maximum singular value of the xy differentiation matrix of the image patch generated by the xy differentiation matrix generation unit for each image patch, and calculates the maximum What is necessary is just to provide the largest singular value calculation part which outputs a singular value as a parameter | index of an image patch.

  In the above, the parameter calculation unit 310 calculates the parameter based on the differentiation of the image patch (that is, the maximum eigenvalue of the differentiation matrix of the image patch or the maximum singular value of the xy differentiation matrix of the image patch) as the index of the image patch. The embodiment has been described. Next, an embodiment of the index calculation unit 310 that calculates the distribution of the image patch as the index of the image patch will be described.

  In this embodiment, the index calculation unit 310 first calculates the distribution of image patches for each image patch based on the following Equations 10 and 11 for all image patches belonging to the image patch set, and then The calculated variance of the image patch is output as an index of the image patch.

ただし、

は入力画像におけるｉ番目の画像パッチ（以下、単に

とも言う。）の分散である。また、

の画素数をＭとする。さらに、

のｋ番目の画素の画素値を

とする。

However,

Is the i-th image patch in the input image (hereinafter simply

Also say. ) Dispersion. Also,

Let M be the number of pixels. further,

The pixel value of the kth pixel

And

  FIG. 13 is a block diagram showing a sixth embodiment of the noise level estimation apparatus according to the present invention (hereinafter also simply referred to as “noise level estimation apparatus 6”). Hereinafter, the noise level estimation device 6 will be described in detail with reference to FIG.

  As shown in FIG. 13, the noise level estimation device 6 is a device for estimating the noise level of an image from an input image, and includes a high frequency component extraction unit 20, a high frequency component image patch set generation unit 11, and a low level. The frequency component extraction unit 25, the low frequency component image patch set generation unit 16, the low frequency component image patch selection unit 35, the corresponding high frequency component image patch extraction unit 22, and the noise level estimation unit 40 are configured.

  As shown in FIG. 13, in the noise level estimation apparatus 6, the high frequency component extraction part 20 extracts the high frequency component image of the said input image by performing a high frequency component extraction process with respect to an input image. The high frequency component image patch set generation unit 11 performs high frequency component image patch set generation processing on the high frequency component image in order to obtain the high frequency component image patch set from the high frequency component image extracted by the high frequency component extraction unit 20. Thus, a high frequency component image patch set is generated.

  Here, since the process performed in the high frequency component extraction unit 20 of the noise level estimation device 6 is the same as the process performed in the high frequency component extraction unit 20 of the noise level estimation device 2 described above, a detailed description thereof will be given. Omitted. Moreover, since the process performed in the high frequency component image patch set generation unit 11 of the noise level estimation device 6 is the same as the process performed in the high frequency component image patch set generation unit 11 of the noise level estimation device 2 described above, Detailed description thereof is omitted.

  Moreover, the low frequency component extraction part 25 extracts the low frequency component image of the said input image by performing a low frequency component extraction process with respect to an input image. In order for the low frequency component image patch set generation unit 16 to obtain a low frequency component image patch set from the low frequency component image extracted by the low frequency component extraction unit 25, the low frequency component image patch is applied to the low frequency component image. By performing the set generation process, a low frequency component image patch set is generated.

  Here, as a specific example of the low-frequency component extraction process, a low-frequency component image of the input image may be extracted by applying a low-pass filter to the input image.

  As another specific example of the low frequency component extraction process, the low frequency component image of the input image may be extracted by subtracting the high frequency component image extracted by the high frequency component extraction unit 20 from the input image. good.

  That is, the “low frequency component extraction process” referred to in the present invention is a process for extracting a low frequency component image of an input image, and is not limited to the above two specific examples. Any processing that can extract component images can be referred to as “low frequency component extraction processing” of the present invention.

  The low-frequency component image patch set generation process is basically the same as the above-described image patch set generation process, and thus detailed description thereof is omitted.

  However, the image patch set generation process is a process of generating a plurality of image patches while shifting the pixel position with respect to the input image, whereas the low frequency component image patch set generation process is performed on the low frequency component image. On the other hand, it is a process of generating a plurality of low frequency component image patches while shifting the pixel position.

  In the present invention, a plurality of low-frequency component image patches obtained by the low-frequency component image patch set generation processing are set as one set (that is, a low-frequency component image patch set).

  Next, the low frequency component image patch selection unit 35 first selects a plurality of low frequency component image patch sets from the low frequency component image patch set generation unit 16 generated by the low frequency component image patch set generation unit 16 based on a predetermined image patch selection method. A frequency component image patch is selected, and the selected plurality of low frequency component image patches are set as a selected low frequency component image patch set.

  Here, the processing performed by the low-frequency component image patch selection unit 35 of the noise level estimation device 6 is basically the same as the processing performed by the image patch selection unit 30 of the noise level estimation device 4 described above. Therefore, the detailed description is abbreviate | omitted.

  However, the process performed in the image patch selection unit 30 is a process for obtaining a selected image patch set from the image patch set, whereas the process performed in the low frequency component image patch selection unit 35 is a low frequency component. This is a process for obtaining a selected low-frequency component image patch set from the component image patch set.

  Next, the corresponding high frequency component image patch extraction unit 22 extracts the corresponding high frequency component image patch for all the low frequency component image patches belonging to the selected low frequency component image patch set obtained by the low frequency component image patch selection unit 35. By performing processing, high frequency component image patches corresponding to all these low frequency component image patches (that is, selected low frequency component image patches) are extracted as corresponding high frequency component image patches, and these corresponding high frequency component patches are extracted. The component image patch is set as a corresponding high-frequency component image patch set.

  Here, the corresponding high-frequency component image patch extraction process refers to a high-frequency component image patch corresponding to the selected low-frequency component image patch (that is, a high-frequency component image patch having the same pixel position as the selected low-frequency component image patch). , Corresponding high frequency component image patch).

  The noise level estimation unit 40 then estimates the noise level of the input image based on the corresponding high frequency component image patch set obtained by the corresponding high frequency component image patch extraction unit 22.

  Here, the processing performed in the noise level estimation unit 40 of the noise level estimation device 6 is basically the same as the processing performed in the noise level estimation unit 40 of the noise level estimation device 1 described above. Detailed description is omitted.

  However, the processing performed by the noise level estimation unit 40 of the noise level estimation device 1 is performed by the noise level estimation unit 40 of the noise level estimation device 6 whereas the processing performed on the image patch set. The processing is processing performed on the corresponding high-frequency component image patch set.

  That is, the first embodiment (see FIG. 3) and the second embodiment (see FIG. 4) of the noise level estimation unit 40 described above can be applied to the noise level estimation unit 40 of the noise level estimation device 6. However, in the first embodiment and the second embodiment of the noise level estimation unit 40, the image patch set is input, whereas in the noise level estimation unit 40 of the noise level estimation device 6, the corresponding high frequency component image patch set is displayed. As input.

  Thus, according to the noise level estimation apparatus 6 of the present invention, the noise level of the image can be estimated from the input image.

  As shown in FIG. 13, the noise level estimation device 6 is an embodiment in which the high frequency component image is extracted by the high frequency component extraction unit 20 and the low frequency component image is extracted by the low frequency component extraction unit 25. However, the high frequency component image and the low frequency component image may be extracted by one extraction processing unit. In the noise level estimation device 6, the high frequency component image patch set is generated by the high frequency component image patch set generation unit 11, and the low frequency component image patch set is generated by the low frequency component image patch set generation unit 16. However, the high frequency component image patch set and the low frequency component image patch set may be generated by one image patch set generation unit.

  The noise level estimation apparatus according to the present invention can be implemented by software (computer program) using a computer system, and can be implemented by an ASIC (Application Specific Integrated Circuit), a GPU (Graphics Processing Unit) or an FPGA (FPGA). Of course, it can also be implemented by hardware such as Field Programmable Gate Array.

Noise level estimator 1, 2, 3, 4, 5, 6
Image patch set generation unit 10,12
High frequency component image patch set generation unit 11
Low frequency component image patch set generation unit 16
High frequency component extraction unit 20
High frequency component image patch extraction unit 21
Corresponding high frequency component image patch extraction unit 22
Low frequency component extraction unit 25
Image patch selection unit 30
Low frequency component image patch selector 35
Noise level estimator 40, 41
Noise level estimation repetition end determination unit 50
Threshold conversion unit 60
Index calculation unit 310
x derivative calculation unit 311
y derivative calculation unit 312
Differential matrix generation unit 313
Maximum eigenvalue calculation unit 314
Image patch selection unit 320 based on index threshold
Image patch selection unit 321 based on higher index
Covariance matrix calculation unit 410
Image patch average correction unit 411
Minimum eigenvalue calculation unit 420
Minimum singular value calculation unit 421
Noise calculation unit 430,431

Claims (49)

A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
A noise level estimation unit that estimates the noise level based on the image patch set generated by the image patch set generation unit;
With
The noise level estimator is
A variance-covariance matrix calculating unit that calculates a variance-covariance matrix corresponding to the image patch set based on the image patch set;
A minimum eigenvalue calculating unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculating unit;
A noise level calculator that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculator;
A noise level estimation apparatus comprising: The variance-covariance matrix calculation unit calculates the variance-covariance matrix by performing a variance-covariance matrix calculation process based on the following formula:

here,

Represents the set of image patches,

Represents the i-th image patch in the image, N represents the number of elements of the image patch set,

Represents the average corrected image patch set,

Represents the average of the image patches,

Represents the variance-covariance matrix,
The minimum eigenvalue calculation unit performs the minimum eigenvalue calculation process, thereby performing the minimum eigenvalue of the variance-covariance matrix.

To calculate
In the noise level calculation unit,

To calculate the noise level by performing a noise level calculation process,
here,

The noise level estimation apparatus according to claim 1, wherein represents the noise level. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
A noise level estimation unit that estimates the noise level based on the image patch set generated by the image patch set generation unit;
With
The noise level estimator is
An image patch average correction unit that generates an average corrected image patch set based on the image patch set;
A minimum singular value calculation unit that calculates a minimum singular value of the average corrected image patch set generated by the image patch average correction unit;
A noise level calculator that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculator;
A noise level estimation apparatus comprising: The image patch average correction unit calculates the average corrected image patch set by performing an average corrected image patch set generation process based on the following equation:

here,

Represents the average corrected image patch set;

Represents the set of image patches,

Represents the i-th image patch in the image, N represents the number of elements of the image patch set,

Represents the average of the image patches,
The minimum singular value calculation unit performs the minimum singular value calculation process, thereby the minimum singular value of the average corrected image patch set.

To calculate
The noise level calculation unit

To calculate the noise level by performing a noise level calculation process,
here,

The noise level estimation apparatus according to claim 3, which represents the noise level. A noise level estimation device for estimating a noise level of an image including noise from a single image,
A high frequency component extraction unit for extracting a high frequency component image from the image;
A high-frequency component image patch set generation unit that generates a high-frequency component image patch set from the high-frequency component image extracted by the high-frequency component extraction unit;
A noise level estimation unit that estimates the noise level based on the high-frequency component image patch set;
With
The noise level estimator is
A variance-covariance matrix calculator that calculates a variance-covariance matrix corresponding to the high-frequency component image patch set based on the high-frequency component image patch set;
A minimum eigenvalue calculating unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculating unit;
A noise level calculator that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculator;
A noise level estimation apparatus comprising: A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
A high-frequency component image patch extraction unit that extracts a high-frequency component image patch set from the image patch set generated by the image patch set generation unit;
A noise level estimation unit that estimates the noise level based on the high-frequency component image patch set;
With
The noise level estimator is
A variance-covariance matrix calculator that calculates a variance-covariance matrix corresponding to the high-frequency component image patch set based on the high-frequency component image patch set;
A minimum eigenvalue calculating unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculating unit;
A noise level calculator that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculator;
A noise level estimation apparatus comprising: A noise level estimation device for estimating a noise level of an image including noise from a single image,
A high frequency component extraction unit for extracting a high frequency component image from the image;
A high-frequency component image patch set generation unit that generates a high-frequency component image patch set from the high-frequency component image extracted by the high-frequency component extraction unit;
A noise level estimation unit that estimates the noise level based on the high-frequency component image patch set;
With
The noise level estimator is
A high frequency component image patch average correction unit that generates an average corrected high frequency component image patch set based on the high frequency component image patch set;
A minimum singular value calculation unit that calculates a minimum singular value of the average corrected high frequency component image patch set generated by the high frequency component image patch average correction unit;
A noise level calculator that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculator;
A noise level estimation apparatus comprising: A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
A high-frequency component image patch extraction unit that extracts a high-frequency component image patch set from the image patch set generated by the image patch set generation unit;
A noise level estimation unit that estimates the noise level based on the high-frequency component image patch set;
With
The noise level estimator is
A high frequency component image patch average correction unit that generates an average corrected high frequency component image patch set based on the high frequency component image patch set;
A minimum singular value calculation unit that calculates a minimum singular value of the average corrected high frequency component image patch set generated by the high frequency component image patch average correction unit;
A noise level calculator that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculator;
A noise level estimation apparatus comprising: A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated by the index calculation unit and a predetermined threshold. An image patch selection unit based on an index threshold, wherein the selected image patch is a set of selected image patches;
With
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculation unit includes an x differentiation calculation unit, a y differentiation calculation unit, a differentiation matrix generation unit, and a maximum eigenvalue calculation unit,
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and calculates the differentiation matrix of the image patch for each image patch. To generate
It said maximum eigenvalue calculating unit calculates the maximum eigenvalue of the derivative matrix of the image patch for each image patch, the noise level estimation apparatus and outputs the maximum eigenvalue calculated as an index of the image patch. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated by the index calculating unit, or the descending order from the largest index among the indices calculated by the index calculating unit, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set;
With
The index calculation unit includes an x differentiation calculation unit, a y differentiation calculation unit, a differentiation matrix generation unit, and a maximum eigenvalue calculation unit,
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and calculates the differentiation matrix of the image patch for each image patch. To generate
It said maximum eigenvalue calculating unit calculates the maximum eigenvalue of the derivative matrix of the image patch for each image patch, the noise level estimation apparatus and outputs the maximum eigenvalue calculated as an index of the image patch. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated by the index calculation unit and a predetermined threshold. An image patch selection unit based on an index threshold, wherein the selected image patch is a set of selected image patches;
With
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculator includes an x derivative calculator, a y derivative calculator, an xy derivative matrix generator, and a maximum singular value calculator.
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The xy differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
The largest singular value calculation unit, wherein the maximum singular value of xy derivative matrix of image patches calculated for each image patch, the noise level estimation apparatus and outputs the maximum singular value calculated as an index of the image patch. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated by the index calculating unit, or the descending order from the largest index among the indices calculated by the index calculating unit, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set;
With
The index calculator includes an x derivative calculator, a y derivative calculator, an xy derivative matrix generator, and a maximum singular value calculator.
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The xy differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
The largest singular value calculation unit, wherein the maximum singular value of xy derivative matrix of image patches calculated for each image patch, the noise level estimation apparatus and outputs the maximum singular value calculated as an index of the image patch. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated by the index calculation unit and a predetermined threshold. An image patch selection unit based on an index threshold, wherein the selected image patch is a set of selected image patches;
With
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculation unit calculates the distribution of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and outputs the calculated distribution of the image patch as an index of the image patch And

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

The noise level estimation apparatus characterized by expressing the pixel value of the kth pixel. A noise level estimation device for estimating a noise level of an image including noise from a single image,
An image patch set generation unit for generating an image patch set from the image;
An image patch selection unit that selects a plurality of image patches from the image patch set generated by the image patch set generation unit, and sets the selected plurality of image patches as a selected image patch set;
A noise level estimation unit that estimates the noise level based on a set of selected image patches obtained by the image patch selection unit;
With
The image patch selection unit
An index calculation unit that calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated by the index calculating unit, or the descending order from the largest index among the indices calculated by the index calculating unit, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set;
With
The index calculation unit calculates the distribution of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and outputs the calculated distribution of the image patch as an index of the image patch And

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

The noise level estimation apparatus characterized by expressing the pixel value of the kth pixel. The noise level estimator is
A variance-covariance matrix calculating unit that calculates a variance-covariance matrix corresponding to the selected image patch set based on the selected image patch set;
A minimum eigenvalue calculating unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculating unit;
A noise level calculator that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculator;
The noise level estimation apparatus according to claim 9, further comprising: The noise level estimator is
A selected image patch average correction unit that generates an average corrected selected image patch set based on the selected image patch set;
A minimum singular value calculation unit that calculates a minimum singular value of the average corrected selection image patch set generated by the selected image patch average correction unit;
A noise level calculator that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculator;
The noise level estimation apparatus according to claim 9, further comprising: A noise level estimation device that estimates a noise level of an image by repeating noise level estimation from a single image including noise,
An image patch set generation unit, an index calculation unit, an image patch selection unit based on an index threshold, a noise level estimation unit, a threshold conversion unit, and a noise level estimation repetition end determination unit,
The image patch set generation unit generates an image patch set from the image,
The index calculation unit calculates an index of an image patch based on a predetermined index calculation method for all image patches belonging to the image patch set,
The image patch selection unit based on the index threshold is set to the index threshold based on the index calculated by the index calculation unit and the threshold from the threshold conversion unit for all image patches belonging to the image patch set. The image patch selection process is performed to select an image patch, and the selected image patch is set as a selected image patch set.
The noise level estimation unit estimates the noise level based on the image patch set or the selected image patch set, and outputs the estimated noise level to the noise level estimation repetition end determination unit,
The threshold conversion unit converts the noise level from the noise level estimation repetition end determination unit to the threshold, and outputs the converted threshold to the image patch selection unit based on the index threshold,
The noise level estimation repetition end determination unit determines the noise level estimation repetition end based on a predetermined determination condition, and determines that the noise level from the noise level estimation unit is not the threshold value when the noise level estimation repetition end is not determined A noise level estimation device that outputs to the conversion unit and determines that the noise level from the noise level estimation unit is the noise level estimated by the noise level estimation device when it is determined that the repetition of noise level estimation has been completed. . The predetermined determination conditions are a first determination condition and a second determination condition,
The first determination condition is k> 0 (where k is the number of repetitions of the noise level estimation process),
The second determination condition is:

(However,

Is a noise level estimated by the noise level estimation unit. ).
The noise level estimation apparatus according to claim 17, wherein the noise level estimation repetition end determination unit determines that the noise level estimation repetition ends when the first determination condition and the second determination condition are satisfied. The index calculation unit includes an x differentiation calculation unit, a y differentiation calculation unit, a differentiation matrix generation unit, and a maximum eigenvalue calculation unit,
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and calculates the differentiation matrix of the image patch for each image patch. To generate
The noise level estimation device according to claim 17 or 18, wherein the maximum eigenvalue calculation unit calculates a maximum eigenvalue of the differentiation matrix of the image patch for each image patch, and outputs the calculated maximum eigenvalue as an index of the image patch. . The index calculator includes an x derivative calculator, a y derivative calculator, an xy derivative matrix generator, and a maximum singular value calculator.
In the index calculation unit, for all image patches belonging to the image patch set, the x derivative calculation unit calculates the x derivative of the image patch for each image patch, and the y derivative calculation unit includes the image patch. Is calculated for each image patch,
The xy differentiation matrix generation unit uses the x differentiation of the image patch calculated by the x differentiation calculation unit and the y differentiation of the image patch calculated by the y differentiation calculation unit, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
The said maximum singular value calculation part calculates the maximum singular value of the xy differentiation matrix of the said image patch for every image patch, and outputs the calculated maximum singular value as a parameter | index of an image patch. Noise level estimation device. The index calculation unit calculates the distribution of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and outputs the calculated distribution of the image patch as an index of the image patch And

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

The noise level estimation apparatus according to claim 17 or 18, which represents a pixel value of a kth pixel. A noise level estimation device for estimating a noise level of an image including noise from a single image,
A high frequency component extraction unit, a high frequency component image patch set generation unit, a low frequency component extraction unit, a low frequency component image patch set generation unit, a low frequency component image patch selection unit, a corresponding high frequency component image patch extraction unit, A noise level estimation unit,
The high frequency component extraction unit extracts a high frequency component image of the image by performing high frequency component extraction processing on the image,
The high-frequency component image patch set generation unit generates a high-frequency component image patch set by performing high-frequency component image patch set generation processing on the high-frequency component image,
The low frequency component extraction unit extracts a low frequency component image of the image by performing a low frequency component extraction process on the image,
The low frequency component image patch set generation unit generates a low frequency component image patch set by performing a low frequency component image patch set generation process on the low frequency component image,
The low frequency component image patch selection unit selects a plurality of low frequency component image patches from the set of low frequency component image patches based on a predetermined image patch selection method, and selects the plurality of selected low frequency component image patches. As a set of selected low-frequency component image patches,
The corresponding high-frequency component image patch extraction unit performs corresponding high-frequency component image patch extraction processing on all the low-frequency component image patches belonging to the selected low-frequency component image patch set, so that all these low-frequency component image patches Are extracted as corresponding high frequency component image patches, and these corresponding high frequency component image patches are extracted as corresponding high frequency component image patch sets,
The noise level estimation unit estimates the noise level based on the corresponding high-frequency component image patch set,
The corresponding high frequency component image patch extraction process is a process for extracting a high frequency component image patch having the same pixel position as that of the low frequency component image patch as a corresponding high frequency component image patch. The noise level estimator is
A variance-covariance matrix calculating unit that calculates a variance-covariance matrix corresponding to the corresponding high-frequency component image patch set based on the corresponding high-frequency component image patch set;
A minimum eigenvalue calculating unit that calculates a minimum eigenvalue of the variance-covariance matrix calculated by the variance-covariance matrix calculating unit;
A noise level calculator that calculates the noise level based on the minimum eigenvalue calculated by the minimum eigenvalue calculator;
The noise level estimation apparatus according to claim 22. The noise level estimator is
Based on the corresponding high frequency component image patch set, a corresponding high frequency component image patch average correction unit that generates an average corrected corresponding high frequency component image patch set;
A minimum singular value calculation unit that calculates a minimum singular value of the average corrected corresponding high frequency component image patch set generated by the corresponding high frequency component image patch average correction unit;
A noise level calculator that calculates the noise level based on the minimum singular value calculated by the minimum singular value calculator;
The noise level estimation apparatus according to claim 22. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
A noise level estimation step for estimating the noise level based on the image patch set generated in the image patch set generation step;
Have
The noise level estimating step includes:
A variance-covariance matrix calculating step for calculating a variance-covariance matrix corresponding to the image patch set based on the image patch set;
A minimum eigenvalue calculating step of calculating a minimum eigenvalue of the variance-covariance matrix calculated in the variance-covariance matrix calculating step;
A noise level calculating step for calculating the noise level based on the minimum eigenvalue calculated in the minimum eigenvalue calculating step;
A noise level estimation method characterized by comprising: In the variance-covariance matrix calculation step, the variance-covariance matrix calculation process is performed based on the following formula to calculate the variance-covariance matrix,

here,

Represents the set of image patches,

Represents the i-th image patch in the image, N represents the number of elements of the image patch set,

Represents the average corrected image patch set,

Represents the average of the image patches,

Represents the variance-covariance matrix,
In the minimum eigenvalue calculation step, the minimum eigenvalue of the variance-covariance matrix is obtained by performing a minimum eigenvalue calculation process.

To calculate
In the noise level calculation step,

To calculate the noise level by performing a noise level calculation process,
here,

The noise level estimation method according to claim 25, wherein represents the noise level. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
A noise level estimation step for estimating the noise level based on the image patch set generated in the image patch set generation step;
Have
The noise level estimating step includes:
An image patch average correction step for generating an average corrected image patch set based on the image patch set;
A minimum singular value calculation step of calculating a minimum singular value of the average corrected image patch set generated in the image patch average correction step;
A noise level calculating step for calculating the noise level based on the minimum singular value calculated in the minimum singular value calculating step;
A noise level estimation method characterized by comprising: In the image patch average correction step, the average corrected image patch set generation processing is performed based on the following formula to calculate the average corrected image patch set,

here,

Represents the average corrected image patch set;

Represents the set of image patches,

Represents the i-th image patch in the image, N represents the number of elements of the image patch set,

Represents the average of the image patches,
The minimum singular value calculation step performs a minimum singular value calculation process, thereby the minimum singular value of the average corrected image patch set.

To calculate
The noise level calculation step includes:

To calculate the noise level by performing a noise level calculation process,
here,

28. The noise level estimation method according to claim 27, which represents the noise level. A noise level estimation method for estimating a noise level of an image including noise from a single image,
A high frequency component extraction step for extracting a high frequency component image from the image;
A high frequency component image patch set generation step for generating a high frequency component image patch set from the high frequency component image extracted in the high frequency component extraction step;
A noise level estimating step for estimating the noise level based on the set of high-frequency component image patches;
Have
The noise level estimating step includes:
A variance covariance matrix calculating step for calculating a variance covariance matrix corresponding to the high frequency component image patch set based on the high frequency component image patch set;
A minimum eigenvalue calculating step of calculating a minimum eigenvalue of the variance-covariance matrix calculated in the variance-covariance matrix calculating step;
A noise level calculating step for calculating the noise level based on the minimum eigenvalue calculated in the minimum eigenvalue calculating step;
A noise level estimation method characterized by comprising : A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
A high-frequency component image patch extraction step for extracting a high-frequency component image patch set from the image patch set generated in the image patch set generation step;
A noise level estimating step for estimating the noise level based on the set of high-frequency component image patches;
Have
The noise level estimating step includes:
A variance covariance matrix calculating step for calculating a variance covariance matrix corresponding to the high frequency component image patch set based on the high frequency component image patch set;
A minimum eigenvalue calculating step of calculating a minimum eigenvalue of the variance-covariance matrix calculated in the variance-covariance matrix calculating step;
A noise level calculating step for calculating the noise level based on the minimum eigenvalue calculated in the minimum eigenvalue calculating step;
A noise level estimation method characterized by comprising : A noise level estimation method for estimating a noise level of an image including noise from a single image,
A high frequency component extraction step for extracting a high frequency component image from the image;
A high frequency component image patch set generation step for generating a high frequency component image patch set from the high frequency component image extracted in the high frequency component extraction step;
A noise level estimating step for estimating the noise level based on the set of high-frequency component image patches;
Have
The noise level estimating step includes:
A high frequency component image patch average correction step for generating an average corrected high frequency component image patch set based on the high frequency component image patch set;
A minimum singular value calculation step of calculating a minimum singular value of the average corrected high frequency component image patch set generated in the high frequency component image patch average correction step;
A noise level calculating step for calculating the noise level based on the minimum singular value calculated in the minimum singular value calculating step;
A noise level estimation method characterized by comprising : A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
A high-frequency component image patch extraction step for extracting a high-frequency component image patch set from the image patch set generated in the image patch set generation step;
A noise level estimating step for estimating the noise level based on the set of high-frequency component image patches;
Have
The noise level estimating step includes:
A high frequency component image patch average correction step for generating an average corrected high frequency component image patch set based on the high frequency component image patch set;
A minimum singular value calculation step of calculating a minimum singular value of the average corrected high frequency component image patch set generated in the high frequency component image patch average correction step;
A noise level calculating step for calculating the noise level based on the minimum singular value calculated in the minimum singular value calculating step;
A noise level estimation method characterized by comprising : A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated in the index calculation step and a predetermined threshold. An image patch selection step based on an index threshold, wherein the selected image patch is a selected image patch set;
Have
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, a differentiation matrix generation step, and a maximum eigenvalue calculation step,
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and calculates the differentiation matrix of the image patch for each image patch. To generate
Noise level estimation method the maximum eigenvalue calculating step calculates the maximum eigenvalue of the derivative matrix of the image patch for each image patch, characterized in that the maximum eigenvalue calculated as an index of the image patch. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated in the index calculating step, or the descending order from the largest index among the indices calculated in the index calculating step, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set, an image patch selection step based on a higher index;
Have
The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, a differentiation matrix generation step, and a maximum eigenvalue calculation step,
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and calculates the differentiation matrix of the image patch for each image patch. To generate
Noise level estimation method the maximum eigenvalue calculating step calculates the maximum eigenvalue of the derivative matrix of the image patch for each image patch, characterized in that the maximum eigenvalue calculated as an index of the image patch. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated in the index calculation step and a predetermined threshold. An image patch selection step based on an index threshold, wherein the selected image patch is a selected image patch set;
Have
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, an xy differentiation matrix generation step, and a maximum singular value calculation step.
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The xy differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
The largest singular value calculating step, wherein the maximum singular value of xy derivative matrix of image patches calculated for each image patch, the noise level estimation method, characterized by a maximum singular value calculated as an index of the image patch. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated in the index calculating step, or the descending order from the largest index among the indices calculated in the index calculating step, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set, an image patch selection step based on a higher index;
Have
The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, an xy differentiation matrix generation step, and a maximum singular value calculation step.
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The xy differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
The largest singular value calculating step, wherein the maximum singular value of xy derivative matrix of image patches calculated for each image patch, the noise level estimation method, characterized by a maximum singular value calculated as an index of the image patch. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
For all image patches belonging to the set of image patches, an image patch is selected and selected by performing an image patch selection process based on the index threshold based on the index calculated in the index calculation step and a predetermined threshold. An image patch selection step based on an index threshold, wherein the selected image patch is a selected image patch set;
Have
In the image patch selection process based on the index threshold, the index is compared with the predetermined threshold, and when the index is equal to or less than the predetermined threshold, an image patch corresponding to the index is selected, and the index is When it is larger than a predetermined threshold, an image patch corresponding to the index is not selected,
The index calculation step calculates the variance of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and uses the calculated variance of the image patch as an index of the image patch:

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

A noise level estimation method characterized by representing a pixel value of a kth pixel. A noise level estimation method for estimating a noise level of an image including noise from a single image,
An image patch set generation step for generating an image patch set from the image;
An image patch selection step of selecting a plurality of image patches from the image patch set generated in the image patch set generation step, and setting the selected plurality of image patches as a selected image patch set;
A noise level estimation step for estimating the noise level based on the selected image patch set obtained in the image patch selection step;
Have
The image patch selection step includes:
An index calculation step of calculating an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set;
From the image patch set, the ascending order from the smallest index among the indices calculated in the index calculating step, or the descending order from the largest index among the indices calculated in the index calculating step, Selecting an image patch corresponding to a predetermined number of indices, and setting the selected image patch as a selected image patch set, an image patch selection step based on a higher index;
Have
The index calculation step calculates the variance of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and uses the calculated variance of the image patch as an index of the image patch:

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

A noise level estimation method characterized by representing a pixel value of a kth pixel. The noise level estimating step includes:
A variance-covariance matrix calculating step for calculating a variance-covariance matrix corresponding to the selected image patch set based on the selected image patch set;
A minimum eigenvalue calculating step of calculating a minimum eigenvalue of the variance-covariance matrix calculated in the variance-covariance matrix calculating step;
A noise level calculating step for calculating the noise level based on the minimum eigenvalue calculated in the minimum eigenvalue calculating step;
The noise level estimation method according to any one of claims 33 to 38. The noise level estimating step includes:
A selected image patch average correction step for generating an average corrected selected image patch set based on the selected image patch set;
A minimum singular value calculation step of calculating a minimum singular value of the average corrected selected image patch set generated in the selected image patch average correction step;
A noise level calculating step for calculating the noise level based on the minimum singular value calculated in the minimum singular value calculating step;
The noise level estimation method according to any one of claims 33 to 38. A noise level estimation method for estimating a noise level of an image by repeating noise level estimation from a single image including noise,
An image patch set generation step, an index calculation step, an image patch selection step based on an index threshold, a noise level estimation step, a threshold conversion step, and a noise level estimation repetition end determination step,
The image patch set generation step generates an image patch set from the image,
The index calculating step calculates an index of the image patch based on a predetermined index calculation method for all image patches belonging to the image patch set,
The image patch selection step based on the index threshold is based on the index calculated in the index calculation step and the threshold converted in the threshold conversion step for all image patches belonging to the image patch set. By performing image patch selection processing based on a threshold, an image patch is selected, and the selected image patch is set as a selected image patch set.
The noise level estimation step estimates the noise level based on the image patch set or the selected image patch set,
The threshold conversion step converts the noise level estimated in the noise level estimation step into the threshold,
The noise level estimation repetition end determination step determines the noise level estimation repetition end based on a predetermined determination condition, and determines that the noise level estimated in the noise level estimation step is the end when the noise level estimation repetition end is determined. A noise level estimation method, wherein the noise level is estimated by the noise level estimation method. The predetermined determination conditions are a first determination condition and a second determination condition,
The first determination condition is k> 0 (where k is the number of repetitions of the noise level estimation process),
The second determination condition is:

(However,

Is the noise level estimated in the noise level estimation step. ).
42. The noise level estimation method according to claim 41, wherein, in the noise level estimation repetition end determination step, it is determined that the noise level estimation repetition end is completed when the first determination condition and the second determination condition are satisfied. The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, a differentiation matrix generation step, and a maximum eigenvalue calculation step,
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and calculates the differentiation matrix of the image patch for each image patch. To generate
43. The noise level estimation method according to claim 41 or claim 42, wherein the maximum eigenvalue calculating step calculates a maximum eigenvalue of the differentiation matrix of the image patch for each image patch, and uses the calculated maximum eigenvalue as an index of the image patch. The index calculation step includes an x differentiation calculation step, a y differentiation calculation step, an xy differentiation matrix generation step, and a maximum singular value calculation step.
For all image patches belonging to the image patch set, the x derivative calculating step calculates the x derivative of the image patch for each image patch, and the y derivative calculating step calculates the y derivative of the image patch to the image patch. Calculated for each
The xy differentiation matrix generation step uses the x differentiation of the image patch calculated in the x differentiation calculation step and the y differentiation of the image patch calculated in the y differentiation calculation step, and converts the xy differentiation matrix of the image patch into an image. Generate for each patch,
43. The noise according to claim 41 or claim 42, wherein the maximum singular value calculation step calculates a maximum singular value of an xy differentiation matrix of the image patch for each image patch, and uses the calculated maximum singular value as an index of the image patch. Level estimation method. The index calculation step calculates the variance of the image patch for each image patch based on the following formula for all image patches belonging to the image patch set, and uses the calculated variance of the image patch as an index of the image patch:

However,

Is an image patch

Where M is the image patch

Represents the number of pixels of

Is an image patch

43. The noise level estimation method according to claim 41 or claim 42, which represents a pixel value of a kth pixel. A noise level estimation method for estimating a noise level of an image including noise from a single image,
A high frequency component extraction step, a high frequency component image patch set generation step, a low frequency component extraction step, a low frequency component image patch set generation step, a low frequency component image patch selection step, a corresponding high frequency component image patch extraction step, A noise level estimation step,
The high frequency component extraction step extracts a high frequency component image of the image by performing high frequency component extraction processing on the image,
The high-frequency component image patch set generation step generates a high-frequency component image patch set by performing high-frequency component image patch set generation processing on the high-frequency component image,
The low frequency component extraction step extracts a low frequency component image of the image by performing a low frequency component extraction process on the image,
The low frequency component image patch set generation step generates a low frequency component image patch set by performing low frequency component image patch set generation processing on the low frequency component image,
The low frequency component image patch selection step selects a plurality of low frequency component image patches from the set of low frequency component image patches based on a predetermined image patch selection method, and selects the plurality of selected low frequency component image patches. As a set of selected low-frequency component image patches,
The corresponding high-frequency component image patch extraction step performs corresponding high-frequency component image patch extraction processing on all the low-frequency component image patches belonging to the selected low-frequency component image patch set, so that all these low-frequency component image patches Are extracted as corresponding high frequency component image patches, and these corresponding high frequency component image patches are extracted as corresponding high frequency component image patch sets,
The noise level estimation step estimates the noise level based on the corresponding high frequency component image patch set,
The corresponding high frequency component image patch extraction process is a process for extracting a high frequency component image patch having the same pixel position as that of the low frequency component image patch as a corresponding high frequency component image patch. The noise level estimating step includes:
A variance-covariance matrix calculating step for calculating a variance-covariance matrix corresponding to the corresponding high-frequency component image patch set based on the corresponding high-frequency component image patch set;
A minimum eigenvalue calculating step of calculating a minimum eigenvalue of the variance-covariance matrix calculated in the variance-covariance matrix calculating step;
A noise level calculating step for calculating the noise level based on the minimum eigenvalue calculated in the minimum eigenvalue calculating step;
47. The noise level estimation method according to claim 46, comprising: The noise level estimating step includes:
Based on the corresponding high-frequency component image patch set, a corresponding high-frequency component image patch average correction step for generating an average corrected corresponding high-frequency component image patch set;
A minimum singular value calculation step of calculating a minimum singular value of the average corrected corresponding high frequency component image patch set generated in the corresponding high frequency component image patch average correction step;
A noise level calculating step for calculating the noise level based on the minimum singular value calculated in the minimum singular value calculating step;
47. The noise level estimation method according to claim 46, comprising: A program for causing a computer to execute the noise level estimation method according to any one of claims 25 to 48.

Images