JP2012059118A - Noise removal device, and method and program therefor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a new technique of accurate noise removal by wavelets using a dyadic lifting scheme while preserving edges.SOLUTION: A noise removal device comprises: a training image information unit 21 that stores noiseless training image information and noisy training image information obtained by adding, for each of one or plural kinds of noise, the noise to the noiseless training image information; a learning area cutout unit 22 that associates the noiseless training image information with the noisy training image information, and cuts out any learning areas for each kind of noise from the respective pieces of training image information; a filter generation unit 23 that generates a filter that has learned to eliminate a difference between both of the cutout learning areas; and a noise removal unit 25 that applies the generated filter to a noise image 26 to be a target of noise removal for each learning area to remove noises of the noise image 26.

Description

  The present invention relates to a noise removing device for removing various noises from a digital image.

  Digital image processing has developed significantly over the last decade due to the usefulness of digital cameras, including webcams and mobile phone cameras. Digital images stored in these devices may be imaged under poor conditions (eg, in dark lighting), resulting in various types of noise. Therefore, an excellent image restoration technique is required.

  Spatial domain filters such as weighted average and median filters that reduce image noise are already used. In particular, over the past several decades, methods based on partial differential equations (PDEs) have been widely used in techniques for removing noise while preserving image edges. These methods are based on a variational approach to the nonlinear diffusion problem or the energy functional minimization problem (see Non-Patent Document 1). In addition to these methods, there is a noise removal method based on minimum mean square estimation (MMSE) that is optimal in the Bayesian sense (see Non-Patent Document 7).

  As an alternative to the spatial domain filter, the frequency domain has been used. In particular, the wavelet transform has been often used in image processing. Donoho has proposed a soft-threshold method. In this method, the wavelet coefficient below a certain threshold gradually decreases to 0 (see Non-Patent Document 3). This method is based on a downsampling wavelet transform and samples both the scale and translation parameters. However, the downsampling wavelet transform is not shift-invariant and tends to fail when processing such as edge detection, feature extraction, and noise removal is executed. In contrast, the dyadic wavelet transform (binary wavelet) is a shift-invariant wavelet transform because only the scale parameter of the continuous wavelet transform is sampled to maintain the shift-invariant. For example, Mallat performed edge extraction using discrete dyadic wavelet transform using quadratic spline dyadic wavelets (see Non-Patent Documents 5 and 6).

  In general, the noise removal algorithm must balance the trade-off between noise removal and image / signal structure preservation. For this reason, in recent years, much research has been conducted on algorithms for retaining edges. Standard algorithms are based on non-linear diffusion or wavelets (see Non-Patent Documents 1 and 4). However, regardless of the advanced proposal methods in recent years, most algorithms have not yet reached the desired level and their application range is narrow. For this reason, there is no definitive version of an effective image noise removal method.

  Under such circumstances, many wavelet-based noise removal methods have been proposed (see Non-Patent Documents 2, 4, 8-10, and 12). Some effective methods consider the dependence of space and scale in a statistical model based on the wavelet domain or Bayesian theory, such as the BLS-GSM method of Non-Patent Document 9. These techniques use a down-sampling discrete wavelet transform to simplify the statistic of the number of signals and make it easier to handle.

On the other hand, techniques using the dyadic wavelet transform as shown in Non-Patent Documents 4 and 12 are rare. In general, the dyadic wavelet transform does not sample the translation parameter and thus produces a very redundant image signal. Therefore, even if the conversion is repeated, the statistic of the number of signals is not simplified. However, since the dyadic wavelet transform is shift invariant, it can be applied as a noise removal method.

  More recently, a dyadic lifting scheme that extends the Swelldens lifting scheme (see Non-Patent Document 11) has been proposed (see Non-Patent Document 12). The Swelldens lifting scheme creates a new bi-orthogonal wavelet filter from a set of bi-orthogonal wavelet filters to improve the characteristics of the downsampling wavelet. These schemes use a bi-orthogonal lifting wavelet transform with controllable free variables. The dyadic lifting scheme creates a new wavelet filter from a dyadic wavelet.

  Turuki et al. Determined the free variables that appear in the dyadic lifting scheme to have a favorable number of burnishing moments (see Non-Patent Document 12). Hereinafter, this method is referred to as a VM method. They designed a spline dyadic wavelet filter with a larger number of burnishing moments for digital image denoising and demonstrated the effectiveness of this approach. However, they are silent about edge retention.

  In addition, as a technique for removing noise from noise images with higher accuracy than each non-patent document, lifting dyadic wavelets are performed using a plurality of noise images, and high frequency components and noise obtained by dyadic wavelet transform of training images are used. Non-Patent Document 13 discloses a technique for determining a free variable so as to eliminate a difference in high-frequency components of a lifting wavelet of an image and creating a new high-frequency filter based on the free variable to remove noise. Yes.

Aubert, G. and Kornprobst, P .: Mathermatical problems in image processing-Partial differential equations and the calculus of variations, Springer, 2nd edition, 2006. Cho, D .: Image Denoising Using Wavelet Transforms, VDM Verlag, 2008. Donoho, D.L .: De-noising by soft-thresholding, IDEE Trans.Inform.Theory, Vol.41, No.3, pp.613-627, 1995. Lazzaro, D. and Montefusco, L.B .: Edge-preserving wavelet thresholding for image denoising, J. Comput.Appl. Math., Vol.210, pp.222-231, 2007. Mallat, S .: A wavelet tour of signal processing, Academic press, 1998. Mallat, S. and Zhong, S .: Characterization of signals from multiscale edges, IEEE Trans.Pattern Anal.Mach.Intell., Vol.14, No.7, pp.710-732, 1992. Papari, G., Petkov, P.C. and Neri, A .: Contour detection by multiresolution sur-round inhibition, Proc. Int. Conf. On Image Processing ICIP 2006, Atlanta, Georgia, pp.749-752, 2006. Portilla, J .: Blind Non-White Noise Removal in Images using Gaussian Scale Mixtures in the Wavelet Domain, Proc. 4th IEEE Benelux Signal Proc.Symposium, Hilvarenbeek, The Netherlands, pp. 17-20, 2004. Portilla, J .: Full Blind Denoising through Noise Covariance Estimation using Gaussian Scale Mixtures in the Wavelet Domain, 10th IEEE Int'l Conf on Image Processing, Singapore, pp. 1217-1220, 2004. Portilla, J., Strela, V., Wainwright, M. and Simoncelli, E .: Image Denoising using Scale Mixtures of Gaussians in the Wavelet Domain, IEEE transactions on Image Processing, Vol.12, No.11, pp.1338- 1351, 2003. Sweldens, W .: The lifting scheme: A construction of second generation wavelets, SIAM J. Math. Anal., Vol.29, No.2, pp.511-546, 1997. Turuki, T.A., Niijima, M.H.K. and Takano, S .: The dyadic lifting schemes and the denoising of digital images, International Journal of Wavelets, Vo1.6, No.3, pp.331-351, 2008. Satoshi Tsuruta, Shinya Minamoto, “Noise Removal of Digital Images Using Dyadic Lifting Scheme”, Proceedings of the National Institute of Visualization Information Society (Yonezawa 2009), 99-100, October 24, 2009

  However, although it is suggested that the technique shown in Non-Patent Document 13 determines a free variable so as to eliminate the difference between the high-frequency component of the training image and the high-frequency component of the noise image, no specific method is described. However, it merely shows that the high-frequency component of the training image and the high-frequency component of the noise image are compared and processed so as to have the same high-frequency component.

  The present invention uses a method for retaining edges proposed in Non-Patent Document 4 and provides a new technique for noise removal by wavelets using a dyadic lifting scheme. Unlike the VM method, the method shown in Non-Patent Document 12 is not used. Further, an appropriate free variable of the dyadic lifting scheme is determined so that the high frequency component of the noise image approaches the original image. Specifically, the free variable is determined so as to be the same as the high frequency component of the training pattern without noise by solving the simultaneous linear equations.

  The noise removal device disclosed in the present application includes noise-free training frequency information that does not include noise, and noise-free training frequency information in which the noise is included for each one or more types of noise in the noise-free training frequency information. Training frequency information storage means for storing the training frequency information and the training frequency information with no noise and the training frequency information with noise are associated with each other, and a learning region is extracted from each training frequency information for each type of noise. And a filter generating means for generating a filter that learns so as to eliminate a difference between the learning area of the training frequency information without noise and the learning area of the training frequency information with noise cut out by the learning area cutting out means, and noise To be removed Noise removing means for removing noise from the processing target frequency information by applying the filter generated by the filter generating means to each learning region cut out by the learning region cutting out means for the target frequency information. It is to be prepared.

  Thus, in the noise removal device disclosed in the present application, training frequency information including noise and training frequency information not including noise are associated with each other to cut out an arbitrary learning region, and training including noise is performed for each learning region. Generates a filter learned so that there is no difference between frequency information and training frequency information that does not contain noise, and applies the generated filter to each extracted learning region to remove noise in the processing target frequency information Therefore, it is possible to remove various types of noise learned in advance, and to create a filter for each learning area, so that noise can be removed while maintaining the characteristics for each area in any image. There is an effect that can be.

In the noise removing device disclosed in the present application, the frequency information is audio information or image information.
Thus, in the noise removal device disclosed in the present application, since the frequency information is audio information or image information, noise is removed in the case of audio information, white noise, ISO noise, sesame salt in the case of image information. There is an effect that clear audio information and image information can be obtained by removing image disturbances such as noise, luminance and contrast defects.

  In the noise removal device disclosed in the present application, the frequency information is image information, and the filter generation unit performs the noiseless training based on high-frequency components in the vertical direction, the horizontal direction, and the diagonal direction by shift-invariant wavelet transform. Learning is performed so as to eliminate the difference between the learning region of frequency information and the learning region of the training frequency information with noise.

  Thus, in the noise removal device disclosed in the present application, when removing noise of image information, not only vertical and horizontal high-frequency components in image information but also oblique high-frequency components in image information by shift-invariant wavelet transform. Since learning is also performed based on the above, it is possible to generate a more sophisticated noise removal filter.

  In the noise removing device disclosed in the present application, the learning area cutting means associates the training frequency information without noise and the training frequency information with noise in a cutting area of a predetermined size set in advance, and The learning area to be cut out is determined according to the characteristics of the high frequency component of the area.

  As described above, in the noise removal device disclosed in the present application, the training frequency information with no noise and the training frequency information with noise are associated with each other in a cut-out area having a predetermined pixel size set in advance, and the high-frequency component of the cut-out area is Since the learning region to be cut out is determined according to the feature, the learning region is determined according to the feature, and it is possible to perform noise removal on the processing target frequency having various features. For example, in the case of image information as an example of features, when attention is paid to the direction in which the hair is flowing, the region flowing in the vertical direction, the region flowing in the horizontal direction, the region flowing in the right oblique direction, the left A region flowing in the oblique direction may be determined as each learning region, and in the case of speech information, a high sound region and a low sound region may be determined as each learning region.

  In the noise removal device disclosed in the present application, the noise removal unit compares the high-frequency component of the learning region cut out by the learning region cut-out unit with the high-frequency component of an arbitrary region in the processing target frequency information. Noise is removed by applying the filter corresponding to the learning region to a region determined to be similar.

  As described above, in the noise removal device disclosed in the present application, the high-frequency component in the learning area is compared with the high-frequency component in an arbitrary area in the processing target image. Since noise is removed by a filter corresponding to the learning region compared with the region, it is possible to remove the noise by making a similar determination in units of the learning region, and the training frequency information and the processing target frequency information are totally different as a whole. Even if it is frequency information, by applying a filter to the similar part in the learning area unit and removing noise, even if it is unknown processing target frequency information, noise can be removed reliably and complicated The processing target frequency information having characteristics also has an effect of being able to cope with noise removal for each learning region.

It is a hardware block diagram of the noise removal apparatus which concerns on 1st Embodiment. It is a functional block diagram of the noise removal apparatus which concerns on 1st Embodiment. It is a flowchart which shows operation | movement of the noise removal apparatus which concerns on 1st Embodiment. It is a benchmark image used in an Example. It is a 1st figure which shows the result of having performed noise removal. It is a 2nd figure which shows the result of having performed noise removal. It is a figure which shows the example identified with a learning area | region. It is a figure which shows the result of having performed noise removal with respect to the image which changed the brightness | luminance. It is a figure which shows the result of having performed noise removal with respect to the image which changed contrast. It is a figure which shows the result of having performed noise removal with respect to the image containing sesame salt noise. It is a figure which shows the result of having performed noise removal with respect to the image containing ISO noise.

  Throughout the present embodiment, the same elements are denoted by the same reference numerals. In the following embodiments, the apparatus will be mainly described. However, as is apparent to those skilled in the art, the present invention can also be implemented as a method and a program for operating a computer. In addition, the present invention can be implemented in hardware, software, or hardware and software embodiments. The program can be recorded on any computer-readable medium such as a hard disk, CD-ROM, DVD-ROM, optical storage device, or magnetic storage device. Furthermore, the program can be recorded on another computer via a network.

(First embodiment of the present invention)
The noise removal apparatus according to the present embodiment will be described with reference to FIGS. FIG. 1 is a hardware configuration diagram of the noise removing device according to the present embodiment, FIG. 2 is a functional block diagram of the noise removing device according to the present embodiment, and FIG. 3 is an operation of the noise removing device according to the present embodiment. It is a flowchart to show.

  In addition, although the noise removal with respect to image information is mainly demonstrated here, the noise removal apparatus which concerns on this embodiment is applicable also to the audio | voice information which consists of frequency information similarly to image information.

  In FIG. 1, a noise removal apparatus 1 includes an operating system and a ROM 13 that stores various programs (for example, a noise removal program, a learning program, an image display program, etc.) and various data (for example, training image information, filter information, HD (hard disk) 14 in which noise image information and the like are stored, a RAM 12 from which various programs and the like are read out as necessary, a CPU 11 that executes an actual calculation based on the read out programs and the like, and other A communication I / F 15 that is an interface for communicating with a device (for example, a server) and an interface for receiving input from an input device such as a keyboard or a mouse, or outputting data to a printer, a monitor, or the like And an input / output I / F 16.

  In FIG. 2, the noise removal device 1 includes a training image information unit 21, a learning region extraction unit 22, a filter generation unit 23, a filter information storage unit 24, and a noise removal unit 25.

  The training image information unit 21 stores one noise-free training image that does not include noise and one or more training images with noise in which noise is inserted into the one noise-free training image. A training image with noise is prepared for each type of noise (for example, white noise, ISO noise, sesame salt noise, etc.).

  The learning region cutout unit 22 scans the training image in a region having a set size (for example, 32 × 32 pixels) while associating the training image without noise with the training image with noise, and the high-frequency component in each region The learning area is determined according to the characteristics of For example, a flat area, a vertical edge area, a horizontal edge area, a diagonal edge area, etc. in a training image are identified based on high-frequency components and learned for each feature. Determine the area.

  For each determined learning region, the filter generation unit 23 learns so that there is no difference between the high-frequency component obtained by performing dyadic wavelet transform on the training image without noise and the high-frequency component of the lifting wavelet for the training image with noise. Determine free variables and generate filters. The generated filter is stored in the filter information storage unit 24. Details of these processes will be described later.

  The noise removal unit 25 restores a noise-removed image 27 obtained by removing noise from the noise image 26 using a filter stored in the filter information storage unit 24 for the noise image 26 to be processed. When applying the filter, the high-frequency component and feature of the filter are compared with the feature of the high-frequency component in an arbitrary region of the noise image 26, and if there is a similar region, the filter is applied to that region. . Since filters having various characteristics are stored in the filter information storage unit 24 for each type of noise, a plurality of them are combined to remove part or all of the noise in the noise image 26.

  Next, the operation of the noise removal device according to the present embodiment will be described. In FIG. 3, the filter size is first designated (S31). The size of the filter may be set to an arbitrary size by the user, or may be calculated and set by the CPU in the apparatus according to the number of pixels used as the training image and the number of pixels of the noise image to be processed. It may be. For example, the calculation may be performed so that the number of pixels of the training image is set to 1/10.

  When the size of the filter is designated, the learning area cutout unit 22 scans the training image in the designated size area and determines the learning area to be learned (S32). The filter generation unit 23 creates a filter by determining a free variable so that there is no difference between the training image with noise and the training image without noise for each determined learning region, and stores the filter in the filter information storage unit 24 (S33). ). The noise removal unit 25 applies the filter of the filter information storage unit 24 to the noise image 26 to be processed to remove noise, restores the noise removal image 27 (S34), and ends the process.

A more specific process will be described below. First, the dyadic lifting scheme will be described.
When f (t) translated by τ is expressed as fτ (t) = f (t−τ), the wavelet transform is

It can be expressed. Where ψ is a real wavelet function. At this time,

So this transformation is shift invariant. In order to construct a shift-invariant wavelet representation, only the portion of the scale s is discretized, and the portion of the translation u is not discretized. Then, the scale portion is sampled at binary sequence {2 ^{_j} j} ∈ _z in order to simplify the calculations on a computer.

Next, dyadic wavelet transform (binary wavelet transform) is introduced according to this concept. Let L ² (R) be a square-integrable space on the real number R. Also, the Fourier transform of the function ψ∈L ² (R)

It expresses. if,

If there are constants A> 0 and B satisfying the condition, the function ψ (t) is called a dyadic wavelet function (binary wavelet function). From equation (1), ψ ^ (0) = 0,

It is. The binary wavelet transform of f (t) uses the binary wavelet function ψ (t),

Is defined. Two-scale relationship to construct a binary wavelet function

A scaling function φ (t) that satisfies the above is used. Normal scaling function φ (t)

And is normalized. The Fourier transform of this scaling function (3) is

It becomes. Where h ^ (ω) is the discrete Fourier transform

It is.

Since φ ^ (0) = 1, when (4) and (5) are used, h ^ (0) = √2 or Σ _k h [k] = √2 is obtained. Using the scaling function φ (t) and the wavelet filter g [k], the binary wavelet function is defined as ψ (t) = Σ _k g [k] (√2) φ (2t−k). An arbitrary function fεL ² (R) expanded with a binary wavelet can be reconfigured if the reconfiguration condition described in Non-Patent Document 5 is satisfied. In order to derive this reconstruction condition, a dual scaling function and a dual wavelet function are required. The dual scaling function φ˜ (t) (“˜” is a tilde) is defined by φ˜ (t) = Σ _l h˜ [l] (√2) φ˜ (2t−l). However,

Δ _{k, l} is the Kronecker delta symbol. The dual wavelet function ψ˜ (t) is given by ψ˜ (t) = Σ _l g˜ [l] (√2) φ˜ (2t−1).

  The discrete Fourier transforms of the filters h [k], g [k], h ~ [k], and g ~ [k] are respectively expressed as h ^ (ω), g ^ (ω), h ~ ^ (ω), and g ~. Representing ^ (ω), the reconstruction condition is

It is expressed. Here, * represents a complex conjugate. The reconstruction condition (6) plays an important role in constructing a lifting dyadic wavelet filter.

(Proposition 1)
The discrete Fourier transforms of the initial filters h ⁰ [k], g ⁰ [k], h ~ ⁰ [k], and g ~ ⁰ [k] are represented as h ^ ⁰ (ω), g ^ ⁰ (ω), h ~ ^, respectively. ^{It is} assumed that ⁰ (ω), g˜ ^ ⁰ (ω), and these satisfy the reconstruction condition (6). At this time,

The Fourier transforms h ^ (ω), g ^ (ω), h ^^ (ω), and g ^^ (ω) of the conjugate lifting binary wavelet filter defined by (5) satisfy the reconstruction condition (6). Here, s [l] is a free variable. The following proposition is useful for computing the binary wavelet transform and its inverse.

(Proposition 2)
Conversion formula under reconstruction condition (6)

And inverse transformation formula

Holds. Where a ₀ [n] is

Given in.

This is a conversion formula and an inverse conversion formula for a one-dimensional signal. In the case of an image, these formulas may be applied in the horizontal direction (horizontal) and the vertical direction (vertical). This will be described in more detail below. Therefore, C ^j [m, n], D ₁ ^j [m, n], D ₂ ^j [m, n], and D ₃ ^j [m, n] are respectively converted into a low frequency component, a horizontal high frequency component, A high-frequency component in the vertical direction and a high-frequency component in the oblique direction are used. m and n represent positions in the horizontal direction and the vertical direction, respectively. First, in order to obtain D ₁ ^{j + 1} [m, n], when Expression (8) is applied in the vertical direction of C ^j [m, n],

Next, when (9) is applied in the horizontal direction of C ^{j, row} [m, n],

Get. Similarly, when (8) is applied in the horizontal direction of C ^j [m, n],

When (9) is applied in the vertical direction,

Get. And when (9) is applied respectively in the horizontal and vertical directions,

Get. At this time, the reconstruction formula for the image is

Given in.

Next, the learning algorithm will be described. Here, how to determine the free variable s [l] in (7) will be described. In order to distinguish the horizontal and vertical variables, the high-frequency component in the horizontal direction is expressed as g _h [k], and the high-frequency component in the vertical direction is expressed as g _v [k]. At this time, from Proposition 1,

Get. However, s ₁ and s ₂ are free variables. Then, s ₁ [l] and s ₂ [l] are determined so as to be the same as the high frequency component of the learning image without noise (the training image without noise). If (15) and (16) are applied to (11) and (12),

Get. However,

It is.

In addition to this, D ₃ ^{j + 1} [m, n] is approximated by the following equation using (13) and the free variable s ₃ .

However,

It is.

To simplify the rest of the discussion, we will limit the wavelet decomposition to level 0 to level 1 decomposition. If C ⁰ [m, n] is a noise image (a training image with noise), its high-frequency components are D — ₁ ¹ [m, n], D — ₂ ¹ [m, n]. From (17)

Therefore, the free variable s _r (r = 1, 2, 3) is determined to be the same as the high frequency component of the original image (no-noise training image) of C ⁰ [m, n]. That is, the high-frequency component corresponding to D _r ¹ [m, n] (the high-frequency component of the noise-free training image) is _D˜r ¹ [m, n] (r = 1, 2, 3),

think of.

Since D _r ¹ [m, n] (r = 1, 2, 3) includes several free variables s _r , 2N similar noise images C — ^{1, v} (v = 1, 2,... ., 2N) and these original images (images without noise) are prepared as learning images (training images)

Imposing. ^{_{However, C_ l 1, v [m}} , n] and ^{_{D_ l 1, v [m,}} n] is a low-frequency and high-frequency components based on the initial filter for each noise ^{_{image, D ~ r 1, v [}} m , N] are high-frequency components of the original image not including these noises.

Although the number of free variables s _r is 2N + 1, since the number of equations (19) is 2N, another equation is required to uniquely determine s _r . Since g _h and g _v are high-pass filters, these filters are conditional

Should be met. Therefore, if Σ _k g ⁰ [k] = 0 and Σ _k h ⁰ [k] is a constant,

Holds. The same condition as (20) is imposed on s ₃ . When (19) and (20) are expressed in matrix form,

It becomes. Therefore, s _r can be uniquely determined by solving the simultaneous linear equations (21) by an appropriate numerical calculation method (for example, Gaussian elimination method).

  Next, the outline storing method will be described. In general, the noise removal algorithm removes not only noise but also important features of an image. As the simulation shows, the method described so far can remove noise without removing image features much compared to the conventional method, but further improves to leave image features. For this purpose, the method proposed in Non-Patent Document 4 is incorporated.

Let C￣ [m, n] = C [m, n] + E [m, n] be a noisy image, and noise C [m, n] and E [m, n] with the corresponding original image. Then, consider the following minimization problem in which a value D · _r ¹ that minimizes the penalty functional F _p (D _r ¹ ) is searched.

Where ‖ · ‖ ₂ is the Euclidean norm, p (·) is a given penalty function, λ is a positive number that adjusts the fidelity of the data and the effect of the penalty term, D￣ _r ¹ and D _r ¹ is a high-frequency component in each direction corresponding to C￣ and C, respectively. Edge strength

As a penalty term to keep S [m, n] as much as possible

Is used. Here, a potential function widely used in the nonlinear diffusion filtering method is selected as φ. In the case of this embodiment,

Will be used. This function is one of the functions described in Non-Patent Document 4, and the parameter μ serves as a contrast parameter depending on a scale representing a threshold value between an edge and a region other than the edge. For an L × L image,

far. This value is often used as a statistic that is not easily affected by outliers, and MAD represents median absolute deviation. At this time, from Non-Patent Document 4, the noise removal algorithm results in solving the following nonlinear equation.

  Solving this equation using a suitable iterative method yields an approximate solution of (23).

Next, a noise removal algorithm based on the method described above will be described. Since the number of elements of s _r is equal to the number of elements of h ⁰ [k] given by (15) and (16), the image is divided into several partial images, and a dyadic lifting scheme for each partial image Must be applied. Therefore, each partial image is represented as C _i ⁰ (1 ≦ i ≦ M). Then, using (17) and (18) described above, C _i ⁰ (1 ≦ i ≦ M) is changed to each frequency component C _i ¹ [m, n], D _{1, i} ¹ [m, n]. , D _{2, i} ¹ [m, n] and D _{3, i} ¹ [m, n]. Then, using the method of Non-Patent Document 4, (23) is solved for each partial image. Specifically, it is as follows.
(1) The initial values (D ₁ ) ₀ , (D ₂ ) ₀ , (D ₃ ) ₀ , k = 0 are determined. k represents the number of iterations.
(2)

Calculate

(3) (D _r ) If _k converges, the calculation is terminated as k = k ^, and the process proceeds to the next step. If not converged, k = k + 1 and return to step (2).
(4) Using (D _r ) _{k ^}

Calculate

(5) C _i ¹ [m, n], D · _{1, i} ¹ [m, n], D · _{2, i} ¹ [m, n], D · _{3, i} ¹ [m, n] and initial filter , And the reconstruction formula (14) is used to construct a noise-removed image C · _i ⁰ .

  The above steps (2) to (4) correspond to the contour storing method.

If the support length of h ~ and g ~ is α, four (3α-2) × (3α-2) decomposed images are required to reconstruct an α × α image from equation (14). It is. This means that a (5α-4) × (5α-4) image is required to decompose the α × α image based on (17) and to reconstruct based on (14). . Therefore, in order to execute the procedure shown above, it is necessary to consider how to expand the partial image C _i ⁰ . In the method according to the present embodiment, when C _i ⁰ is decomposed using (17), the neighborhood of C _i ⁰ whose size is a (5α-4) × (5α-4) image is used, and the boundary is used. Then, a mirror image (an image that appears in a mirror) is used.

  As described above, the above method can be applied not only to image information but also to audio information. That is, noise in a noisy environment can be removed by calculating two-dimensional frequency information such as an image instead of one-dimensional frequency information such as sound. In this case, the training image information unit 21 is the training sound information unit 21a, the noise image 26 is the noise sound 26a, the noise-removed image 27 is the noise-removed sound 27a, and the two-dimensional image information in each process is one-dimensional sound information. By replacing with, it can be applied to noise removal of voice.

  Although the present invention has been described with the above embodiment, the technical scope of the present invention is not limited to the scope described in the embodiment, and various modifications or improvements can be added to this embodiment.

An example is shown below regarding the said embodiment. A 256 × 256 grayscale image is used as a benchmark. FIG. 4 is a benchmark image used in the embodiment. Here, four types of images (Lenna: FIG. 4 (A), Barbara: FIG. 4 (B), Boat: FIG. 4 (C), Title: FIG. 4 (D)) are used. In addition, FIG. 5A shows an image obtained by adding Gaussian noise to these images. Further, FIG. 6 shows an image cut into 80 × 80 in order to show that edge information (contour information) can be held. Details of each figure will be described later.

  In this embodiment, a spline binary wavelet filter is used as an initial filter. The coefficients are shown in Table 1 below.

  The value of N shown above is N = 3. This means that six partial original images and an image containing noise are used for learning (determined as a learning region). Each partial image is a 32 × 32 gray scale image, and an example is shown in FIG.

Since the support length of g ⁰ is considered to be too short to construct the learned filters g _h and g _v including noise information, first, the partial image not including important edge information and the support length of g ⁰ are selected. In order to stretch, (15) and (16) are used. Then, filters g ⁰ _h and g ⁰ _v are created, and these filters are used as initial filters for noise removal. Further, in order to determine the free variable s _r , a learning region is selected according to the image characteristics. For example, when it is desired to remove noise from a region including hair, a region similar to the training image is selected as a learning region. In this way, after preparing several types of learned filters, an optimal filter is selected according to the characteristics of the noise image to be processed. That is, in order to effectively remove noise, an appropriate learning region is determined, and an optimal learned filter is selected according to the image characteristics.

Since the support length of h ⁰ [n] is 4, α is set to 4. Although Non-Patent Document 4 states that the parameter λ in Expression (23) is obtained manually (trial and error) in accordance with the characteristics of the image, no specific numerical value is shown. Therefore, in this embodiment, λ = 11 is determined as a result of a preliminary experiment. The calculation of this example was performed using a commercially available standard CPU, and the execution time of the algorithm was 4.89 seconds. The execution time of the learning process depends on the number of learning regions, which can be ignored. This is because, in order to determine s _r , it is only necessary to solve a small-size simultaneous linear equation, and it generally takes 0.1 seconds to solve a simultaneous linear equation of about 10 elements.

  PSNR (peak signal to noise ratio) is used as an objective evaluation value of image quality. this is,

Defined by Here, f (i, j) is an L × L original image, and f￣ (i, j) is an approximation thereof. Table 2 below shows the coefficients of the spline wavelet filter, and Table 3 shows PSNR values. As a comparison with conventional methods, the BLS-GSM method and the VM method are taken up. LMEP includes a contour holding method, and LM does not include a contour holding method.

  The spline wavelet filter of Table 1 has 1 burnishing moment, and the filter of Table 2 has 3 burnishing moments (see Non-Patent Document 12 for details). In Non-Patent Document 12, a sigmoid function is used to determine a threshold value.

Is used, and if the decomposition coefficient becomes smaller than a certain threshold value, it does not suddenly become 0 but gradually becomes 0. In order to realize this, the values of the parameters a and b are important. However, in Non-Patent Document 12, the determination method is not described, and only the value is described. For this reason, in this embodiment, trial and error were independently performed and a = 145 and b = 4.6 were determined. If a and b are optimally determined according to the image, better results may be obtained.

  From Table 3, the LM, LMEP, and BLS-GSM methods, which are the methods of the present application, are superior to the VM method, and the PSNR is almost equivalent. However, unlike the BLS-GSM method, the method of the present application retains contour information as shown in FIGS. Here, FIG. 5A shows a noise image, FIG. 5B shows the result of noise removal by the BLS-GSM method, and FIG. 5C shows the result of noise removal by the VM method. ) Shows the result of noise removal by the LMEP method. 6A shows the result of noise removal by the BLS-GSM method, FIG. 6B shows the result of noise removal by the VM method, and FIG. 6C shows the result of noise removal by the LM method. As a result, FIG. 6D shows the result of noise removal by the LMEP method.

  Actually, an image from which noise has been removed by the BLS-GSM method has a blurred outline as compared with the method of the present application. The LM method and the LMEP method are almost the same in terms of PSNR and contour maintenance, but the LMEP method has a better PSNR value although it is a little. In any case, the effect of the technique of the present application appears.

  In the VM method, in order to determine the free variable so that the number of wavelet coefficients close to zero is maximized, the binary wavelet is assumed to have one burnishing moment. However, noise information may not be fully considered. On the other hand, since the method of the present application uses a noise image as a learning image in order to determine a free variable, noise information is sufficiently taken into consideration. Therefore, it is considered superior to the VM method. In general, it can be seen that the method of the present application is superior to the VM method and the BLS-GSM method in that the contour is maintained.

  Furthermore, noise removal was performed by applying the method of the present application to images with different brightness and contrast. The results are shown in FIGS. 8 and 9 and Table 4. The upper part of FIG. 8 shows the result of noise removal performed on the high brightness image, and the lower part of FIG. 8 shows the result of noise removal performed on the low brightness image. FIG. 8A shows the result of noise removal by the BLS-GSM method, FIG. 8B shows the result of noise removal by the VM method, and FIG. 8C shows the result of noise removal by the LM method. FIG. 8D shows the result of noise removal by the LMEP method. Further, the upper part of FIG. 9 shows the result of noise removal performed on the high contrast image, and the lower part of FIG. 9 shows the result of noise removal performed on the low contrast image. 9A shows the result of noise removal by the BLS-GSM method, FIG. 9B shows the result of noise removal by the VM method, and FIG. 9C shows the result of noise removal by the LM method. FIG. 9D shows the result of noise removal by the LMEP method.

  Except for the case of high brightness and high contrast, the results were the same as or better than other methods. The method of the present application has a poor PSNR value compared to the BLS-GSM method, but unlike the BLS-GSM method, it can learn the characteristics of noise, so it can be used not only for Gaussian noise but also for other noises. Can be applied. In order to show this advantage, the result of applying the method of the present application to the sesame salt noise image is shown in FIG. 10A shows a sesame salt noise image, FIG. 10B shows the result of noise removal by the VM method, FIG. 10C shows the result of noise removal by the LM method, and FIG. 10D shows the LMEP method. Shows the result of noise removal. As is apparent from FIG. 10, it can be seen that sesame salt noise is accurately removed by the LM method and the LMEP method.

  Furthermore, the technique of the present application was also applied to ISO noise. In this case, since there is no original image that does not contain noise, select a region that does not contain important information as a learning partial image (where there is little change), and let the free variable be learned so that the high-frequency component is zero. It was. The result is shown in FIG. 11A and 11B show ISO noise images, and FIGS. 11C and 11D show the results of noise removal. As is clear from the results of FIG. 11, it can be seen that the technique of the present application can also be applied to ISO noise.

In the above description, symbols such as “^,...” Are described after the characters for convenience, but actually indicate that they are added on the immediately preceding character. The symbol “_” is also shown after the character for the sake of convenience, but actually indicates that it is attached below the immediately preceding character.

1 Noise removal device 11 CPU
12 RAM
13 ROM
14 HD
15 Communication I / F
16 Input / output I / F
DESCRIPTION OF SYMBOLS 21 Training image information 22 Learning area extraction part 23 Filter production | generation part 24 Filter information storage part 25 Noise removal part 26 Noise image 27 Noise removal image

Claims (7)

Training frequency information storage means for storing training frequency information without noise, noise-free training frequency information, and training frequency information with noise in which the noise is included for each one or a plurality of types of noise in the noise-free training frequency information; ,
A learning region cutout unit that associates the training frequency information without noise and the training frequency information with noise, and cuts out an arbitrary learning region for each type of noise from each training frequency information,
Filter generating means for generating a filter that learns to eliminate the difference between the learning area of the training frequency information without noise and the learning area of the training frequency information with noise cut out by the learning area cutting out means;
The filter generated by the filter generation unit is applied to each of the learning regions cut out by the learning region extraction unit with respect to the processing target frequency information to be subjected to noise removal, and the noise of the processing target frequency information is applied. A noise removing device comprising noise removing means for performing removal. In the noise removal apparatus of Claim 1,
The noise removing apparatus, wherein the frequency information is audio information or image information. In the noise removal apparatus of Claim 1,
The frequency information is image information;
A difference between the learning area of the training frequency information without noise and the learning area of the training frequency information with noise based on the high-frequency components in the vertical direction, the horizontal direction, and the diagonal direction by the shift invariant wavelet transform. A noise removing device that learns to eliminate the noise. In the noise removal apparatus in any one of Claim 1 thru | or 3,
The learning area cutout means associates the noise-free training frequency information with the noise-trained training frequency information in a predetermined size cutout area set in advance, and according to the characteristics of the high-frequency component of the cutout area A noise removing device that determines a learning region to be cut out. In the noise removal apparatus in any one of Claims 1 thru | or 4,
The noise removing unit compares the high frequency component of the learning region extracted by the learning region extracting unit with the high frequency component of an arbitrary region in the processing target frequency information, and determines that the regions are determined to be similar to each other. On the other hand, a noise removing apparatus that removes noise by applying the filter corresponding to the learning region. Computer
The training frequency information without noise and the training frequency information without noise and the training frequency information with noise including the noise for each of one or more types of noise are associated with each training frequency. A learning region extraction step for extracting an arbitrary learning region for each type of noise from information,
A filter generation step of generating a filter that has been learned so as to eliminate the difference between the learning region of the training frequency information without noise cut out in the learning region extraction step and the learning region of the training frequency information with noise;
For the processing target frequency information to be subjected to noise removal, the filter generated by the filter generation step is applied to each learning region extracted by the learning region extraction step, and the noise of the processing target frequency information is And a noise removal step for performing the removal. Training frequency information storage means for storing noise-free training frequency information that does not include noise, and noise-free training frequency information in which the noise is included for each of one or more types of noise in the noise-free training frequency information,
A learning area cutout unit that associates the training frequency information without noise and the training frequency information with noise, and cuts out an arbitrary learning area for each type of noise from each training frequency information,
Filter generation means for generating a filter that learns to eliminate a difference between the learning area of the training frequency information without noise and the learning area of the training frequency information with noise cut out by the learning area cutout means;
The filter generated by the filter generation unit is applied to each of the learning regions cut out by the learning region extraction unit with respect to the processing target frequency information to be subjected to noise removal, and the noise of the processing target frequency information is applied. A noise removal program that causes a computer to function as noise removal means for performing removal.