RU2623898C2 - Method of removal of noise on raster images based on full variation - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: restore the original image of u(x,y), (x,y)∈R² for a specified noisy image ν(x,y), calculating the total variation of the brightness function, formulating the problem of minimizing the total variation of the brightness function with a restriction on the noise intensity. Considering this constraint on the basis of a linear combination of a mixture of Gaussian and Poisson noise, reducing the obtained optimisation problem with the restriction to the optimisation problem without restriction in the form of the Lagrange functional, constructing the Euler-Lagrange equation for solving the unconditional optimisation problem, building a numerical scheme for solving this equation, looking for the best parameters, comparing deviations of values of brightness functions of two images on two consecutive steps of iteration with a specified accuracy ε to check the stopping of the iteration process, determining the optimal parameters λ_1, λ₂, μ, σ, at each step of the iteration to eliminate noise on real images.

EFFECT: effective elimination of both Gaussian and Poisson noise alone, and a linear combination of these noises.

4 tbl, 2 dwg

Description

This invention relates to the field of image processing and can be used to eliminate noise on raster images in the scientific and technical field, medicine, biology, etc.

In modern research, bitmaps are an important type of information. In raster images, various effects are observed, including noise. Noise reduces the quality of images and the quality of their processing. Therefore, the problem of eliminating noise in digital images is still relevant. Many methods have now been developed to eliminate noise in bitmap images (Chan TF, Shen J. Image processing and analysis: Variational, PDE, Wavelet, and stochastic methods. - SIAM, 2005, 400 c; Burger M., Mennuci ACG, Osher S ., Rumpf M. Level set and PDE based reconstruction methods in imaging. - Springer, 2008, 319 c).

Close to the claimed invention is a noise elimination method proposed by the authors: Rudin, Osher, Fatemi (LI Rudin, S. Osher, E. Fatemi. Nonlinear total variation based noise removal algorithms. Physica D, 1992, Vol. 60, 259-268) . They proposed to use the full variation of the brightness function of the raster image to eliminate noise:

,

,

, ∇u=(u_x, u_y),

,

,

, λ>0 - множитель Лагранжа,

- полная вариация.where u (x, y) ∈R ² is ideal (without noise) and ν (x, y) ∈R ² is a real (noisy) image,

, ∇u = (u _x , u _y ),

,

,

, λ> 0 is the Lagrange multiplier,

- full variation.

The disadvantage of this method is the effective elimination of only Gaussian noise, and for other types of noise this method is not very effective.

Also close to the claimed invention is the method of Gannan Yuan and co-authors (Yuan Gannan, Han Zifa, Zhang Jie, Dong Jing, Zhao Yuxin, Li Tao, Song Chengye, Li Qianf, Guo Ruiliang - Chinese patent CN103198455 A, published July 10, 2013). This method consists in performing the following steps:

- гауссовское ядро, σ² - дисперсия распределения Гаусса.1. Determine the Gaussian filter of the noisy image H = ν * G _σ , where

is the Gaussian core, σ ² is the dispersion of the Gaussian distribution.

2. Determine four matrices of coincidence of gradations of gray (Gray-Level Co-Occurrence Matrix - GLCM) when processed by a Gaussian filter in different directions (0 °, 45 °, 90 °, 135 °). Four GLCM matrices are determined based on neighboring pairs of pixels defined by a given direction and distance. So, if the distance is one pixel, and the direction is horizontal (90 °), then all pairs of pixels in the window located horizontally next are taken to determine the conjugacy. Similarly, pairs are determined for the remaining directions.

3. Calculate the contrast image as the average of the four GLCM matrices. The noise elimination model is based on minimizing the energy functional of the full variation of the brightness function of a noisy image and the brightness function of a contrast image.

4. Construct the Euler-Lagrange equation and solve it by the iterative method.

The disadvantage of this method is to filter only Gaussian noise. This method is also not designed to effectively eliminate other types of noise.

Another invention that is close to the claimed invention is a noise reduction method proposed by the authors: Lee, Chartrand and Asaki (T. Le, R. Chartrand, TJ Asaki. A variational approach to reconstructing images corrupted by Poisson noise. Journal of mathematical imaging and vision, 2007, Vol. 27, Is. 3, 257-263):

,

,

where β> 0 is the regularization coefficient. The disadvantage of this method is to eliminate only Poisson noise, and for other types of noise this method is also not very effective.

All of the above methods consist in the fact that during the noise elimination procedure a characteristic property of the full variation of the brightness function was used: the full variation of noisy images is always greater than the full variation of the corresponding smooth images. The essence of these methods of noise elimination based on the full variation of the brightness function is to perform the following actions:

задана ограниченная область

. Если функция и является гладкой, то ее полная вариация имеет вид:

.1. Enter the full variation of the brightness function of the noisy image. Let in space

limited area set

. If the function u is smooth, then its full variation has the form:

.

2. Determine the constraint for solving the optimization problem V _T [u] → min.

3. Reduce the problem of conditional optimization (with restriction) to the problem of unconditional optimization (without restriction).

4. Construct the Euler-Lagrange equation of the optimization problem without restriction.

5. Build a numerical scheme for solving the Euler-Lagrange equation.

6. Evaluate the quality of noise elimination through assessing the quality of the restored image.

The main disadvantage of the above methods is the lack of guarantee that the common types of noise will be effectively eliminated, since these methods are intended only to eliminate Gaussian noise or only to eliminate Poisson noise. The considered methods inefficiently eliminate other types of noise and especially a mixture of Gaussian and Poisson noise. A mixture of these two types of noise is often considered in medicine, biology, astronautics, etc.

The task to which the invention is directed is to provide guarantees for the effective elimination of both a mixture of noise and separately Gaussian and Poisson noise.

This problem is solved due to the fact that the claimed invention solves the problem of minimizing the full variation of the brightness function of a noisy image with a restriction on the noise intensity. For the claimed method, parameters are calculated that determine the optimal solution.

The method of eliminating noise based on full variation is characterized in that the original image is restored and for a given noisy image ν as follows:

;1. the full variation of the brightness function is calculated

;

2. The problem of minimizing the total variation of the brightness function with a restriction on the noise intensity is formulated;

3. the resulting conditional optimization problem with restriction is reduced to the problem of unconditional optimization without restriction in the form of the Lagrange functional;

4. the Euler-Lagrange equation is constructed to solve the problem of unconditional optimization;

5. A numerical scheme for solving this equation is constructed: optimal input parameters are searched, deviations of the brightness functions of two images at two successive iteration steps are compared with a given accuracy ε, if Δu≤ε, then the solution is found, if Δu> ε, then the iteration process continues.

The structural diagram for a method of eliminating noise (Fig. 1) contains: an input unit 1 to which a noisy image is received. The output of block 1 is connected to the input of preparation block 2, in which the full variation of the brightness function of the noisy image is calculated and a restriction on the noise intensity is determined. The output of the preparation unit 2 is the input of the conversion unit 3, the output of which is connected to the input of the calculation and optimization unit 4. In the conversion unit 3, the optimization problem with restriction is reduced to the problem without restriction and the Euler-Lagrange equation is constructed. The numerical scheme is determined in block 4. The output of block 4 is connected to the input of the correction block 5 with feedback. Based on a comparison of the deviations of the brightness functions of two images at two consecutive iteration steps with a given accuracy, which are determined in block 4, the stopping condition is checked. The output of block 5 is connected to the second input of block 4. The output of block 5 is the result of eliminating noise in the original image.

The technical result is the effective elimination of both Gaussian and Poisson noise separately, and a linear combination of these noises.

The invention is illustrated by drawings, which depict:

FIG. 1 is a block diagram for eliminating noise;

FIG. 2 - comparison of the efficiency of using several methods of noise elimination using real images with artificial noise as an example.

The method of eliminating noise in raster images is as follows.

1. Take a noisy image of ν.

2. Calculate the full variation of the brightness function of the noisy image ν and formulate the optimization problem V _T [u] → min.

3. Identify restrictions. It is assumed that for a given image, the noise variation is constant (Poisson noise does not change, and Gaussian noise depends only on its variation):

where p (ν | u) is the conditional probability of observing a real image ν for a given ideal image u.

Consider a Gaussian noise. The distribution density is defined as:

.

.

The Poisson noise density is defined as: p ₂ (ν | u) = exp (-u) u ^ν / ν !.

Note that the values of the image brightness functions u and ν are integers (for example, for an eight-bit image, the brightness interval is determined by values from 0 to 255).

To eliminate the mixture of Gaussian and Poisson noise, we consider their linear combination:

ln (p (ν | u)) = λ ₁ ln (p _l (ν | u)) + λ ₂ ln (p ₂ (ν | u)), where λ ₁ > 0, λ ₂ > 0, λ ₁ + λ ₂ = 1. According to (1), we obtain the noise elimination problem with the restriction:

,

,

where κ is a constant value.

4. To reduce the problem of conditional optimization with restriction to the problem of unconditional optimization using the Lagrange functional:

,

,

to find a solution:

where τ> 0 is the Lagrange multiplier.

In this model, if λ ₁ = 0 and λ ₂ = 1, then with β = τλ ₂ = τ, a modified ROF model will be obtained to eliminate Poisson noise. If λ ₂ = 0 and λ ₁ = 1, then with λ = τλ ₁ / (2σ ² ) = τ / (2σ ² ), the ROF model will be obtained to eliminate Gaussian noise. If λ ₁ > 0, λ ₂ > 0, then a model will be obtained to eliminate the mixture of Gaussian and Poisson noise.

5. Build a numerical scheme for the model of eliminating a mixture of noise. To solve problem (2), we can use the Lagrange multiplier method. But here the Euler-Lagrange equation is used.

Let a function ƒ (x, y) be defined in a bounded domain Ω⊂R ² and be continuously differentiable up to the second order in x and y for (x, y) ∈Ω.

,

. Решение задачи оптимизации

удовлетворяет уравнению Эйлера-Лагранжа:Let F (x, y, ƒ, ƒ _x , ƒ _y ) be a convex functional, where

,

. The solution to the optimization problem

satisfies the Euler-Lagrange equation:

,

,

,

,

.Where

,

,

.

Then the solution to problem (2) satisfies the following Euler-Lagrange equation:

where μ = 1 / τ.

We represent equation (3) in the following form:

.Where

.

To obtain a discrete model (4), we add an artificial time parameter u = u (x, y, t). Equation (4) corresponds to the following diffusion equation:

Then the discrete form of equation (5) has the form:

,

,

,

,

,

,

,

,

,

,

,

,

;

;

;

; i=1, …, N₁; j=1, …, N_2;

;

;

;

; i = 1, ..., N ₁ ; j = 1, ..., N _2;

k = 0, 1, ..., K; Δx = Δy = 1; 0 <ξ <1, K is a sufficiently large number, K = 500.

,

, μ^k на каждой итерации k. В новой процедуре такие параметры будут вычисляться на каждом шаге итерации.6. Find the optimal model parameters for eliminating the noise mixture. Procedure (6) can be used to eliminate noise in images if the values of the parameters λ ₁ , λ ₂ , μ, σ are given. Often, in practice, these parameters are unknown, and they need to be evaluated. Then the parameters λ ₁ , λ ₂ , μ in the process (6) must be represented as

,

, μ ^k at each iteration k. In the new procedure, such parameters will be calculated at each step of the iteration.

. Данное условие позволяет вычислить оптимальные параметры линейной комбинации шумов λ₁, λ₂:Let (u, τ) be a solution to problem (2). Then we get the condition

. This condition allows you to calculate the optimal parameters of a linear combination of noise λ ₁ , λ ₂ :

, λ₂=1-λ_1.

, λ ₂ = 1-λ _1.

The discrete form for calculating these parameters has the form:

,

, k=0,1, …, K.

,

, k = 0,1, ..., K.

To find the optimal parameter μ, we multiply (3) by (ν-u) and integrate by parts over the entire domain Ω. As a result, we obtain the formula for finding the optimal parameter μ:

.

.

The discrete form of computing this parameter is:

,

,

Where

,

,

,

,

,

,

,

,

,

;

;

;

;

,

;

;

;

;

;

;ν _0j = ν _1j ;

;

;

k = 0, 1, ..., K; Δx = Δy = 1.

To calculate the parameter about here, the Immerker method (J. Immerker, Fast noise variance estimation. Computer vision and image understanding, 1996, Vol. 64, No. 2, 300-302) was used:

- маска изображения, N₁ и N₂ определяют размер изображения.Where

- image mask, N ₁ and N ₂ determine the size of the image.

The * operator is a convolution operator that performs the conversion:

where i = 1, ..., N _l ; j = 1, ..., N ₂ ; u _ij = 0 if i = 0, or j = 0, or i = N ₁ +1, or j = N ₂ +1.

The parameter σ is calculated immediately at the first iteration.

7. Evaluate image quality and noise reduction quality. To assess the quality of images after noise elimination, the criteria PSNR, MSE and SSIM were used:

,

,

,

,

,

,

,

,

,Where

,

,

,

,

,

,

,

C ₁ = (K ₁ L) ² , C ₂ = (K ₂ L) ² ; K ₁ <<1; K ₂ << 1.

For example, K ₁ = K ₂ = 10 ^-6 , L = 2 ⁸ -1 = 255 - the brightness of the 8-bit gray image.

The higher the Q _PSNR , the better the image quality. If the Q value of _PSNR lies between 20 and 25, then the image quality is acceptable, for example, for wireless transmission.

The Q _MSE value was used to estimate the difference between the two images, where Q _MSE is the standard error. The lower the _MSE Q value, the better the recovery result. The Q value of _{MSE is} directly related to the Q value of _PSNR .

The Q _SSIM value was used to evaluate image quality by comparing the similarity of the two images. Its value lies in the interval [-1, 1]. The higher the Q _SSIM , the better the image quality.

,

, μ⁰. Если сначала задать параметры

,

, μ⁰, то неудачные значения определят не очень хорошие оценки u_ij, а через них - оценки параметров распределений. Случайный выбор параметров

,

, μ⁰ также неприемлем, т.к., фактически, вносит дополнительный шум в изображение.8. To evaluate the dependence of the noise elimination result on the initial solution. Obviously, in the local iterative procedure (6), the result in the general case depends on the initial values of the parameters

,

, μ ⁰ . If you first set the parameters

,

, μ ⁰ , then the unsuccessful values will determine not very good estimates u _ij , and through them - estimates of the distribution parameters. Random selection of parameters

,

, μ ^{0 is} also unacceptable, because, in fact, introduces additional noise into the image.

,

, μ⁰ должны быть, по возможности, достаточно близки к тем значениям, которые будут найдены. Поэтому оценим параметры

,

, μ⁰ как средние по соседним пикселям изображения, используя, например, метод Иммеркера.Obviously, the initial values of the parameters

,

, μ ⁰ should be as close as possible to those values that will be found. Therefore, we evaluate the parameters

,

, μ ⁰ as averages over neighboring pixels of the image, using, for example, the Immerker method.

Consider an example implementation of the method. The proposed model was tested on real images. As an example, the result is shown for the image of a human skull with a size of 300 × 300 pixels (Fig. 2A). The remaining images (Fig. 2b-2z) show an enlarged fragment of this image.

To obtain a noisy image, first a Gaussian noise was added (Fig. 2c), then a Poisson noise (Fig. 2e). In FIG. 2g shows a noisy image for a mixture of two noise with parameters λ ₁ = 0.8, λ ₂ = 0.2. The parameters of the linear combination of λ ₁ and λ ₂ were determined as follows.

, относительно среднего u_ij в каждом пикселе с координатами (i, j) i=1, …, N_l; j=1, …, N₂.First, Poisson noise with a distribution density p ₂ (ν | u) and variation

, relative to the average u _ij in each pixel with coordinates (i, j) i = 1, ..., N _l ; j = 1, ..., N ₂ .

.The brightness function of such an image is denoted by ν ⁽²⁾ . Its values should be in the range from 0 to 255. If the value is outside this range, then it does not change ν ⁽²⁾ _ij = u _ij . In this image, there were only five of these values (0.0056%). The total variation in Poisson noise is defined as the mean

.

находятся за пределами этого диапазона. Результирующее изображение (фиг. 1ж) образовано двумя зашумленными изображениями в пропорции 0,5 для ν⁽¹⁾ и 0,5 для ν⁽²⁾. Это означает, что ν=0,5ν⁽¹⁾+0,5ν⁽²⁾. Следовательно:Further, we assumed that the variation of the Gaussian noise σ ₁ = 40.2412 will be four times greater than σ ₂ . The brightness function of such an image is denoted by ν ⁽¹⁾ . As before, the values of the brightness function ν ⁽¹⁾ should also be in the range from 0 to 255. In this case, it turned out that 5780 (6.42%) pixels with a brightness value

are out of this range. The resulting image (Fig. 1g) is formed by two noisy images in the proportion of 0.5 for ν ⁽¹⁾ and 0.5 for ν ⁽²⁾ . This means that ν = 0.5ν ⁽¹⁾ + 0.5ν ⁽²⁾ . Hence:

.

.

As a result, we obtain that the coefficients of the linear combination have values, respectively, λ ₁ = 4/5 = 0.8 and λ ₂ = 1/5 = 0.2.

For a noisy image, the quality criteria are, respectively, Q _PSNR = 21.4168, Q _MSE = 427.9526 and Q _SSIM = 0.4246.

In the table. Figure 1-3 shows the results of noise reduction in this image for cases of predefined and automatically determined parameters.

Note that in this case, the Q _PSNR value after eliminating the noise for predefined (ideal) parameters is better than for automatically obtained estimates, although the opposite is sometimes observed for the Q _SSIM value.

To create the initial image, the convolution operator (7) was used. In the table. 4 shows the dependence of the restored result on the initial solution, where:

, μ=1;(a) initial parameters

, μ = 1;

, μ=1;(b) initial parameters

, μ = 1;

(c) the initial solution μ ⁰ is a random matrix of a given size;

.(d) the initial solution μ ^{0 is} found as averaging of neighboring pixels u ⁰ = ν * A by the convolution operator, where

.

In the table. 4 shows that the best result of eliminating the combined noise corresponds to case (d) of the choice of the initial solution according to the criteria of PSNR and MSE.

Thus, a method for eliminating a mixture of Gaussian and Poisson noise based on the known variational approach is proposed here. The quality of the noise elimination result depends on the values of the linear combination coefficients λ ₁ and λ ₂ . Their values must be predefined, or they must be automatically determined, which is important in the case of processing real images. For real images, this method with automatically detected parameters gives a result that is close in quality to ideal when the correct parameter values are set in advance.

This method can be used to separately eliminate both Gaussian and Poisson noise. In this case, the quality of processing is practically not inferior to the quality of methods specially designed to eliminate noise of only one kind.

Tab. 1. Comparison of the quality of noise elimination methods in a real image for a noise mixture

Tab. 2. Comparison of the quality of noise elimination methods in a real image with Gaussian noise

Tab. 3. Comparison of the quality of noise elimination methods in a real image with Poisson noise

Tab. 4. The dependence of the result of eliminating the combined noise from the initial solution

Claims (5)

The method of eliminating noise based on the full variation, which consists in restoring the original image u (x, y) for a given noisy image ν (x, y),

, where u (x, y) and ν (x, y) are images in x and y coordinates,

- two-dimensional coordinate space, calculate the full variation of the brightness function, formulate the problem of minimizing the full variation of the brightness function with a restriction on the noise intensity, characterized in that they consider this restriction on the basis of a linear combination of a mixture of Gaussian and Poisson noise, reduce the resulting optimization problem to effectively eliminate as Gaussian and Poisson noise separately, and their linear combination, to the problem of unconditional optimization, in accordance with which the Aile equation is constructed era-Lagrange to solve the problem of unconditional optimization, build a numerical scheme for solving this equation, look for the optimal input parameters λ ₁ , λ ₂ , μ, σ for the original image, compare the deviation of the brightness functions of two images at two consecutive iteration steps with a given accuracy s for checking the stop of the iteration process, and determine the optimal parameters λ ₁ , λ ₂ as:

, where σ is the noise variation parameter; u and ν are the values of the image brightness functions; and parameter μ as:

, where λ ₁ , λ ₂ are the optimal parameters, σ is the parameter of noise variation, at each step of the iteration of eliminating noise in real images.

Images