KR100754219B1 - An extended method and system to estimate noise variancestandard deviation from a video sequence - Google Patents

Abstract

The noise estimator uses an image structure remover and a noise variance (standard deviation) calculator. The image structure remover calculates the difference across local windows within two consecutive frames. The noise variance (standard deviation) calculator estimates the noise variance (standard deviation) from the distribution of local differences. If there is no motion or little motion between two consecutive frames, the image structure can be eliminated by calculating local differences. Therefore, a very resistant estimation result can be obtained.

Noise, estimation, variance, standard deviation, image

Description

An extended method and system to estimate noise variance (standard deviation) from a video sequence}

1 is a block diagram of a video noise estimator in accordance with an embodiment of the present invention.

FIG. 2 shows an example of calculating the mean absolute error (MAE) adjusted as the time local difference in FIG. 1.

3 shows an example of a distribution (histogram) of the adjusted mean absolute error (MAE) in the absence of motion between two consecutive frames.

4 shows an example of calculating a mean square error (MSE) adjusted as the time local difference of FIG.

The present invention generally relates to video processing, and more particularly to noise estimation in video signals.

Noise estimation is required by many algorithms to optimally process an image or video. For example, in TV systems, noise reduction is often applied as the first step in obtaining a noise-free video sequence. The noise reduction optimization algorithm is noise adaptive, which first estimates the noise variance of the input video sequence and then performs the noise reduction. Noise estimation is very important in this case. This is because overestimation causes image blurring and underestimation causes insufficient noise attenuation.

를 순간적인 시간 t에서의 들어오는 비디오 프레임으로 놓고,

가 세로좌표,

가 가로좌표를 나타낼때

를좌표

에서의 대응하는 픽셀 값으로 놓는다. 일반적으로 우리는 입력 비디오 시퀀스는 분산으로

를 가지고 독립적이면서, 항등 분포의 가법적이고 정상의 0평균 가우시안 잡음(additive and stationary zero-mean Gaussian noise) 에 의하여 손상되는 것으로 가정한다. 즉,

가 잡음 손상이 없는 진정한 픽셀 값을 나타내고,

가 To describe the problem of noise estimation

With the incoming video frame at the instant t,

Is the ordinate,

Indicates abscissa

Coordinate

Set to the corresponding pixel value at. Normally we input as the video sequence is distributed

Independently, we assume that we are impaired by additive and stationary zero-mean Gaussian noise of the identity distribution. In other words,

Represents a true pixel value with no noise damage,

end

[Equation 2]

는Any pixel when the Gaussian distributed noise component satisfies

Is

[Equation 1]

로 나타내어질 수 있다.

It can be represented as.

의 선행 정보 없이 손상된 이미지

의 잡음 분산

를 측정하는 것이다. Therefore, the problem of noise estimation is that the original image

Corrupted image without prior information from

Noise variance

To measure.

의 국부 분산(local variance)의 기대값을 계산하는 것이다. 이 방법은 이미지 구조(image structure)로부터 손상을 겪는데, 즉, 과대추정을 유발하게 된다. 이 문제를 극복하기 위해서 몇몇의 방법이 제안되었다. 하나의 방법은 대응하는 픽셀의 그래디언트 크기(gradient magnitude)가 미리 조정한 임계치보다 더 큰 경우 국부 분산을 제외시키는 것이다. 그러나 그래디언트 크기는 또한 잡음 분산과 연관되어 있기 때문에 정확한 임계치를 찾는것은 어렵다. 다른 종래의 방법들은 먼저 손상된 이미지

에 고역 통과 필터를 적용하여, 작은 구조(little structure)를 가지고 잡음 요소를 추출한 후 잡음 요소에 잡음 추정을 수행하는 것이다. 하나의 예는 잡음 분산이 각 레벨에서의 네 개의 가장 작은 블록 기반의 국부 분산들의 시퀀스로부터 추정될 때, 이미지를 서로 다른 블록 크기의 피라미드 구조로 분해하는 것이다. 또 다른 예는 잡음 분산이 레일리 확률 밀도 함수(Rayleigh probability density function)가

에서 최대값에 이르는 속성에 기초하여 잡음 분산이 추정될 때, 레일리 분포를 강도 그래디언트(intensity gradient)의 크기에 일치시키는 것이다. 다른 방법들은 가법잡음(additive noise) 뿐만 아니라 배수사 잡음(multiplcative noise)을 추정하는 것이다. Simple method of noise estimation image

Calculate the expected value of the local variance of. This method suffers damage from the image structure, i.e. causes overestimation. Several methods have been proposed to overcome this problem. One method is to exclude local variance if the gradient magnitude of the corresponding pixel is greater than the preset threshold. However, it is difficult to find the exact threshold because gradient size is also associated with noise variance. Other conventional methods first damage the image

The high pass filter is applied to the noise element to extract the noise element with a little structure and then perform noise estimation on the noise element. One example is to decompose an image into pyramid structures of different block sizes when the noise variance is estimated from the sequence of the four smallest block based local variances at each level. Another example is that the noise variance is the Rayleigh probability density function

When the noise variance is estimated based on the property up to the maximum value at, the Rayleigh distribution is matched to the magnitude of the intensity gradient. Other methods are estimating multiplcative noise as well as additive noise.

All of the above methods use spatial local statistics to estimate the noise variance. The accuracy of estimation depends on the degree of separation of the noise component from the actual image signal. Robustness drops dramatically when most images contain complex structures.

The present invention solves the above problems. One embodiment of the present invention provides a noise estimator using an image structure remover and a noise variance (standard deviation) calculator. The image structure eliminator calculates the difference over the local window in two consecutive frames of each pixel. For example, mean absolute error (MAE), mean square error (MSE), and the like. The noise variance calculator estimates the noise variance (standard deviation) from the distribution of the temporal local difference. If there is no motion or little motion between two consecutive frames, the image structure can be eliminated by calculating such local time difference. As a result, very resistant estimation results can be obtained.

Other embodiments, features, and advantages of the present invention will become apparent from the accompanying description in conjunction with the following drawings.

With reference to the accompanying drawings, embodiments of the present invention will be described. In one embodiment, the present invention provides a noise estimator using an image structure remover and a noise variance calculator, as described in more detail below. The image structure remover calculates the temporal difference over the local window in two consecutive frames of each pixel. For example, mean absolute error (MAE), mean square error (MSE), and the like. The noise variance calculator estimates the noise variance (standard deviation) from the distribution of time local differences. If there is no motion or little motion between two consecutive frames, the image structure can be eliminated by calculating such a time local difference. As a result, very resistant estimation results can be obtained.

Initially, an analysis of the Gaussian distribution signal is provided. An example of removing corrupted noise from a video sequence is provided in Commonly Assigned US Patent Application No. 11 / 025,173, "Method of Temporal Noise Reduction in Video Sequences," which is hereby incorporated by reference.

According to one embodiment of the invention in the cited reference, temporal noise cancellation is applied to two video frames, one video frame is the noisy frame of the current input, and the other video frame is a The filtered frame. Once the current frame is filtered, the filtered result is stored in memory to filter the next incoming frame. A motion adaptive temporal filtering method is applied for noise cancellation. Motion information in pixels between the current frame and the previous filtered frame in memory is checked. The pixels in the current frame are then classified into motion regions and non-motion regions relative to the previous filtered frame. In the non-motion region, pixels in the current frame are filtered along the time axis based on Maximum Likelihood Estimation where the filtered output is inherently optimal. In the motion domain, the temporal filter is switched off to avoid motion blur.

가 n ~ N(0,

) 로 표시되는 독립적, 항등 분포의 가법적이고 정상의 0평균 가우시안 잡음(additive and stationary zero-mean Gaussian noise)으로 손상된다고 가정한다. 관찰된 값 x는 Hereinafter, the analysis of the Gaussian distribution signal and the maximum likelihood estimation method will be discussed. Unknown constant value

Is n to N (0,

We assume that we are impaired by additive and stationary zero-mean Gaussian noise of an independent, identity distribution, denoted by. The observed value x is

는 몇 개의 관찰 값들로부터 최대 우도 추정법을 이용하여 추정될 수 있다. It can also be defined as a Gaussian distribution random variable. Constant value

Can be estimated from several observations using the maximum likelihood estimation method.

에 대하여, 우도(likelihood) 함수

는k observed values

Likelihood function

Is

는 잡음 표준편차이다. 관계식 (2)로부터

는 Can be expressed as

Is the noise standard deviation. From relation (2)

Is

를 최소화하는

값에 의해 최대화될 것이다. 미분

를 계산하여, 이 미분값을 0으로 놓고 방정식을

에 대하여 풀면, 그때 최대 우도 추정은 다음과 같이 정의된다.sign

To minimize

It will be maximized by the value. differential

Calculate, set this derivative to zero and solve the equation

Solving for, then the maximum likelihood estimate is defined as

The above results can also be obtained sequentially as follows.

Furthermore, using cyclic weights, the relation (5) can be modified as follows.

는 평균값

가 얻어지는 픽셀들의 수를 나타내는 최적의 가중치이다. 관계식 (6)은 추정된 값을 순차적으로 갱신하고 관계식 (4)보다 더 적은 메모리를 요구하기 때문에 이점이 많다. 더욱이, 관계식 (4)는

값을 추정하기 위하여 모든 본래의 데이터를 메모리에 저장할 필요가 있지만, 관계식 (6)은 단지 현재의 관찰된 값

, 이전의 추정된 값

및 최적의 가중치

만을 저장할 필요가 있다.here,

Is the average value

Is the optimal weight indicating the number of pixels to be obtained. Relation (6) is advantageous because it updates the estimated values sequentially and requires less memory than relation (4). Moreover, relation (4)

It is necessary to store all the original data in memory to estimate the value, but relation (6) only shows the current observed value.

, Previous estimated value

And optimal weights

Only need to store.

In temporal noise cancellation, it is generally assumed that the input video is corrupted by the same type of noise as in the above analysis. Thus, each pixel can be considered a constant value (true pixel value) corrupted by Gaussian noise. In the non-motion region, since the pixels along the time axis have the same true pixel values, they can be optimally estimated using relation (4) or relation (6). Since video frames enter the video system sequentially, Equation (6) is convenient for removing noise on a frame-by-frame basis, furthermore essentially reducing memory requirements.

에서 잡음 감쇄기는 잡음 요소

를 제거하고 실제 픽셀 값

를 추정한다. 추정된 값

는 다음과 같이 얻어진다.That is, no pixels

Noise attenuator in the noise element

Remove the actual pixel values

Estimate Estimated value

Is obtained as follows.

[Equation 3]

And

[Equation 4]

는, 초기치로

을 가지고 다음 프레임

에서의 픽셀들을 필터링 하는데 있어서의

의 잠재적인 가중치이고,

는 픽셀

의 관련된 가중치이고, 또한 픽셀

의 움직임 검출 함수로서

의 조정이다. 그리고

는

값을 초과할 수 없는 임계치이다. At this time

Is the initial value

Take the next frame

In filtering pixels in

Is the potential weight of,

Is a pixel

Is the associated weight of

As a motion detection function of

Is the adjustment of. And

Is

The threshold cannot exceed the value.

가 움직임 영역이라면,

은 0으로 설정된다. 만약 움직임 영역이 아니라면

는

로 설정된다. 이와 동등하게, 필터링된 픽셀

는

의 가중치를 갖는 원본 픽셀들의 평균이라고 말할 수 있다.If pixel

If is a moving area,

Is set to zero. If it's not a moving area

Is

Is set to. Equivalently, filtered pixels

Is

It can be said that the average of the original pixels having a weight of.

의 연속적인 움직임 레벨을 가리키는 부동 소수점 값(floating point value)을 출력하면 , 그때에

는 0과

사이에 삽입된다. 이 경우에 간단히 설명하면 필터링된 픽셀

는

의 가중치를 갖는 원본 픽셀들의 평균 이라고 말할 수 있다. If the motion detection pixel

If you print a floating point value that indicates a continuous level of motion, then

Is 0 and

Is inserted in between. In this case, simply explain the filtered pixel

Is

It can be said that the average of the original pixels with a weight of.

의 잡음 분산이 추정될 때에, 과거 프레임(previous frame)은

을 얻기 위해 상기 방법에 의하여 이미 필터링 되고, 각 픽셀

의 잠재적인 가중치

또한 얻어지고, 메모리에 저장된다. 픽셀

의 잡음 분산은 [수학식5]와 같이 얻을 수 있다. frame

When the noise variance of is estimated, the previous frame

Each pixel already filtered by the above method to obtain

Potential weight of

It is also obtained and stored in memory. pixel

The noise variance of can be obtained as in [Equation 5].

[Equation 5]

로 놓고, 만약

와

사이에 움직임이 없다고 한다면,

는 잡음 분산

을 가지고,평균이 0인 가우시안 분포 확률변수가 된다.

Put it on, if

Wow

If there is no movement in between,

Is noise dispersion

, Which is a Gaussian distribution random variable with a mean of zero.

로 놓으면

If you put it

이므로, 만약

와

사이에 움직임이 없다고 한다면

의 확률 밀도함수(p.d.f)는 [수학식 6]과 같이 얻어질 수 있다.

If so,

Wow

If there is no movement between

The probability density function of (pdf) can be obtained as shown in [Equation 6].

[Equation 6]

의 분포가 픽셀 값과 픽셀의 기하학적, 시간적 위치에독립적이라는 것을 안다, 즉, 우리는

를

로 표시한다.

의 1차 기대치는 [수학식7]과 같이 계산될 수 있다.We are such

We know that the distribution of is independent of the pixel value and the geometric and temporal position of the pixel, that is, we

To

To be displayed.

The first expectation of can be calculated as shown in [Equation 7].

[Equation 7]

의 2차 기대치는 [수학식8]과 같이 계산될 수 있다.

The second expectation of can be calculated as shown in [Equation 8].

[Equation 8]

의 분산은 [수학식9]와 같이 얻어질 수 있다.

The variance of can be obtained as shown in [Equation 9].

[Equation 9]

로 놓으면

If you put it

이므로, 만약

와

사이에 움직임이 없다고 한다면

의 분포는 픽셀값과 픽셀의 기하학적, 시간적 위치에 독립적이다.

의 1차 기대값은 [수학식 10]과 같이 얻어질 수 있다.

If so,

Wow

If there is no movement between

The distribution of is independent of the pixel value and its geometric and temporal position.

The first expected value of can be obtained as shown in [Equation 10].

[Equation 10]

Based on the analysis, a block diagram of one embodiment of a noise estimation system 100 according to the present invention is shown in FIG. 101), a noise canceller 106 to filter the image for noise attenuation, and memory devices 108,110.

와 과거의 필터링된 프레임

사이의 시간 국부 차이

를 계산한다. 잡음 분산/표준 편차 계산기(104)는 시간 국부 차이의 분포를 분석하고 잡음 분산/표준편차

를 추정한다. 한 예로서 시간 국부 차이

는 평균 절대 오차(MAE), 평균 자승 오차(MSE)와 같이 두개의 연속적인 이미지에서의 H X W 크기를 가지는 국부 윈도우에 걸쳐서 계산된다. 다음의 설명은 본 발명에 따른 몇몇의 예시적인 잡음 추정의 실시예들을 제공한다. 다른 예들도 가능하다.Image structure remover 102 is the current frame

And past filtered frames

Time local difference between

Calculate The noise variance / standard deviation calculator 104 analyzes the distribution of time local differences and noise variance / standard deviation.

Estimate As an example, time local differences

Is calculated over a local window having an HXW size in two consecutive images, such as mean absolute error (MAE) and mean square error (MSE). The following description provides some exemplary noise estimation embodiments in accordance with the present invention. Other examples are also possible.

를 계산하는 일 실시예를 제공한다. Referring to FIG. 2, an example difference unit 200 in the image structure remover 102 of FIG. 1 is a time local difference using an average absolute error (MAE) adjusted as shown in Equation 11 below.

Provides an embodiment for calculating.

[Equation 11]

에 대한 상기 관계를 구현한다.Example difference device 200 uses difference means 202, absolute value means 204, calculation block 206, multiplier 208, adder 210, divider 212.

Implement the above relationship for.

를 계산하고, 절대값 수단(204)은

을 계산하며, 계산 블록(206)은

(즉

)이 잡음 감쇄기(106)에 의하여 생성될 때

을 계산하고, 배율기(208)는

을 계산하며, 덧셈기(210)는 The differential means 202

And absolute value means 204

, And the calculation block 206

(In other words

Is generated by the noise attenuator 106

And the multiplier 208

, Adder 210 is

를 계산하고, 분할기(212)는

에 대한 마지막 결과를 출력한다.

And divider 212

Print the last result for.

는 그러한 잡음이 제거된

의 평균이다. 추정된 분산은

와 관계가 있고 그 관계는 계산 블록(206)에 의해 지시된다. Filtered Past Frame Pixels

Is such noise removed

Is the average. The estimated variance is

And the relationship is indicated by calculation block 206.

가 서로 매우 가깝다는 것을 안다. 즉, 조정된 평균 절대 오차(MAE)는 [수학식 12]와 같이 단순 화할 수 있다.We also use the weighted values in the local domain

Know that is very close to each other. That is, the adjusted mean absolute error (MAE) can be simplified as shown in [Equation 12].

[Equation 12]

는

의 평균이다. 그러므로

의 첫번째 기대값은 [수학식 13]과 같이 계산될 수 있다.From the previous analysis, if there is no movement in the local window

Is

Is the average. therefore

The first expected value of can be calculated as shown in Equation 13.

[Equation 13]

의 분산은 [수학식 14]와 같이 계산될 수 있다.

The variance of can be calculated as shown in [Equation 14].

[Equation 14]

로부터 잡음 분산/표준편차를 추정하는 많은 방법이 있다. 세가지의 예가 아래에 나열되어 있다. 다른 실시예들도 또한 가능하다.Obtained

There are many ways to estimate the noise variance / standard deviation from. Three examples are listed below. Other embodiments are also possible.

을 계산하고 [수학식 15]와 같은 잡음 표준 편차를 추정한다. 1. In the first example, the noise variance / standard deviation calculator 104 (FIG. 1) is the first expected value.

And estimate the noise standard deviation as shown in Equation 15.

[Equation 15]

를 계산하고 [수학식 16]과 같은 잡음 분산을 추정한다. 2. In the second example, the noise variance / standard deviation calculator 104 is a variance.

Calculate and estimate the noise variance as shown in Equation 16.

[Equation 16]

의 최대 히스토그램 값에 대응하는 위치

를 [수학식 17]과 같이 계산한다. 3. In the third example, the noise variance / standard deviation calculator 104 first

Corresponding to the maximum histogram value of

Is calculated as shown in Equation 17.

[Equation 17]

는

의 히스토그램이다.

가 시간 국부 차이인

를 계산하기 위한 국부 윈도우의 크기일 때,

의 히스토그램(또는 분포)(300)의 예가 도 3에 도시되어 있다. 히스토그램(300)은, k가

를 계산하기 위한 블록(블록 크기)에서의 픽셀들의 숫자이고, 각각의 k= 1, 2, 4, 8일 경우에 대응하는, 301에서 304까지의 4개의 예시적인 그래프들을 도시한다. At this time

Is

Histogram of.

Local time difference

Is the size of the local window for

An example of a histogram (or distribution) 300 of is shown in FIG. In the histogram 300, k is

Four exemplary graphs from 301 to 304 are shown, which are the number of pixels in the block (block size) for computing P and corresponding to the respective k = 1, 2, 4, 8 cases.

가 충분히 클 때, 위치

는

에 매우 가깝다는 것을 주의해야 한다. 그러한 점에서, 이와 같은 경우에는 잡음 표준 편차는 [수학식18]과 같이 추정된다.Block size

When is big enough, position

Is

It should be noted that it is very close to. In that case, the noise standard deviation is estimated as shown in Equation 18 in this case.

Equation 18

와 과거 프레임

사이에 움직임이 없다고 한다면 상기 방법에 의하여 매우 내성 있는 결과를 얻을 수 있다. 만약 움직임이 존재한다면,

의 분포는 예상할 수 없고 신뢰할 수 없는 추정으로 이끌게 된다. 움직임 효과를 제거하기 위하여 시간 국부 통계 계산기(temporal local statistics calculator)는 움직임 보상 방법으로 확장되어 진다. 예를 들어 움직임이 보상 되도록 조정된 평균 절대 오차(MAE)

가 [수학식 19]와 같이 얻어질 수 있다. If the current frame

And past frame

If there is no movement between them, a very tolerable result can be obtained by the above method. If there is movement,

The distribution of can lead to unpredictable and unreliable estimates. To remove the motion effect, the temporal local statistics calculator is extended to the motion compensation method. For example, the mean absolute error (MAE) adjusted to compensate for motion

Can be obtained as shown in Equation 19.

[Equation 19]

는 움직임 추정에 의하여 얻어지는, 처리된 블록의 움직임 벡터이다. 같은 이유로 이 관계는 [수학식 20]과 같이 단순화될 수 있다. At this time

Is the motion vector of the processed block, obtained by motion estimation. For the same reason, this relationship can be simplified as shown in Equation 20.

[Equation 20]

의 분포로부터의 잡음 분산/표준 편차를 추정하는데 이용될 수 있다. 상기 방법들에 있어서 움직임 추정이 이용되는데 제한이 없다는 것을 주의하라.In addition, the same methods were obtained

Can be used to estimate the noise variance / standard deviation from the distribution of. Note that there is no limit to the use of motion estimation in the above methods.

Next, another noise estimation method exists. Referring to the example difference device 400 of FIG. 4, the time local difference is implemented using Equation 21 using the adjusted mean square error (MSE).

[Equation 21]

를 계산하고,절대값 수단들(404)은

을 계산하며, 계산 블 록(206)은

(즉

)이 잡음 감쇄기(106)(도1)에 의하여 생성될 때

을 계산하고, 배율기(408)는

을 계산하며, 덧셈기(410)는The example difference apparatus 400 uses the difference means 402, squaring means 404, the calculation block 406, the multiplier 408, the adder 410, the divider 412 and the above mathematical expression. Implement the relationship of equation 21. The differential means 402

, The absolute means means 404

, And the calculation block 206

(In other words

Is generated by the noise attenuator 106 (FIG. 1).

And the multiplier 408

, Adder 410 is

를 계산하고, 분할기(412)는

에 대한 마지막 결과를 출력한다.

, Divider 412

Print the last result for.

가 서로간에 매우 가깝다는 것을 안다. 즉, 조정된 평균 자승 오차(MSE)는 [수학식 22]와 같이 단순화될 수 있다.We also weighted in the local area

Know that are very close to each other. That is, the adjusted mean square error (MSE) can be simplified as shown in Equation 22.

[Equation 22]

는

의 평균 값이다. 그러므로,

의 첫번째 기대값은 [수학식 23]과 같이 계산될 수 있다. From the above analysis, if there is no movement in the local window

Is

Is the average value. therefore,

The first expected value of can be calculated as shown in Equation 23.

[Equation 23]

로부터의 잡음 분산/표준편차를 추정하는 많은 방법들이 있다. 두 개의 예들이 다음에 나열되어 있다. 다른 구현예들도 또한 가능하다.Obtained

There are many ways to estimate the noise variance / standard deviation from. Two examples are listed below. Other implementations are also possible.

를 계산하고 [수학식 24]와 같은 잡음 분산을 추정한다. 1. As a first example, the noise variance / standard deviation calculator 104 (FIG. 1) is the first expected value.

And estimate the noise variance as shown in Equation 24.

[Equation 24]

의 최대 히스토그램 값에 대응하는 위치

를 [수학식 25]와 같이 계산한다.2. As a second example, the noise variance / standard deviation calculator 104 (FIG. 1) first looks as follows.

Corresponding to the maximum histogram value of

Is calculated as shown in Equation 25.

[Equation 25]

는

의 히스토그램이다. 블록의 크기

가 충분히 클 때 위치

는

에 매우 가깝다는 것을 주의해야 한다. 그러한 점에 서 이와 같은 경우에는 잡음 표준 편차는 [수학식 26]과 같이 추정된다.At this time

Is

Histogram of. Block size

When is large enough

Is

It should be noted that it is very close to. In this case, the noise standard deviation is estimated as shown in Equation 26.

[Equation 26]

는 [수학식 27]과 같이 얻어질 수 있다.This method can also be extended to a motion compensation method. For example, the mean squared error (MSE) adjusted to compensate for motion

May be obtained as shown in Equation 27.

[Equation 27]

는 움직임 추정에 의하여 얻어지는, 처리된 블록의 움직임 벡터이다. 같은 이유로 이 관계는 [수학식 28]과 같이 단순화될 수 있다At this time

Is the motion vector of the processed block, obtained by motion estimation. For the same reason, this relationship can be simplified as shown in Equation 28.

[Equation 28]

의 분포로부터의 잡음 분산/표준 편차를 추정하는데 이용될 수 있다. 상기 방법들에 있어서 움직임 추정이 이용되는데 제한이 없다는 것을 주의하라.In addition, the same methods were obtained

Can be used to estimate the noise variance / standard deviation from the distribution of. Note that there is no limit to the use of motion estimation in the above methods.

As will be appreciated by those skilled in the art, the present invention can be used for both progressive ive and interlaced video.

Even and odd fields in interlaced video can be treated as two separate progressive video sequences. Alternatively, the fields may be merged into a single frame prior to processing.

The invention has been described in great detail with reference to the specific version which is optimal. However, other versions are possible. Therefore, the spirit and scope of the appended claims should not be limited to the description of the optimal version contained herein.

The present invention is generally related to video processing and more particularly to noise estimation in video signals. The noise estimator of the present invention is an image structure remover and a noise variance (standard deviation) calculator. There is an effect of estimating noise using.

Claims (28)

In the video noise estimation method in a video frame sequence, Calculating temporal local statistics of the current frame and the past filtered frame; Estimating a distribution of the time local statistics; And Estimating video noise from the estimated distribution. The method of claim 1, Estimating the video noise comprises calculating a noise standard deviation as a function of the estimated distribution. The method of claim 1, And estimating the video noise includes calculating noise variance as a function of the estimated distribution. The method of claim 1, Calculating the time local statistics comprises determining a frame difference from a difference between the current frame and the past filtered frame in the frame sequence. The method of claim 4, wherein And calculating the time local statistics further comprises calculating a mean absolute error of the difference to produce the time local statistics. The method of claim 4, wherein Calculating the time local statistics further comprises calculating a sum of an absolute error of the difference to generate the time local statistics. The method of claim 4, wherein Calculating the time local statistics further comprises calculating a mean square error of the difference to produce the time local statistics. The method of claim 4, wherein Calculating the time local statistics further comprises calculating a sum of squared errors of the difference to produce the time local statistics. The method of claim 4, wherein Calculating the time local statistics further comprises calculating the time local statistics over an H x W window between the current frame and the past filtered frame. The method of claim 1, Computing the time local statistics includes the time of the two matching blocks in a current frame and a past filtered frame when two matching blocks are retrieved by motion estimation for motion. And calculating each of the local statistics. The method of claim 1, Estimating a distribution of temporal local statistics comprises estimating a first order expectation of the temporal local statistics. The method of claim 1, Estimating the distribution of time local statistics comprises estimating a variance of the time local statistics. The method of claim 1, Estimating the distribution of temporal local statistics comprises estimating a maximum histogram position of the temporal local statistics. The method of claim 1, Estimating the distribution of time local statistics comprises estimating noise-level-related parameters of the time local statistics. A video noise estimation system for estimating video noise in a video frame sequence, An image structure remover that calculates time local statistics from a current frame and a past filtered frame; And And a noise estimator for estimating the distribution of time local statistics and estimating video noise from the estimated distribution. The method of claim 15, And the noise estimator estimates video noise by calculating a noise standard deviation as a function of the estimated distribution. The method of claim 15, And the noise estimator estimates video noise by calculating a noise variance as a function of the estimated distribution. The method of claim 15, And the image structure remover calculates the time local statistics by determining a frame difference from the difference between the current frame and the past filtered frame in the sequence of frames. The method of claim 18, And the image structure remover calculates the time local statistics by calculating an average absolute error of the difference to produce the time local statistics. The method of claim 18, And the image structure remover calculates the time local statistics by calculating a sum of an absolute error of the difference to produce the time local statistics. The method of claim 18, And the image structure remover calculates the time local statistics by calculating an average squared error of the difference to produce the time local statistics. The method of claim 18, And the image structure remover calculates a sum of squared errors of the differences to produce the time local statistics. The method of claim 18, And the image structure eliminator calculates the time local statistics over an H x W window between the current frame and the past filtered frame. The method of claim 15, The image structure eliminator calculates the time local statistics of the two matching blocks, respectively, in the current frame and the past filtered frame when the two matching blocks are retrieved by motion estimation for motion. Video temporal statistics by calculating the time local statistics. The method of claim 15, And the noise estimator estimates a distribution of the time local statistics by estimating a first expected value of the time local statistics. The method of claim 15, And the noise estimator estimates a distribution of the time local statistics by estimating a variance of the time local statistics. The method of claim 15, And the noise estimator estimates a distribution of the time local statistics by estimating a maximum histogram position of the time local statistics. The method of claim 15, And the noise estimator estimates a distribution of time local statistics by estimating noise level related parameters of the time local statistics.

Images