CN102129670B - Method for detecting and repairing movie scratch damage - Google Patents

Abstract

The invention relates to a method for detecting and repairing movie scratch damage and belongs to the technical field of image processing. The method comprises two processes of scratch detection and scratch repair, wherein in the process of scratch detection, two modules, i.e., a scratch position location module and a scratch detail obtainment module are provided. The method comprises the following steps of: locating positions of scratches in a horizontal direction by utilizing airspace luminance feature models of vertical scratches and obtaining the detail information of the scratches in a vertical direction by utilizing morphology features of the scratches; and the repair of the scratches comprises the steps of processing an image in a time-space domain and repairing image deletion parts through combining the information of front and back frames, wherein both the self-similarity of the image in a frame and the relativity between the image frames are utilized. In the invention, the system repair process is completely automated; the repair effect on the scratches of an old movie is improved; and the manpower cost and the time cost are reduced; therefore, the method can be widely applied to the scratch repair of all kinds of video sequences.

Description

The detection restorative procedure of movie scratch damage

Technical field

What the present invention relates to is a kind of method of technical field of image processing, specifically a kind of detection restorative procedure of movie scratch damage.

Background technology

Cinefilm is a kind of easily aging chemical substance, and because inappropriate preservation, broadcast and copy, so that many old films or damage or lose.Cut is the damage that often occurs in the cinefilm, forms reason and mainly contains two: the one, and video particle in shooting process has produced scuffing to film, and the 2nd, video film and video camera friction in playing process have produced damage.Therefore, cut usually is wire and is distributed in the cinefilm, is generally 3～10 pixels wide, different in size, or is white or for black, institute's overlay area image information is by havoc.Research is the important process in this field for the detection restorative procedure of scratch damage in the old film.

Scratch detection is an important step before the scratch removal, and the degree of accuracy of detection method directly has influence on the effect of scratch removal.Classical scratch detection method has the detection method based on the cut Intensity model, such as the Bruni model, can comparatively accurately detect the position of vertical cut on the image level direction, but owing to not considering the morphological character of cut, therefore can't obtain vertical cut length and the detailed information such as particular location in vertical direction.Other methods have utilized the local edge of cut to detect, although can obtain the detailed information of cut, but owing to acting on entire image and lacking quantitative analysis, subjectivity is larger, many interfering objects that have equally local edge be can detect inevitably, flase drop and undetected phenomenon produced.

Utilize the resulting cut mask of scratch detection can carry out scratch removal.Most of scratch removal methods all adopt the method based on the interpolation of certain model, and polynomial interpolation hypothesis image disappearance part satisfies multinomial model with its surrounding pixel, selects the polynomial expression exponent number to carry out match, thereby obtains suitable model coefficient.Polynomial interpolation is calculated simple, and effectively the Recovery image low-frequency information still is difficult to the Recovery image high-frequency information, can't keep the image texture details.Random (MRF) model of probabilistic model such as autoregression (AR) model and Markov is match disappearance partial pixel better, especially recover high-frequency information, but calculated amount is large, realizes complicated.2002, BaHester and Bertalmio have proposed the concept of " repairing ", with the method for recurrence image disappearance zone are repaired to inside from the periphery, until fill up whole zone, this method is applicable to contain in the image situation of less disappearance information, is not suitable for scratch removal.

Summary of the invention

The present invention is directed to the prior art above shortcomings, a kind of detection restorative procedure of movie scratch damage is provided, can effectively repair scratch damage, with respect to traditional detection and restorative procedure better effect and lower method complexity be arranged, more wide application prospect is arranged.

The present invention is achieved by the following technical solutions, the present invention includes the detection of cut and two parts of reparation of cut.

Wherein, the detection of cut comprises that locate the cut position and the cut details is obtained two modules, utilize first the brightness model orientation cut position in the horizontal direction, spatial domain of vertical cut, the morphological feature of recycling cut obtains these cuts detailed information in vertical direction, location, cut position and cut details are obtained between these two modules and are complemented each other, thereby the former has removed a large amount of interference noises that the latter detects for the reach that the pre-determined bit of cut horizontal level has been dwindled the latter, thereby and the latter has carried out refinement for the former has extracted the length of vertical cut and the positional information on the vertical direction;

The concrete steps of scratch detection are as follows:

1) image A is carried out vertical projection, the average of each row of computed image matrix obtains an one dimension horizontal vector, does the mean filter on the horizontal direction, obtains the brightness cross section cross (j) of image, and this step has been amplified the characteristic of cut cosine decay;

2) choose all extreme points of cross (j), according to the cut characteristic, the extreme point that obtains is carried out wide constraint, brightness constraint and weber energy constraint, thereby filter out the extreme point that meets the cut characteristic;

3) according to the cut horizontal level of obtaining and the width of cut, obtain the elementary mask of cut;

4) original image A is done histogram equalization, the contrast of cut and background image is carried out rim detection with the canny operator to image in the reinforcement image, and do morphological dilation, increase the cut characteristic along horizontal axis, extract the edge skeleton of image, be denoted as D;

5) use respectively construction operator B _L, B _R, B _HBe denoted as after D opened operation

The three is carried out getting after the OR operation D and M are subtracted each other, obtain to comprise noise in interior cut detail view;

6) with step 3) in elementary mask and the step 5 of the cut that obtains) in the effective combination of cut detail view that obtains, and do aftertreatment, obtain the final mask of cut.

Described cut cosine characteristic model is formulated as:

$L_p(i,j)=\frac{\mathrm{sgn}(i-r_s\left(j\right))-\mathrm{sgn}(i-r_{\mathrm{\θ}}\left(j\right))}{2}\×b_p{k_p}^{\frac{\mathrm{sgn}(j-(p_i-{\mathrm{\ω}}_p))-\mathrm{sgn}(j-(p_i+{\mathrm{\ω}}_p))}{2}|j-c_p|}\cos \frac{\mathrm{\π}|j-p_i|}{{\mathrm{\ω}}_p}$

I is image x axle in the formula, and j is image y axle.r _s(j) be the reference position of image j row cut, r _e(j) be the terminal location of image j row cut, b _pBe cut minimum brightness, p _lRepresent the extreme point position of cosine curve, k _pBe the attenuation coefficient of cut, ω _pFor the cut width half, sgn is step function.

Step 6) described in the elementary mask of cut and the effective combination of cut detail view, namely two figure are carried out and operation, utilize the noisy cut figure of elementary mask restraint strap of cut, remove cut position interference noise on every side, utilize simultaneously with the cut detail view of noise and extract cut detailed information in the elementary mask, obtain Mask _Previous, then carry out aftertreatment, obtain the final mask of cut.

The method of scratch detection part of the present invention is compared with existing method, advantage is: degree of accuracy is high, location, cut position and cut details are obtained between these two modules and are complemented each other, thereby the former has removed a large amount of interference noises that the latter detects for the reach that the pre-determined bit of cut horizontal level has been dwindled the latter, thereby and the latter has carried out refinement for the former has extracted the length of vertical cut and the positional information on the vertical direction; Computation complexity is low, and the method complexity of location, cut position only has O (N) (N is picturewide to be detected), much smaller than the method for carrying out scratch detection by method and the wavelet field of neural network prescreen cut.

Scratch removal part of the present invention expands to existing method for shielding error code-block-based bilateral filtering (BBF) method in the time domain, has proposed the 3D-BBF scratch removal method based on the time-space domain, and concrete steps are:

At first judge the cut type, if cut repeats near the frame same position of front and back, the cut type is I class cut, select the BBF method of two dimension, otherwise if cut only exists at present frame, the cut type is II class cut, image sequence is carried out shot boundary cut apart, adopt again time-space domain 3D-BBF method.

Described time-space domain 3D-BBF method is divided into many sub-blocks with image, be (x if contain the sub-block to be repaired of cut information in the center position of present frame, y), size is m * n, and then the region of search also is (x in the center position of front and back frame, y), size is p * q, (p＞m, q＞n), the size of p and q is selected as required, is generally 2m * 2n.The zone means greatly the increase of method complexity, but the tolerance for front and back vertical shift degree improves simultaneously, method is assessed from the geographic distance between sub-block to be repaired and the region of search repairing sub-block and difference two aspects of pixel value, and then calculate suitable repairing sub-block weights, after process 3D-BBF method was carried out scratch removal, the pixel value of scored area was that all sizes are the weighted mean sum of the sub-block of m * n in present frame and the frame search zone, front and back.

The method of scratch removal part of the present invention, compare with existing method, advantage is: repairing effect is good, 3D-BBF processes image in the time-space domain, has both utilized the self-similarity of image in a frame, has utilized again the correlativity between picture frame, it is when repairing the cut of continuous a few frame existence, effect is slightly better than BBF, but when repairing only at the cut that present frame exists, effect significantly improves; Computation complexity is low, and great majority need to be used Motion estimation and compensation based on the restorative procedure of time domain, greatly increased the method complexity, and the 3D-BBF method do not need cut is carried out motion compensation.

Compared with prior art, the beneficial effect of scratch detection repair system of the present invention is: the repair process full automation, improved the repairing effect of old movie scratch, reduced human cost and time cost, can be widely used in the scratch removal of various video sequences.

Description of drawings

Fig. 1 is schematic diagram of the present invention.

Fig. 2 is the scratch detection schematic flow sheet.

Fig. 3 is the scratch removal schematic flow sheet.

Fig. 4 is scratch removal Method And Principle schematic diagram.

Fig. 5 is classical cut test pattern.

Fig. 6 is the image after Fig. 5 repairs.

Fig. 7 is the film image frame that contains trickle cut.

Fig. 8 is the image after Fig. 7 repairs.

Fig. 9 is the film image frame that contains obvious cut.

Figure 10 is the image after Fig. 9 repairs.

Embodiment

The below elaborates to embodiments of the invention, and the present embodiment is implemented under take technical solution of the present invention as prerequisite, provided detailed embodiment and concrete operating process, but protection scope of the present invention is not limited to following embodiment.

Embodiment

As shown in Figure 1, the input original sequence obtains the cut mask through scratch detection, passes through scratch removal, the image sequence after obtaining repairing again.

As shown in Figure 2, scratch detection may further comprise the steps:

1) image A (as shown in Figure 5) is carried out vertical projection, it is the average of each row of computed image matrix, obtain an one dimension horizontal vector sig (j), sig (j) is done mean filter on the horizontal direction, obtain the brightness cross section cross (j) (as shown in Figure 6) of image, this step has been amplified the characteristic of cut cosine decay, and formula is as follows:

$\mathrm{sig}\left(j\right)=\frac{1}{M}\underset{i=1}{\overset{M}{\mathrm{\Σ}}}A(i,j)$

$\mathrm{cross}\left(j\right)=\left\{\begin{array}{cc}\mathrm{sig}\left[j\right]-\frac{1}{S+1}{\mathrm{\&Sigma;}}_j^{j+S}\mathrm{sig}\left[j\right]& (j\&le;S)\\ \mathrm{sig}\left[j\right]-\frac{1}{21}{\mathrm{\&Sigma;}}_{j-S}^{j+S}\mathrm{sig}\left(j\right)& (S<j<N-S)\\ \mathrm{sig}\left[j\right]-\frac{1}{S+1}{\mathrm{\&Sigma;}}_{j-S}^j\mathrm{sig}\left[j\right]& (j\&GreaterEqual;N-S)\end{array}\right.$

2) choose all extreme points of cross (j), according to the cut characteristic, the extreme point that obtains is carried out wide constraint, brightness constraint and weber energy constraint, thereby filter out the extreme point that meets the cut characteristic;

3) according to the cut horizontal level of obtaining and the width of cut, obtain the elementary mask (shown in Fig. 5-3) of cut;

4) original image A is done histogram equalization, the contrast of cut and background image in the reinforcement image, with the canny operator image is carried out rim detection, and do morphological dilation, increase the cut characteristic along horizontal axis, extract the edge skeleton of image, be denoted as D (shown in Fig. 5-4);

5) use respectively construction operator B _L, B _R, B _HBe denoted as after D opened operation The three is carried out getting after the OR operation

D and M are subtracted each other, obtain to comprise noise at interior cut detail view (shown in Fig. 5-5);

6) with step 3) in elementary mask and the step 5 of the cut that obtains) in the effective combination of cut detail view that obtains, and do aftertreatment, obtain the final mask (shown in Fig. 5-6) of cut.

Step 2) in extreme point has been carried out wide constraint, brightness constraint and weber energy constraint, concrete constrained procedure is as follows:

If the horizontal ordinate position that certain minimum extreme point is corresponding is j _h, j then _hAbout the horizontal ordinate position of two maximum extreme points be respectively j _H-1And j _K+1, cut width width=|j then _H+1-j _H-1|.Wide according to 3～13 pixels of previously described cut width characteristic, namely 3≤width≤10 filter out the minimum extreme point that meets the cut width characteristic, and the horizontal ordinate position is designated as j _IndcThen choose j _IndcAbout all maximum extreme points, the horizontal ordinate position is designated as j _Indc1

The mean flow rate en of each extreme point of the sequence of calculation:

Calculate the mean flow rate enmed of all minimum extreme points

$\mathrm{enmed}=\frac{1}{\mathrm{lindc}}\underset{\mathrm{indc}=1}{\overset{\mathrm{lindc}}{\mathrm{\&Sigma;}}}\mathrm{en}\left(j_{\mathrm{indc}}\right)$

Wherein lindc is the length of array indc.

Calculate the mean flow rate enmedlat of all maximum extreme points

$\mathrm{enmedlat}=\frac{1}{\mathrm{lindc}1}\underset{\mathrm{indc}=1}{\overset{\mathrm{lindc}1}{\mathrm{\&Sigma;}}}\mathrm{en}\left(j_{\mathrm{indc}1}\right)$

Wherein lindc1 is the length of array indc1.

From j _Indc1In 1, filter out the extreme point j that meets the brightness constraint _Jind, so that cross (j _Jind)＞enmed.

Filter out qualified extreme point according to Weber's law at last: if the brightness of an object is f ₀, background luminance is f _sIf, Object as seen.

Step 6) elementary mask and the effective combination of cut detail view with cut described in, namely two figure are carried out and operation, utilize the noisy cut figure of elementary mask restraint strap of cut, remove cut position interference noise on every side, utilize simultaneously with the cut detail view of noise and extract cut detailed information in the elementary mask, obtain Mask _Previous, then carrying out aftertreatment, concrete methods of realizing is:

At first carry out the expansive working on vertical direction and the horizontal direction, the large young pathbreaker figure level according to figure is divided into the N section again, and for scored area, vertical initial value of establishing each section is top, and final value is bottom.Record pixel sum on each section with array T, and setting threshold is threshold

$T_N=\underset{i=\mathrm{top}}{\overset{\mathrm{bottom}}{\mathrm{\&Sigma;}}}{\mathrm{Mask}}_{\mathrm{previous}}(i,j)$

${\mathrm{Mask}}_N(i,j)=\left\{\begin{array}{c}1(T_N\left(j\right)>\mathrm{threshold})\\ \begin{array}{cc}0& \mathrm{else}\end{array}\end{array}\right.$

As shown in Figure 3, scratch removal may further comprise the steps:

At first judge the cut type, if cut repeats near the frame same position of front and back, the cut type is I class cut, select the block-based bilateral filtering method (BBF method) of conventional two-dimensional, otherwise if cut only exists at present frame, the cut type is II class cut, image sequence is carried out shot boundary cut apart, adopt again time-space domain 3D-BBF method.

In the scratch detection stage, all obtained corresponding cut mask for each frame.If the horizontal reference position of each section cut is start on the i frame cut mask _i, horizontal final position is end _i, vertical reference position is top _i, vertical final position is r.If multi-frame to be repaired is the l frame, the repairing frame is the m frame, and the cinematic data statistics according to a large amount of draws following cut type judgment condition:

1) if [start on the m frame horizontal direction _l-σ, end _i+ σ] scope in no marking, cut is II class cut;

2) if 1) do not satisfy i.e. [start _l-σ, end _l+ σ] cut is arranged in the scope, but top _m＜bottom _lOr bottom _m＞top _l, cut is II class cut;

3) otherwise, cut is I class cut

The region of search scope of 3D-BBF method is frames before and after the image, and searching is similar in a certain size zone of present frame and front and back frame.Sub-block to be repaired is (x, y) in the center position of present frame, and size is m * n, and then the region of search also is (x in the center position of front and back frame, y), size is p * q, (p＞m, q＞n), the size of p and q is selected as required, is generally 2m * 2n.The zone means greatly the increase of method complexity, but the while is for the tolerance raising of front and back vertical shift degree.Method is assessed from the geographic distance between sub-block to be repaired and the region of search repairing sub-block and difference two aspects of pixel value, and then calculates suitable repairing sub-block weights.

Fig. 4 has showed three continuous two field pictures, and the black square of intermediate frame is sub-block to be repaired, establishes size and is m * n, and the grey square frame is region of search, and size is p * q.After process 3D-BBF method was carried out scratch removal, the pixel value of black sub-block was that all sizes are the weighted mean sum of the sub-block of m * n in present frame and the front and back frame grey square frame.3D-BBF can be formulated as follows:

${B_{k_1}}^{\&prime;}=\frac{1}{W_i}\underset{t\&Element;N_{I0}\&cup;N_{I1}\&cup;N_{I2}}{\mathrm{\&Sigma;}}\{c\left(\right||k_0-l_0|\left|\right)s\left(\right||B_{k_0}-B_{t_0}|\left|\right)B_{t_0}+c\left(\right||k_1-l_1|\left|\right)s\left(\right||B_{k_1}-B_{t_1}|\left|\right)B_{t_1}+c\left(\right||k_2-l_2|\left|\right)s\left(\right||B_{k_2}-B_{t_2}|\left|\right)B_{t_2}\}$

$\mathrm{Wi}=\underset{l\&Element;N_{I0}\&cup;N_{I1}\&cup;N_{I2}}{\mathrm{\&Sigma;}}\{c\left(\right||k_0-l_0|\left|\right)s\left(\right||B_{k_0}-B_{l_0}|\left|\right)B_{l_0}+c\left(\right||k_1-l_1|\left|\right)s\left(\right||B_{k_1}-B_{l_1}|\left|\right)B_{l_1}+c\left(\right||k_2-l_2|\left|\right)s\left(\right||B_{k_2}-B_{l_2}|\left|\right)\}$

The sub-block (black sub-block among Fig. 4) that is repaired for present frame, Be respectively the sub-block (the grey sub-block in Fig. 4 grey square frame) that is used for reparation in continuous three frames, c (|| k _0,1,2-l _0,1,2||) be || k _0,1,2-l _0,1,2|| the variance of (sub-block distance, namely the interior each point of sub-block is apart from the absolute value sum) is σ _dGaussian function,

For

The variance of (pixel value poor, i.e. the absolute value sum of the difference of each point pixel value in the sub-block) is σ _rGaussian function.

Be the neighborhood of block of pixels l at present frame and front and back frame.

The present invention can be applicable to the various video sequence.

Fig. 7 is the cut test pattern of the test scratch removal method of a width of cloth classics, and this figure has together top-down cut, and overlay image sky and armor part are damaged very obvious.Fig. 8 is the image after repairing, and picture is smooth, and detail textures information also keeps finely.

Fig. 9 is an image in the old film, has some tiny cuts.Figure 10 is the image after repairing, and has repaired the cut in the image.

Fig. 8-the 1st, the image in the old film of another, more existing tiny cuts also have an obvious cut.Fig. 8-the 2nd, the image after repairing has been repaired the various cuts among the figure effectively.

Significantly cut easily detects difficult the reparation, and trickle cut is easily repaired difficult the detection, and the result shows, the present invention all has preferably repairing effect to these two kinds of cuts.

Claims (6)

1. the detection restorative procedure of a movie scratch damage comprises the detection of cut and two parts of reparation of cut, it is characterized in that:

A. the detection of cut: the testing process of cut is divided into location, cut position and obtains two steps with the cut details: the brightness model orientation cut position in the horizontal direction, spatial domain that utilizes first vertical cut, the morphological feature of recycling cut obtains these cuts detailed information in vertical direction, location, cut position and cut details are obtained between these two steps and are complemented each other, thereby the former has removed a large amount of interference noises that the latter detects for the reach that the pre-determined bit of cut horizontal level has been dwindled the latter, thereby and the latter has carried out refinement for the former has extracted the length of vertical cut and the positional information on the vertical direction;

B. the reparation of cut: method for shielding error code-block-based bilateral filtering method is expanded in the time domain, and be applied in the scratch removal, 3D-BBF scratch removal method based on the time-space domain has been proposed, the 3D-BBF method is processed image in the time-space domain, both utilized the self-similarity of image in a frame, utilized again the correlativity between picture frame, because some cuts continued presence in several frames of film sequence, other cuts only exist at present frame, the 3D-BBF method is when repairing the former, effect is slightly better than BBF, but when repairing the latter, effect significantly improves.

2. the detection restorative procedure of movie scratch damage according to claim 1 is characterized in that, for 1) described in the scratch detection method, be divided into following step:

1) image A is carried out vertical projection, the average of each row of computed image matrix obtains an one dimension horizontal vector, does the mean filter on the horizontal direction, obtains the brightness cross section cross (j) of image, and this step has been amplified the characteristic of cut cosine decay;

2) choose all extreme points of cross (j), according to the cut characteristic, the extreme point that obtains is carried out wide constraint, brightness constraint and weber energy constraint, thereby filter out the extreme point that meets the cut characteristic;

3) according to the cut horizontal level of obtaining and the width of cut, obtain the elementary mask of cut;

4) original image A is done histogram equalization, the contrast of cut and background image is carried out rim detection with the canny operator to image in the reinforcement image, and do morphological dilation, increase the cut characteristic along horizontal axis, extract the edge skeleton of image, be denoted as D;

5) use respectively construction operator B _L, B _R, B _HBe denoted as after D opened operation

The three is carried out getting after the OR operation

D and M are subtracted each other, obtain to comprise noise in interior cut detail view;

6) with step 3) in elementary mask and the step 5 of the cut that obtains) in the effective combination of cut detail view that obtains, and do aftertreatment, obtain the final mask of cut.

3. the detection restorative procedure of movie scratch damage according to claim 2 is characterized in that step 1) described in cut cosine, its model represents with following formula:

I is image x axle in the formula, and j is image y axle, r _s(j) be the reference position of image j row cut, r _θ(j) be the terminal location of image j row cut, b _pBe cut minimum brightness, p _lRepresent the extreme point position of cosine curve, k _pBe the attenuation coefficient of cut, ω _pFor the cut width half, sgn is step function.

4. the detection restorative procedure of movie scratch damage according to claim 2, it is characterized in that, step 6) elementary mask and the effective combination of cut detail view with cut described in, namely two figure are carried out and operation, utilize the noisy cut figure of elementary mask restraint strap of cut, remove the interference noise around the cut position, utilize simultaneously with the cut detail view of noise and extract cut detailed information in the elementary mask, obtain Mask _Previous, then carrying out aftertreatment, concrete methods of realizing is:

At first carry out the expansive working on vertical direction and the horizontal direction, the large young pathbreaker figure level according to figure is divided into the N section again, and for scored area, vertical initial value of establishing each section is top, and final value is bottom, uses array T _NRecord the pixel sum on each section, and setting threshold is threshold

5. the detection restorative procedure of movie scratch damage according to claim 1, it is characterized in that, 2) the 3D-BBF scratch removal method described in, specifically comprise: traditional BBF method is a kind of method for shielding error code, utilized the Self-similar Feature of image, has preferably effect, consider that cut is the part disappearance information in the image, and some cut is continued presence in several frames, some cut only exists at present frame, therefore, the BBF method is expanded in the time domain, the 3D-BBF method has been proposed, made up the complete scratch removal system of a cover: at first judge the cut type, if cut repeats near the frame same position of front and back, the cut type is I class cut, select the BBF method of two dimension, otherwise if cut only exists at present frame, the cut type is II class cut, image sequence is carried out shot boundary cut apart, adopt again time-space domain 3D-BBF method.

6. the detection restorative procedure of movie scratch damage according to claim 5 is characterized in that, the concrete steps that 3D-BBF realizes:

Image is divided into many sub-blocks, be (x if contain the sub-block to be repaired of cut information in the center position of present frame, y), size is m * n, then the region of search also is (x in the center position of front and back frame, y), size is p * q, wherein, p＞m, q＞n, the size of p and q is selected as required, the zone means greatly the increase of method complexity, but simultaneously for the tolerance raising of front and back vertical shift degree, method is assessed from the geographic distance between sub-block to be repaired and the region of search repairing sub-block and difference two aspects of pixel value, and then calculate suitable repairing sub-block weights, after process 3D-BBF method was carried out scratch removal, the pixel value of scored area was that all sizes are the weighted mean sum of the sub-block of m * n in present frame and the frame search zone, front and back, and formula is expressed as follows:

The sub-block that is repaired for present frame,

Be respectively the sub-block that is used for reparation in continuous three frames, c (‖ k _0,1,2-l _0,1,2‖) be ‖ k _0,1,2-l _0,1,2The variance of ‖ is σ _dGaussian function, ‖ k _0,1,2-l _0,1,2‖ is the sub-block distance, and namely the interior each point of sub-block is apart from the absolute value sum;

For

Variance be σ _rGaussian function,

Poor for pixel value, i.e. the absolute value sum of the difference of each point pixel value in the sub-block; N _I0, N _I1, N _I2Be the neighborhood of block of pixels l at present frame and front and back frame.

Images