CN106303156A - Method, application and mobile terminal to video denoising - Google Patents

Abstract

The invention discloses the method to video denoising, application and mobile terminal.Wherein the application to video denoising includes that image acquisition unit, noise grade calculate unit, smoothing factor computing unit, metrics calculation unit, weight calculation unit and luma processing unit.Image acquisition unit is suitable to obtain the picture frame sequence about video.Noise grade calculates unit and is suitable to calculate each noise grade treating denoising pixel.Smoothing factor computing unit is suitable to according to each noise grade treating denoising pixel and brightness thereof, determines that corresponding this treats the brightness smoothing factor of denoising pixel.Metrics calculation unit is suitable to calculate this based on block matching algorithm and treats denoising pixel and selected the distance of each associated pixel point.Weight calculation unit is suitable to calculate the weight that each associated pixel point is corresponding.Luma processing unit, treats denoising pixel for each, calculates all luminance weighted meansigma methodss being associated with pixel and treats the brightness through denoising of denoising pixel as this.

Description

Method, application and mobile terminal to video denoising

Technical field

The present invention relates to technical field of image processing, particularly relate to the method to video denoising, application and mobile terminal.

Background technology

In the application scenarios of picture or video acquisition, acquired image unavoidably has signal noise.Such as, exist Picture captured in the scenes such as night or video generally have obvious noise.Image denoising is image procossing (single width figure Sheet process or the process of sequence of frames of video) in highly important link.

At present, the processing mode of main flow image denoising includes multiple known guarantor limit filtering algorithm, such as bilateral filtering side Formula etc..When in video, picture frame sequence carries out denoising, traditional video denoising mode is to pixel in picture frame Carry out Block-matching operation with its pixel adjacent on room and time, and then complete filtering operation.

Existing video denoising process typically requires cost more time.For this situation, configure more much higher media Equipment is through carrying out denoising acceleration frequently with hardware modes such as GPU.But, at multiple portable electric appts (such as mobile terminal Deng) in use existing video denoising mode time, image denoising process need take considerable time.

To this end, the present invention proposes a kind of new technical scheme to video denoising.

Summary of the invention

To this end, the present invention provides a kind of new technical scheme to video denoising, effectively solve above at least one Problem.

According to an aspect of the present invention, it is provided that a kind of method to video denoising, be suitable to perform in the terminal.Should Method comprises the steps.Obtain the picture frame sequence about video.Calculate each denoising pixel treated in each picture frame Noise grade.According to each noise grade treating denoising pixel and brightness thereof, determine that corresponding this treats the bright of denoising pixel Degree smoothing factor.From each this frame treated residing for denoising pixel and the most adjacent multiple image, select and treat denoising picture with this Vegetarian refreshments association pixel, and based on block matching algorithm calculate this treat denoising pixel with selected each associated pixel point away from From.It is associated with, with each, the distance that pixel is corresponding according to each brightness smoothing factor treating denoising pixel, calculates each The weight that associated pixel point is corresponding.Denoising pixel is treated, according to each brightness being associated with pixel and power for each Weight, calculates all luminance weighted meansigma methodss being associated with pixel and treats the brightness through denoising of denoising pixel as this.

Alternatively, according in the method to video denoising of the present invention, the step of the picture frame sequence about video is obtained Suddenly include: obtain the original frame sequence of rgb pixel form sequentially in time；Original frame sequence is converted into brightness and colourity is divided From sequence of data frames and as picture frame sequence.

Alternatively, according in the method to video denoising of the present invention, calculate and each in each picture frame treat denoising picture The step of the noise grade of vegetarian refreshments includes: treat centered by denoising pixel by this, selects the image block conduct of predetermined window size This treats denoising pixel neighborhood of a point block；Calculate the variance of brightness in this field block, to determine that this treats the noise etc. of denoising pixel Level.

Alternatively, according in the method to video denoising of the present invention, from each this frame treated residing for denoising pixel In the most adjacent multiple image, select and treat the pixel that denoising pixel associates with this, and should based on block matching algorithm calculating Treat that denoising pixel includes with the step of the distance of selected each associated pixel point: add up this and treat the figure before denoising pixel As in the pixel associated by pixel of correspondence position in frame and the highest at least one of the pixel similarity of correspondence position Point；Pixel according to correspondence position and the position relationship of at least some of pixel added up, select and treat denoising picture with this Vegetarian refreshments keeps the pixel of this position relationship as treating the pixel that denoising pixel is associated with this；Calculate this and treat denoising picture The distance of vegetarian refreshments and each pixel being associated.

Alternatively, according in the method to video denoising of the present invention, calculate this and treat that denoising pixel is relevant to each The step of distance of the pixel of connection includes: obtain this neighborhood block to be matched treating denoising pixel about brightness feature to Amount；Obtain each pixel neighborhood of a point block being associated characteristic vector about brightness；Calculate neighborhood block to be matched with each The spacing of the characteristic vector of the pixel neighborhood of a point block being associated is as distance corresponding to this pixel being associated.

Alternatively, according in the method to video denoising of the present invention, put down according to each brightness treating denoising pixel Sliding coefficient is associated with, with each, the distance that pixel is corresponding, and the step calculating weight corresponding to each associated pixel point includes Following manner:

$w(x,y)=\frac{1}{W\left(x\right)}\exp \left(\frac{-1}{h^2}\right|\left|u\right(N^d\left(x\right))-u(N^d\left(y\right)\left)\right|{|}^2)$

Wherein, x represents the pixel treating denoising, and y represents the pixel that is associated with x,

||u(N^d(x))-u(N^d(y)) | | represent x neighborhood block character pair vector and y character pair vector between away from From, d is characterized the dimension of vector, and h represents the brightness smoothing factor of x, and W (x) represents that all of pixel being associated with x is corresponding 'sSum, (x y) is the weight of y pixel to w.

Alternatively, according in the method to video denoising of the present invention, treat denoising pixel for each, according to its Associate brightness and the weight of each pixel, calculate and be associated with the luminance weighted meansigma methods of pixel and treat denoising picture as this The step through the brightness of denoising of vegetarian refreshments includes following manner:

$f\left(x\right)=\underset{y}{\Σ}w(x,y)u\left(y\right)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x, and (x y) is the power of y pixel to w Weight, u (y) represents the brightness value of y.

Alternatively, also include according to the method to video denoising of the present invention: smooth described picture frame according to following formula The colourity of each pixel in sequence:

$f_t\left(x\right)=\underset{\mathrm{\Δ}=-n}{\overset{n}{\Σ}}\left(\frac{1}{W\left(x\right)}\exp \right(\left|\right|u_t\left(x\right)-u_{t+\mathrm{\Δ}}\left(x\right)|{|}^2\left)u_{t+\mathrm{\Δ}}\right(x\left)\right)$

Wherein, f_tX () represents the chromatic value after pixel x is smooth on t two field picture, exp (| | u_t(x)-u_t+Δ(x)||² Represent x with on adjacent Δ frame between the pixel of additional space position about neighborhood block colour difference away from, W (x) be with from-n to n All consecutive frames on the colour difference of pixel of additional space position away from sum, u_t+ΔX () represents corresponding on Δ frame adjacent to x The chromatic value of the pixel of locus,Represent the pixel of this additional space position The weight of chromatic value.

According to another aspect of the present invention, it is provided that a kind of application to video denoising, be suitable to resident in the terminal, should Application includes that image acquisition unit, noise grade calculate unit, smoothing factor computing unit, metrics calculation unit, weight calculation Unit and luma processing unit.Image acquisition unit is suitable to obtain the picture frame sequence about video.Noise grade calculates unit Be suitable to calculate each noise grade treating denoising pixel in each picture frame.Smoothing factor computing unit is suitable to treat according to each The noise grade of denoising pixel and brightness thereof, determine that corresponding this treats the brightness smoothing factor of denoising pixel.Distance calculates Unit is suitable to, from each this frame treated residing for denoising pixel and the most adjacent multiple image, select and treat denoising pixel with this Association pixel, and based on block matching algorithm calculate this treat denoising pixel with selected the distance of each associated pixel point. Weight calculation unit is suitable to according to each brightness smoothing factor treating denoising pixel with each to be associated with pixel corresponding Distance, calculates the weight that each associated pixel point is corresponding.Luma processing unit, treats denoising pixel for each, is suitable to root According to each brightness being associated with pixel and weight, calculate all luminance weighted meansigma methodss being associated with pixel conduct This treats the brightness through denoising of denoising pixel.

According to another aspect of the present invention, it is provided that a kind of mobile terminal, including video denoising is answered according to the present invention With.

To sum up, according to the noise-removed technology scheme of the present invention, the picture high with treating denoising pixel similarity can accurately be selected Vegetarian refreshments (in other words, exclude the pixel that a large amount of similarity is low) is as associated pixel point, and then is calculated really by Block-matching The weight of fixed each associated pixel point.It addition, the noise grade that the noise-removed technology scheme of the present invention treats denoising pixel is entered Row assessment, and then determine the brightness smoothing factor of corresponding noise grade.In conjunction with this noise grade and each each pixel that is associated Point weight and brightness, the noise-removed technology scheme of the present invention can by with treat the luminance weighted of denoising pixel associated pixel point Meansigma methods is as the brightness through denoising.It should be noted that the noise-removed technology scheme of the present invention can be to associated pixel point Accurately select (utilizing similar block transmission operation) and exclude the lowest pixel of similarity and (can ignore in denoising operates Disregard), denoising speed can be greatly improved.It addition, the noise-removed technology scheme of the present invention is while quickly reducing noise, Still video details can be effectively maintained.Particularly, when being applied to the portable multimedia apparatus such as such as mobile terminal, this Bright noise-removed technology scheme, by being efficiently completed denoising task, can be greatly improved user experience.

To single treat denoising pixel for, that the noise-removed technology scheme of the present invention is accurately selected, associated there Pixel (being positioned at this frame at the pixel place treating denoising and front and back on frame) quantity has abundant.Therefore, even if treating denoising Pixel has the highest noise grade, and the noise-removed technology scheme of the present invention can also obtain good denoising effect.

It is further to note that in existing denoising scheme, often there is one not through the broadcasting pictures of denoising rear video Continuous print is beated sense.Brightness and colourity can be filtered by the noise-removed technology scheme of the present invention simultaneously, can improve video and broadcast Put the fluency of picture.

Accompanying drawing explanation

In order to realize above-mentioned and relevant purpose, herein in conjunction with explained below and accompanying drawing, some illustrative side is described Face, these aspects indicate can to put into practice the various modes of principles disclosed herein, and all aspects and equivalence aspect It is intended to fall under in the range of theme required for protection.By reading in conjunction with the accompanying detailed description below, the disclosure above-mentioned And other purpose, feature and advantage will be apparent from.Throughout the disclosure, identical reference generally refers to identical Parts or element.

Fig. 1 shows the schematic diagram of mobile terminal 100 according to some embodiments of the invention；

Fig. 2 shows the schematic diagram of the application 200 to video denoising according to some embodiments of the invention；

Fig. 3 shows the schematic diagram of picture frame sequence according to an embodiment of the invention；

Fig. 4 shows the schematic diagram of the application 400 to video denoising according to yet other embodiments of the invention；

Fig. 5 shows the flow chart of the method 500 to video denoising according to some embodiments of the invention；

Fig. 6 shows the flow chart of a kind of implementation 600 of step S540 in Fig. 5；And

Fig. 7 shows the flow chart of the method 700 to video denoising according to some embodiments of the invention.

Detailed description of the invention

It is more fully described the exemplary embodiment of the disclosure below with reference to accompanying drawings.Although accompanying drawing shows the disclosure Exemplary embodiment, it being understood, however, that may be realized in various forms the disclosure and should be by embodiments set forth here Limited.On the contrary, it is provided that these embodiments are able to be best understood from the disclosure, and can be by the scope of the present disclosure Complete conveys to those skilled in the art.

Fig. 1 is the structured flowchart of mobile terminal 100.Mobile terminal 100 can include memory interface 102, or many Individual data processor, image processor and/or CPU 104, and peripheral interface 106.

Memory interface 102, one or more processor 104 and/or peripheral interface 106 both can be discrete components, also Can be integrated in one or more integrated circuit.In the mobile terminal 100, various elements can be by one or more communication Bus or holding wire couple.Sensor, equipment and subsystem are alternatively coupled to peripheral interface 106, in order to help realizes multiple Function.

Such as, motion sensor 110, light sensor 112 and range sensor 114 are alternatively coupled to peripheral interface 106, Facilitating orientation, illuminate and the function such as range finding.Other sensors 116 are equally connected with peripheral interface 106, such as, position system System (such as GPS), temperature sensor, biometric sensor or other sensor devices, thus can help to implement phase The function closed.

Camera sub-system 120 and optical pickocff 122 may be used for the camera of convenient such as recording photograph and video clipping The realization of function, wherein said camera sub-system and optical pickocff can be such as charge-coupled image sensor (CCD) or complementary gold Belong to oxide semiconductor (CMOS) optical pickocff.Can help to realize by one or more radio communication subsystem 124 Communication function, wherein radio communication subsystem can include radio-frequency transmitter and transmitter and/or light (the most infrared) receiver And transmitter.The particular design of radio communication subsystem 124 and embodiment can depend on that mobile terminal 100 is supported Individual or multiple communication networks.Such as, mobile terminal 100 can include being designed to supporting LTE, 3G, GSM network, GPRS network, EDGE network, Wi-Fi or WiMax network and Bluebooth^TMThe communication subsystem 124 of network.

Audio subsystem 126 can be coupled with speaker 128 and mike 130, in order to helps enforcement to enable voice Function, such as speech recognition, speech reproduction, digital record and telephony feature.I/O subsystem 140 can include touch screen control Device 142 processed and/or other input controllers 144 one or more.Touch screen controller 142 is alternatively coupled to touch screen 146.Lift For example, this touch screen 146 and touch screen controller 142 can use any one of multiple touch-sensing technology to detect The contact carried out therewith and movement or time-out, wherein detection technology is including, but not limited to capacitive character, resistive, infrared and table Face technology of acoustic wave.Other input controllers 144 one or more are alternatively coupled to other input/control devicess 148, such as one Or the pointer device of multiple button, rocker switch, thumb wheel, infrared port, USB port and/or instruction pen etc.Described One or more button (not shown)s can include for controlling speaker 128 and/or the up/down of mike 130 volume Button.

Memory interface 102 can be coupled with memorizer 150.This memorizer 150 can include that high random access is deposited Reservoir and/or nonvolatile memory, the most one or more disk storage equipment, one or more optical storage apparatus, and/ Or flash memories (such as NAND, NOR).Memorizer 150 can store operating system 172, such as Android, iOS or The operating system of Windows Phone etc.This operating system 172 can include for processing basic system services and execution Depend on the instruction of the task of hardware.Memorizer 150 can also store application 174.When mobile device is run, can be from memorizer Load operating system 172 in 150, and performed by processor 104.Application 174 operationally, also can add from memorizer 150 Carry, and performed by processor 104.Application 174 operates on operating system 172, utilizes operating system 172 and bottom hardware The interface provided realizes the desired function of various user, such as instant messaging, web page browsing, pictures management etc..Application 174 can be There is provided independent of operating system, it is also possible to be that operating system carries.It addition, application 174 is installed in mobile terminal 100 Time, it is also possible to add to operating system and drive module.

In above-mentioned various application 174, a kind of application therein is the application 200 to video denoising according to the present invention. Application 200 can carry out denoising to the picture frame sequence about video.Below in conjunction with Fig. 2, the video according to the present invention is gone Mode of making an uproar is illustrative.

Fig. 2 shows the schematic diagram of the application 200 to video denoising according to some embodiments of the invention.As in figure 2 it is shown, Application 200 includes that image acquisition unit 210, noise grade calculate unit 220, smoothing factor computing unit 230, distance calculating list Unit 240, weight calculation unit 250 and luma processing unit 260.

Image acquisition unit 210 is suitable to obtain the picture frame sequence about video.Here, treat that the original video of denoising is permissible It is the picture frame sequence of mobile terminal (100) captured in real-time, it is also possible to carry out the video data of automatic network, or locally stored The video data stored in device (150), this is not done too much restriction by the present invention.The picture frame of original video can be such as The various pixel format such as RGB, YUV or YCbCr.

It should be noted that application 200 is suitable to the picture frame to brightness and chrominance separation performs denoising.According to this A bright embodiment, during the pixel format that the picture frame at original video is the brightness such as YUV or YCbCr and chrominance separation represents, Original image frame sequence can directly be processed object as apply 200 by image acquisition unit 210.According to the present invention another Embodiment, when the picture frame for example, RGB etc. of original video does not separates the pixel format of brightness and colourity, image acquisition unit 210 pairs of original frame sequence obtained sequentially in time carry out conversion operation.Original frame sequence the most in an rgb format changes into As a example by YCbCr, the conversion operation of image acquisition unit 210 is illustrated.

First illustrating, YCbCr format is as follows with the relational expression of rgb format:

Y=K_RR+K_GG+K_BB

$C_b=\frac{1}{2(1-K_B)}(B-Y)$

$C_r=\frac{1}{2(1-K_R)}(R-Y)$

Wherein, K_RWith K_BIt is predefined, K_G=1-K_B-K_R.Such as, according to the definition in ITU-R BT.601: K_B= 0.114, K_R=0.299, K_G=1-K_B-K_R=0.587.

Therefore, in the present embodiment, image acquisition unit 210 can carry out conversion operation according to following Matrix Formula.

$\left[\begin{array}{c}Y\\ C_b\\ C_r\end{array}\right]=\left[\begin{array}{ccc}0.299& 0.587& 0.114\\ -0.169& -0.331& 0.5\\ 0.5& -0.419& -0.081\end{array}\right]\left[\begin{array}{c}R\\ G\\ B\end{array}\right]$

Wherein, Y is brightness value, C_bAnd C_rIt is respectively chromatic value.

The brightness provided for image acquisition unit 210 and the picture frame sequence of chrominance separation, noise grade calculates single Unit 220 is suitable to calculate each noise grade treating denoising pixel in each picture frame.In other words, noise grade calculates unit To each, 220 can treat that denoising pixel carries out noise grade estimation (noise level estimation).Here, noise grade Computing unit 220 can use multiple known algorithm to carry out noise grade estimation, and this is not made too many restrictions by the present invention.

In an embodiment in accordance with the invention, noise calculating unit 220 can use " Practical Signal- Dependent Noise Parameter Estimation From a Single Noisy Image”(IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL.23, NO.10, OCTOBER 2014) disclosed in noise grade estimate Calculating method.

According in another embodiment of the present invention, in order to assess a noise grade treating denoising pixel, noise meter Calculate unit 220 and treat centered by denoising pixel by this, select the image block of predetermined window size to treat denoising pixel as this Neighborhood block.On this basis, noise calculating unit 220 calculates the variance of brightness in this field block, to determine that this treats denoising pixel The noise grade of point.Generally, variance is the biggest, noise intensity the biggest (that is, noise grade is the highest).

Smoothing factor computing unit 230 is suitable to according to each noise grade treating denoising pixel and brightness thereof, and it is right to determine This is answered to treat the brightness smoothing factor of denoising pixel.

Metrics calculation unit 240 is suitable to from each this frame treated residing for denoising pixel and the most adjacent multiple image, Selected treat the pixel that denoising pixel associate with this, and calculate this based on block matching algorithm and treat denoising pixel and selected often The distance of individual associated pixel point.Here, block matching algorithm such as can use " Nonlocal Image and Movie Denoising " (Int J Comput Vis (2008) 76:123-139DOI 10.1007/s11263-007-0052-1) institute public affairs The algorithm opened, but it is not limited to this.

In an embodiment in accordance with the invention, metrics calculation unit 240 is being selected and the picture treating that denoising pixel associates Before vegetarian refreshments, carry out pretreatment operation.Here, pretreatment operation is to get rid of and treat that the pixel similarity degree of denoising is relatively low (i.e. The degree of association is relatively low) pixel.The denoising in later stage is affected the least by the pixel that these similarity degrees are low.In other words, institute Later stage denoising impact is negligible by the pixel got rid of.So, metrics calculation unit 240 passes through pretreatment operation The quantity that can reduce in a large number and treat pixel that denoising pixel associates, thus the time reducing metrics calculation unit 240 disappear Consumption.

According in another embodiment of the present invention, metrics calculation unit 240 is except performing to carry in above-described embodiment Beyond the pretreatment operation arrived, it is also possible to use the mode of operation of similar block transmission (patch propagation) to association picture Vegetarian refreshments is selected.Subsequently, according to the position relationship of the pixel of correspondence position with at least some of pixel added up, away from Can select from computing unit 240 and treat that denoising pixel keeps the pixel of this position relationship to treat denoising picture as with this with this The pixel that vegetarian refreshments is associated.Specifically, metrics calculation unit 240 can be added up this and be treated the picture frame before denoising pixel In pixel associated by the pixel of middle correspondence position the highest at least some of with the pixel similarity of correspondence position. Process below in conjunction with the Fig. 3 selected associated pixel point to transmitting based on similar block is illustrative.

Fig. 3 shows the pixel a treating denoising₀The picture frame T at place₀With at T₀Before by metrics calculation unit 240 carry out the picture frame T that distance calculates_-1、T_-2、T_-3, at T₀The most pending picture frame T₁、T₂、T₃.Pixel a is at image Frame T_-1、T_-2、T_-3Additional space position on pixel be respectively a_-1、a_-2、a_-3.Typically, at the image frame sequence of video In picture frame adjacent before and after in row, the pixel of additional space position (pixel locus in picture frame) is (such as, a₀、a_-1、a_-2And a_-3) there is the least time interval.Accordingly, the motion change of these pixels is the least.Generally, a₀Similarity The distributing position of the pixel of higher (the most close together) and a_-1、a_-2And a_-3These 3 point respective similitude distributing positions are basic Unanimously.Therefore, metrics calculation unit 240 can first be added up and a_-1、a_-2And a_-3The highest pixel of at least some of similarity Distributing position.So, metrics calculation unit 240 can be selected and a₀Maintain like the pixel of distributing position relation as with Its pixel being associated.Obviously, the selected mode that metrics calculation unit 240 is transmitted by above-mentioned similar block, can accurately and It is quickly found out and a₀The pixel that similarity is high, and eliminate usual and a₀The pixel that similarity is low, thus greatly reduce The follow-up amount of calculation carrying out Block-matching operation.Subsequently, metrics calculation unit 240 can calculate the pixel a treating denoising₀With each The distance of the pixel being associated.Here, the calculation about the distance between two pixels e.g. calculates two pictures Distance between element neighborhood of a point block.Specifically, this neighborhood block to be matched treating denoising pixel is first obtained about brightness Characteristic vector.Treat denoising pixel a₀Neighborhood block to be matched with a₀Centered by, the image block of predetermined window size.a₀Corresponding Characteristic vector include the brightness (each brightness value is a characteristic component) of each pixel in the neighborhood block of its correspondence.Similar Ground, metrics calculation unit 240 obtains each pixel neighborhood of a point block being associated characteristic vector about brightness.Basis at this On, metrics calculation unit 240 can calculate the feature of neighborhood block to be matched and each pixel neighborhood of a point block being associated to The spacing of amount is as distance corresponding to this pixel being associated.Typically, the spacing of characteristic vector is the most European Distance, but it is not limited to this.Here, distance is the nearest, and the similarity between pixel is the highest.

To sum up, smoothing factor computing unit 230 determines the brightness smoothing factor treating denoising pixel, metrics calculation unit 240 determine the distance treating that denoising pixel is corresponding with each associated pixel point.On this basis, weight calculation unit 250 is fitted In being associated with, with each, the distance that pixel is corresponding according to each brightness smoothing factor treating denoising pixel, calculate each phase The weight that associated pixel point is corresponding.For each denoising pixel for the treatment of, luma processing unit 260 is suitable to be associated with according to each The brightness of pixel and weight, calculate all luminance weighted meansigma methodss being associated with pixel and treat denoising pixel as this The brightness through denoising.

In an embodiment in accordance with the invention, weight calculation unit 250 carries out weight calculation according to following formula, but not It is limited to this.

$w(x,y)=\frac{1}{W\left(x\right)}\exp \left(\frac{-1}{h^2}\right|\left|u\right(N^d\left(x\right))-u(N^d\left(y\right)\left)\right|{|}^2)$

Wherein, x represents the pixel treating denoising, and y represents the pixel that is associated with x.

||u(N^d(x))-u(N^d(y)) | | represent x neighborhood block character pair vector and y character pair vector between away from From, d is characterized the dimension of vector, and h represents the brightness smoothing factor of x, and W (x) represents that all of pixel being associated with x is corresponding 'sSum, (x y) is the weight of y pixel to w.

On this basis, luma processing unit 260 can calculate according to following formula and treat bright through denoising of denoising pixel Degree.

$f\left(x\right)=\underset{y}{\Σ}w(x,y)u\left(y\right)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x, and (x y) is the power of y pixel to w Weight, u (y) represents the brightness value of y.

To sum up, application 200 can accurately select the pixel high with treating denoising pixel similarity as associated pixel Point, and then carry out Block-matching calculating with the weight determining each associated pixel point.It addition, application 200 can treat denoising picture The noise grade of vegetarian refreshments is estimated, and then determines the brightness smoothing factor of corresponding noise grade.In conjunction with this noise grade with every The weight of individual each pixel that is associated and brightness, application 200 can by with the brightness treating denoising pixel associated pixel point Weighted mean is as the brightness through denoising.It should be noted that metrics calculation unit 240 is to associated pixel in application 200 The accurate selection of point, can be greatly improved the denoising speed of application 200.It addition, application 200 is quickly reducing the same of noise Time, still can be effectively maintained video details.To single treat denoising pixel for, application 200 accurately select with its phase Pixel (being positioned at this frame at the pixel place treating denoising and front and back on the frame) quantity of association has abundant.Therefore, even if treating The pixel of denoising has the highest noise grade, application 200 can also obtain good denoising effect.

Fig. 4 shows the schematic diagram of the application 400 to video denoising according to yet other embodiments of the invention.Such as Fig. 4 institute Showing, application 400 includes that image acquisition unit 410, noise grade calculate unit 420, smoothing factor computing unit 430, distance meter Calculate unit 440, weight calculation unit 450, luma processing unit 460 and colourity processing unit 470.Wherein, image acquisition unit 410, noise grade calculates unit 420, smoothing factor computing unit 430, metrics calculation unit 440, weight calculation unit 450 and Luma processing unit 460 respectively with image acquisition unit 210, noise grade calculate unit 220, smoothing factor computing unit 230, Metrics calculation unit 240, weight calculation unit 250 are consistent with the embodiment of luma processing unit 260, repeat no more here.

It addition, colourity processing unit 470 is suitable to according to the colourity of each pixel in following formula smoothed image frame sequence:

$f_t\left(x\right)=\underset{\mathrm{\Δ}=-n}{\overset{n}{\Σ}}\left(\frac{1}{W\left(x\right)}\exp \right(\left|\right|u_t\left(x\right)-u_{t+\mathrm{\Δ}}\left(x\right)|{|}^2\left)u_{t+\mathrm{\Δ}}\right(x\left)\right)$

Wherein, f_tX () represents the chromatic value after pixel x is smooth on t two field picture.exp(||u_t(x)-u_t+Δ(x)||² Represent x with on adjacent Δ frame between the pixel of additional space position about neighborhood block colour difference away from.Here, each pixel Colourity can include two components (such as, C in YCbCr format_bAnd C_r).Each component can be carried out by above-mentioned formula Smoothing processing.It addition, colour difference away from above in connection with the distance treating denoising pixel and associated pixel point in brightness space Calculate similar, but amount of calculation becomes colourity from brightness, repeats no more here.

W (x) is away from sum with the colour difference of the pixel of additional space position on all consecutive frames of-n to n.u_t+Δ X () represents the chromatic value of the pixel of additional space position on Δ frame adjacent with x.Represent The weight of the chromatic value of the pixel of this additional space position.

It should be noted that in existing denoising scheme, often have a kind of discontinuous through the broadcasting pictures of denoising rear video Sense of beating.Brightness and colourity can be filtered by the application 400 of the present invention simultaneously, can improve the stream of video playback picture Smooth property.

Fig. 5 shows the flow chart of the method 500 to video denoising according to some embodiments of the invention.Method 500 is fitted In performing in the equipment such as mobile terminal (100).

As it is shown in figure 5, method 500 starts from step S510.In step S510, obtain the picture frame sequence about video. According to one embodiment of the invention, in step S510, it is first according to time sequencing and obtains the primitive frame sequence of rgb pixel form Row.Subsequently, original frame sequence is converted into brightness and the sequence of data frames of chrominance separation and as described picture frame sequence.Step S510 more specifically embodiment is consistent with above image acquisition unit 210, repeats no more here.

For the picture frame sequence obtained in step S510, method 500 performs step S520.In step S520, calculate Each noise grade treating denoising pixel in each picture frame.According to one embodiment of the invention, step S520 is more specifically Embodiment is as follows.First, treat centered by denoising pixel by this, select the image block of predetermined window size to treat denoising as this Pixel neighborhood of a point block.Subsequently, calculate the variance of brightness in this field block, to determine that this treats the noise grade of denoising pixel. Here, it is consistent that the more specifically embodiment of step S520 calculates unit 220 with above noise grade, repeats no more here.

The noise grade of denoising pixel and the brightness of this pixel, method is treated determined by step S520 500 can perform step S530, determine that corresponding this treats the brightness smoothing factor of denoising pixel.Step S530 is the most real Execute mode consistent with above smoothing factor computing unit 230, repeat no more here.

Method 500 also includes step S540.In step S540, from each this frame treated residing for denoising pixel with front and back In adjacent multiple image, selected treat the pixel that denoising pixel associates with this, and calculate this based on block matching algorithm and treat denoising Pixel and the distance being selected each associated pixel point.

According to one embodiment of the invention, step S540 may be implemented as method 600 as shown in Figure 6.Such as Fig. 6 institute Showing, method 600 starts from step S610, adds up this and treats in the picture frame before denoising pixel associated by the pixel of correspondence position Pixel in the highest at least some of with the pixel similarity of correspondence position.Subsequently, method 600 performs step S620, Pixel according to correspondence position and the position relationship of at least some of pixel added up, select and treat denoising pixel with this Keep the pixel of this position relationship as treating the pixel that denoising pixel is associated with this.Subsequently, method 600 performs step Rapid S630, calculates this distance treating denoising pixel and each pixel being associated.According to one embodiment of the invention, in step In rapid S630, first obtain this neighborhood block to be matched characteristic vector about brightness treating denoising pixel.Subsequently, obtain often The individual pixel neighborhood of a point block being associated is about the characteristic vector of brightness.On this basis, neighborhood block to be matched is calculated with every The spacing of the characteristic vector of the individual pixel neighborhood of a point block being associated is as distance corresponding to this pixel being associated.Step Rapid S540 more specifically embodiment is consistent with above metrics calculation unit 240, repeats no more here.

Based on each associated pixel point pair determined in the brightness smoothing factor determined in step S530 and step S540 The distance answered, method 500 can perform step S550.In step S550, calculate the weight that each associated pixel point is corresponding. According to one embodiment of the invention, step S550 calculates, according to following formula, the weight that each associated pixel point is corresponding.

$w(x,y)=\frac{1}{W\left(x\right)}\exp \left(\frac{-1}{h^2}\right|\left|u\right(N^d\left(x\right))-u(N^d\left(y\right)\left)\right|{|}^2)$

Wherein, x represents the pixel treating denoising.Y represents the pixel that is associated with x.

||u(N^d(x))-u(N^d(y)) | | represent x neighborhood block character pair vector and y character pair vector between away from From.D is characterized the dimension of vector.H represents the brightness smoothing factor of x.W (x) represents that all of pixel being associated with x is corresponding 'sSum.(x y) is the weight of y pixel to w.Step S550 more specifically embodiment party Formula is consistent with above weight calculation unit 250, repeats no more here.

For each denoising pixel for the treatment of, method 500 can perform step S560.In step S560, according to each with The brightness of its associated pixel point and weight, calculate all luminance weighted meansigma methodss being associated with pixel and treat denoising as this The brightness through denoising of pixel.According to one embodiment of the invention, step S560 calculates pixel through going according to following formula The brightness made an uproar.

$f\left(x\right)=\underset{y}{\Σ}w(x,y)u\left(y\right)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x, and (x y) is the power of y pixel to w Weight, u (y) represents the brightness value of y.Step S560 more specifically embodiment is consistent, here with above luma processing unit 260 Repeat no more.

Fig. 7 shows the flow chart of the method 700 to video denoising according to some embodiments of the invention.Method 700 is wrapped Include step S710, S720, S730, S740, S750, S760 and S770.Wherein, the embodiment of step S710-S760 respectively with In Fig. 5, step S510-S560 is consistent, repeats no more here.

In step S770, smooth the colourity of each pixel in described picture frame sequence according to following formula:

$f_t\left(x\right)=\underset{\mathrm{\Δ}=-n}{\overset{n}{\Σ}}\left(\frac{1}{W\left(x\right)}\exp \right(\left|\right|u_t\left(x\right)-u_{t+\mathrm{\Δ}}\left(x\right)|{|}^2\left)u_{t+\mathrm{\Δ}}\right(x\left)\right)$

Wherein, f_tX () represents the chromatic value after pixel x is smooth on t two field picture.exp(||u_t(x)-u_t+Δ(x)||² Represent x with on adjacent Δ frame between the pixel of additional space position about neighborhood block colour difference away from.W (x) be with from-n to n All consecutive frames on the colour difference of pixel of additional space position away from sum.u_t+ΔX () represents corresponding on Δ frame adjacent to x The chromatic value of the pixel of locus.Represent the pixel of this additional space position The weight of chromatic value.Step S770 more specifically embodiment is consistent with above colourity processing unit 470, the most superfluous State.

A10, application as described in A9, wherein, described image acquisition unit is suitable to obtain about video according to following manner Picture frame sequence:

Obtain the original frame sequence of rgb pixel form sequentially in time；

Original frame sequence is converted into brightness and the sequence of data frames of chrominance separation and as described picture frame sequence.

A11, application as described in A9 or A10, wherein, described noise grade calculates unit and is suitable to calculate according to following manner Each noise grade treating denoising pixel in each picture frame:

Treating centered by denoising pixel by this, the image block of selection predetermined window size treats the neighbour of denoising pixel as this Territory block；

Calculate the variance of brightness in this field block, to determine that this treats the noise grade of denoising pixel.

A12, application as according to any one of A9-A11, wherein, described metrics calculation unit is suitable to according to following manner From each this frame treated residing for denoising pixel and the most adjacent multiple image, select and treat the picture that denoising pixel associates with this Vegetarian refreshments, and calculate this based on block matching algorithm and treat denoising pixel and selected the distance of each associated pixel point:

Add up in the picture frame before this treats denoising pixel in the pixel associated by pixel of correspondence position with right Answer the highest at least some of of the pixel similarity of position；

Pixel according to correspondence position and the position relationship of at least some of pixel added up, select and treat with this Pixel of making an uproar keeps the pixel of this position relationship as treating the pixel that denoising pixel is associated with this；And

Calculate this distance treating denoising pixel and each pixel being associated.

A13, application as described in A12, wherein, described metrics calculation unit is suitable to calculate this according to following manner and treats denoising The distance of pixel and each pixel being associated:

Obtain this neighborhood block to be matched characteristic vector about brightness treating denoising pixel；

Obtain each pixel neighborhood of a point block being associated characteristic vector about brightness；And

Calculate the spacing conduct of neighborhood block to be matched and the characteristic vector of each pixel neighborhood of a point block being associated The distance that this pixel being associated is corresponding.

A14, application as according to any one of A9-13, wherein, described weight calculation unit performs institute according to following formula State and be associated with, with each, the distance that pixel is corresponding according to each brightness smoothing factor treating denoising pixel, calculate each phase The weight that associated pixel point is corresponding:

$w(x,y)=\frac{1}{W\left(x\right)}\exp \left(\frac{-1}{h^2}\right|\left|u\right(N^d\left(x\right))-u(N^d\left(y\right)\left)\right|{|}^2)$

Wherein, x represents the pixel treating denoising, and y represents the pixel that is associated with x,

||u(N^d(x))-u(N^d(y)) | | represent x neighborhood block character pair vector and y character pair vector between away from From, d is characterized the dimension of vector, and h represents the brightness smoothing factor of x, and W (x) represents that all of pixel being associated with x is corresponding 'sSum, (x y) is the weight of y pixel to w.

A15, application as described in A14, wherein, described luma processing unit, treat denoising pixel for each, be suitable to base , according to brightness and the weight being associated with each pixel, to calculate and be associated with the luminance weighted flat of pixel in following manner Average also treats the brightness through denoising of denoising pixel as this:

$f\left(x\right)=\underset{y}{\Σ}w(x,y)u\left(y\right)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x, and (x y) is the power of y pixel to w Weight, u (y) represents the brightness value of y.

A16, application as according to any one of A9-15, also include colourity processing unit, is suitable to smooth according to following formula The colourity of each pixel in described picture frame sequence:

$f_t\left(x\right)=\underset{\mathrm{\Δ}=-n}{\overset{n}{\Σ}}\left(\frac{1}{W\left(x\right)}\exp \right(\left|\right|u_t\left(x\right)-u_{t+\mathrm{\Δ}}\left(x\right)|{|}^2\left)u_{t+\mathrm{\Δ}}\right(x\left)\right)$

Wherein, f_tX () represents the chromatic value after pixel x is smooth on t two field picture, exp (| | u_t(x)-u_t+Δ(x)||² Represent x with on adjacent Δ frame between the pixel of additional space position about neighborhood block colour difference away from, W (x) be with from-n to n All consecutive frames on the colour difference of pixel of additional space position away from sum, u_t+ΔX () represents corresponding on Δ frame adjacent to x The chromatic value of the pixel of locus,Represent the pixel of this additional space position The weight of chromatic value.

In description mentioned herein, illustrate a large amount of detail.It is to be appreciated, however, that the enforcement of the present invention Example can be put into practice in the case of not having these details.In some instances, it is not shown specifically known method, knot Structure and technology, in order to do not obscure the understanding of this description.

Similarly, it will be appreciated that one or more in order to simplify that the disclosure helping understands in each inventive aspect, exist Above in the description of the exemplary embodiment of the present invention, each feature of the present invention is grouped together into single enforcement sometimes In example, figure or descriptions thereof.But, the method for the disclosure should not be construed to reflect an intention that i.e. required guarantor The application claims protected is than the feature more features being expressly recited in each claim.More precisely, as following As claims are reflected, inventive aspect is all features less than single embodiment disclosed above.Therefore, abide by The claims following detailed description of the invention are thus expressly incorporated in this detailed description of the invention, the most each claim itself Independent embodiment as the present invention.

Those skilled in the art are to be understood that the module of the equipment in example disclosed herein or unit or group Part can be arranged in equipment as depicted in this embodiment, or alternatively can be positioned at and the equipment in this example In different one or more equipment.Module in aforementioned exemplary can be combined as a module or be segmented into multiple in addition Submodule.

Those skilled in the art are appreciated that and can carry out the module in the equipment in embodiment adaptively Change and they are arranged in one or more equipment different from this embodiment.Can be the module in embodiment or list Unit or assembly are combined into a module or unit or assembly, and can put them in addition multiple submodule or subelement or Sub-component.In addition at least some in such feature and/or process or unit excludes each other, can use any Combine all features disclosed in this specification (including adjoint claim, summary and accompanying drawing) and so disclosed appoint Where method or all processes of equipment or unit are combined.Unless expressly stated otherwise, this specification (includes adjoint power Profit requires, summary and accompanying drawing) disclosed in each feature can be carried out generation by providing identical, equivalent or the alternative features of similar purpose Replace.

Although additionally, it will be appreciated by those of skill in the art that embodiments more described herein include other embodiments Some feature included by rather than further feature, but the combination of the feature of different embodiment means to be in the present invention's Within the scope of and form different embodiments.Such as, in the following claims, embodiment required for protection appoint One of meaning can mode use in any combination.

Additionally, some in described embodiment be described as at this can be by the processor of computer system or by performing The method of other device enforcement of described function or the combination of method element.Therefore, have for implementing described method or method The processor of the necessary instruction of element is formed for implementing the method or the device of method element.Additionally, device embodiment This described element is the example of following device: this device is for implementing by performed by the element of the purpose in order to implement this invention Function.

As used in this, unless specifically stated so, ordinal number " first ", " second ", " the 3rd " etc. is used Describe plain objects and be merely representative of the different instances relating to similar object, and be not intended to imply that the object being so described must Must have the time upper, spatially, sequence aspect or in any other manner to definite sequence.

Although the embodiment according to limited quantity describes the present invention, but benefits from above description, the art In it is clear for the skilled person that in the scope of the present invention thus described, it can be envisaged that other embodiments.Additionally, it should be noted that The language that uses in this specification primarily to the readable and purpose of teaching and select rather than in order to explain or limit Determine subject of the present invention and select.Therefore, in the case of without departing from the scope of the appended claims and spirit, for this For the those of ordinary skill of technical field, many modifications and changes will be apparent from.For the scope of the present invention, to this The disclosure that invention is done is illustrative and not restrictive, and it is intended that the scope of the present invention be defined by the claims appended hereto.

Claims (10)

1. the method to video denoising, is suitable to perform in the terminal, and the method includes:

Obtain the picture frame sequence about video；

Calculate each noise grade treating denoising pixel in each picture frame；

According to each noise grade treating denoising pixel and brightness thereof, determine that corresponding this treats that the brightness of denoising pixel smooths Coefficient；

From each this frame treated residing for denoising pixel and the most adjacent multiple image, select and treat that denoising pixel associates with this Pixel, and based on block matching algorithm calculate this treat denoising pixel with selected the distance of each associated pixel point；

It is associated with, with each, the distance that pixel is corresponding according to each brightness smoothing factor treating denoising pixel, calculates each The weight that associated pixel point is corresponding；And

Treat denoising pixel for each, according to each brightness being associated with pixel and weight, calculate all being associated with The luminance weighted meansigma methods of pixel also treats the brightness through denoising of denoising pixel as this.

The most described acquisition includes about the step of the picture frame sequence of video:

Obtain the original frame sequence of rgb pixel form sequentially in time；

Original frame sequence is converted into brightness and the sequence of data frames of chrominance separation and as described picture frame sequence.

3. method as claimed in claim 1 or 2, wherein, each in each picture frame of described calculating treats making an uproar of denoising pixel The step of sound grade includes:

Treat centered by denoising pixel by this, select the image block of predetermined window size to treat denoising pixel neighborhood of a point as this Block；

Calculate the variance of brightness in this field block, to determine that this treats the noise grade of denoising pixel.

4. the method as according to any one of claim 1-3, wherein, described from each this frame treated residing for denoising pixel and In the most adjacent multiple image, selected treat the pixel that denoising pixel associates with this, and calculate this based on block matching algorithm and treat Denoising pixel includes with the step of the distance of selected each associated pixel point:

Add up in the picture frame before this treats denoising pixel in the pixel associated by pixel of correspondence position with corresponding position The highest at least some of of the pixel similarity put；

Pixel according to correspondence position and the position relationship of at least some of pixel added up, select and treat denoising picture with this Vegetarian refreshments keeps the pixel of this position relationship as treating the pixel that denoising pixel is associated with this；And

Calculate this distance treating denoising pixel and each pixel being associated.

5. method as claimed in claim 4, wherein, described calculating this treat denoising pixel and each pixel being associated The step of distance includes:

Obtain this neighborhood block to be matched characteristic vector about brightness treating denoising pixel；

Obtain each pixel neighborhood of a point block being associated characteristic vector about brightness；And

Calculate the neighborhood block to be matched spacing with the characteristic vector of each pixel neighborhood of a point block being associated as this The distance that the pixel that is associated is corresponding.

6. the method as according to any one of claim 1-5, wherein, described smooths according to each brightness treating denoising pixel Coefficient is associated with, with each, the distance that pixel is corresponding, calculates under the step of weight corresponding to each associated pixel point includes State mode:

$w(x,y)=\frac{1}{W\left(x\right)}\exp \left(\frac{-1}{h^2}\right|\left|u\right(N^d\left(x\right))-u(N^d\left(y\right)\left)\right|{|}^2)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x,

||u(N^d(x))-u(N^d(y)) | | represent the distance between neighborhood block character pair vector and the y character pair vector of x, d Being characterized the dimension of vector, h represents the brightness smoothing factor of x, and W (x) represents that all of pixel being associated with x is correspondingSum, (x y) is the weight that y pixel is corresponding to w.

7. method as claimed in claim 6, wherein, described treats denoising pixel for each, according to being associated with each picture The brightness of vegetarian refreshments and weight, calculate be associated with pixel luminance weighted meansigma methods and as this treat denoising pixel through going The step of the brightness made an uproar includes following manner:

$f\left(x\right)=\underset{y}{\mathrm{\Σ}}w(x,y)u\left(y\right)$

Wherein, x represents the pixel treating denoising, and y represents the pixel being associated with x, and (x y) is the weight of y pixel, u to w Y () represents the brightness value of y.

8. the method as according to any one of claim 1-7, also includes: smooth in described picture frame sequence according to following formula The colourity of each pixel:

$f_t\left(x\right)=\underset{\mathrm{\Δ}=-n}{\overset{n}{\mathrm{\Σ}}}\left(\frac{1}{W\left(x\right)}\exp \right(\left|\right|u_t\left(x\right)-u_{t+\mathrm{\Δ}}\left(x\right)|{|}^2\left)u_{t+\mathrm{\Δ}}\right(x\left)\right)$

Wherein, f_tX () represents the chromatic value after pixel x is smooth on t two field picture, exp (| | u_t(x)-u_t+Δ(x)||²Represent X with on adjacent Δ frame between the pixel of additional space position about neighborhood block colour difference away from, W (x) for the institute from-n to n There is on consecutive frame the colour difference of the pixel of additional space position away from sum, u_t+ΔX () represents additional space on Δ frame adjacent with x The chromatic value of the pixel of position,Represent the colourity of the pixel of this additional space position The weight of value.

9. the application to video denoising, be suitable to resident in the terminal, this application includes:

Image acquisition unit, is suitable to obtain the picture frame sequence about video；

Noise grade calculates unit, is suitable to calculate each noise grade treating denoising pixel in each picture frame；

Smoothing factor computing unit, is suitable to according to each noise grade treating denoising pixel and brightness thereof, determine corresponding this Treat the brightness smoothing factor of denoising pixel；

Metrics calculation unit, is suitable to from each this frame treated residing for denoising pixel and the most adjacent multiple image, selected with This treats the pixel that denoising pixel associates, and calculates this based on block matching algorithm and treat denoising pixel and selected each association The distance of pixel；

Weight calculation unit, is suitable to according to each brightness smoothing factor treating denoising pixel and each is associated with pixel pair The distance answered, calculates the weight that each associated pixel point is corresponding；And

Luma processing unit, treats denoising pixel for each, is suitable to according to each brightness being associated with pixel and weight, Calculate all luminance weighted meansigma methodss being associated with pixel and treat the brightness through denoising of denoising pixel as this.

10. a mobile terminal, including: the application to video denoising as claimed in claim 9.