CN102129689A - Method for modeling background based on camera response function in automatic gain scene - Google Patents

Abstract

The invention discloses a method for modeling a background based on a camera response function in an automatic gain scene, which comprises the following steps of: performing automatic gain progressiveness-based analysis to obtain a roughly divided background area, obtaining low-noise training data by using a joint histogram method, and performing recovery once to obtain a globally optimal camera response function by the method based on maximum likelihood estimation and parameter constraints; online calculating a gain ratio frame by frame by utilizing correlation between a foreground and background difference and the gain ratio and the homography of a grayscale difference function relative to the gain ratio; and if the gain ratio is not 1, performing updating to obtain a background reference frame the same as a current reference frame by using the camera response function and the gain ratio, otherwise determining the background reference frame is unchanged, and obtaining the background reference frame with a gain coefficient the same as that of the current frame along with the change of the gain coefficient of a camera. By the method, the shortcomings of high background change speed, caused by difficulties in realizing automatic gain along with the camera, of the conventional methods are overcome, thereby ensuring high-efficiency motion detection.

Description

Under the automatic gain scene based on the background modeling method of camera response function

Technical field

The present invention relates to Flame Image Process and computer vision field, refer in particular to a kind of in camera automatic gain scene accurately the follower with gain index variation to obtain the background modeling method of accurate motion detection.

Background technology

Motion detection is the important research direction of computer vision, also is the crucial and module on basis during numerous computer visions are used, as the video semanteme mark, and pattern-recognition, traffic video monitoring, human body tracking.The motion detection purpose is with interested moving object complete cutting apart from video.Cut apart the whether accurate precision that directly influences subsequent module.

Method for testing motion can be classified as following a few class ^[1]: optical flow method, frame difference method, background subtraction method.In the scene of fixed cameras, the background subtraction method is widely studied owing to it all has good effect on speed and precision, and it subtracts each other present frame and reference background frame, passing threshold judgement again, thereby the sport foreground of being partitioned into.The effect of background subtraction method depends on the precision of background modeling, and promptly whether the reference background frame can truly reflect current scene.And existing background modeling method is confined to consider the dynamic change of video camera internal thermal noise, scene, as sleet, the water surface, vegetation rock, the interference of illumination variation, shade.Yet actual interference is not limited to above-mentioned, disturbs as the camera automatic gain.Automatic gain is the inherent function of most cameras, and majority can not manually be cancelled, when uprising suddenly owing to the average irradiance that blocks, reason such as switch lamp causes camera sensor (CCD or CMOS) to receive or during step-down, automatic gain is by adjusting aperture size, aperture time etc., make the gradation of image average reach best visual effect to changing inversely, just as the function of pupil.Automatic gain causes large-scale motion false retrieval, is the background behind prospect or the automatic gain because common background modeling method can't be judged the quick variation of grey scale pixel value on a large scale.

Cucchiara ^[2]Compensate gray-value variation behind the automatic gain by an empirical model, some important parameters provides with empirical value, and different model camera compensation effect difference is big.Kim ^[3]The simple hypothesis automatic gain causes the gray-scale value linear change, extrapolate the reference background frame after the variation, and this hypothesis has very mistake when high gray-scale value.Yet above-mentioned algorithm all is from experience and hypothesis, does not recognize that gradation of image value under the automatic gain changes by camera response function CRF to be determined that CRF is by the nonlinear function of the artificial design of producer, can not simply be similar to linear function.So above-mentioned algorithm lacks theory support and versatility.Soh ^[4]Control automatic gain with the gray-scale value average of reference background frame and change, but need to change the video camera internal circuit configuration, be difficult to general realization.

Because different camera CRF differences, and producer is difficult to also in addition know that for the consideration of maintaining secrecy is unwilling to announce CRF video is by that a camera output.Usually the existing common issue with of CRF recovery algorithms is: operand is big, for the CRF that obtains enough accuracy must increase number of parameters and need repeatedly iteration, and to noise-sensitive.And by Grossberg ^[5]Few parameter camera response function EMoR (Empirical Model of Response) that proposes, the design constraint of CRF and each the model camera CRF database D oRF (Database ofResponse Functions) that collects are in advance combined, obtain a function that contains N parameter.The advantage of EMoR is not need iteration, so reduced operand, and only need seldom parameter just can accurately recover CRF with respect to other algorithms, but shortcoming is still to need manually to choose in advance training data, be unfavorable for the full-automatic realization of total system, and when training data contained noise, then the poor robustness of EMoR was absorbed in local optimum easily.

List of references:

1.Hu?W?M，Tan?T?N，Wang?L，Maybank?S.A?survey?on?visual?surveillance?of?object?motion?and?behaviors.Ieee?Transactions?on?Systems?Man?and?Cybernetics?Part?C-Applications?and?Reviews，2004，34(3)：334-352

2.Cucchiara?R，Melli?R，Prati?A.Auto-iris?compensation?for?traffic?surveillance?systems.In：Proceedings?of?the?IEEE?Intelligent?Transportation?Systems?Conference.Italy：IEEE，2005.851-856

3.Kim?Z.Real?time?object?tracking?based?on?dynamic?feature?grouping?with?background?subtraction.In：Proceedings?of?the?IEEE?Computer?Society?Conference?on?Computer?Vision?and?Pattern?Recognition.Anchorage，USA：IEEE，2008.1626-1633

4.Soh?Y?S，Kwon?Y，Wang?Y.A?new?iris?control?mechanism?for?traffic?monitoring?system.In：Proceedings?of?the?9th?Pacific?Rim?International?Conference?on?Artificial?Intelligence.Guilin，China：Springer，2006.1227-1231

5.Grossberg?M?D，Nayar?S?K.Determining?the?camera?response?from?images：What?is?knowable？Ieee?Transactions?on?Pattern?Analysis?and?Machine?Intelligence，2003，25(11)：1455-1467

Summary of the invention

The problem to be solved in the present invention is: in the background subtraction method motion detection, existing background modeling method can't judge that the quick variation of grey scale pixel value on a large scale is the background behind prospect or the automatic gain, exist error big, need collect defective such as training data in advance, be unfavorable for the full-automatic realization of background subtraction method motion detection, and be vulnerable to influences such as noise.

Technical scheme of the present invention is: under the automatic gain scene based on the background modeling method of camera response function, it is characterized in that in background subtraction method motion detection, under the camera automatic gain scene, the reference background frame is followed the variation of camera gain coefficient in real time, obtain the reference background frame identical, may further comprise the steps with the present frame gain coefficient:

1) passes through based on the gradual analysis of automatic gain, the establishing target function, set the critical flase drop threshold value of automatic gain, the gray-value variation of the critical flase drop threshold value of described automatic gain during with the critical flase drop of system's generation automatic gain is characterized as according to setting, detect frame by frame whether the critical flase drop of automatic gain takes place, if take place then obtain the background area of rough segmentation, and use the method for joint histogram to obtain training data, be specially:

11) the average item that utilizes weight maximum among the parameter camera response function EMoR approximate as camera response function CRF obtains gray scale difference value function BDF, and then the postiive gain when obtaining critical flase drop compares k _PpWith negative ratio of gains k _Nn:

As 1＜k _c/ k _r＜k _Pp, postiive gain then takes place but do not cause the motion flase drop as yet, work as k _Nn＜k _c/ k _r＜1, then generation is born gain but is not caused motion flase drop, k as yet _r, k _cIt is respectively the gain coefficient of reference background frame R and present frame C;

12) be respectively when the ratio of gains: critical flase drop postiive gain k _Pp, 1, k during the negative gain of critical flase drop _Nn, obtain the corresponding gray scale difference functions respectively, according to k _Pp, 1, k _NnCorresponding BDF curve is divided into four parts with image-region, constructs objective function with this, and when objective function takes place greater than the then critical flase drop of the critical flase drop threshold value of setting, and rough segmentation goes out the background area of present frame;

13) background area pixels of rough segmentation is via the noise reduction process based on joint histogram, and removes and contain 0,255 data item, obtains the training data of low-dimensional;

2) with resulting training data in the step 1) as the input data, by disposable recovery obtains the camera response function of global optimum based on the method for maximal possibility estimation and restriction on the parameters;

3) maximal value of gray scale difference value is the monotonic increasing function about the ratio of gains, by the correlativity of the preceding background difference and the ratio of gains, and aforesaid monotonic increasing function, by preceding background frames and step 2) in the camera response function that recovered, ask for the ratio of gains frame by frame; Preceding background difference refers to the difference of present frame and reference background frame, and preceding background frames is the general designation of present frame and reference background frame;

4) if the ratio of gains that step 3) is determined is not 1, then by the ratio of gains and step 2) the camera response function that recovers, obtain the reference background frame identical with the gain coefficient of present frame, otherwise the reference background frame is constant, upgrade the reference background frame thus frame by frame, obtain the reference background frame identical with the present frame gain coefficient.

Step 1) is specially:

11) establishing that the camera automatic gain causes in the entire image pixel quantity takes place is the absolute flase drop of N ', positive and negative ratio of gains k _Pp, k _NnFor

k _pp＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＞σ(p _i))/N＜N′；1≤i≤N} (1)

k _nn＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＜-σ(p _i))/N＜N′；1≤i≤N} (2)

Wherein the collection of pixels in the entire image is P={p ₁, p ₂...., p _N, N is a total number of image pixels; B _r(p _i), B _c(p _i) be respectively pixel p _iThe gray-scale value of corresponding reference background frame R and the gray-scale value of present frame C; k _r, k _cIt is respectively the gain coefficient of reference background frame R and present frame C; σ (p _i) be p _iThe preceding background decision threshold that point is corresponding; Can obtain BDF by CRF; Num () represents qualified number of pixels;

The distribution character of each image-region is respectively k by the ratio of gains during 12) according to critical flase drop _Pp, 1, k _NnThe time obtain corresponding BDF curve, structure objective function T is divided into four classes with image-region, when T takes place greater than the then critical flase drop of threshold value, and rough segmentation goes out background:

Make x=I _i, y=BDF (I _i, k _j/ k _i), k wherein _i, k _jBe respectively automatic gain two two field picture i of front and back, the pairing gain coefficient of j, k take place _j/ k _iBe the ratio of gains, I _iBe the gray-scale value of i two field picture, k _j/ k _iBe respectively k _Pp, k _Nn, 1 o'clock, obtain curve y=Lp (x), y=Ln (x), y=0 is divided into four parts with image-region, P=PA ∪ PB ∪ PC ∪ PD, when automatic postiive gain taking place and be in critical flase drop, PA is the current background zone, and B is arranged _c(p _i)-B _r(p _i)＞0, B _c(p _i)-B _r(p _i)＜Lp (B _r(p _i)); Automatic postiive gain when taking place in PB, and the former highlight regions of having powerful connections that the moving object of low gray-scale value shelters from has B _c(p _i)-B _r(p _i)＜Ln (B _r(p _i)); PC when automatic postiive gain takes place for, the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _r(p _i))＜B _c(p _i)-B _r(p _i)＜0; PD when automatic postiive gain takes place for, the moving object of high gray-scale value shelters from former have powerful connections than dark areas, B is arranged _c(p _i)-B _r(p _i)＞Lp (B _r(p _i)), and satisfy num (PA)＞＞num (PD), num (PB)＞num (PC); When negative automatically gain taking place and be in critical flase drop, PA is that the moving object of high gray-scale value shelters from former gray-scale value upper zone of having powerful connections, and 0＜B is arranged _c(p _i)-B _r(p _i)＜Lp (B _r(p _i)); PB for the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _r(p _i))＜B _c(p _i)-B _r(p _i); PC is the current background zone, and Ln (B is arranged _r(p _i))＜B _c(p _i)-B _r(p _i)＜0; Automatically negative gain takes place shelter from former low gray-scale value zone of having powerful connections because of the moving object that is high gray-scale value, PD is this zone, because cause strong gray scale difference value, B is arranged _c(p _i)-B _r(p _i)＞Lp (B _r(p _i)), and satisfy num (PC)＞＞num (PB), num (PD)＞num (PA),

Set up objective function thus:

$T=\frac{\mathrm{num}\left(\mathrm{PA}\right)}{\mathrm{num}\left(\mathrm{PA}\right)+\mathrm{num}\left(\mathrm{PD}\right)}-\frac{\mathrm{nun}\left(\mathrm{PC}\right)}{\mathrm{num}\left(\mathrm{PB}\right)+\mathrm{num}\left(\mathrm{PC}\right)}---\left(3\right)$

The T absolute value is big more, and then the probability that takes place of automatic gain is big more, and setting critical flase drop threshold value t is 0.75, makes that PBG is the background area pixels set of rough segmentation, and when T＞t, postiive gain takes place but do not cause motion flase drop, PBG=PA automatically; When T＜-during t, negative automatically gain takes place but does not cause motion flase drop, PBG=PC;

13) based on the noise reduction process of joint histogram:

Make H (IX, PX, the X) element number of the gray-scale value that is illustrated in collection of pixels PX among the image X from 0 to IX, promptly

B (px _i, X) be pixel p x among the image X _iGray-scale value, make joint histogram be:

Q_BTF＝{(m，IC(m))|H(IC(m)，PBG，C)＝H(m，PBG，R)}(5)

Wherein m ∈ { 0,1,2, ...., 255}, 0≤IC (m)≤255, R, C are respectively reference background frame and present frame, the character that is not subtracted by the CRF dullness, the Q_BTF element number is 256, the element that contains 0 or 255 among the Q_BTF is removed,, obtained gathering P_BTF to remove saturated and the error that is caused that end, element number M＜255, P_BTF is the training data of low-dimensional.

Step 2) be specially:

21) in the EMoR framework, based on logarithm and contrafunctional computing, gain coefficient is separated from CRF with scene illumination, in mathematical analysis, change into the input training data set V=P_BTF that linear regression problem: CRF recovers, V satisfies

IV _iBe that gain coefficient is k _iThe time the gradation of image value, IV _jBe that gain coefficient is k _jThe time the gradation of image value, M is the training data number, IV _i, IV _jSatisfy:

IV _i+ε＝BTF _ij(IV _j) (8)

Wherein BTF is a luminance transfer function, and ε is a Gaussian noise, recovers based on the CRF of maximal possibility estimation and restriction on the parameters under the EMoR framework, obtains globally optimal solution, to the general type negate function of EMoR with take the logarithm:

$\ln k+\ln q=\ln \left({\mathrm{RCF}}^{-1}\left(I\right)\right)=g_0\left(I\right)+\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{d}_n{l}_n\left(I\right)---\left(9\right)$

l ₁(I) ... .l _N(I) be to CRF database D oRF negate function and the back of taking the logarithm use major component that pivot analysis PCA obtains by by main to time arrangement, g ₀(I) be CRF database D oRF negate function and take the logarithm after average; Under desirable noise-free case, IV _i, IV _jCorresponding brightness value q is identical, and gain coefficient k difference, with IV _i, IV _jSubstitution formula (9) is also subtracted each other,

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))---\left(10\right)$

Under actual conditions, i.e. IV _i, IV _jSatisfy formula (8), then formula (10) is deformed into:

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n({\mathrm{IV}}_j+\mathrm{\ϵ}))---\left(11\right)$

Because ε is a Gaussian noise, has additive property, so formula (11) has:

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\Σ}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\ϵ}}^{\′}---\left(12\right)$

Wherein the Gaussian noise that obtains for the linear operation of ε in formula (11) of ε ' makes d ₀-ln (k _i/ k _j), have:

$g_0\left({\mathrm{IV}}_j\right)-g_0\left({\mathrm{IV}}_i\right)={\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HDA0000047852530000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_0+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\&epsiv;}}^{\&prime;}---\left(13\right)$

Order:

t(m)＝g ₀(IV _j(m))-g ₀(IV _i(m))

${\mathrm{\&phi;}}_n\left(m\right)=\left\{\begin{array}{cc}{l}_n\left({\mathrm{IV}}_i\left(m\right)\right)-l_n\left({\mathrm{IV}}_j\left(m\right)\right)& n\&NotEqual;0\\ 1& n=0\end{array}\right.$

Then formula (13) becomes

$t\left(m\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n{\mathrm{\&phi;}}_n\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}=d^T\mathrm{\&Phi;}\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}---\left(14\right)$

D wherein ^T=(d ₀, d ₁, d ₂... .., d _N), Φ=(φ ₀, φ ₁, φ ₂... .., φ _N) ^T, then following formula becomes the linear regression of standard;

22) use the CRF based on maximal possibility estimation and restriction on the parameters to ask for, smallest error function is:

$E_D\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2---\left(15\right)$

Wherein, M is the element number of set V, know by EMoR, and when formula (14) is got different n values, basis function l _n(I) weight is different, and n is big more, and then the weight of Dui Ying basis function in expression formula is more little, and the respective weights coefficient is more little, the smallest error function under the operation parameter constraint:

E(d)＝E _D(d)+λE _d(d) (16)

λ is constrained parameters, is diagonal matrix, 0＜λ ₁＜λ ₂＜... ..＜λ _NBe the element on the matrix λ diagonal line,

$E_d\left(d\right)=\frac{1}{2}{d}^{T}d---\left(17\right)$

With formula (15), (17) substitution formula (16):

$E\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2+\frac{1}{2}\mathrm{\&lambda;}{d}^{T}d---\left(18\right)$

By maximal possibility estimation, formula (18) is differentiated to d

$\&dtri;E\left(d\right)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;}{d}^T---\left(19\right)$

Make formula (19) be 0 and the distortion:

$0=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}t\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T-d^T(\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\mathrm{\&Phi;}\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;})---\left(20\right)$

Obtain:

d＝(λ+Φ ^TΦ) ^-1Φ ^Tt (21)

Wherein T=(t (1), t (2) ...., t (M))

With formula (21) substitution formula (9), obtain CRF through asking the exponential sum function of negating.

Step 3) is specially:

31) analyze that to obtain the BDF maximal value be monotonic increasing function about the ratio of gains, promptly both singly answer;

32) ask for based on the automatic gain of homography:

Make that the BDF maximal value is Δ MI (k _j/ k _i)), corresponding horizontal ordinate is MI (k _j/ k _i):

(MI(k _j/k _i)＝I _i，ΔMI(k _j/k _i)＝ΔI _ji)|max{ΔI _ji＝(I _i，k _j/k _i)}，0≤I _i≤255 (22)

If the gray difference of the R of present frame C and reference background frame is only caused by automatic gain, all pixel p in the image so _iCorresponding coordinate (x (i)=B _r(p _i), y (i)=B _c(p _i)-B _r(p _i)) formed distribution DC drop on curve D L:{ (x=I, y=Δ I) | Δ I=BDF (I, k _c/ k _r) on, and Δ MI (k _c/ k _r)=max (y (i)) by the homography of Δ MI, can obtain k _c/ k _rIf the prospect of doing exercises exists, k _c/ k _rInterval s _kBe [k _C-1/ k _R-1-k_th, k _C-1/ k _R-1+ k_th], k _C-1, k _R-1Be the gain coefficient of previous frame C and B, k_th is a ratio of gains gradual change scope, gets 0.12, then obtains the interval s of MI correspondence _m, ask DC at interval s _mPeak value coordinate (MB, Δ MB):

(MB＝x(i)，ΔMB＝y(i))|max{y(i)}，x(i)Osm (23)

Make MI (k _j/ k _i)=MB obtains k by homography _j/ k _i=k _m, in the ideal case, if the peak value that is caused by automatic gain then satisfies k simultaneously _m∈ s _k, MI (k _m)=MB, the new ratio of gains is k _c/ k _r=k _m, consider noise effect, when | MI (k _m)-MB＜TM, and k _m∈ s _k, then to upgrade the ratio of gains, otherwise be the peak value that causes by sport foreground, the ratio of gains is constant, and TM gets 5 here;

33) according to step 32) ask for the ratio of gains frame by frame.

The present invention need not to collect in advance training data, makes system automation, and unmanned realization on duty becomes possibility, effectively increases work efficiency, and saves financial resources.The present invention has following advantage compared with prior art:

(1) compatible all types of camera, highly versatile:

Camera model is numerous, and camera automatic gain characteristic has nothing in common with each other, and disturbs based on the automatic gain that the empirical fit method can only be removed limited several cameras, has limited its range of application.And the present invention is from camera gray-scale value output principle, from recovering the camera response function and asking for the ratio of gains in real time and set about, obtain the reference background frame after automatic gain disturbs, theoretical complete and highly versatile, the automatic gain that can eliminate all kinds of cameras is to the background modeling adverse effect, obtain correct motion detection result, be convenient to implement on a large scale;

(2) need not to change hardware configuration, function is independent, and interface is simple:

Than the method for disturbing with the removal automatic gain by change camera hardware structure, the present invention realizes for software, does not need to change hardware configuration, does not influence other functions of modules of system, whether the independent detection automatic gain takes place, and the image of automatic gain is eliminated in output.And low with system other module coupling, the reference background frame output interface after the input interface of present frame and reference background frame only need being provided and eliminating automatic gain gets final product;

(3) fully automatic operation, the precision height:

Previous methods is recovered the training data that the camera response function needs manually to select input, and is random on the one hand big, and the artificial easily noise of introducing needs the people on duty on the other hand, wastes time and energy.And ask for based on the parameter of least square and to be subjected to noise easily and to obtain locally optimal solution.The automatic rough segmentation of the present invention background area, and obtain low noise training data by joint histogram, by the camera response function that recovers to obtain global optimum based on the method for maximal possibility estimation and restriction on the parameters, whole process automatically realizes, unmanned.

(4) operand is little, and real-time is good:

The camera response function is constant for definite camera, so only need once to recover, and the ratio of gains is dynamic change with the variation of automatic gain, need ask for frame by frame, previous methods is asked for the ratio of gains and is similar to the method that the camera response function recovers, operand is huge, can't realize asking in real time of the ratio of gains, the present invention utilizes the relation between the ratio of gains, front and back background frames, the brightness transfer function three, the online ratio of gains of asking for, and then the reference background frame after the automatic gain interference that is eliminated, and operand is little, real-time implementation.

Description of drawings

Fig. 1 is the process flow diagram that the present invention is based on the background modeling method of automatic gain.

The present frame of Fig. 2 (a) when causing critical false retrieval for automatic gain, Fig. 2 (b) causes the pixel distribution of critical false retrieval for automatic gain.

Fig. 3 (a) and Fig. 3 (b) they are at the exemplary video sequence, the present invention and additive method accuracy statistical graph, and wherein: Fig. 3 (a) compares for false drop rate; Fig. 3 (b) compares for loss.

Fig. 4 (a)-Fig. 4 (e) is at the exemplary video sequence, the motion detection comparison diagram of the present invention and additive method, and wherein: Fig. 4 (a) is a present frame, Fig. 4 (b)-Fig. 4 (e) is that the motion detection binary map of the present invention and additive method compares.

Embodiment

Below in conjunction with accompanying drawing the present invention is described in detail, described embodiment is intended to be convenient to the understanding of the present invention.

Fig. 1 is based on the process flow diagram of camera automatic gain background modeling method.According to flow sequence, the specific implementation process of each step of the inventive method is as follows:

1, obtains image sequence

System at first obtains image sequence, and sequence is inputed to two parallel modules: background subtraction method module and automatic gain background modeling method module, automatic gain background modeling method module is implemented the inventive method.

2, judge whether the camera response function recovers, if do not recover, then by the critical flase drop objective function T of structure, and the critical flase drop threshold value of the automatic gain t that sets, the gray-value variation of the critical flase drop threshold value of described automatic gain during with the critical flase drop of system's generation automatic gain is characterized as according to setting, detect the critical flase drop whether automatic gain takes place frame by frame, up to | T|＞t, promptly critical flase drop takes place, and then isolates the background area, automatic gain is the process of a quantitative change, when less than threshold value t, can not cause the flase drop of motion detection, during only greater than threshold value t, the flase drop that just can cause motion detection, both critical conditionss are " the critical flase drop of automatic gain ".

[21] utilize the approximate value of the average item of weight maximum among the EMoR, obtain gray scale difference value function BDF, the positive and negative ratio of gains k when obtaining critical flase drop as CRF _Pp, k _Nn:

Making the camera automatic gain cause in the entire image pixel quantity taking place is the absolute flase drop of N ', and N ' takes a morsel and gets final product, the absolute flase drop as 3%, and the positive and negative ratio of gains of this moment is k _Pp, k _NnFor

k _pp＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＞σ(p _i))/N＜3％；1≤i≤N}(1)

k _nn＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＜-σ(p _i))/N＜3％；1≤i≤N}(2)

Wherein the collection of pixels in the entire image is P={p ₁, p ₂...., p _N, N is a total number of image pixels; B _r(p _i), B _c(p _i) be respectively pixel p _iThe gray-scale value of corresponding reference background frame R and the gray-scale value of present frame C; k _r, k _cIt is respectively the gain coefficient of reference background frame R and present frame C; σ (p _i) be p _iThe preceding background decision threshold that point is corresponding, " preceding background " is the general designation of present frame and reference background frame; BDF is the grey scale change difference functions under the different gains ratio, ask for BDF and need known CRF, and this stage CRF waits to ask, and learns f by EMoR ₀(k * be the topmost component of CRF, and be the average of DoRF q), only need rough segmentation to go out the background area because of this stage again, not high to the accuracy requirement of CRF, so use CRF ≈ f ₀(k * q), and then obtain BDF; Num () represents qualified number of pixels, and again because the background subtraction method all has based on morphologic aftertreatment, so pixel that can a small amount of flase drop of filtering is k _Pp, k _NnBe considered as motion flase drop critical gain ratio.Then as 1＜k _c/ k _r＜k _Pp, postiive gain then takes place but do not cause the motion flase drop as yet, work as k _Nn＜k _c/ k _r＜1, then generation is born gain but is not caused the motion flase drop as yet.BDF is the grey scale change difference functions under the different gains ratio.

The distribution character of each image-region is respectively k by the ratio of gains during [22] according to critical flase drop _Pp, 1, k _NnThe time obtain corresponding BDF curve, structure objective function T is divided into four classes with image-region, when T takes place greater than the then critical flase drop of threshold value, and rough in present frame which is partitioned into partly is the zone at background place.The present frame in any moment all can be divided into two big classes: background, prospect (or claiming moving object), wherein background is the zone that does not change in scene, if automatic gain does not take place, the area relative gray-scale value of this part is constant, but because the generation of automatic gain, can cause the gray-scale value in the zone of this part also to change, therefore first rough segmentation goes out the background area:

Make x=I _i, y BDF (I _i, k _j/ k _i), k wherein _i, k _jBe respectively automatic gain two two field picture i of front and back, the pairing gain coefficient of j, k take place _j/ k _iBe the ratio of gains, I _iBe the gray-scale value of i two field picture, k _j/ k _iBe respectively k _Pp, k _Nn, 1 o'clock, obtain curve y=Lp (x), y=Ln (x), y=0 is divided into four parts with image-region, P=PA ∪ PB ∪ PC ∪ PD.

When automatic postiive gain taking place and being in critical flase drop, image-region can be divided into four parts, P=PA ∪ PB ∪ PC ∪ PD.PA is the current background zone, because postiive gain takes place, so B is arranged _c(p _i)-B _r(p _i)＞0 is not again because cause the motion flase drop as yet, so B is arranged _c(p _i)-B _r(p _i) Lp＜(B _r(p _i)); Automatic postiive gain takes place because be that the moving object of hanging down gray-scale value shelters from former highlight regions of having powerful connections, PB is this zone, because cause strong gray scale difference value, B is arranged _c(p _i)-B _r(p _i)＜Ln (B _r(p _i)); PC for the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _c(p _i))＜B _c(p _i)-B _c(p _i)＜0; PD be the moving object of high gray-scale value shelter from former have powerful connections than dark areas, B is arranged _c(p _i)-B _c(p _i)＞Lp (B _c(p _i)).PA accounts for major part in the zone as a setting in image when automatic critical gain takes place, and PD accounts for very zonule as highlighted prospect in image, otherwise can suppress the generation of postiive gain, thus num (PA)＞＞num (PD).Again because cause the enough area of space PB of generation needs of automatic postiive gain, otherwise the entire image gray average can abrupt change, and PC is the common factor of current vehicle penumbra zone and the low gray areas of reference background frame, and shared zone is also very little in image, so num (PB)＞num (PC).When negative automatically gain taking place and be in critical flase drop, PA is that the moving object of high gray-scale value shelters from former gray-scale value upper zone of having powerful connections, and 0＜B is arranged _c(p _i)-B _c(p _i)＜Lp (B _c(p _i)); PB for the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _c(p _i))＜B _c(p _i)-B _c(p _i); PC is the current background zone, because negative gain takes place, so B is arranged _c(p _i)-B _c(p _i)＜0 is not again because cause the motion flase drop as yet, so B is arranged _c(p _i)-B _c(p _i)＞Ln (B _c(p _i)); Automatically negative gain takes place shelter from former low gray-scale value zone of having powerful connections because of the moving object that is high gray-scale value, PD is this zone, because cause strong gray scale difference value, B is arranged _c(p _i)-B _c(p _i)＞Lp (B _c(p _i)), and satisfy num (PC)＞＞num (PB), num (PD)＞num (PA).In like manner can proper negative gain critical moment, set up objective function thus:

$T=\frac{\mathrm{num}\left(\mathrm{PA}\right)}{\mathrm{num}\left(\mathrm{PA}\right)+\mathrm{num}\left(\mathrm{PD}\right)}-\frac{\mathrm{nun}\left(\mathrm{PC}\right)}{\mathrm{num}\left(\mathrm{PB}\right)+\mathrm{num}\left(\mathrm{PC}\right)}---\left(3\right)$

The T absolute value is big more, and then the probability of automatic gain generation is big more, considers noise and CRF ≈ f ₀(kq) approximate error, t gets 0.75, makes that PBG is the background area pixels set of rough segmentation.When T＞t, postiive gain takes place but does not cause motion flase drop, PBG=PA automatically.When T＜-during t, negative automatically gain takes place but does not cause motion flase drop, PBG=PC.

3. the background area pixels of rough segmentation is via the noise reduction process based on joint histogram, and removes and contain 0,255 data item, avoids the error that camera is full or end, and obtains the training data of low-dimensional.

[31] based on the noise reduction process of joint histogram:

Make H (IX, PX, the X) element number of the gray-scale value that is illustrated in collection of pixels PX among the image X from 0 to IX, promptly

B (px _i, X) be pixel p x among the image X _iGray-scale value.Make joint histogram be:

Q_BTF＝{(m，IC(m))|H(IC(m)，PBG，C)＝H(m，PBG，R)} (5)

Wherein m ∈ 0,1,2 ...., 255}, 0≤IC (m)≤255, R, C are respectively reference background frame and present frame.By the character that the CRF dullness does not subtract, than PBG, Q_BTF is more accurate training data, and element number reduces to 256.

[32] for removing the error that is caused saturated and that end, the element that contains 0 or 255 among the Q_BTF is removed, obtained gathering P_BTF, element number M＜255.P_BTF promptly is required training data.

4. with the training data of P_BTF, by the camera response function that recovers to obtain global optimum based on the method for maximal possibility estimation and restriction on the parameters as input.

[41] in the EMoR framework, based on logarithm and contrafunctional computing, gain coefficient is separated from CRF with scene illumination, and then change into the linear regression problem.

The input training data set V=P_BTF that CRF recovers, V satisfies

IV _iBe that gain coefficient is k _iThe time the gradation of image value, IV _jBe that gain coefficient is k _jThe time the gradation of image value.M is the training data number.Have in the ideal case:

IV _i＝BTF _ij(IV _j) (7)

But because the noise and the existence of falsely dropping, actual as shown in the formula, wherein ε is a Gaussian noise:

IV _i+ε＝BTF _ij(IV _j) (8)

It is local minimum that noise is absorbed in general CRF recovery algorithms easily, and for this reason, proposition CRF based on maximal possibility estimation and restriction on the parameters under the EMoR framework recovers, and obtains globally optimal solution.For gain coefficient k is separated, to the general type negate function of EMoR with take the logarithm:

$\ln k+\ln q=\ln \left({\mathrm{RCF}}^{-1}\left(I\right)\right)=g_0\left(I\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n{l}_n\left(I\right)---\left(9\right)$

l ₁(I) ... .L _M(I) be to DoRF negate function and the back of taking the logarithm use major component that PCA obtains by by main to time arrangement, g ₀(I) be DoRF negate function and take the logarithm after average.Under desirable noise-free case, IV _i, IV _jSatisfy formula (7), i.e. IV _i, IV _jCorresponding brightness value q is identical, and gain coefficient k difference, with IV _i, IV _jSubstitution formula (9) is also subtracted each other,

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))---\left(10\right)$

Under actual conditions, i.e. IV _i, IV _jSatisfy formula (8), then formula (10) is deformed into:

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n({\mathrm{IV}}_j+\mathrm{\&epsiv;}))---\left(11\right)$

Because ε is a Gaussian noise, has additive property, so formula (11) has:

$\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\&epsiv;}}^{\&prime;}---\left(12\right)$

Wherein the Gaussian noise that obtains for the linear operation of ε in formula (11) of ε ' is consolidation form, makes d ₀=-ln (k _i/ k _j), have:

$g_0\left({\mathrm{IV}}_j\right)-g_0\left({\mathrm{IV}}_i\right)={\mathrm{<figure-callout\; id="0"\; label="d"\; filenames="HDA0000047852530000031.png"\; state="\{\{state\}\}">d</figure-callout>}}_0+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\&epsiv;}}^{\&prime;}---\left(13\right)$

Order:

t(m)＝g ₀(IV _j(m))-g ₀(IV _i(m))

${\mathrm{\&phi;}}_n\left(m\right)=\left\{\begin{array}{cc}{l}_n\left({\mathrm{IV}}_i\left(m\right)\right)-l_n\left({\mathrm{IV}}_j\left(m\right)\right)& n\&NotEqual;0\\ 1& n=0\end{array}\right.$

The m here is a m sample in the formula (6), (IV in the formula (13) _i, IV _j) promptly be the general reference of sample in the formula (6).

Then formula (13) becomes

$t\left(m\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n{\mathrm{\&phi;}}_n\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}=d^T\mathrm{\&Phi;}\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}---\left(14\right)$

D wherein ^T=(d ₀, d ₁, d ₂... .., d _N), Φ=(φ ₀, φ ₁, φ ₂... .., φ _N) ^T, d ^T, Φ is for formula (14) form is further simplified, and write as the form of multiplication of vectors, with convenient follow-up based on matrix operation.

Then formula (14) becomes the linear regression of standard.

[42] basis function l ₁(I) ... .l _N(I) be the major component that obtains by pivot analysis PCA by by main to time arrangement gained, weight is successively decreased successively, the basis function of different weights is constrained in separately the weight coefficient scope, avoid over-fitting taking place because of noise causes little weight basis function to obtain big weight coefficient, use maximal possibility estimation rather than the least square method in the algorithm in the past, to obtain separating of global optimum, accurately recover CRF.

Formula (14) is to ask parameter d _nThe linear regression problem, get N 3, knows by EMoR CRF is recovered precision＞99%, but this precision is to be prerequisite with the accurate coupling that does not contain noise.In order effectively to offset noise effect, use CRF to ask for based on maximal possibility estimation and restriction on the parameters, smallest error function is:

$E_D\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2---\left(15\right)$

Wherein, M is the element number of match point set V, know by EMoR, and when formula (14) is got different n values, basis function l _n(I) weight in expression formula is different, and n is big more, and then the weight of Dui Ying basis function in expression formula is more little, and the respective weights coefficient is more little.For the basis function that prevents little weight obtains big weight coefficient over-fitting takes place and is absorbed in local optimum, the smallest error function under the operation parameter constraint:

E(d)＝E _D(d)+λE _d(d) (16)

λ is constrained parameters, is diagonal matrix, 0＜λ ₁＜λ ₂＜... ..＜λ _NIt is the element on the matrix λ diagonal line.

$E_d\left(d\right)=\frac{1}{2}{d}^{T}d---\left(17\right)$

With formula (15), (17) substitution formula (16):

$E\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2+\frac{1}{2}\mathrm{\&lambda;}{d}^{T}d---\left(18\right)$

By maximal possibility estimation, formula (18) is differentiated to d

$\&dtri;E\left(d\right)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;}{d}^T---\left(19\right)$

Make formula (19) be 0 and the distortion:

$0=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}t\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T-d^T(\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\mathrm{\&Phi;}\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;})---\left(20\right)$

Solve:

d＝(λ+Φ ^TΦ) ^-1Φ ^Tt (21)

Wherein

t＝(t(1)，t(2)，....，t(M))

With formula (21) substitution formula (9), obtain CRF through asking the exponential sum function of negating.

5. ask for the ratio of gains frame by frame.

[51] automatic gain can cause brightness value to change, and (I k) is the function of the gray-scale value I before taking place about ratio of gains k and automatic gain to the grey scale change difference DELTA.When k determines, obtain Δ (I, k) maximal value is max Δ (k), we obtain max Δ (k) by analysis is monotonic increasing function about k, analyzing and obtaining the BDF maximal value is monotonic increasing function about the ratio of gains, promptly both singly answer:

If the gray-scale value of two width of cloth images is respectively I _iAnd I _j, the ratio of gains is k _j/ k _i, the relation of the ratio of gains and gain coefficient is: if the gain coefficient of image i is k _i, the gain coefficient of image j is k _j, then the ratio of gains of image i and image j is k _j/ k _iIf promptly the gain coefficient of definite two width of cloth images then can be determined the ratio of gains; If but the known gain ratio then has a lot of gain coefficient value possibilities.If I _i, I _jGain coefficient is respectively 1 and k _j/ k _i, BDF=I then _j-I _i=f (k _j/ k _iQ)-and f (q), q is the scene brightness value, establishes and works as k _j/ k _i=k ₁The time, q gets q ₁Obtain BDF maximal value max (BDF _K1)=f (k ₁q ₁)-f (q ₁); If k ₂＞k ₁, have by the CRF monotone increasing: f (k ₂q ₁)＞f (k ₁q ₁), max (BDF then _K2) 〉=f (k ₂q ₁)-f (q ₁)＞f (k ₁q ₁)-f (q ₁)=max (BDF _K1), so the maximal value of BDF is the monotonically increasing function about the ratio of gains, promptly the BDF maximal value and the ratio of gains are single should the relations.

[52] ask for based on the automatic gain of homography:

Make that the BDF maximal value is Δ MI (k _j/ k _i)), corresponding horizontal ordinate is MI (k _j/ k _i):

(MI(k _j/k _i)＝I _i，ΔMI(k _j/k _i)＝ΔI _ji)|max{ΔI _ji＝BDF(I _i，k _j/k _i)}，0≤I _i≤255(22)

If the gray difference of the R of present frame C and reference background frame is only caused by automatic gain, all pixel p in the image so _iCorresponding coordinate (x (i)=B _r(p _i), y (i)=B _r(p _i)-B _r(p _i)) formed distribution DC drop on curve D L:{ (x=I, y=Δ I) | Δ I=BDF (I, k _c/ k _r) on, and Δ MI (k _c/ k _r)=max (y (i)) by the homography of Δ MI, can obtain k _c/ k _rB wherein _r(p _i), B _r(p _i) be respectively pixel p _iCorresponding reference background frame R, the gray-scale value of present frame C, k _r, k _cIt is respectively the gain coefficient of R and C.Prospect exists even do exercises, and still can obtain k _c/ k _r, because automatic gain is a progressive process, so k _c/ k _rInterval s _kBe [k _C-1/ k _R-1-k_th, k _C-1/ k _R-1+ k_th], k _C-1, k _R-1Be the gain coefficient of previous frame C and B, k_th is a ratio of gains gradual change scope, gets 0.12 here.Then obtain the interval s of MI correspondence _m, ask DC at interval s _mPeak value coordinate (MB, Δ MB):

Make MI (k _j/ k _i)=MB obtains k by homography _j/ k _i=k _mIn the ideal case, if the peak value that is caused by automatic gain then satisfies k simultaneously _m∈ s _k, MI (k _m)=MB, the new ratio of gains is k _c/ k _r=k _mConsider noise effect, when | MI (k _m)-MB|＜TM, and k _m∈ s _k, then to upgrade the ratio of gains, otherwise be the peak value that causes by sport foreground, the ratio of gains is constant, and TM gets 5 here.

6. upgrade the reference background frame

[61] if the ratio of gains equals 1, automatic gain not taking place then, directly enters background subtraction method module.

[62] if the ratio of gains is not equal to 1, automatic gain takes place then, upgraded obtaining the reference background frame reference background frame identical with the present frame gain coefficient with the ratio of gains by the camera response function.

Fig. 2 correspondence be the critical flase drop that causes of automatic gain constantly, shown in Fig. 2 (a), a large car enters guarded region, automatic postiive gain takes place and be in critical flase drop.Shown in Fig. 2 (b), image is by curve y=Lp (x) at this moment, and y=Ln (x), y=0 are divided into four regional P=PA ∪ PB ∪ PC ∪ PD.

Fig. 3 is at the exemplary video sequence, the present invention and additive method accuracy statistical graph.Shown in Fig. 3 (a), loss average of the present invention is 4.1%, and it is little to fluctuate, and Cucchiara algorithm loss average is 18.3%, and Kim algorithm loss average is 27%, and fluctuation range is bigger, MoG loss average 52.4%.Shown in Fig. 6 (b), false drop rate average of the present invention is 3.2%, and fluctuation is slight, and Cucchiara algorithm false drop rate is about 25.6%, and fluctuation is violent, and Kim algorithm loss is about 23.5%, MoG loss about 48.3%.To sum up can obtain, motion detection precision of the present invention is high and stable, and Cucchiara and Kim take second place respectively, and MoG false drop rate and loss all about 50%, lost efficacy.

Fig. 4 is at the exemplary video sequence, the foreground detection binary map of the present invention and additive method, shown in Fig. 4 (a), the car of low gray scale enters scene, block highlight regions, cause taking place automatic postiive gain, shown in Fig. 4 (b), by the testing result of MoG algorithm, flase drop takes place in the background area of high gray scale easily as can be seen, and omission takes place the foreground area of low gray scale easily, this is because the background area of high gray scale gray scale after automatic postiive gain takes place is bigger, and change violently, the decision threshold of MoG is less than this variation, thereby causes flase drop.And the background of low gray scale foreground area since automatically postiive gain make gray-scale value become big, make it the reference background frame gray scale of convergence region, when difference less than the decision threshold of MoG omission takes place just.Shown in Fig. 4 (c), Cucchiara can eliminate certain flase drop.Shown in Fig. 4 (d), the relative Cucchiara of Kim detects better effects if, but loss is still too high.Cucchiara and Kim are difficult to take into account simultaneously omission and false drop rate, and detect the effect instability.Shown in Fig. 4 (e), this paper method all can be complete under various scenes is partitioned into sport foreground.

The present invention is the new module group on existing background subtraction method basis, whether have automatic gain take place, if do not take place, use existing ripe algorithm to guarantee the motion detection accuracy rate if detecting, only when automatic gain takes place, just call corresponding module, saved calculation resources.The CPU of testing hardware platform is Intel Core P8700, internal memory 2G, and operating system is Linux Suse 11.1.Wherein CRF asks for needs 1.3s approximately, takes place if detect no automatic gain, and then average 0.3ms consuming time automatic gain takes place and upgrade the reference background frame, then average 2.2ms consuming time if detect.The average 19.8ms consuming time of MoG.And CRF only needs once to recover, and takes about 1～2s, and monitoring can be ignored during with respect to length, so satisfy the requirement of motion detection real-time; The present invention simultaneously is by the test of a large amount of typical video sequences, and the result shows that the present invention has high generality and accuracy.

The above only is the embodiment among the present invention, but scope of the present invention should not described by this and limits.It should be appreciated by those skilled in the art,, all belong to claim of the present invention and come restricted portion in any modification or partial replacement that does not depart from the scope of the present invention.

Claims (4)

1. Under the automatic gain scene based on the background modeling method of camera response function, it is characterized in that in background subtraction method motion detection, under the camera automatic gain scene, the reference background frame is followed the variation of camera gain coefficient in real time, obtain the reference background frame identical, may further comprise the steps with the present frame gain coefficient:

   1) passes through based on the gradual analysis of automatic gain, the establishing target function, set the critical flase drop threshold value of automatic gain, the gray-value variation of the critical flase drop threshold value of described automatic gain during with the critical flase drop of system's generation automatic gain is characterized as according to setting, detect frame by frame whether the critical flase drop of automatic gain takes place, if take place then obtain the background area of rough segmentation, and use the method for joint histogram to obtain training data, be specially:

   11) the average item that utilizes weight maximum among the parameter camera response function EMoR approximate as camera response function CRF obtains gray scale difference value function BDF, and then the postiive gain when obtaining critical flase drop compares k _PpWith negative ratio of gains k _Nn:

   As 1＜k _c/ k _r＜k _Pp, postiive gain then takes place but do not cause the motion flase drop as yet, work as k _Nn＜k _c/ k _r＜1, then generation is born gain but is not caused motion flase drop, k as yet _r, k _cIt is respectively the gain coefficient of reference background frame R and present frame C;

   12) be respectively when the ratio of gains: critical flase drop postiive gain k _Pp, 1, k during the negative gain of critical flase drop _Nn, obtain the corresponding gray scale difference functions respectively, according to k _Pp, 1, k _NnCorresponding BDF curve is divided into four parts with image-region, constructs objective function with this, and when objective function takes place greater than the then critical flase drop of the critical flase drop threshold value of setting, and rough segmentation goes out the background area of present frame;

   13) background area pixels of rough segmentation is via the noise reduction process based on joint histogram, and removes and contain 0,255 data item, obtains the training data of low-dimensional;

   2) with resulting training data in the step 1) as the input data, by disposable recovery obtains the camera response function of global optimum based on the method for maximal possibility estimation and restriction on the parameters;

   3) maximal value of gray scale difference value is the monotonic increasing function about the ratio of gains, by the correlativity of the preceding background difference and the ratio of gains, and aforesaid monotonic increasing function, by preceding background frames and step 2) in the camera response function that recovered, ask for the ratio of gains frame by frame; Preceding background difference refers to the difference of present frame and reference background frame, and preceding background frames is the general designation of present frame and reference background frame;

   4) if the ratio of gains that step 3) is determined is not 1, then by the ratio of gains and step 2) the camera response function that recovers, obtain the reference background frame identical with the gain coefficient of present frame, otherwise the reference background frame is constant, upgrade the reference background frame thus frame by frame, obtain the reference background frame identical with the present frame gain coefficient.
2. 2. based on the background modeling method of camera response function, it is characterized in that step 1) is specially under the automatic gain scene according to claim 1:

   11) establishing that the camera automatic gain causes in the entire image pixel quantity takes place is the absolute flase drop of N ', positive and negative ratio of gains k _Pp, k _NnFor

   k _pp＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＞σ(p _i))/N＜N′；1≤i≤N}(1)

   k _nn＝min{k _c/k _r|num(BDF(B _r(p _i)，k _c/k _r)＜-σ(p _i))/N＜N′；1≤i≤N}(2)

   Wherein the collection of pixels in the entire image is P={p ₁, p ₂...., p _N, N is a total number of image pixels; B _r(p _i), B _r(p _i) be respectively pixel p _iThe gray-scale value of corresponding reference background frame R and the gray-scale value of present frame C; k _r, k _cIt is respectively the gain coefficient of reference background frame R and present frame C; σ (p _i) be p _iThe preceding background decision threshold that point is corresponding; Can obtain BDF by CRF; Num () represents qualified number of pixels;

   The distribution character of each image-region is respectively k by the ratio of gains during 12) according to critical flase drop _Pp, 1, k _NnThe time obtain corresponding BDF curve, structure objective function T is divided into four classes with image-region, when T takes place greater than the then critical flase drop of threshold value, and rough segmentation goes out background:

   Make x=I _i, y BDF (I _i, k _j/ k _i), k wherein _i, k _jBe respectively automatic gain two two field picture i of front and back, the pairing gain coefficient of j, k take place _j/ k _iBe the ratio of gains, I _iBe the gray-scale value of i two field picture, k _j/ k _iBe respectively k _Pp, k _Nn, 1 o'clock, obtain curve y=Lp (x), y=Ln (x), y=0 is divided into four parts with image-region, P=PA ∪ PB ∪ PC ∪ PD, when automatic postiive gain taking place and be in critical flase drop, PA is the current background zone, and B is arranged _r(p _i)-B _r(p _i)＞0, B _r(p _i)-B _r(p _i)＜LP (B _r(p _i)); Automatic postiive gain when taking place in PB, and the former highlight regions of having powerful connections that the moving object of low gray-scale value shelters from has B _r(p _i)-B _r(p _i)＜Ln (B _r(p _i)); PC when automatic postiive gain takes place for, the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _r(p _i))＜B _r(p _i)-B _r(p _i)＜0; PD when automatic postiive gain takes place for, the moving object of high gray-scale value shelters from former have powerful connections than dark areas, B is arranged _r(p _i)-B _r(p _i)＞Lp (B _r(p _i)), and satisfy num (PA)＞＞num (PD), num (PB)＞num (PC); When negative automatically gain taking place and be in critical flase drop, PA is that the moving object of high gray-scale value shelters from former gray-scale value upper zone of having powerful connections, and 0＜B is arranged _r(p _i)-B _r(p _i)＜Lp (B _r(p _i)); PB for the moving object of low gray-scale value shelters from former have powerful connections than dark areas, Ln (B is arranged _r(p _i))＜B _r(p _i)-B _r(p _i); PC is the current background zone, and Ln (B is arranged _r(p _i))＜B _r(p _i)-B _r(p _i)＜0; Automatically negative gain takes place shelter from former low gray-scale value zone of having powerful connections because of the moving object that is high gray-scale value, PD is this zone, because cause strong gray scale difference value, B is arranged _r(p _i)-B _r(p _i)＞Lp (B _r(p _i)), and satisfy num (PC)＞＞num (PB), num (PD)＞num (PA),

   Set up objective function thus:

   $T=\frac{\mathrm{num}\left(\mathrm{PA}\right)}{\mathrm{num}\left(\mathrm{PA}\right)+\mathrm{num}\left(\mathrm{PD}\right)}-\frac{\mathrm{nun}\left(\mathrm{PC}\right)}{\mathrm{num}\left(\mathrm{PB}\right)+\mathrm{num}\left(\mathrm{PC}\right)}---\left(3\right)$

   The T absolute value is big more, and then the probability that takes place of automatic gain is big more, and setting critical flase drop threshold value t is 0.75, makes that PBG is the background area pixels set of rough segmentation, and when T＞t, postiive gain takes place but do not cause motion flase drop, PBG=PA automatically; When T＜-during t, negative automatically gain takes place but does not cause motion flase drop, PBG=PC;

   13) based on the noise reduction process of joint histogram:

   Make H (IX, PX, the X) element number of the gray-scale value that is illustrated in collection of pixels PX among the image X from 0 to IX, promptly

   

   B (px _i, X) be pixel p x among the image X _iGray-scale value, make joint histogram be:

   Q_BTF＝{(m，IC(m))|H(IC(m)，PBG，C)＝H(m，PBG，R)} (5)

   Wherein m ∈ { 0,1,2, ...., 255}, 0≤IC (m)≤255, R, C are respectively reference background frame and present frame, the character that is not subtracted by the CRF dullness, the Q_BTF element number is 256, the element that contains 0 or 255 among the Q_BTF is removed,, obtained gathering P_BTF to remove saturated and the error that is caused that end, element number M＜255, P_BTF is the training data of low-dimensional.
3. 3. based on the background modeling method of camera response function, it is characterized in that step 2 under the automatic gain scene according to claim 2) be specially:

   21) in the EMoR framework, based on logarithm and contrafunctional computing, gain coefficient is separated from CRF with scene illumination, in mathematical analysis, change into the input training data set V=P_BTF that linear regression problem: CRF recovers, V satisfies

   

   IV _iBe that gain coefficient is k _iThe time the gradation of image value, IV _jBe that gain coefficient is k _jThe time the gradation of image value, M is the training data number, IV _i, IV _jSatisfy:

   IV _i+ε＝BTF _ij(IV _j) (8)

   Wherein BTF is a luminance transfer function, and ε is a Gaussian noise, recovers based on the CRF of maximal possibility estimation and restriction on the parameters under the EMoR framework, obtains globally optimal solution, to the general type negate function of EMoR with take the logarithm:

   $\ln k+\ln q=\ln \left({\mathrm{RCF}}^{-1}\left(I\right)\right)=g_0\left(I\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n{l}_n\left(I\right)---\left(9\right)$

   l ₁(I) ... .l _N(I) be to CRF database D oRF negate function and the back of taking the logarithm use major component that pivot analysis PCA obtains by by main to time arrangement, g ₀(I) be CRF database D oRF negate function and take the logarithm after average; Under desirable noise-free case, IV _i, IV _jCorresponding brightness value q is identical, and gain coefficient k difference, with IV _i, IV _jSubstitution formula (9) is also subtracted each other,

   $\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))---\left(10\right)$

   Under actual conditions, i.e. IV _i, IV _jSatisfy formula (8), then formula (10) is deformed into:

   $\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n({\mathrm{IV}}_j+\mathrm{\&epsiv;}))---\left(11\right)$

   Because ε is a Gaussian noise, has additive property, so formula (11) has:

   $\ln (k_i/k_j)=g_0\left({\mathrm{IV}}_i\right)-g_0\left({\mathrm{IV}}_j\right)+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\&epsiv;}}^{\&prime;}---\left(12\right)$

   Wherein the Gaussian noise that obtains for the linear operation of ε in formula (11) of ε ' makes d ₀-ln (k _i/ k _j), have:

   $g_0\left({\mathrm{IV}}_j\right)-g_0\left({\mathrm{IV}}_i\right)=d_0+\underset{n=1}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n(l_n\left({\mathrm{IV}}_i\right)-l_n\left({\mathrm{IV}}_j\right))+{\mathrm{\&epsiv;}}^{\&prime;}---\left(13\right)$

   Order:

   t(m)＝g ₀(IV _j(m))-g ₀(IV _i(m))

   ${\mathrm{\&phi;}}_n\left(m\right)=\left\{\begin{array}{cc}{l}_n\left({\mathrm{IV}}_i\left(m\right)\right)-l_n\left({\mathrm{IV}}_j\left(m\right)\right)& n\&NotEqual;0\\ 1& n=0\end{array}\right.$

   Then formula (13) becomes

   $t\left(m\right)=\underset{n=0}{\overset{N}{\mathrm{\&Sigma;}}}{d}_n{\mathrm{\&phi;}}_n\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}=d^T\mathrm{\&Phi;}\left(m\right)+{\mathrm{\&epsiv;}}^{\&prime;}---\left(14\right)$

   D wherein ^T=(d ₀, d ₁, d ₂... .., d _N), Φ=(φ ₀, φ ₁, φ ₂... .., φ _N) ^T, then following formula becomes the linear regression of standard;

   22) use the CRF based on maximal possibility estimation and restriction on the parameters to ask for, smallest error function is:

   $E_D\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2---\left(15\right)$

   Wherein, M is the element number of set V, know by EMoR, and when formula (14) is got different n values, basis function l _n(I) weight is different, and n is big more, and then the weight of Dui Ying basis function in expression formula is more little, and the respective weights coefficient is more little, the smallest error function under the operation parameter constraint:

   E(d)＝E _D(d)+λE _d(d) (16)

   λ is constrained parameters, is diagonal matrix, 0＜λ ₁＜λ ₂＜... ..＜λ _NBe the element on the matrix λ diagonal line,

   $E_d\left(d\right)=\frac{1}{2}{d}^{T}d---\left(17\right)$

   With formula (15), (17) substitution formula (16):

   $E\left(d\right)=\frac{1}{2}\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}{\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}}^2+\frac{1}{2}\mathrm{\&lambda;}{d}^{T}d---\left(18\right)$

   By maximal possibility estimation, formula (18) is differentiated to d

   $\&dtri;E\left(d\right)=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\{t\left(m\right)-d^T\mathrm{\&Phi;}\left(m\right)\}\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;}{d}^T---\left(19\right)$

   Make formula (19) be 0 and the distortion:

   $0=\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}t\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T-d^T(\underset{m=1}{\overset{M}{\mathrm{\&Sigma;}}}\mathrm{\&Phi;}\left(m\right)\mathrm{\&Phi;}{\left(m\right)}^T+\mathrm{\&lambda;})---\left(20\right)$

   Obtain:

   d＝(λ+Φ ^TΦ) ^-1Φ ^Tt (21)

   Wherein

   

   T=(t (1), t (2) ...., t (M))

   With formula (21) substitution formula (9), obtain CRF through asking the exponential sum function of negating.
4. According under claim 2 or the 3 described automatic gain scenes based on the background modeling method of camera response function, it is characterized in that step 3) is specially:

   31) analyze that to obtain the BDF maximal value be monotonic increasing function about the ratio of gains, promptly both singly answer;

   32) ask for based on the automatic gain of homography:

   Make that the BDF maximal value is Δ MI (k _j/ k _i)), corresponding horizontal ordinate is MI (k _j/ k _i):

   (MI(k _j/k _i)＝I _i，ΔMI(k _j/k _i)＝ΔI _ji)|max{ΔI _ji＝BDF(I _i，k _j/k _i)}，0≤I _i≤255 (22)

   If present frame C and reference background frame gray difference only cause all pixel p in the image so by automatic gain _iCorresponding coordinate (x (i)=B _r(p _i), y (i)=B _r(p _i)-B _r(p _i)) formed distribution DC drop on curve D L:{ (x=I, y=Δ I) | Δ I=BDF (I, k _c/ k _r) on, and Δ MI (k _c/ k _r)=max (y (i)) by the homography of Δ MI, can obtain k _c/ k _rIf the prospect of doing exercises exists, k _c/ k _rInterval s _kBe [k _C-1/ k _R-1-k_th, k _C-1/ k _R-1+ k_th], k _C-1, k _R-1Be the gain coefficient of previous frame C and B, k_th is a ratio of gains gradual change scope, gets 0.12, then obtains the interval s of MI correspondence _m, ask DC at interval s _mPeak value coordinate (MB, Δ MB):

   

   Make MOI (k _j/ k _i)=MB obtains k by homography _j/ k _i=k _m, in the ideal case, if the peak value that is caused by automatic gain then satisfies k simultaneously _m∈ s _k, MI (k _m)=MB, the new ratio of gains is k _c/ k _r=k _m, consider noise effect, when | MI (k _m)-MB＜TM, and k _m∈ s _k, then to upgrade the ratio of gains, otherwise be the peak value that causes by sport foreground, the ratio of gains is constant, and TM gets 5 here;

   33) according to step 32) ask for the ratio of gains frame by frame.

Images