CN103716579A - Video monitoring method and system - Google Patents

Abstract

The invention provides a video monitoring method. The video monitoring method comprises the following steps that: image frames of a plurality of cameras are acquired and a moving foreground is extracted according to the image frames; pixel coordinates of the moving foreground in the image frames are obtained; three-dimensional space coordinates of the moving foreground are calculated according to the pixel coordinates as well as the focal lengths and special distances of the plurality of cameras; and abnormal events of the moving foreground are triggered according to the three-dimensional space coordinates. In addition, the invention also provides a video monitoring system. The video monitoring method and system can improve the accuracy of abnormal event triggering, such that safety can be improved.

Description

Video frequency monitoring method and system

Technical field

The present invention relates to technical field of image processing, particularly relate to a kind of video frequency monitoring method and system.

Background technology

Video frequency monitoring method in conventional art, arranges camera in monitoring site conventionally in advance, and by the video data of this camera collection monitoring site, video data is generally the form of continuous picture frame.Picture frame comprises background and sport foreground, and sport foreground is the region that in picture frame, pixel changes.For example, as a pedestrian from monitoring site through out-of-date, the region of the motion that in the continuous picture frame getting, pedestrian occupies is sport foreground, relatively static monitoring site is the background parts of picture frame.

In conventional art, conventionally first extract the sport foreground of picture frame, then Detection and Extraction to sport foreground whether enter in the deathtrap of delimiting in advance, thereby judged whether that abnormal conditions occur and send corresponding alarm.

Yet, video frequency monitoring method in conventional art, the picture frame obtaining by camera can only reflect the plane information of monitoring site, when sport foreground is in camera focal length direction during axial motion, cannot judge whether it enters deathtrap, while making to monitor, understand holiday abnormal conditions and the warning that gives the alarm, thereby reduced fail safe.

Summary of the invention

Based on this, be necessary to provide a kind of video frequency monitoring method that can improve fail safe.

A video frequency monitoring method, comprising:

Gather the picture frame of a plurality of cameras, according to described picture frame, extract sport foreground;

Obtain the pixel coordinate of described sport foreground in described picture frame;

According to the focal length of described pixel coordinate, a plurality of cameras and space length, calculate the three dimensional space coordinate of described sport foreground;

According to described three dimensional space coordinate, trigger the anomalous event of described sport foreground.

In an embodiment, the described step according to described picture frame extraction sport foreground is therein:

According to mixed Gauss model, by background difference, extract the sport foreground of described picture frame.

In an embodiment, the quantity of described camera is 2, and is horizontally disposed with therein;

The described picture frame getting is left picture frame and the right picture frame being gathered respectively by described 2 cameras the same time;

The described step of calculating the three dimensional space coordinate of described sport foreground according to the focal length of described pixel coordinate, a plurality of cameras and space length is:

According to formula:

Disparity＝X _left-X _right

Calculate the inspection of described left picture frame and right picture frame, wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in described left picture frame, X _rightfor the horizontal coordinate of sport foreground in described right picture frame;

According to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of described sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of described sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is the horizontal range between described two cameras, the focal length that f is described camera, and Y is that sport foreground obtains vertical coordinate in described left picture frame and described right picture frame, Disparity is the parallax of left picture frame and right picture frame.

In an embodiment, the described step that triggers the anomalous event of described sport foreground according to described three dimensional space coordinate is therein:

Judge that whether described three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of described sport foreground.

Therein in an embodiment, the described step that judges whether described three dimensional space coordinate is positioned at default deathtrap is:

Obtain the depth image that described picture frame is corresponding, described depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates;

By described depth image, judge whether described three dimensional space coordinate is positioned at default deathtrap.

In an embodiment, before the step of the anomalous event of the described sport foreground of described triggering, also comprise therein:

According to described depth information, judge whether described sport foreground is human body images of gestures, if so, continue the step of the anomalous event of the described sport foreground of the described triggering of execution.

Therein in an embodiment, describedly according to described depth information, judge that whether described sport foreground is that the step of human body images of gestures is:

According to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

Calculate the depth value of described sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, the coordinate that u and v are any two points chosen at random in threshold range, and z is the depth information in sport foreground;

Obtain the training classifier of default human body images of gestures;

According to the depth value of described sport foreground and described training classifier, by decision tree, judge whether described sport foreground is human body images of gestures.

In addition, be also necessary to provide a kind of video monitoring system that can improve fail safe.

A video monitoring system, comprising:

Many orders camera, for acquired image frames;

Sport foreground extraction element, for extracting sport foreground according to described picture frame;

Coordinate transformation device, for obtaining described sport foreground at the pixel coordinate of described picture frame, the three dimensional space coordinate that calculates described sport foreground according to the focal length of described pixel coordinate, many orders camera and space length;

Anomalous event trigger equipment, for triggering the anomalous event of described sport foreground according to described three dimensional space coordinate.

In an embodiment, described sport foreground extraction element is also for extracting the sport foreground of described picture frame by background difference according to mixed Gauss model therein.

In an embodiment, described many orders camera is horizontally disposed binocular camera shooting head therein;

The described picture frame getting is left picture frame and the right picture frame being gathered by described binocular camera shooting head the same time;

Described coordinate transformation device is also for according to formula:

Disparity＝X _left-X _right

Calculate the inspection of described left picture frame and right picture frame, wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in described left picture frame, X _rightfor the horizontal coordinate of sport foreground in described right picture frame, and according to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of described sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of described sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is the horizontal range between described two cameras, the focal length that f is described camera, and Y is that sport foreground obtains vertical coordinate in described left picture frame and described right picture frame, Disparity is the parallax of left picture frame and right picture frame.

In an embodiment, described anomalous event trigger equipment is also for judging that whether described three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of described sport foreground therein.

Therein in an embodiment, described anomalous event trigger equipment also for

Obtain the depth image that described picture frame is corresponding, described depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates, and judge by described depth image whether described three dimensional space coordinate is positioned at default deathtrap.

In an embodiment, before the step of the anomalous event of the described sport foreground of described triggering, also comprise therein:

According to described depth information, judge whether described sport foreground is human body images of gestures, if so, continue the step of the anomalous event of the described sport foreground of the described triggering of execution.

Therein in an embodiment, described anomalous event trigger equipment is also for according to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

Calculate the depth value of described sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, the coordinate that u and v are any two points chosen at random in threshold range, and z is the depth information in sport foreground; Obtain the training classifier of default human body images of gestures; According to the depth value of described sport foreground and described training classifier, by decision tree, judge whether described sport foreground is human body images of gestures.

Above-mentioned video frequency monitoring method and device, by many orders camera collection picture frame, and extract sport foreground according to picture frame, then sport foreground is obtained in picture frame to the three dimensional space coordinate that pixel coordinate is converted into its residing physical location, and trigger anomalous event according to the residing three dimensional space coordinate of sport foreground, compare with conventional art, not only can be according to sport foreground the location triggered anomalous event on two dimensional surface, in the three dimensional space coordinate of the sport foreground that also can obtain according to conversion, obtain depth information (apart from the distance of camera) and trigger anomalous event, make the triggering of anomalous event more accurate, thereby improved fail safe.

Accompanying drawing explanation

Fig. 1 is the flow chart of video frequency monitoring method in an embodiment;

Fig. 2 is that in an embodiment, 2 camera imagings are coordinate transformation schematic diagram;

Fig. 3 is depth image schematic diagram in an embodiment;

Fig. 4 is the structural representation of video monitoring system in an embodiment;

Fig. 5 is the structural representation of binocular camera shooting head in an embodiment.

Embodiment

In one embodiment, as shown in Figure 1, a kind of method of obtaining pictorial information, comprises the following steps:

Step S102, gathers the picture frame of a plurality of cameras, according to picture frame, extracts sport foreground.

The data mode of the image of camera collection is continuous picture frame.Sport foreground is the pixel region changing in continuous picture frame.

In one embodiment, can by background difference, extract according to mixed Gauss model the sport foreground of picture frame.

In the present embodiment, can carry out in advance the definition of mixed Gauss model, the mixed Gauss model after definition is:

$P\left(X_t\right)=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{w}_{k,t}N(X_t,{\mathrm{\μ}}_{k,t},{\mathrm{\Σ}}_{k,t});$

$N(X_t,{\mathrm{\μ}}_{k,t},{\mathrm{\Σ}}_{k,t})=\frac{1}{{\left(2\mathrm{\π}\right)}^{\frac{n}{2}}{\left|\mathrm{\Σ}\right|}^{\frac{1}{2}}}{e}^{-\frac{1}{2}(X_t-{\mathrm{\μ}}_t){\mathrm{\Σ}}^{-1}(X_t-{\mathrm{\μ}}_t)};$

${\mathrm{\Σ}}_{k,t}={\mathrm{\σ}}_k^{2}E;$

Wherein, X _tfor moment t(is because video camera is when the capture video image, the sample frequency of sampled image frames is fixed value, so t can be also frame number) time pixel (x, y) pixel value, K is the number (being generally 3 to 5) of default mixed Gauss model, w _{k, t}the weights of k Gaussian Profile while being the moment t presetting, μ _{k, t}the average of k Gaussian Profile while being moment t, ∑ _{k, t}the covariance matrix of k Gaussian Profile while being moment t, N is Gaussian Profile probability density function,

be the variance of k Gaussian Profile, the desired value that E is Gaussian Profile.

After getting picture frame, can to Gauss model, upgrade according to current time.Can adopt online K mean algorithm is similar to parameter is estimated.Online K mean algorithm is mated the pixel of current time respectively with K Gaussian Profile, if the match is successful, upgrades average and the variance of this distribution, increases the weights that distribute; If coupling, does not produce a new distribution and removes to replace the less item of weights in existing mixed distribution.

Can, by an aforementioned K Gaussian Profile by the arranging than from big to small of weights and variance, then select with the immediate Gaussian Profile of distribution average as the Gaussian Profile of mating, that is:

$M_{k,t}=\left\{\begin{array}{cc}1& |X_t-{\mathrm{\&mu;}}_{k,t}|<\mathrm{\&lambda;\&sigma;}\\ 0& \mathrm{otherwise}\end{array}\right.$

Wherein, M _{k, t}for matching factor, i.e. M _k,tthe k value of=1 o'clock is the k value of the Gauss model of the final coupling of selecting, X _tthe pixel value of pixel (x, y) during for moment t, μ _{k, t}the average of k Gaussian Profile during for moment t, σ is variance, λ is default coefficient.

If do not find the coupling of current pixel to distribute in K distributes, the distribution of possibility minimum substitutes the distribution by new so.The average of new distribution is set to current pixel value, has larger variance and less weights.

Then can to t K the weights that distribute constantly, adjust according to the Gauss model of choosing, can pass through formula:

w _k,t+1＝(1-α)w _k,t+αM _k,t

w _k，t+1＝w _k,t+α(M _k,t-w _k,t)

Weights are adjusted.The not default coefficient of α wherein, M _k,tfor aforesaid matching factor, w _{k, t}the weights of k Gaussian Profile during for moment t.After weights are adjusted, the summation of weights remains unchanged, and is still 1.Renewal process is equivalent to matching result to carry out the second-order low-pass filter of cause and effect, is also equivalent to estimate current weight with the index windowing of past data.

That is to say, for the i.e. not distribution of coupling of not choosing, weights remain unchanged; Following formula is passed through in distribution for the i.e. coupling of choosing:

μ _t+1(x,y)＝(1-α)u(x,y)+αI _t(x,y)；

${\mathrm{\&sigma;}}_{t+1}^2(x,y)=(1-\mathrm{\&beta;}){\mathrm{\&sigma;}}_t^2(x,y)+\mathrm{\&beta;}{(I_t(x,y)-{\mathrm{\&mu;}}_t(x,y))}^T(I_t(x,y)-{\mathrm{\&mu;}}_t(x,y))\text{;}$

β＝αN(X _t,μ _k,σ _k)；

Current distribution is upgraded.

After aforesaid mixed Gauss model is upgraded, can be by calculating the w of each Gaussian Profile in aforesaid mixed Gauss model _k,t/ σ _{k, t}, and it is sorted, ratio w _k,t/ σ _k,tlarger, represent to have larger w _k,tless σ _{k, t}, the more front Gaussian Profile that therefore sorts, more applicable description background.Generally choose the preceding M of a sequence Gaussian Profile as a setting, M is tried to achieve by following formula:

$M=\arg \underset{m}{\min }(\underset{k=1}{\overset{m}{\mathrm{\&Sigma;}}}{w}_k>T_R)$

Wherein, threshold value T _rrepresent to represent background distribution weights and in integral body shared minimum scale.M is the quantity of distribution that can reach " best " of this ratio, before m most probable distribution.

By by each pixel and above-mentioned front m definite Gaussian Profile matching operation one by one, until find coupling think background dot, if without any a Gaussian Profile and X _tcoupling, is judged to be sport foreground.Each pixel to picture frame is carried out identical decision, thereby obtains sport foreground by mixture Gaussian background model.

Step S104, obtains the pixel coordinate of sport foreground in picture frame.

Pixel coordinate is the residing position of pixel in picture frame.The picture frame that a plurality of cameras obtain simultaneously has a plurality of, and sport foreground can have different pixel coordinates in each picture frame.

Step S106, according to the three dimensional space coordinate of the focal length of pixel coordinate, a plurality of cameras and space length calculating sport foreground.

In one embodiment, the quantity of camera is 2, and is horizontally disposed with.The picture frame getting is left picture frame (left) and the right picture frame (right) being gathered respectively by 2 cameras the same time.In the present embodiment, can pass through binocular camera shooting head acquired image frames.

According to the step of the three dimensional space coordinate of the focal length of pixel coordinate, a plurality of cameras and space length calculating sport foreground, can be specially:

According to formula:

Disparity＝X _left-X _right

Calculate the inspection of left picture frame and right picture frame.Wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in left picture frame, X _rightfor the horizontal coordinate of sport foreground in right picture frame;

According to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is two horizontal ranges between camera, f is the focal length of camera, Y is that sport foreground obtains vertical coordinate (because camera is horizontally disposed with in left picture frame and right picture frame, therefore in left picture frame, the Y value of pixel is identical with the Y value of pixel in right picture frame), Disparity is the parallax of left picture frame and right picture frame.

For example, as shown in Figure 2, L and R are respectively left picture frame and the right picture frame that horizontally disposed binocular camera shooting head intercepts simultaneously, C _leftand C _rightbe respectively the camera optical axis (axis through focal length perpendicular to camera mirror plane) of left and right camera, B is the distance (baseline distance) between camera, f _leftand f _rightbe respectively the focal length of left and right cameras, generally for convenience of calculating, can be by f _leftand f _rightbe made as equally, be f.(X _left, Y) and (

y) be respectively the pixel coordinate of the projection that same object produces respectively on the picture frame of left and right, (x _c, y _c, z _c) calculate the actual coordinate of this object in three dimensions.

It should be noted that in other embodiments, camera also can vertically be placed.When camera is vertically placed, B is the vertical range between camera, can be by X _left/ X _rightexchange the three dimensional space coordinate that can obtain sport foreground with Y.

In other embodiments, also can be by the plural camera collection picture frame of putting in space.Can in plural camera, choose many group cameras, the number of every group of camera is two.The picture frame that can simultaneously gather by every group of camera calculates the three dimensional space coordinate of sport foreground, then by the three dimensional space coordinate averaged of sport foreground corresponding to the every group of camera calculating, thereby improves certainty of measurement.

Step S108, according to the anomalous event of three dimensional space coordinate triggering sport foreground.

The anomalous event of sport foreground is the event that sport foreground triggers while entering into default deathtrap, can send corresponding alarm according to the event triggering.

In one embodiment, according to the step of the anomalous event of three dimensional space coordinate triggering sport foreground, can be specially: judge that whether three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of sport foreground.

For example, can in the background of picture frame, the close region of the dangerous facilities such as high-tension bus-bar be delimited for deathtrap (being represented by three dimensional space coordinate equally) in advance, if sport foreground is human body image, when the three dimensional space coordinate that human body image detected enters the deathtrap that high-tension bus-bar closes on, trigger anomalous event, thereby warn or notify related personnel to process to pedestrian.

Further, judge that the step whether three dimensional space coordinate is positioned at default deathtrap can be specially: obtain the depth image that picture frame is corresponding, depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates; By depth image, judge whether three dimensional space coordinate is positioned at default deathtrap.

For example, as shown in Figure 3, Fig. 3 has shown the depth image generating according to the three dimensional space coordinate of sport foreground in picture frame.In Fig. 3, determine the pixel coordinate of depth image according to the visible planar coordinate in the three dimensional space coordinate of pixel, the gray value that then this pixel coordinate is corresponding is set to the depth information corresponding with visible planar coordinate.For example,, if the three dimensional space coordinate of sport foreground P is (x _p, y _p, z _p), at the pixel (x of depth image _p, y _p) gray value located is γ z _p, wherein, γ is default coefficient, can be according to z _presiding number range is determined.For example,, if the z of all sport foregrounds that get _pspan be 0 to 100, γ can value be 255/100, gray value minimum can value be 0, maximum can value 255.

In the present embodiment, the step that triggers the anomalous event of sport foreground also can judge whether sport foreground is human body images of gestures according to depth information before, if so, continues to carry out the step of the anomalous event that triggers sport foreground.

Further, according to depth information, judge that whether sport foreground is that the step of human body images of gestures is:

According to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

The depth value that calculates sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, and u and v are the coordinate of any two points chosen at random in threshold range, and z is the depth information in sport foreground; Obtain the training classifier of default human body images of gestures; According to the depth value of sport foreground and training classifier, by decision tree, judge whether sport foreground is human body images of gestures.

In the present embodiment, can travel through in depth image I to obtain each pixel, then by above-mentioned formula, calculate the depth value d of each pixel, then by decision tree, according to the depth value d of the pixel traversing, judge whether this pixel belongs to the fringe region of sport foreground (judging that whether depth value d is in corresponding threshold interval), thereby can obtain all pixel coordinates residing relative position in sport foreground in sport foreground, for example, if pixel a corresponding edge can be made as 1 by the relative position property value of this pixel, if the deep inside (being non-edge) of the corresponding sport foreground of pixel b, the relative position property value of this pixel can be made as to 0.Finally all pixel coordinates can be inputted to training classifiers, thereby judge whether sport foreground is human body images of gestures, same, also can obtain the pixel coordinate scope that belongs to human body images of gestures in sport foreground.

Training classifier can generate by training sample in advance.For example, the depth image that can have human body images of gestures according to several generates training classifier, thereby obtains the classification kernel function of training classifier.

In one embodiment, as shown in Figure 4, a kind of video monitoring system, comprises many orders camera 102, sport foreground extraction element 104, coordinate transformation device 106 and anomalous event trigger equipment 108, wherein:

Many orders camera 102, for acquired image frames.

Sport foreground extraction element 104, for extracting sport foreground according to picture frame.

Many orders camera has the image collecting device of a plurality of cameras.A plurality of cameras that many orders camera can carry by it at one time carry out the IMAQ of multi-angle to same object.

Many orders camera collection to the data mode of image be continuous picture frame.Sport foreground is the pixel region changing in continuous picture frame.

In one embodiment, sport foreground extraction element 104 can be used for extracting by background difference according to mixed Gauss model the sport foreground of picture frame.

In the present embodiment, mixed Gauss model is pre-defined, for:

$P\left(X_t\right)=\underset{k=1}{\overset{K}{\mathrm{\&Sigma;}}}{w}_{k,t}N(X_t,{\mathrm{\&mu;}}_{k,t},{\mathrm{\&Sigma;}}_{k,t});$

$N(X_t,{\mathrm{\&mu;}}_{k,t},{\mathrm{\&Sigma;}}_{k,t})=\frac{1}{{\left(2\mathrm{\&pi;}\right)}^{\frac{n}{2}}{\left|\mathrm{\&Sigma;}\right|}^{\frac{1}{2}}}{e}^{-\frac{1}{2}(X_t-{\mathrm{\&mu;}}_t){\mathrm{\&Sigma;}}^{-1}(X_t-{\mathrm{\&mu;}}_t)};$

${\mathrm{\&Sigma;}}_{k,t}={\mathrm{\&sigma;}}_k^{2}E;$

Wherein, X _tfor moment t(is because video camera is when the capture video image, the sample frequency of sampled image frames is fixed value, so t can be also frame number) time pixel (x, y) pixel value, K is the number (being generally 3 to 5) of default mixed Gauss model, w _{k, t}the weights of k Gaussian Profile while being the moment t presetting, μ _{k, t}the average of k Gaussian Profile while being moment t, ∑ _{k, t}the covariance matrix of k Gaussian Profile while being moment t, N is Gaussian Profile probability density function, be the variance of k Gaussian Profile, the desired value that E is Gaussian Profile.

After getting picture frame, sport foreground extraction element 104 can be used for according to current time, Gauss model being upgraded.Can adopt online K mean algorithm is similar to parameter is estimated.Online K mean algorithm is mated the pixel of current time respectively with K Gaussian Profile, if the match is successful, upgrades average and the variance of this distribution, increases the weights that distribute; If coupling, does not produce a new distribution and removes to replace the less item of weights in existing mixed distribution.

Sport foreground extraction element 104 can be used for an aforementioned K Gaussian Profile, by the arranging than from big to small of weights and variance, then selecting with the immediate Gaussian Profile of distribution average as the Gaussian Profile of mating, that is:

$M_{k,t}=\left\{\begin{array}{cc}1& |X_t-{\mathrm{\&mu;}}_{k,t}|<\mathrm{\&lambda;\&sigma;}\\ 0& \mathrm{otherwise}\end{array}\right.$

Wherein, M _{k, t}for matching factor, i.e. M _k,tthe k value of=1 o'clock is the k value of the Gauss model of the final coupling of selecting, X _tthe pixel value of pixel (x, y) during for moment t, μ _{k, t}the average of k Gaussian Profile during for moment t, σ is variance, λ is default coefficient.

If do not find the coupling of current pixel to distribute in K distributes, the distribution of possibility minimum substitutes the distribution by new so.The average of new distribution is set to current pixel value, has larger variance and less weights.

Sport foreground extraction element 104 also can be used for according to the Gauss model of choosing, t K the weights that distribute constantly being adjusted, and can pass through formula:

w _k,t+1＝(1-α)w _k，t+αM _k,t

w _k，t+1＝w _k，t+α(M _k，t-w _k，t)

Weights are adjusted.The not default coefficient of α wherein, M _{k, t}for aforesaid matching factor, w _{k, t}the weights of k Gaussian Profile during for moment t.After weights are adjusted, the summation of weights remains unchanged, and is still 1.Renewal process is equivalent to matching result to carry out the second-order low-pass filter of cause and effect, is also equivalent to estimate current weight with the index windowing of past data.

That is to say, for the i.e. not distribution of coupling of not choosing, weights remain unchanged; Following formula is passed through in distribution for the i.e. coupling of choosing:

μ _t+1(x,y)＝(1-α)u(x,y)+αI _t(x,y)；

${\mathrm{\&sigma;}}_{t+1}^2(x,y)=(1-\mathrm{\&beta;}){\mathrm{\&sigma;}}_t^2(x,y)+\mathrm{\&beta;}{(I_t(x,y)-{\mathrm{\&mu;}}_t(x,y))}^T(I_t(x,y)-{\mathrm{\&mu;}}_t(x,y))\text{;}$

β＝αN(X _t,μ _k,σ _k)；

Current distribution is upgraded.

After sport foreground extraction element 104 also can be used for aforesaid mixed Gauss model to upgrade, by calculating the w of each Gaussian Profile in aforesaid mixed Gauss model _k,t/ σ _{k, t}, and it is sorted, ratio w _k,t/ σ _{k, t}larger, represent to have larger w _{k, t}less σ _{k, t}, the more front Gaussian Profile that therefore sorts, more applicable description background.Generally choose the preceding M of a sequence Gaussian Profile as a setting, M is tried to achieve by following formula:

$M=\arg \underset{m}{\min }(\underset{k=1}{\overset{m}{\mathrm{\&Sigma;}}}{w}_k>T_R)$

Wherein, threshold value T _rrepresent to represent background distribution weights and in integral body shared minimum scale.M is the quantity of distribution that can reach " best " of this ratio, before m most probable distribution.

Sport foreground extraction element 104 also can be used for by by each pixel and above-mentioned front m definite Gaussian Profile matching operation one by one, until find coupling think background dot, if without any a Gaussian Profile and X _tcoupling, is judged to be sport foreground.Each pixel to picture frame is carried out identical decision, thereby obtains sport foreground by mixture Gaussian background model.

Coordinate transformation device 106, for obtaining described sport foreground at the pixel coordinate of described picture frame, the three dimensional space coordinate that calculates described sport foreground according to the focal length of described pixel coordinate, many orders camera and space length.

Pixel coordinate is the residing position of pixel in picture frame.The picture frame that many orders camera 102 obtains simultaneously has a plurality of, and sport foreground can have different pixel coordinates in each picture frame.

In one embodiment, many orders camera 102 is horizontally disposed binocular camera shooting head.The picture frame that many orders camera 102 gets is left picture frame (left) and the right picture frame (right) being gathered respectively by binocular camera shooting head the same time.

As shown in Figure 5, wherein, left and right are respectively two of the left and right camera lens of horizontally disposed binocular camera shooting head, and B is the distance between left and two camera lens focuses of right, is also baseline distance.It should be noted that, left and right just, for distinguishing two camera lenses in left and right, do not limit the particular location of camera lens.

In the present embodiment, coordinate transformation device 106 also can be used for according to formula:

Disparity＝X _left-X _right

Calculate the inspection of left picture frame and right picture frame.Wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in left picture frame, X _rightfor the horizontal coordinate of sport foreground in right picture frame.

Coordinate transformation device 106 also can be used for according to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is two horizontal ranges between camera, f is the focal length of camera, Y is that sport foreground obtains vertical coordinate (because camera is horizontally disposed with in left picture frame and right picture frame, therefore in left picture frame, the Y value of pixel is identical with the Y value of pixel in right picture frame), Disparity is the parallax of left picture frame and right picture frame.

For example, as shown in Figure 2, L and R are respectively left picture frame and the right picture frame that horizontally disposed binocular camera shooting head intercepts simultaneously, C _leftand C _rightbe respectively the camera optical axis (axis through focal length perpendicular to camera mirror plane) of left and right camera, B is the distance (baseline distance) between camera, f _leftand f _rightbe respectively the focal length of left and right cameras, generally for convenience of calculating, can be by f _leftand f _rightbe made as equally, be f.(

y) and (

y) be respectively the pixel coordinate of the projection that same object produces respectively on the picture frame of left and right, (x _c, y _c, z _c) calculate the actual coordinate of this object in three dimensions.

It should be noted that in other embodiments, binocular camera shooting head also can vertically arrange.When camera is vertically placed, B is the vertical range between camera, can be by X _left/ X _rightexchange the three dimensional space coordinate that can obtain sport foreground with Y.

In other embodiments, also can be by a plurality of camera lens acquired image frames of many orders camera 102.A plurality of camera lenses of many orders camera 102 can be divided into many groups, the number of every group is two.The picture frame that can simultaneously collect by every arrangement of mirrors head calculates the three dimensional space coordinate of sport foreground, and by the three dimensional space coordinate averaged of sport foreground corresponding to the every arrangement of mirrors head calculating, thereby improve certainty of measurement.

Anomalous event trigger equipment 108, for triggering the anomalous event of sport foreground according to three dimensional space coordinate.

The anomalous event of sport foreground is the event that sport foreground triggers while entering into default deathtrap, can send corresponding alarm according to the event triggering.

In one embodiment, anomalous event trigger equipment 108 can be used for judging whether three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of sport foreground.

For example, can in the background of picture frame, the close region of the dangerous facilities such as high-tension bus-bar be delimited for deathtrap (being represented by three dimensional space coordinate equally) in advance, if sport foreground is human body image, when the three dimensional space coordinate that human body image detected enters the deathtrap that high-tension bus-bar closes on, trigger anomalous event, thereby warn or notify related personnel to process to pedestrian.

Further, anomalous event trigger equipment 108 also can be used for obtaining the depth image that picture frame is corresponding, and depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates; By depth image, judge whether three dimensional space coordinate is positioned at default deathtrap.

For example, as shown in Figure 3, Fig. 3 has shown the depth image generating according to the three dimensional space coordinate of sport foreground in picture frame.In Fig. 3, determine the pixel coordinate of depth image according to the visible planar coordinate in the three dimensional space coordinate of pixel, the gray value that then this pixel coordinate is corresponding is set to the depth information corresponding with visible planar coordinate.For example,, if the three dimensional space coordinate of sport foreground P is (xp _,y _p, z _p), at the pixel (x of depth image _p, y _p) gray value located is γ z _p, wherein, γ is default coefficient, can be according to z _presiding number range is determined.For example,, if the z of all sport foregrounds that get _pspan be 0 to 100, γ can value be 255/100, gray value minimum can value be 0, maximum can value 255.

In the present embodiment, anomalous event trigger equipment 108 also can be used for judging according to depth information whether sport foreground is human body images of gestures, if so, carries out the anomalous event that triggers sport foreground.

Further, anomalous event trigger equipment 108 also can be used for according to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

The depth value that calculates sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, and u and v are the coordinate of any two points chosen at random in threshold range, and z is the depth information in sport foreground; Obtain the training classifier of default human body images of gestures; According to the depth value of sport foreground and training classifier, by decision tree, judge whether sport foreground is human body images of gestures.

In the present embodiment, anomalous event trigger equipment 108 can be used for traveling through in depth image I obtaining each pixel, then by above-mentioned formula, calculate the depth value d of each pixel, then by decision tree, according to the depth value d of the pixel traversing, judge whether this pixel belongs to the fringe region of sport foreground (judging that whether depth value d is in corresponding threshold interval), thereby can obtain all pixel coordinates residing relative position in sport foreground in sport foreground, for example, if pixel a corresponding edge can be made as 1 by the relative position property value of this pixel, if the deep inside (being non-edge) of the corresponding sport foreground of pixel b, the relative position property value of this pixel can be made as to 0.Finally all pixel coordinates can be inputted to training classifiers, thereby judge whether sport foreground is human body images of gestures, same, also can obtain the pixel coordinate scope that belongs to human body images of gestures in sport foreground.

Training classifier can generate by training sample in advance.For example, the depth image that can have human body images of gestures according to several generates training classifier, thereby obtains the classification kernel function of training classifier.

Above-mentioned video frequency monitoring method and device, by many orders camera collection picture frame, and extract sport foreground according to picture frame, then sport foreground is obtained in picture frame to the three dimensional space coordinate that pixel coordinate is converted into its residing physical location, and trigger anomalous event according to the residing three dimensional space coordinate of sport foreground, compare with conventional art, not only can be according to sport foreground the location triggered anomalous event on two dimensional surface, in the three dimensional space coordinate of the sport foreground that also can obtain according to conversion, obtain depth information (apart from the distance of camera) and trigger anomalous event, make the triggering of anomalous event more accurate, thereby improved fail safe.

The above embodiment has only expressed several execution mode of the present invention, and it describes comparatively concrete and detailed, but can not therefore be interpreted as the restriction to the scope of the claims of the present invention.It should be pointed out that for the person of ordinary skill of the art, without departing from the inventive concept of the premise, can also make some distortion and improvement, these all belong to protection scope of the present invention.Therefore, the protection range of patent of the present invention should be as the criterion with claims.

Claims (14)

1. a video frequency monitoring method, comprising:

Gather the picture frame of a plurality of cameras, according to described picture frame, extract sport foreground;

Obtain the pixel coordinate of described sport foreground in described picture frame;

According to the focal length of described pixel coordinate, a plurality of cameras and space length, calculate the three dimensional space coordinate of described sport foreground;

According to described three dimensional space coordinate, trigger the anomalous event of described sport foreground.

2. video frequency monitoring method according to claim 1, is characterized in that, the described step according to described picture frame extraction sport foreground is:

According to mixed Gauss model, by background difference, extract the sport foreground of described picture frame.

3. video frequency monitoring method according to claim 1, is characterized in that, the quantity of described camera is 2, and is horizontally disposed with;

The described picture frame getting is left picture frame and the right picture frame being gathered respectively by described 2 cameras the same time;

The described step of calculating the three dimensional space coordinate of described sport foreground according to the focal length of described pixel coordinate, a plurality of cameras and space length is:

According to formula:

Disparity＝X _left-X _right

Calculate the inspection of described left picture frame and right picture frame, wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in described left picture frame, X _rightfor the horizontal coordinate of sport foreground in described right picture frame;

According to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of described sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of described sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is the horizontal range between described two cameras, the focal length that f is described camera, and Y is that sport foreground obtains vertical coordinate in described left picture frame and described right picture frame, Disparity is the parallax of left picture frame and right picture frame.

4. video frequency monitoring method according to claim 3, is characterized in that, the described step that triggers the anomalous event of described sport foreground according to described three dimensional space coordinate is:

Judge that whether described three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of described sport foreground.

5. video frequency monitoring method according to claim 4, is characterized in that, the described step that judges whether described three dimensional space coordinate is positioned at default deathtrap is:

Obtain the depth image that described picture frame is corresponding, described depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates;

By described depth image, judge whether described three dimensional space coordinate is positioned at default deathtrap.

6. video frequency monitoring method according to claim 3, is characterized in that, before the step of the anomalous event of the described sport foreground of described triggering, also comprises:

According to described depth information, judge whether described sport foreground is human body images of gestures, if so, continue the step of the anomalous event of the described sport foreground of the described triggering of execution.

7. video frequency monitoring method according to claim 6, is characterized in that, describedly according to described depth information, judges that whether described sport foreground is that the step of human body images of gestures is:

According to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

Calculate the depth value of described sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, the coordinate that u and v are any two points chosen at random in threshold range, and z is the depth information in sport foreground;

Obtain the training classifier of default human body images of gestures;

According to the depth value of described sport foreground and described training classifier, by decision tree, judge whether described sport foreground is human body images of gestures.

8. a video monitoring system, is characterized in that, comprising:

Many orders camera, for acquired image frames;

Sport foreground extraction element, for extracting sport foreground according to described picture frame;

Coordinate transformation device, for obtaining described sport foreground at the pixel coordinate of described picture frame, the three dimensional space coordinate that calculates described sport foreground according to the focal length of described pixel coordinate, many orders camera and space length;

Anomalous event trigger equipment, for triggering the anomalous event of described sport foreground according to described three dimensional space coordinate.

9. video monitoring system according to claim 8, is characterized in that, described sport foreground extraction element is also for extracting the sport foreground of described picture frame by background difference according to mixed Gauss model.

10. video monitoring system according to claim 8, is characterized in that, described many orders camera is horizontally disposed binocular camera shooting head;

The described picture frame getting is left picture frame and the right picture frame being gathered by described binocular camera shooting head the same time;

Described coordinate transformation device is also for according to formula:

Disparity＝X _left-X _right

Calculate the inspection of described left picture frame and right picture frame, wherein, Disparity is parallax, X _leftfor the horizontal coordinate of sport foreground in described left picture frame, X _rightfor the horizontal coordinate of sport foreground in described right picture frame, and according to formula:

$\left\{\begin{array}{c}{x}_c=\frac{{\mathrm{BX}}_{\mathrm{left}}}{\mathrm{Disparity}}\\ y_c=\frac{\mathrm{BY}}{\mathrm{Disparity}}\\ z_c=\frac{\mathrm{Bf}}{\mathrm{Disparity}}\end{array}\right.$

Calculate the three dimensional space coordinate of described sport foreground; Wherein, (x _c, y _c, z _c) be the three dimensional space coordinate of described sport foreground and x _cand y _cfor visible planar coordinate, z _cfor depth information, B is the horizontal range between described two cameras, the focal length that f is described camera, and Y is that sport foreground obtains vertical coordinate in described left picture frame and described right picture frame, Disparity is the parallax of left picture frame and right picture frame.

11. video monitoring systems according to claim 10, is characterized in that, described anomalous event trigger equipment is also for judging that whether described three dimensional space coordinate is positioned at default deathtrap, if so, triggers the anomalous event of described sport foreground.

12. video monitoring systems according to claim 11, is characterized in that, described anomalous event trigger equipment also for

Obtain the depth image that described picture frame is corresponding, described depth image is gray-scale map, the visible planar coordinate of the coordinate corresponding three-dimensional space coordinates of its pixel, the depth information of its gray value corresponding three-dimensional space coordinates, and judge by described depth image whether described three dimensional space coordinate is positioned at default deathtrap.

13. video monitoring systems according to claim 10, is characterized in that, before the step of the anomalous event of the described sport foreground of described triggering, also comprise:

According to described depth information, judge whether described sport foreground is human body images of gestures, if so, continue the step of the anomalous event of the described sport foreground of the described triggering of execution.

14. video monitoring systems according to claim 13, is characterized in that, described anomalous event trigger equipment is also for according to formula:

$d(I,c)=z(c+\frac{u}{z\left(c\right)})-z(c+\frac{v}{z\left(c\right)})$

Calculate the depth value of described sport foreground, wherein d (I, c) is the depth value of coordinate c place pixel in depth image I, the coordinate that u and v are any two points chosen at random in threshold range, and z is the depth information in sport foreground; Obtain the training classifier of default human body images of gestures; According to the depth value of described sport foreground and described training classifier, by decision tree, judge whether described sport foreground is human body images of gestures.

Images