CN104378582A - Intelligent video analysis system and method based on PTZ video camera cruising - Google Patents

Abstract

The invention provides an intelligent video analysis system and method based on PTZ video camera cruising. The system comprises a front-end PTZ video camera and a rear-end server. The rear-end server comprises a cruising configuration module, a PTZ video camera control module, a video analysis configuration module, a system control module, an intelligent video analysis module and an alarm management module. The cruising configuration module is used for setting cruising groups and cruising points of the system to generate a cruising list. The PTZ video camera control module is used for analyzing the cruising list and generating a cruising execution list automatically. The video analysis configuration module is used for configuring related intelligent video analytical algorithms for all the cruising points and configuring the cruising points into the cruising list. The system control module is used for calibrating video camera parameters of each cruising point, calling a video stitching module and automatically generating a panorama splicing map in the whole cruising period according to the execution sequence of the cruising points. The intelligent video analysis module is used for conducting target detection and event analysis according to related setting and giving a real-time alarm to a detected event. The alarm management module is used for conducting corresponding local management on the alarm.

Description

A kind of intelligent video analysis system and method cruised based on Pan/Tilt/Zoom camera

Technical field

The invention belongs to video monitoring, video analysis, area of pattern recognition, especially relate to a kind of intelligent video analysis system and method cruised based on Pan/Tilt/Zoom camera.

Background technology

Video monitoring is the important component part of safety and protection system, and along with the development of Video Supervision Technique, video camera has been widely used for monitoring in real time various environment, region and place.Because Pan/Tilt/Zoom camera has the advantage such as variable visual angle and variable focal length compared to fixed cameras, have that monitoring scene is larger, tracking target is wider, obtained applying more and more widely.

Pan/Tilt/Zoom camera tracking technique is that one utilizes image processing techniques, and realize target finds and control Pan/Tilt/Zoom camera to position in certain scene domain moving target, the monitoring technique followed the tracks of and capture.This technology may be used for monitoring road conditions, public place security monitoring, multiple field such as forest fire protection.But in current monitoring field, a normally video camera only responsible region oneself monitored, and although every platform Pan/Tilt/Zoom camera can move, but monitoring range is still very limited, because the informational needs of PTZ tri-variablees of Pan/Tilt/Zoom camera places one's entire reliance upon the feedback of track algorithm in tracing process, be difficult to control Pan/Tilt/Zoom camera accurately, and at present automatically PTZ track algorithm to Small object and environmental suitability poor, at present also cannot popularization and application on a large scale.

Based on the target detection track algorithm relative maturity of still camera, but because the view field of single camera is limited, larger monitoring visual field needs multiple video camera could realize covering, in the Target Tracking System of reality, more employing multi-camera systems, inevitably to add the cost of Iarge-scale system when budget is certain or reduce the quality of video camera, and fix due to camera scene, in order to the visual field taking into account video camera covers, intelligent video analysis algorithm generally cannot obtain the minutia detecting target, as the license board information of the vehicle of the facial information and tracking that obtain the people of tracking.

In view of introduction before, we find that system common is at present the automatic tracking system of base Pan/Tilt/Zoom camera or all there is larger problem based on the target detection tracking system of fixed cameras, cannot meet the as far as possible few video camera of video monitoring system, realize wider covering, carry out the demand of more accurate intelligent video analysis effect, how fully to excavate the characteristic of Pan/Tilt/Zoom camera, the framework how carrying out intelligent video analysis system in conjunction with the characteristic of Pan/Tilt/Zoom camera becomes the key problem that we will solve.

Summary of the invention

In order to solve the problem, the invention provides a kind of intelligent video analysis system of cruising based on Pan/Tilt/Zoom camera, this system comprises front end Pan/Tilt/Zoom camera and back-end server, back-end server comprises: cruise configuration module, to cruise list for a little carrying out setting generation to the group and cruising of cruising of system, a presetting bit of a corresponding Pan/Tilt/Zoom camera that cruises of each group of cruising is cruising some configuration cruise mode and a cruise time of each group of cruising; Pan/Tilt/Zoom camera control module, to cruising, list is analyzed, and automatically generates execution list of cruising, and makes Pan/Tilt/Zoom camera between each presetting bit, carry out detection of cruising according to the order of cruising preset; Video analysis configuration module, the intelligent video analysis algorithm relevant for configuration of cruising for each is also configured to and cruises in list; System control module, perform the relevant video camera open function cruised a little, according to the algorithm configuration that each in configured list cruises a little, enable the relevant video analysis algorithm cruised a little, for each cruising a little is carried out camera parameters demarcation and call video-splicing module, automatically generate the panoramic mosaic figure in whole cycle of cruising according to the execution sequence cruised a little; Intelligent video analysis module, carries out target detection and event analysis according to relevant setting, and produces Real-time Alarm to the event detected; Alarm management module, carries out corresponding local management function to alarm.

It is exactly all or part of parameter being gone out projection matrix P by given reference substance backwards calculation that described camera parameters is demarcated, after demarcation, two-dimensional coordinate in the two dimensional image of being caught at camera by it, and the projection matrix P obtained, can ask the positional information of certain target in three-dimensional.

Described intelligent video analysis module comprises further: image pre-processing module, and adopt the self adaptation rapid image noise reduction algorithm of wavelet transformation to carry out filtering noise reduction to image, greyscale transformation operates; Module of target detection, for carrying out moving object detection, clarification of objective is extracted, pedestrian/vehicle detection, face/car plate detection and location, and carries out Target Recognition Algorithms according to clarification of objective; Target tracking module, utilizes two-way optical flow method to carry out target following; Target's feature-extraction module, the target detected previous frame sets up the joint histogram masterplate based on color and HOG feature, and this joint histogram combines the Gradient Features of color characteristic and HOG; Feature detection matching module, is carried out search coupling at present frame, is compared by Pasteur's distance, around the target location of previous frame, namely carries out searching for coupling within the scope of certain radius find best matched position to be the position of possible target at present frame; Event checking module, judges whether generation event based on the target location change detected.

Described module of target detection is, image conversion poor based on frame, the triplicity of mixed Gaussian probabilistic model detects target.

A kind of method that described pedestrian/vehicle detection adopts optical flow field relative motion and Hog+SVM model training to combine extracts the target of specified type.

Described video-splicing module is configured to further: the coupling of carrying out feature point extraction and characteristic point, be specially: represent the grey scale change situation of this pixel and vicinity points with minimal gray variance on four of pixel Main way, the i.e. interest value of pixel, then select the point with maximum interest value as characteristic point in the local of image, 4 regions are chosen in the lap of reference picture, each region utilizes Moravec operator to find out characteristic point, choose the region of the fixed size centered by characteristic point, the most similar coupling is found in search graph, utilize the central point of the characteristic area of coupling, substitute into following equation to solve, required solution is the conversion coefficient M between two width images:

$\left[\begin{array}{c}{x}^{\′}\\ y^{\′}\\ 1\end{array}\right]=M\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{m}_0& m_1& m_2\\ m_3& m_4& m_5\\ m_6& m_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right].$

Present invention also offers a kind of intelligent video analysis method of cruising based on Pan/Tilt/Zoom camera, it comprises:

Step (1) first carries out a little setting the group and cruising of cruising of system, generates list of cruising, a presetting bit of a corresponding Pan/Tilt/Zoom camera that cruises of each group of cruising, and is cruising some configuration cruise mode and a cruise time of each group of cruising;

Step (2) is called by presetting bit, Pan/Tilt/Zoom camera is moved to and cruises accordingly a little, system control module is that each cruising a little carries out camera parameters demarcation for current scene, and configures relevant intelligent video analysis algorithm by video analysis configuration module and add to and cruise in list;

After step (3) start up system, Pan/Tilt/Zoom camera control module, by the analysis to list of cruising, generates execution list of cruising automatically, makes Pan/Tilt/Zoom camera between each presetting bit, carry out detection of cruising according to the order of cruising preset;

Step (4) system control module calls video-splicing module, automatically generates the panoramic mosaic figure in whole cycle of cruising according to the execution sequence cruised a little;

Step (5) intelligent video analysis module carries out target detection and event analysis according to relevant setting, and produces real-time alarm to the event detected.

In described step (2), camera parameters is demarcated is exactly all or part of parameter being gone out projection matrix P by given reference substance backwards calculation, after demarcation, two-dimensional coordinate in the two dimensional image of being caught at camera by it, projection matrix P with obtaining, can ask the positional information of certain target in three-dimensional.

Described step (5) comprises further: adopt the self adaptation rapid image noise reduction algorithm of wavelet transformation to carry out filtering noise reduction to image, greyscale transformation operates; Carry out moving object detection, clarification of objective is extracted, pedestrian/vehicle detection, face/car plate detection and location, and carries out Target Recognition Algorithms according to clarification of objective; Two-way optical flow method is utilized to carry out target following; The target detected previous frame sets up the joint histogram masterplate based on color and HOG feature, and this joint histogram combines the Gradient Features of color characteristic and HOG; Carry out search coupling at present frame, compared by Pasteur's distance, around the target location of previous frame, namely within the scope of certain radius, carry out searching for coupling find best matched position to be the position of possible target at present frame; Generation event is judged whether based on the target location change detected.

Described target detection is, image conversion poor based on frame, the triplicity of mixed Gaussian probabilistic model detects target.

A kind of method that described pedestrian/vehicle detection adopts optical flow field relative motion and Hog+SVM model training to combine extracts the target of specified type.

Described video-splicing specifically comprises: represent the grey scale change situation of this pixel and vicinity points with minimal gray variance on four of pixel Main way, the i.e. interest value of pixel, then select the point with maximum interest value as characteristic point in the local of image, 4 regions are chosen in the lap of reference picture, each region utilizes Moravec operator to find out characteristic point, choose the region of the fixed size centered by characteristic point, the most similar coupling is found in search graph, utilize the central point of the characteristic area of coupling, substitute into following equation to solve, required solution is the conversion coefficient M between two width images: $\left[\begin{array}{c}{x}^{\′}\\ y^{\′}\\ 1\end{array}\right]=M\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{m}_0& m_1& m_2\\ m_3& m_4& m_5\\ m_6& m_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right].$

Accompanying drawing explanation

Fig. 1 is the structured flowchart according to analytical system of the present invention;

Fig. 2 is the function diagram of the configuration module that cruises according to analytical system of the present invention;

Fig. 3 is the structure chart of the intelligent video analysis module according to analytical system of the present invention;

Fig. 4 is the module map that analytical system according to the present invention is carried out descriptive system control module and called;

Figure 5 shows that image coordinate system, the schematic diagram of camera coordinate system and world coordinate system.

Embodiment

For making above-mentioned purpose of the present invention, feature and advantage become apparent more, and below in conjunction with the drawings and specific embodiments, the present invention is further detailed explanation:

Present invention employs the front-end collection equipment of Pan/Tilt/Zoom camera as system, by cruising in conjunction with 360 degree of Pan/Tilt/Zoom camera and presetting bit fixed point cruise function exploitation video analysis algorithm, carry out target detection tracking, make a video camera can monitor larger field range, and by the Automatic Targets of video analysis algorithm and tracking, reach and can monitor multiple region with a video camera simultaneously and carry out Automatic Targets and identification requirement in a wider context.Both can save the lower deployment cost of system in actual applications, automatic target detection Tracking Recognition demand can be realized again, there is very large theory innovation and application innovation, and have great Social benefit and economic benefit.

The invention provides a kind of intelligent video analysis system of cruising based on Pan/Tilt/Zoom camera, primarily of front end Pan/Tilt/Zoom camera and back-end server composition, concrete configuration operation divides following steps:

Step 1: first carry out a little setting the group and cruising of cruising of system, generate by system list of cruising.A presetting bit of a corresponding Pan/Tilt/Zoom camera that cruises of each group of cruising.

Step 2: be cruising some configuration cruise mode and a cruise time of each group of cruising.

1. be a little configured to 360 degree of automatic cruise modes if cruised, need the direction that setting level (P) rotates, the cruise time, cruising speed class information.Set all presetting bits that will monitor successively, to be automatically generated to according to configuration information by system and to cruise in list.

2. be a little configured to cruise mode of fixing a point if cruised, then need to set the cruise time.Automatically to be generated to according to configuration information by Pan/Tilt/Zoom camera control module and to cruise in list.

Step 3: called by presetting bit, Pan/Tilt/Zoom camera is moved to and cruises accordingly a little, system control module is that each cruising a little carries out camera parameters demarcation for current scene, and configure relevant intelligent video analysis algorithm by video analysis configuration module: comprise behavioral value, vehicle detection, legacy detects, article remove detection, flame and Smoke Detection, traffic incidents detection, and cruise in list configuring to add to accordingly.

Step 4: after start up system, Pan/Tilt/Zoom camera control module, by the analysis to list of cruising, generates execution list of cruising automatically, makes Pan/Tilt/Zoom camera between each presetting bit, carry out detection of cruising according to the order of cruising preset.And system control module calls video-splicing module, automatically generate the panoramic mosaic figure in whole cycle of cruising according to the execution sequence cruised a little.What have overlapping region completes splicing automatically, splicing according to sequencing of zero lap region.

Step 5: intelligent video analysis module carries out target detection and event analysis according to relevant setting, and real-time alarm is produced to the event detected, alarm management module carries out the corresponding local management function of alarm, as video recording, grabgraf, bullet screen, and upload warning information by network and remind Surveillance center to carry out analyzing and processing to Surveillance center.

In described step 1, according to demand different time sections being monitored to different scene operation different intelligent video analysis algorithm, carry out cruise group and a configuration division of cruising, system generates list of cruising automatically.

In described step 2, to each cruise a little carry out 360 degree of automatic cruisings and fixed point cruise mode freely configure, Pan/Tilt/Zoom camera control module is automatically generated to and cruises in list.

In described step 2, cruise a fixed point cruise mode, then need control Pan/Tilt/Zoom camera in scene, carry out the setting of cruising a little, and to the order of cruising of cruising a little, time of staying information sets.Automatically to be generated to according to configuration information by Pan/Tilt/Zoom camera control module and to cruise in list.

In described step 3, demarcating the corresponding camera parameters cruised a little, is by determining the demarcation of diverse location people in scene.

In described step 3, the relevant intelligent video analysis algorithm for a configuration of cruising accordingly: comprise behavioral value, vehicle detection, legacy detects, and article remove detection, flame and Smoke Detection, traffic incidents detection.

In described step 4, after configuring all corresponding cruise parameter and intelligent video analysis algorithms cruised a little, start up system, makes system between each cruises a little, carry out detection of cruising according to the order of cruising preset.By the Pan/Tilt/Zoom camera control sequence in corresponding configured list rise time sequence, and corresponding intelligent video analysis algorithm calling sequence a little of cruising.

In described step 4, the cruise mode a little of cruising controls to be carried out according to a configuration of cruising by the Pan/Tilt/Zoom camera control module of server end, and especially 360 automatic cruisings are by horizontally rotating speed P with certain, rotates realization according to fixed direction.

In described step 4, system carries out the splicing of video image automatically according to the sequencing cruised a little, and the panoramic picture that realizing cruises organizes is shown.Particularly relate to have an overlapping region to 360 degree of panoramic mosaic rotated and picture automatically complete splicing, zero lap region carry out according to sequencing the strategy that splices.

In described step 5, this locality of carrying out alarm and corresponding video recording grabgraf stores and uploads to Surveillance center and reminds Surveillance center to carry out analyzing and processing.

The invention provides a kind of intelligent video analysis system of cruising based on Pan/Tilt/Zoom camera, by cruise group and the cruise management of a little carrying out system and call of the presetting bit of Pan/Tilt/Zoom camera according to setting, 360 degree can be carried out a little and cruise and cruise function of fixing a point cruising, the deployment of combined with intelligent video analysis algorithm, realize single camera Automatic Targets in larger scope and affair alarm, there is wider video monitoring and the ability of intelligent video analysis.Particularly for the video analysis under camera motion scene, by carrying out the detection of algorithm for pattern recognition to specific target, and by the demarcation of video camera and scene, determine the shape size of target, velocity information, makes the precision of target detection be greatly increased.360 degree of detections of cruising of Pan/Tilt/Zoom camera can also obtain panoramic picture, can indicate target and the position of event in scene more clearly, user friendly practical application.Can realize comprising behavioral value, vehicle detection by the cruise function of coupling system and intelligent video analysis algorithm, legacy detects, and article remove detection, flame and Smoke Detection, traffic incidents detection.

The invention reside in and a kind of intelligent video analysis system of cruising based on Pan/Tilt/Zoom camera is provided, the software and hardware architecture of system as shown in Figure 1, comprise front end Pan/Tilt/Zoom camera and back-end server, to cruise configuration module in back-end server deploy, Pan/Tilt/Zoom camera control module, video analysis configuration module, system control module, intelligent video analysis module, alarm management module, by carrying out cruising group and the configuration of cruising a little in conjunction with the cruise function of Pan/Tilt/Zoom camera and presetting bit function, and a little carry out the deployment of intelligent video analysis algorithm for each cruising, single camera is made to have wider video monitoring and intelligent video analysis ability, the framework of system has very large innovation, make system architecture simpler relative to multiple-camera monitoring, reduce deployment and the maintenance cost of system, there is very large economy and social value.

Below in conjunction with accompanying drawing, performing step of the present invention is described in further detail:

Step 1: in step 1, as shown in Figure 2, first we carry out the configuration of cruising of system, carry out dividing with cruising group according to the time period of system monitoring, under the scene needing monitoring, set presetting bit, and presetting bit is added to cruise in group accordingly, we claim presetting bit for cruising a little.As being group 1 of cruising at setting-up time section 8:00-12:00, setting-up time section 12:00-18:00 is group 2 of cruising, and we need Pan/Tilt/Zoom camera to be moved to A, B, C, D, E, F carry out the setting of presetting bit, we are with A, B, C, D, E, F represent corresponding presetting bit, by configuration of cruising successively A, B, C, D, presetting bit is cruised a little to add to as 4 and is cruised in group 1, D, E, F presetting bit is cruised a little to add to as 3 and is cruised in group 2, and system generates the Groups List that cruises automatically according to configuration.

Step 2: as shown in Figure 2, is followed successively by cruising of each group of cruising and a little carries out cruising the configuration of parameter, and configuration information is synchronized to cruises in list, and namely setting Pan/Tilt/Zoom camera is in corresponding parameter of cruising of cruising a little, comprises cruise mode, cruise time information.As the A that cruises for first group is set as degree automatic cruise mode, left-handed rotation, the cruise time is 5 minutes, and cruising speed rank is 3; A B that cruises is fixed point cruise mode, and the cruise time is 5 minutes.

Step 3: as shown in Figure 2, by calling a little corresponding presetting bit of cruising, Pan/Tilt/Zoom camera is realized to move to the operation of cruising accordingly a little, carrying out corresponding camera parameters demarcation for a current scene of cruising, is an intelligent video analysis algorithm that configuration is relevant that cruises accordingly: comprise behavioral value, vehicle detection, legacy detects, article remove detection, flame and Smoke Detection, traffic incidents detection.Finally scene calibration, the configuration information of intelligent analysis process is corresponding with the configuration of cruising a little, is synchronized to and cruises in list.

Camera parameters scaling method:

Extract the crown and the sole point pair at pedestrian diverse location place in scene to be calibrated in one section of video, by these to the one group of vertical line section forming vertical scene ground, the end point of vertical direction and horizontal vanishing line can be calculated by this group vertical line section.If known the length of one group of orthogonal line segment on ground, can organize with this two other axis that orthogonal line segment is three-dimensional coordinate, calculate in these two axis two other end points on a horizontal.By three orthogonal end points, and the principal point coordinate of the video camera calculated, the inside and outside parameter of video camera can be calculated, then calculate the projection matrix of video camera, complete the demarcation of camera parameters.

Be illustrated in figure 5 image coordinate system, the spatial distribution of camera coordinate system and world coordinate system, wherein image coordinate system (o ₀uv) with world coordinate system (O ₁xY) be all two-dimensional coordinate system, and be same plane, difference be the origin of coordinates of image coordinate system in the most upper left corner, and the origin of coordinates of camera coordinate system is in the middle of image; World coordinate system is three-dimensional system of coordinate, 1 P(x in real space, y, z) by imaging, obtain some p (X, Y) in camera coordinate system.

Pin-hole imaging model is typical linear model, any point, space P(x, y, z) image in p (X, Y) point in camera coordinate system, can be obtained by similar triangles:

$\left\{\begin{array}{c}X=\frac{\mathrm{fx}}{z}\\ Y=\frac{\mathrm{fy}}{z}\end{array}\right.$

In formula, f is camera focal length.Joint image coordinate system is also expressed as by next coordinate form:

$s\left[\begin{array}{c}u\\ v\\ 1\end{array}\right]=\begin{array}{c}\left[\begin{array}{cccc}{\mathrm{\α}}_x& 0& u_0& 0\\ 0& {\mathrm{\α}}_y& v_0& 0\\ 0& 0& 1& 0\end{array}\right]\left[\begin{array}{cc}R& t\\ 0^T& 1\end{array}\right]\end{array}\left[\begin{array}{c}x\\ y\\ z\\ 1\end{array}\right]=M_1{M}_2{X}_w$

In formula, s is scale factor; α _x=f/dX is the scale factor on u axle; α _y=f/dY is the scale factor on v axle; u ₀, v ₀be respectively the position of camera coordinate system initial point in image coordinate system; R, t are respectively camera coordinates and lie in spin matrix between world coordinate system and translation vector;

Matrix M ₁parameter alpha _x, α _y, u ₀, v ₀only relevant with intrinsic parameters of the camera, therefore these parameters are called intrinsic parameters of the camera; M ₂parameter R, t is determined by the orientation of the relative world coordinate system of video camera, therefore video camera external parameter is called, the process that then camera parameters is demarcated then can be converted to and solve these parameters, and it is exactly all or part of parameter being gone out projection matrix P by given reference substance backwards calculation that camera parameters is demarcated.After having demarcated, if wonder the positional information of certain target in three-dimensional, the two-dimensional coordinate in the two dimensional image of being caught at camera by it, and the projection matrix P just now obtained, can be asked.

The flow process of intelligent video analysis algoritic module is as shown in Figure 3:

Image pre-processing module: the real-time video of collection is inevitably subject to light, rain, snow, the impact of mist and system interference, image can exist certain fuzzy, noise jamming problem.First will carry out filtering noise reduction to image, greyscale transformation operates.The present invention adopts the self adaptation rapid image noise reduction algorithm of wavelet transformation to carry out preliminary treatment to image.

Module of target detection: image is after noise reduction process, intelligent video analysis algorithm configuration according to presetting bit carries out invocation target detection module, carry out moving object detection, clarification of objective is extracted, pedestrian/vehicle detection, face/car plate detection and location, and carry out Target Recognition Algorithms according to clarification of objective.

Wherein module of target detection, for the static background of fixed point cruise mode, the present invention proposes a kind of background modeling method based on transform domain image, adopts based on having merged that frame is poor, image conversion (embossment conversion), mixed Gaussian probabilistic model detect target.Wherein frame is poor, embossment converts, and Gaussian modeling all has certain light adaptation to a certain extent, carries out background modeling process in conjunction with three, further enhancing the adaptability of algorithm to complex situations such as light, the more complete moving target extracting scene.

1. frame difference can be the difference of consecutive frame or the difference of a little interframe.The method has stronger scene changes adaptive capacity, anti-illumination variation and noise resisting ability strong;

F(x,y)=abs(I _n(x，y)I _(n-i)(x，y))

Wherein, I _nthe gray value that (x, y) puts for n moment (x, y), I _(n-i)(x, y) represents the gray value at the n-th-i two field picture coordinate (x, y) place, and i gets 3 ~ 5 usually.Image, through the process of embossment mapping algorithm, also has certain anti-light photograph changing capability; Embossment algorithm carries out process of convolution to each point of image to adopt following matrix to carry out:

$\left[\begin{array}{ccc}1& 0& 0\\ 0& 0& 0\\ 0& 0& -1\end{array}\right]$

For coordinate (i, i) point, the algorithm of its anaglyph figure is:

Y(i，j)=X(i-1，j-1)-X(i-1，j+1)+128

Wherein, X (i, j) and Y (i, j) is respectively the original pixel value of (i, j) coordinate points and the pixel value after converting.

2. utilize frame difference image, embossment changing image, original-gray image combines as the input source image of mixed Gaussian background modeling, sets up probabilistic model, carries out the detection of foreground target.The theoretical foundation of the method is sturdy, and can add priori, Detection results is good.

The basic thought of mixed Gaussian background modeling is that the color that each pixel presents is represented by K state, and usual K gets between 3-5.The pixel value obtaining video image at each moment T is the sampled value of stochastic variable X.Gauss model has three parameters, is respectively average μ ^kvariances sigma ^k, weights omega ^k, l≤k≤K.。

K the weights being distributed in moment t can upgrade with following formula:

The more new formula of weights:

${\mathrm{\ω}}_{t+1}^k=(1-\mathrm{\α}){\mathrm{\ω}}_t^k+\mathrm{\α}\left(M_t^k\right)$

Model modification formula is:

${\mathrm{\μ}}_{t+1}^k=(1-\mathrm{\α}){\mathrm{\μ}}_t^k+\mathrm{\α}{X}_t$

${{\mathrm{\σ}}_{t+1}^k}^2=(1-\mathrm{\α}){\mathrm{\σ}}_t^{k^2}+\mathrm{\α}(X_t-{\mathrm{\μ}}_t^k{)}^2$

Wherein, α is turnover rate, 0 < α <, 1l k lK, when the model of the 1st Satisfying Matching Conditions is k, and M ^k(x, y)-_ 1, otherwise M ^k(x, y)=0.

When the model number of a pixel is k, and during k>1, to this k model according to priority size sort, priority computing formula is when mating, mate from the model that priority is maximum, if the model of first Satisfying Matching Conditions is k, then namely k puts the Matching Model in this moment for this, does not need the Model Matching little with priority ratio k again.

Through context update and the study of limited frame, set up a background model.

$B_0(x,y)=\frac{1}{m}\underset{i=1}{\overset{m}{\mathrm{\&Sigma;}}}{M}_i^K(x,y)$

Highest priority is nursed: the present invention mainly develops according to the fixed target patrolled on waypoint location, edge between double detection and graded determine the degree changed between twice detection, produce alarm when the intensity of variation between twice detection is greater than the threshold value of setting.

The present invention adopts Canny convolution operator to carry out computing to image, adopts local maximum policy filtering to fall most of non-edge point.

$s_x=\left[\begin{array}{cc}-1& 1\\ -1& 1\end{array}\right],s_y=\left[\begin{array}{cc}1& 1\\ -1& -1\end{array}\right]$

Its x to, y to first-order partial derivative matrix, the mathematic(al) representation of gradient magnitude and gradient direction is:

P [ij]=(f [i, j+1]-f [i, j]+f [i+1, j 10]-f [i, j])/2

Q[i，j]＝(f[i，j]-f[i+1，j]+f[i，j+1]-f[i+1，j+1])/2

$M[i,j]=\sqrt{P[i,j{]}^2+Q[i,j{]}^2}$

θ[i，j]＝arctan(Q[i，j]/P[i，j])

The gradient magnitude of M [i, j] presentation video at coordinate [i, j] place in above formula, θ [i, j] represents the gradient direction at coordinate [i, j] place.

Pedestrian/vehicle detection: for 360 degree of cruise modes, the present invention proposes the target that a kind of method that optical flow field relative motion and Hog+SVM model training combine extracts specified type, in the detection of pedestrian and vehicle, achieve good Detection results, and be widely used in unattended project.HOG is exactly the one abbreviation of finger direction histogram of gradients (Histogram of Oriented Gradient, HOG), is a kind of profiler being used for carrying out object detection in computer vision and image procossing.It carrys out constitutive characteristic by the gradient orientation histogram of calculating and statistical picture regional area; Its main thought is in a sub-picture, and the presentation of localized target and shape can be described well by the direction density distribution at gradient or edge.Its concrete implementation method is: first image is divided into little connected region, we are cell factory it.Then gather the gradient of each pixel in cell factory or edge direction histogram.Finally altogether just can constitutive characteristic describer these set of histograms.

SVM is the abbreviation of SVMs (Support Vector Machine), that Corinna Cortes and Vapnik8 equals nineteen ninety-five and first propose, it shows many distinctive advantages in solution small sample, non-linear and high dimensional pattern identification, and can promote the use of in the other machines problems concerning study such as Function Fitting.Support vector machine method is that the VC being based upon Statistical Learning Theory ties up on theoretical and Structural risk minization basis, between the complexity (namely to the study precision of specific training sample) and learning ability (namely identifying the ability of arbitrary sample error-free) of model, optimal compromise is sought, in the hope of obtaining best Generalization Ability according to limited sample information.

Carry out Hog+SVM training and detect dividing following step:

1). collect training sample, comprise a large amount of positive samples and negative sample.Manual cutting sample, unification zooms to fixed size.

2). extract the feature of all positive samples and negative sample respectively.

3). give sample label to all positive negative samples, positive sample labeling is 1, and negative sample is labeled as-1.

4). by the Hog feature of positive negative sample, positive and negative sample label, is input in Linear SVM grader and trains.

5). utilize the grader trained to detect the target in scene.

Two-way optical flow method: optical flow method concept is derived from optical flow field, the video pattern further from the teeth outwards of moving object is exactly so-called optical flow field, is a two-dimension speed field.If I is (x, y, t) be the pixel value of picture point (x, y) at moment t, if u is (x, y) with v (x, y) be x and the y component of this light stream, postulated point pixel value when t+ δ t moves to (x+ δ xly+ δ y) remains unchanged, δ x=u δ t, δ y=v δ t, then have optical flow equation:

I(x+uδt，y+uδt，t+δt)＝Ｉ（x，y，t)

The corresponding relation that the correlation between change and consecutive frame utilizing pixel in image sequence in time-domain exists between present frame to find previous frame to follow, thus a kind of method optical flow method calculating the movable information of object between consecutive frame actual be by the intensity of detected image pixel over time and then infer the method in object translational speed and direction.

The candidate target of Hog+SVM detection of classifier is carried out to the calculating in bi-directional light flow field, the people and Che etc. that have relative motion can be detected in scene more accurately.Improve accuracy of detection and reduce flase drop.

Target tracking module: target following is exactly analyze the interesting target detected in time domain, obtain target state parameter as change in location movement locus time/spatial feature, to carry out next step Treatment Analysis, as behavioural analysis etc.The two-way optical flow method that the present invention mentions above using carries out target following, effectively can utilize the information in the Time domain and Space territory of moving target, thus make to follow the tracks of accurate stable more, to some extent solve the collision separation between the internal object of visual field and occlusion issue.

Comparatively speaking, the general unobstructed problem of the method based on model, but be difficult to set up a general template (as deforming template).How to define in addition coupling measure make tracking be more accurately again a great problem.

Target's feature-extraction module: the target detected previous frame sets up the joint histogram masterplate based on color and HOG feature, and this joint histogram combines the Gradient Features of color characteristic and HOG, can be good at than more completely describing target signature information.

Feature detection matching module: carry out search coupling at present frame, compared by Pasteur's distance (a kind of method of measuring histogram difference), namely around the target location of previous frame, within the scope of certain radius, carry out search coupling (finding that radius is that 20 pixel effects are relatively good in our practice), find best matched position to be the position of possible target at present frame.

Event checking module: for judging whether generation event according to the target location change detected.

Step 4: as shown in Figure 4, after configuring all cruise parameter and the intelligent video analysis algorithm that cruise a little, start up system, by the analysis controlling list of cruising, automatic generation is cruised execution list, makes Pan/Tilt/Zoom camera between each cruises a little, carry out detection of cruising according to the order of cruising preset.And system call video-splicing module, generates the panoramic mosaic figure in whole cycle of cruising automatically according to the execution sequence cruised a little.What have overlapping region completes splicing automatically, splicing according to sequencing of zero lap region.

Video automatic Mosaic algorithm: the image mosaic technology of distinguished point based, as the core technology of video-splicing technology, is mainly divided into coupling two step of feature point extraction and characteristic point.

The present invention adopts Moravec operator to carry out feature point extraction, its basic thought is, the grey scale change situation of this pixel and vicinity points is represented with minimal gray variance on four of pixel Main way, the i.e. interest value of pixel, then selects the point (the obvious point of grey scale change) with maximum interest value as characteristic point in the local of image.

$V_1=\underset{i=-k}{\overset{k}{\mathrm{\&Sigma;}}}(g_{c+i,r}-g_{c+i+1,r}{)}^2$

$V_2=\underset{i=-k}{\overset{k}{\mathrm{\&Sigma;}}}(g_{c+i,r+i}-g_{c+i+1,r+i+1}{)}^2$

$V_3=\underset{i=-k}{\overset{k}{\mathrm{\&Sigma;}}}(g_{c,r+i}-g_{c,r+i+1}{)}^2$

$V_4=\underset{i=-k}{\overset{k}{\mathrm{\&Sigma;}}}(g_{c+i,r-i}-g_{c+i+1,r-i+1}{)}^2$

Wherein g _{c+i, r}presentation video is at the gray value at coordinate [c+i, r] place, and by that analogy, getting wherein reckling is the interest value of pixel IV (c, r):

IV(c，r)＝V＝min{V ₁，V ₂，V ₃，V ₄}

According to given threshold value, interest value is selected to be greater than the candidate point of point as characteristic point of this threshold value.If V _tfor the threshold value set in advance, if V > is V _t, then V is the candidate point of characteristic point.Local modulus maxima is chosen as the characteristic point needed in candidate point.

On the basis of the feature point extraction more than having had, feature based point matching algorithm key step is as follows:

(1) in the lap of reference picture T, choose 4 regions, each region utilizes Moravec operator to find out characteristic point.

(2) choose the region centered by characteristic point, the present invention of this region select size be 7 × 7 region, in search graph S, find the most similar coupling.Because there are 4 characteristic points, therefore there are 4 characteristic areas, find the coupling of corresponding characteristic area also to have 4 pieces.

(3) utilize the central point of these the 4 groups characteristic areas mated, namely 4 to the characteristic point of mating, and substitute into following equation and solve, required solution is the conversion coefficient between two width images.

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\\ 1\end{array}\right]=M\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{m}_0& m_1& m_2\\ m_3& m_4& m_5\\ m_6& m_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]$

Step 5:

System control module performs the relevant video camera open function cruised a little, then according to the algorithm configuration that each in algorithm configuration list cruises a little, enables the relevant video analysis algorithm cruised a little, namely deploy troops on garrison duty to the algorithm that each cruises a little.

Video analysis algorithm carries out target detection and event detection according to relevant setting, and produces real-time alarm to the time detected, carries out alarm and corresponding video recording, and grabgraf carries out this locality storage and upload to Surveillance center reminding Surveillance center to carry out analyzing and processing.Control platform receives video analysis result, issues various management and control order according to analysis result.

Be more than the detailed description of carrying out the preferred embodiments of the present invention, but those of ordinary skill in the art it should be appreciated that within the scope of the present invention, and guided by the spirit, various improvement, interpolation and replacement are all possible.These are all in the protection range that claim of the present invention limits.

Claims (12)

1. based on the intelligent video analysis system that Pan/Tilt/Zoom camera cruises, this system comprises front end Pan/Tilt/Zoom camera and back-end server, it is characterized in that back-end server comprises:

Cruise configuration module, and cruising a presetting bit of list, a corresponding Pan/Tilt/Zoom camera that cruises of each group of cruising for a little carrying out setting generation to the group and cruising of cruising of system, is cruising some configuration cruise mode and a cruise time of each group of cruising;

Pan/Tilt/Zoom camera control module, to cruising, list is analyzed, and automatically generates execution list of cruising, and makes Pan/Tilt/Zoom camera between each presetting bit, carry out detection of cruising according to the order of cruising preset;

Video analysis configuration module, the intelligent video analysis algorithm relevant for configuration of cruising for each is also configured to and cruises in list;

System control module, perform the relevant video camera open function cruised a little, according to the algorithm configuration that each in configured list cruises a little, enable the relevant video analysis algorithm cruised a little, for each cruising a little is carried out camera parameters demarcation and call video-splicing module, automatically generate the panoramic mosaic figure in whole cycle of cruising according to the execution sequence cruised a little;

Intelligent video analysis module, carries out target detection and event analysis according to relevant setting, and produces Real-time Alarm to the event detected;

Alarm management module, carries out corresponding local management function to alarm.

2. system according to claim 1, is characterized in that:

It is exactly all or part of parameter being gone out projection matrix P by given reference substance backwards calculation that described camera parameters is demarcated, after demarcation, two-dimensional coordinate in the two dimensional image of being caught at camera by it, and the projection matrix P obtained, can ask the positional information of certain target in three-dimensional.

3. system according to claim 1, is characterized in that described intelligent video analysis module comprises further:

Image pre-processing module, adopt the self adaptation rapid image noise reduction algorithm of wavelet transformation to carry out filtering noise reduction to image, greyscale transformation operates;

Module of target detection, for carrying out moving object detection, clarification of objective is extracted, pedestrian/vehicle detection, face/car plate detection and location, and carries out Target Recognition Algorithms according to clarification of objective;

Target tracking module, utilizes two-way optical flow method to carry out target following;

Target's feature-extraction module, the target detected previous frame sets up the joint histogram masterplate based on color and HOG feature, and this joint histogram combines the Gradient Features of color characteristic and HOG;

Feature detection matching module, is carried out search coupling at present frame, is compared by Pasteur's distance, around the target location of previous frame, namely carries out searching for coupling within the scope of certain radius find best matched position to be the position of possible target at present frame;

Event checking module, judges whether generation event based on the target location change detected.

4. system according to claim 3, is characterized in that described module of target detection is, image conversion poor based on frame, the triplicity of mixed Gaussian probabilistic model detects target.

5. system according to claim 3, is characterized in that: a kind of method that described pedestrian/vehicle detection adopts optical flow field relative motion and Hog+SVM model training to combine extracts the target of specified type.

6. system according to claim 1, it is characterized in that: described video-splicing module is configured to further: the coupling of carrying out feature point extraction and characteristic point, be specially: represent the grey scale change situation of this pixel and vicinity points with minimal gray variance on four of pixel Main way, the i.e. interest value of pixel, then select the point with maximum interest value as characteristic point in the local of image, 4 regions are chosen in the lap of reference picture, each region utilizes Moravec operator to find out characteristic point, choose the region of the fixed size centered by characteristic point, the most similar coupling is found in search graph, utilize the central point of the characteristic area of coupling, substitute into following equation to solve, required solution is the conversion coefficient M between two width images:

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\\ 1\end{array}\right]=M\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{m}_0& m_1& m_2\\ m_3& m_4& m_5\\ m_6& m_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right].$

7., based on the intelligent video analysis method that Pan/Tilt/Zoom camera cruises, it is characterized in that:

Step (1) first carries out a little setting the group and cruising of cruising of system, generates list of cruising, a presetting bit of a corresponding Pan/Tilt/Zoom camera that cruises of each group of cruising, and is cruising some configuration cruise mode and a cruise time of each group of cruising;

Step (2) is called by presetting bit, Pan/Tilt/Zoom camera is moved to and cruises accordingly a little, system control module is that each cruising a little carries out camera parameters demarcation for current scene, and configures relevant intelligent video analysis algorithm by video analysis configuration module and add to and cruise in list;

After step (3) start up system, Pan/Tilt/Zoom camera control module, by the analysis to list of cruising, generates execution list of cruising automatically, makes Pan/Tilt/Zoom camera between each presetting bit, carry out detection of cruising according to the order of cruising preset;

Step (4) system control module calls video-splicing module, automatically generates the panoramic mosaic figure in whole cycle of cruising according to the execution sequence cruised a little;

Step (5) intelligent video analysis module carries out target detection and event analysis according to relevant setting, and produces real-time alarm to the event detected.

8. method according to claim 7, is characterized in that:

In described step (2), camera parameters is demarcated is exactly all or part of parameter being gone out projection matrix P by given reference substance backwards calculation, after demarcation, two-dimensional coordinate in the two dimensional image of being caught at camera by it, projection matrix P with obtaining, can ask the positional information of certain target in three-dimensional.

9. method according to claim 7, is characterized in that described step (5) comprises further:

Adopt the self adaptation rapid image noise reduction algorithm of wavelet transformation to carry out filtering noise reduction to image, greyscale transformation operates;

Carry out moving object detection, clarification of objective is extracted, pedestrian/vehicle detection, face/car plate detection and location, and carries out Target Recognition Algorithms according to clarification of objective;

Two-way optical flow method is utilized to carry out target following;

The target detected previous frame sets up the joint histogram masterplate based on color and HOG feature, and this joint histogram combines the Gradient Features of color characteristic and HOG;

Carry out search coupling at present frame, compared by Pasteur's distance, around the target location of previous frame, namely within the scope of certain radius, carry out searching for coupling find best matched position to be the position of possible target at present frame;

Generation event is judged whether based on the target location change detected.

10. method according to claim 9, is characterized in that described target detection is, image conversion poor based on frame, the triplicity of mixed Gaussian probabilistic model detects target.

11. methods according to claim 9, is characterized in that: a kind of method that described pedestrian/vehicle detection adopts optical flow field relative motion and Hog+SVM model training to combine extracts the target of specified type.

12. methods according to claim 7, it is characterized in that: described video-splicing specifically comprises: represent the grey scale change situation of this pixel and vicinity points with minimal gray variance on four of pixel Main way, the i.e. interest value of pixel, then select the point with maximum interest value as characteristic point in the local of image, 4 regions are chosen in the lap of reference picture, each region utilizes Moravec operator to find out characteristic point, choose the region of the fixed size centered by characteristic point, the most similar coupling is found in search graph, utilize the central point of the characteristic area of coupling, substitute into following equation to solve, required solution is the conversion coefficient M between two width images:

$\left[\begin{array}{c}{x}^{\&prime;}\\ y^{\&prime;}\\ 1\end{array}\right]=M\left[\begin{array}{c}x\\ y\\ 1\end{array}\right]=\left[\begin{array}{ccc}{m}_0& m_1& m_2\\ m_3& m_4& m_5\\ m_6& m_7& 1\end{array}\right]\left[\begin{array}{c}x\\ y\\ 1\end{array}\right].$