CN105447458B - A kind of large-scale crowd video analytic system and method - Google Patents

A kind of large-scale crowd video analytic system and method Download PDF

Info

Publication number
CN105447458B
CN105447458B CN201510791068.0A CN201510791068A CN105447458B CN 105447458 B CN105447458 B CN 105447458B CN 201510791068 A CN201510791068 A CN 201510791068A CN 105447458 B CN105447458 B CN 105447458B
Authority
CN
China
Prior art keywords
crowd
event
scene
module
msub
Prior art date
Application number
CN201510791068.0A
Other languages
Chinese (zh)
Other versions
CN105447458A (en
Inventor
彭彬绪
张帆
杨延生
佘忠华
张广程
郝景山
Original Assignee
深圳市商汤科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市商汤科技有限公司 filed Critical 深圳市商汤科技有限公司
Priority to CN201510791068.0A priority Critical patent/CN105447458B/en
Publication of CN105447458A publication Critical patent/CN105447458A/en
Application granted granted Critical
Publication of CN105447458B publication Critical patent/CN105447458B/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00335Recognising movements or behaviour, e.g. recognition of gestures, dynamic facial expressions; Lip-reading
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06KRECOGNITION OF DATA; PRESENTATION OF DATA; RECORD CARRIERS; HANDLING RECORD CARRIERS
    • G06K9/00Methods or arrangements for reading or recognising printed or written characters or for recognising patterns, e.g. fingerprints
    • G06K9/00624Recognising scenes, i.e. recognition of a whole field of perception; recognising scene-specific objects
    • G06K9/00711Recognising video content, e.g. extracting audiovisual features from movies, extracting representative key-frames, discriminating news vs. sport content
    • G06K9/00718Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G06K9/00724Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items of sport video content

Abstract

This disclosure relates to a kind of large-scale crowd video analytic system and method, the system includes crowd density computing module, crowd's foreground segmentation module, crowd's tracking module, crowd state analysis module, event determination module;Wherein:The crowd density computing module, crowd's foreground segmentation module, crowd's tracking module respectively obtain crowd's quantity, crowd region, crowd movement direction and speed after vedio data is handled;The crowd state analysis module carries out Treatment Analysis based on obtained crowd's quantity, crowd region, crowd movement direction and speed, and analysis result is sent into event determination module;The event determination module is used to judge whether crowd's event is abnormal;Crowd's event refers to that in monitor area certain population characteristic sexual behaviour occurs in the crowd for reaching certain scale.The disclosure can help monitoring personnel to complete the tasks such as intelligent trend prediction, characteristic event positioning, be that the offers such as accident prevents, suspicious clue is traced effectively help.

Description

A kind of large-scale crowd video analytic system and method

Technical field

This disclosure relates to computer vision field, particularly a kind of large-scale crowd video analytic system and method.

Background technology

Intensive place, such as railway station, bus station, cinema, large supermarket etc. are easier in crowd, is often occurred The injures and deaths event caused by crowd tramples.Although the existing video monitoring system in China has possessed a certain degree of scale, this A little video monitoring systems play an important role in occurred events of public safety and public security guarantee is timely responded to, and various video monitoring systems System is widely used in various scenes, but monitoring by artificial monitoring, needs exploration badly and research intelligence regards mostly now Frequency is monitored for security protection, is helped monitoring personnel to complete the tasks such as intelligent trend prediction, characteristic event positioning, is that accident is pre- Anti-, suspicious clue the offer such as is traced and effectively helped.

The content of the invention

For above-mentioned subproblem, present disclose provides a kind of large-scale crowd video analytic system and method.

A kind of large-scale crowd video analytic system, the system include crowd density computing module, crowd's foreground segmentation Module, crowd's tracking module, crowd state analysis module, event determination module;Wherein:

The crowd density computing module covers field range for obtaining certain two field picture in monitor video view data Interior crowd's quantity;

Crowd's foreground segmentation module covers field range for obtaining certain two field picture in monitor video view data After interior foreground and background segmentation, the crowd region in prospect;

Crowd's tracking module be used for obtain certain two field picture in monitor video view data cover it is within the vision Crowd movement direction and speed;

The crowd state analysis module is entered based on obtained crowd's quantity, crowd region, crowd movement direction and speed Row Treatment Analysis, and analysis result is sent into event determination module;

The event determination module is used to judge whether crowd's event is abnormal;

Crowd's event refers to that in monitor area certain population characteristic row occurs in the crowd for reaching certain scale For.

A kind of method for realizing large-scale crowd video analytic system, methods described comprise the steps:

The disclosure can help monitoring personnel to complete the tasks such as intelligent trend prediction, characteristic event positioning, be accident Prevention, suspicious clue the offer such as are traced and effectively helped.

Brief description of the drawings

On large-scale crowd video analytic system data processing flow diagram in Fig. 1 disclosure one embodiment;

The schematic network structure of complete convolutional neural networks in one embodiment of Fig. 2 disclosure systems;

The schematic network structure of complete convolutional neural networks in one embodiment of Fig. 3 method of disclosure.

Embodiment

A kind of large-scale crowd video analytic system is provided in a basic embodiment, the system includes crowd Density Calculation Module, crowd's foreground segmentation module, crowd's tracking module, crowd state analysis module, event determination module;Its In:

The crowd density computing module covers field range for obtaining certain two field picture in monitor video view data Interior crowd's quantity;

Crowd's foreground segmentation module covers field range for obtaining certain two field picture in monitor video view data After interior foreground and background segmentation, the crowd region in prospect;

Crowd's tracking module be used for obtain certain two field picture in monitor video view data cover it is within the vision Crowd movement direction and speed;

The crowd state analysis module is entered based on obtained crowd's quantity, crowd region, crowd movement direction and speed Row Treatment Analysis, and analysis result is sent into event determination module;

The event determination module is used to judge whether crowd's event is abnormal;

Crowd's event refers to that in monitor area certain population characteristic row occurs in the crowd for reaching certain scale For.

In this embodiment, the system can be that monitoring personnel completes intelligent trend prediction, characteristic event positioning etc. Task provides data and supported, is that the offers such as accident prevents, suspicious clue is traced effectively help.Described image data can be straight It is the complete video interception of a frame or processed into the view data for facilitating corresponding module to be further processed to connect, It can also be and be compressed the view data after processing for convenience of transmission, then carry out phase in the module for receiving the view data Decompression answered, etc..In crowd state analysis module, its Treatment Analysis carried out is mainly based upon obtained crowd Quantity, crowd region, crowd movement direction and speed are come to analyze crowd's event residing for the crowd for determining monitor area be what Crowd's event of type, and then the conclusion of crowd's event is sent to event determination module, by event determination module according to The judgment rule of formulation judges crowd's event with the presence or absence of abnormal.The type of crowd's event may be because of different system pair The classification of crowd's event is different and different.

In one embodiment, there is provided crowd density computing module calculates the preferred computation model of crowd's quantity, i.e.,:Institute State crowd density computing module and use depth convolutional neural networks (DCNN, Deep Convolutional Neural Network) model carries out crowd density estimation and then obtains crowd's quantity.

The learning objective of the depth convolutional neural networks model is mapping F:X → D, wherein X represent image in training set Pixel characteristic, D represent crowd density image.The model has following features:Have on study crowd characteristic with good Effect property and robustness, it is not necessary to extra mark work, and be independent with display foreground segmentation, therefore can more be closed The result of calculation of reason.The model is used in crowd density computing module, reliable data can be provided for follow-up analysis and supported, And then when abnormal crowd's event occurs, help can be provided for the decision-making of supervisor.

In one embodiment, there is provided the preferred parted pattern of crowd's foreground segmentation module, i.e.,:Crowd's prospect point Module is cut using complete convolutional neural networks (FCNN, Fully Convolutional Neural Network) model to calculate Crowd region.

The relatively conventional convolutional neural networks of complete convolutional neural networks, the situation of picture dimensional variation can be applied to, Using more flexible and convenient.

In one embodiment, there is provided the method in the preferred calculating crowd movement direction and speed of crowd's tracking module, I.e.:Crowd's tracking module obtains crowd movement direction and speed using KLT algorithms (Kanade-Lucas-Tomasi).

It is time-consuming that although the problem of carrying out moving object segmentation using optical flow method essentially consists in most of optical flow methods calculating, in real time Property and practicality are all poor, but are that light stream not only carries the movable information of moving object the advantages of optical flow method, but also The abundant information about scenery three-dimensional structure is carried, it can be detected in the case where not knowing any information of scene Moving Objects.

In one embodiment, crowd density image mapping expression formula is embodied as one by the crowd density computing module Core cell, i.e.,:The crowd density computing module includes crowd density image map unit, the crowd density image mapping The expression formula of unit is:

Wherein:

Di(p) it is the point p around i-th of people in density image D;

Z is normalized parameter;

PiFor the people of i-th of mark;

NhIt is the normalized 2D gaussian kernel functions as head model;

PhFor the position on head;

σhFor NhVariance;

NbIt is the two-variable normal distribution as body model;

PbFor the position of human body;

∑ is NbCovariance matrix.

Above-mentioned mapping can ensure that all density values are equal to crowd's number in artwork on the whole in a density mappings Amount.

In one embodiment, the population characteristic sexual behaviour for producing crowd's event is divided, further clearly originally The open system crowd behaviour to be monitored, i.e.,:The population characteristic sexual behaviour include the crowd is dense, crowd massing, crowd be detained, Crowd runs to drive in the wrong direction with crowd;

It is described the crowd is dense refer to the number average value in past T1 seconds monitor area exceed set parameter threshold;

The crowd massing refers to the number stopped in monitor area in the crowd region to flock together and exceedes setting Threshold value;

The crowd is detained to refer to be tracked to the crowd characteristic point in monitoring image, the motion of the crowd characteristic point away from From the threshold value for being less than setting in time T2;

The crowd, which runs, to be referred to the quantity of people of the movement velocity more than threshold speed V in monitor area and exceedes amount threshold N1;

The quantity of the people for the retrograde direction motion of entry region interior edge that the crowd drives in the wrong direction in finger monitor area exceedes threshold value N2;

Wherein, T1, T2, V, N1 and N2 are preset value or the self-defining value that can be changed.

According to the division of above-mentioned population characteristic sexual behaviour, the disclosure crowd's event to be monitored is including the crowd is dense, crowd Aggregation, crowd are detained, crowd runs and driven in the wrong direction with crowd, and then can be everyone group in systems in order to identify crowd's event Event distributes event id.

Further, by the embodiment, how the event judge module being also implied by system judges that crowd's event is different Often.

For the crowd is dense event, the event determination module to number average value in T past, inner region second by carrying out Statistics is come, if more than the parameter threshold of setting, judges that the crowd is dense in region;Wherein T be preset value or can change from Definition value.In one embodiment, the T is 10 seconds.

For crowd massing event, the event determination module passes through to stopping the crowd to flock together in image Number in region is counted, if it exceeds setting threshold value, then judges abnormal aggregation occurs in region.

It is detained event for crowd, the event determination module to the crowd characteristic point in image by being tracked;One The move distance shorter explanation residence time is more long in the section time, when the residence time exceeding the threshold value set, then illustrates region Inside there is crowd's delay.

Run event for crowd, the event determination module by calculating the movement velocity of tracking characteristics point under scene, The number of characteristic point of the movement velocity more than threshold value T1 is counted, when feature point number is more than threshold value T2, shows to occur under scene Exception is run;Wherein T1, T2 are preset value or the self-defining value that can be changed.

Driven in the wrong direction event for crowd, the event determination module passes through entry region interior edge and driven in the wrong direction direction motion characteristics point Number counted, if it exceeds threshold value is N number of, then judge that region occurs to drive in the wrong direction;Wherein N be preset value or can change from Definition value, such as N are 10.

More preferably, the system determine in one section of event continuous monitoring to crowd's event whether be same crowd's thing The method of part, i.e.,:Certain behavior in the population characteristic sexual behaviour is when continuous several times occur, if between the adjacent time twice Every more than setting time, then adjacent this kind of behavior occurring twice is considered as event twice, otherwise it is adjacent occur twice should Kind behavior is considered as an event.Such as in the crowd is dense event, if if adjacent the crowd is dense twice event interval More than 60 seconds, then it is determined as event twice.In crowd massing event, if the crowd is dense the twice event in phase overlay area is such as Fruit interval is more than 60 seconds, then is determined as event twice.In crowd's delay event, if the delay thing occurred in identical retention areas If part was separated by 10 seconds, it is determined as same delay event.In crowd runs event, if adjacent crowd twice runs If event interval is more than 60 seconds, it is determined as event twice.In crowd drives in the wrong direction event, if identical retrograde region it is adjacent two If secondary crowd drives in the wrong direction event, interval is more than 60 seconds, is determined as event twice.

In one embodiment, the system also includes real-time monitoring module, and the real-time monitoring module is used to reflect and worked as Preceding time point, certain all the way corresponding to video in scene crowd situation, the dense degree of crowd is shown by way of diagram. Wherein, the figure of the mode institute foundation of the diagram can be based on the image in monitor video.

Preferably, the figure used in the real-time monitoring module includes crowd and stops colored graph and Crowds Distribute thermodynamic chart; The crowd stops colored graph according to residence time length, and Jet dyeing is carried out to the crowd in scene;The Crowds Distribute heating power Figure shows the real-time distribution situation of crowd in scene according to crowd's density in the form of thermodynamic chart.

Further, the Jet dyeing is maximum to be dyed according to personage's residence time length in scene to target Value shows as red, and minimum value shows as blueness.Wherein:Blueness is personage on the move, and yellow is to have short stay behavior Personage, as personage's residence time is increasingly longer, its color also can increasingly convergence it is red.

This is that is, blueness is personage on the move, and yellow is to have the personage of short stay behavior, with the residence time It is increasingly longer, its color also can increasingly convergence it is red.By this mode, monitoring personnel can find out scene at a glance Crowd's residence time length of interior regional, the crowd for the stop that notes abnormalities in time.

Further, the redder region of the color of the Crowds Distribute thermodynamic chart shows that crowd density herein is higher;Face Color is as the reduction of crowd density is from red to blue gradual change.

In one embodiment, the system also includes trend discovery module, and the trend discovery module passes through diagram Mode comes the region that positioning scene servant's clustering is held together.Wherein, the figure of the mode institute foundation of the diagram can be regarded based on monitoring Image in frequency.By trend discovery module monitoring personnel being helped to understand the detailed event of a certain passage, a situation arises, finds Potential rule and aid decision making.After a certain passage is selected, system is i.e. according to time, two, space dimension on this passage A situation arises carries out analysis displaying for event.Monitoring personnel can pass through page turning mode Switch Video channel selecting window.

Preferably, the figure used in the trend discovery module includes crowd's quantity line chart and crowd is evenly distributed heating power Figure;Crowd's quantity line chart represents the number of the crowd within a period of time, including peak value number and average number;Superintendent Member can also select in 1 hour as needed, in 1 day, time granularity is checked in 7 days etc..The crowd is evenly distributed heating power Figure illustrates using seclected time section as dimension to the situation that is evenly distributed of crowd, and according to different color regions, can hold very much Easily navigate to and the region that people's clustering is disturbed easily occurs under scene.

Further, the crowd is evenly distributed the red area of thermodynamic chart and shows to hold together region herein for people's clustering;Color As the reduction of number in a period of time is from red to blue gradual change.

In one embodiment, the system, which also includes the system, also includes event statistics module, the event statistics Module is based on the nearest W weeks Frequency of a certain crowd's anomalous event in periodically temporal projection, to check that event occurs to exist Periodically temporal rule;Wherein W is preset value or the self-defining value that can be changed.For example W is 4.

Preferably, the event statistics module includes event query unit, and the event query unit can be to each monitoring The event occurred in video is traced and positioned, such as by selecting video channel, event type, the period occurred, The event occurred can be inquired.

Preferably, the Query Result of the event query unit is believed with the word of the sectional drawing combination corresponding event of historical events The form of breath is shown.In the displaying result of one embodiment, the sectional drawing except showing historical events, crucial letter can be also shown The time of breath, such as event generation, aggregation number.

Optionally, the Query Result comprises at least key frame sectional drawing, and the key frame passes through to the whole of the event Forming process is tracked to obtain.In one embodiment, can be opened when selection during some event inquired in new window The details of corresponding event.Details are made a summary by key message, key frame sectional drawing forms, and wherein key frame is judged by event Module is tracked to obtain to whole event forming process.Monitoring personnel can understand that event occurs by checking key frame Beginning, climax, the whole generating process terminated, have basic insight to the overall condition of event.

In one embodiment, the event determination module provides early warning it is determined that during crowd's event anomalies.

Optionally, the form of the early warning includes the combining form of following a kind of or any various ways:Static text Word, pattern or dynamic word, dynamic pattern, sound.Such as the crowd is dense event, by calculating crowd's saturation come reaction density feelings Condition.Crowd's saturation therein can be according to real-time scene number N and scene number capacity T ratio r=N/T:It is divided into 5 etc. Level:Sparse (crowd's saturation degree is 0~30%), normal (crowd's saturation degree is 30~60%), it is higher (crowd's saturation degree 60~ 80%), exceeded (crowd's saturation degree is 80~120%), supersaturation (crowd's saturation degree is higher than 120%).When crowd's saturation degree is low When 60%, for green;When crowd's saturation degree 60~80% is yellow, when crowd's saturation degree is shown in red higher than 80%, Real-time display saturation degree simultaneously.Three Estate can be divided into according to the number of current aggregator for crowd massing event:Normally (20 people Below), merit attention (below 50 people), abnormal aggregation (more than 50 people), crowd's event of running extremely can be according to the play run Strong degree is divided into 5 grades, in addition to color tips, while shows grade of running.Delay event can be according to crowd with being detained Time length be divided into three different degree of meriting attention, while real-time display holdup time length.Retrograde event only has two Individual state-event:Occur with not occurring.So contribute to monitoring personnel according to the order of severity of event corresponding measure can be taken to enter Row is intervened.

In one embodiment, the system also includes video acquisition and decoder module, is regarded for what is shot to camera Frequency image is acquired and decoded.

In one embodiment, monitoring personnel is quick, finds 5 class anomalous events in time to help, and the camera is clapped The video image taken the photograph is showed in the form of real-time panel;The panel shows content in the matrix form, and each of which is classified as one Individual video channel, and the picture of the first behavior camera of matrix shooting, remaining each behavior one kind crowd's event.Passing through will The picture of camera shooting is shown, helps intuitively to see the actual conditions of scene in real time.

In one embodiment, panel is shown according to crowd's event class.Crowd's event class point includes face Plate directly displays saturation degree, panel directly displays crowd's incident duration, the number divided rank according to group of participants's event With without associated ratings.For example in the crowd is dense event, event class is that panel directly displays saturation degree, is shown as more than 80% Orange warning, it is red alarm event more than 100%.In crowd massing event, event class is to show that aggregation is held in panel The continuous time.In crowd's delay event, event class is that Display panel is detained incident duration.In crowd runs event, Event class is the relation according to feature point number on image and threshold value T2, will run and is divided into 5 grades, the T2 is number. Crowd drives in the wrong direction in event, and event class is that, when there is crowd's event, the direct use of panel can distinguish nobody without associated ratings Color when group's event occurs is identified.

Optionally, the panel is automatically by the video channel number progress split screen displaying that display is set in a screen.This is easy to Can effectively it be realized to the extending transversely of video channel number to being monitored while multi-channel video, while by this mode, system.This Outside, optionally, also support to be used with the full frame mode for fully supporting whole monitoring display device, facilitate monitoring personnel quickly to find crucial letter Breath.

Optionally, the video channel can carry out the monitoring parameter setting of scene, and the monitoring parameter of the scene is set Initialize installation including scene, and with the related parameter setting of early warning;The Initialize installation of the scene includes specifying Global response hot-zone in a certain video channel scene, mode are to choose a polygon in schematic picture by mouse to make For the region of population analysis;It is described be included in the related parameter setting of early warning in schematic picture choose it is one or more polygon Shape region is come to be monitored and/or set a direction be response direction or to be not responding to direction.

Optionally, the Initialize installation of the scene also includes demarcating scene, and its mode is:Choose at least two Height 170cm or so adult;If the pedestrian without the condition that meets in scene, need to reselect the image for demarcation.One In individual embodiment, the adult for choosing 2 height 170cm is demarcated.In one embodiment, 2 height 175cm are chosen Adult demarcated.

Optionally, the Initialize installation of the scene also includes demarcating scene, and its mode is:According to people in difference Different scale size under far and near scene, is estimated under same level coordinate, 100cm is mapped in image in actual scene Pixel count.

In one embodiment, rectangle is marked by mouse to select the people chosen in demarcation just frame, and it is on the scene according to people The far and near different scale size of scape, is estimated under same level coordinate, one meter of pixel being mapped in image in actual scene Number.

By demarcation, it is estimated that the parameter such as the real area of scene, gradient, suggestion galleryful.

In one embodiment, generation perspective matrix, the perspective matrix are mainly used to normalize scene after demarcation The size of different far and near crowds.At least two adult is randomly chosen from the single-frame images of scene, if fewer than 2, is then changed One uncalibrated image, and assume that everyone average height is 175cm, estimate one using a linear regression model (LRM) Perspective matrix M.Pixel value M (p) in perspective matrix M represents in pixel p opening position actual scene 1 meter of shared picture in the picture Prime number amount.

In one embodiment, video channel can be increased, and be regarded by inserting monitoring at the interface of addition video channel Frequency address, you can complete the connection to new video passage.

Optionally, all existing video channels are shown with tabular form.

In one embodiment, the event determination module is additionally operable to the data write into Databasce of crowd's event.

The disclosure is illustrated below in conjunction with the accompanying drawings.

In one embodiment, as shown in Figure 1, there is provided a kind of large-scale crowd video analytic system, within the system Including camera and front end display screen, the image that camera gathers is acquired by reconciliation by video acquisition and deciphering module Code, and decoded image is delivered separately to crowd density computing module, crowd's foreground segmentation module and crowd's tracking module. Wherein, the crowd density computing module carries out crowd density estimation using depth convolutional neural networks model, finally gives people Group's quantity;Crowd's foreground segmentation module estimated the monitor area in image using complete convolutional network model, most Crowd region is obtained eventually;Crowd's tracking module is tracked using KLT optical flow methods, finally gives crowd movement direction and speed. Obtained crowd's quantity, crowd region and crowd movement direction and speed are delivered to crowd state analysis module, in the mould The data based on reception are subjected to crowd's event analysis in block, determine monitor area there occurs what crowd's event, and should Crowd's event passes to event judge module, for whether further determining that crowd's event there occurs exception, if producing exception, Front end early warning is then provided on the display screen of front end, while warning information write into Databasce is stored;No matter but crowd Whether event occurs exception, and the event determination module all the data of crowd's event is stored in into database in case inquiry etc. Use.

In one embodiment, the schematic network structure of the complete convolutional neural networks used in the system is as schemed Shown in 2, the figure is once to train calculating process or target identification process.In figure, first layer P0 is artwork, and the second layer is original Scheme the sub-regions P0 ' in P0, followed by 3 layers of convolutional layer, parameter 7*7 therein, 5*5 represent the size of convolution kernel.From The binary file blob for the subgraph P0 ' that P0 is obtained dimension is 72 × 72 × 3, after first convolutional layer conv1 is handled Obtained binary file blob dimension is the result 72 × 72 × 32 of a 3-dimensional, is handled by second convolutional layer conv2 To the result 36 × 36 × 32 of individual 3-dimensional, the result 18 × 18 × 64 of a 3-dimensional is obtained by the 3rd convolutional layer conv3 processing, this In pond layer has been lain in convolutional layer, it can be seen that the image size that convolutional layer obtains all be preceding layer half.The layer Enter two layers of full linking layer afterwards, fc4 and fc5 is two layers of full linking layer, and fc4 is 1000 × 1 result, and fc5 is 400 × 1 As a result, fc6 is the output of regression target, and fc7 is the output of classifying type target, last Denisty Map and Gobal number The calibration result of original image, manually provide mark and it is generated, network generation fc6 (324 × 1) result can be by figure As rows of density images arranged as (18 × 18), and with demarcating the density_map counting loss concentrated, for training Network;The result that fc7 is 1 × 1 contrasts with gobal number is lost, and can be used for training network.As needed may be used To be switched between the two of training pattern convergence target, convergence target loss loss measurements, the loss's Computational methods are as follows:

Loss=fc7 result-globa number

Another convergence target loses Eucldiean loss measurements with Euler, and the Euler loses Eucldiean loss Computational methods it is as follows:

Eucldiean loss=fc6 result-density map

When loss is unsatisfactory for requiring, this process of iteration.

In one embodiment, there is provided the method for realizing said system, i.e.,:A kind of large-scale crowd video analysis Method, methods described comprise the steps:

S100, calculating parameter crowd quantity, crowd region, crowd movement direction and speed:

Calculate certain two field picture in monitor video view data and cover crowd's quantity within the vision;

Calculate certain two field picture in monitor video view data cover foreground and background within the vision split after, prospect In crowd region;

Calculate certain two field picture in monitor video view data and cover crowd movement direction within the vision and speed;

S200, analysis judgement crowd's event category:Based on obtain crowd's quantity, crowd region, crowd movement direction with Speed carries out Treatment Analysis, and judges crowd's event generic;

Crowd's event refers to that in monitor area certain population characteristic row occurs in the crowd for reaching certain scale For.

In this embodiment, can be that monitoring personnel completes intelligent trend prediction, characteristic event etc. by methods described Task provides data and supported, is that the offers such as accident prevents, suspicious clue is traced effectively help.Described image data can be straight It is the complete video interception of a frame or processed into the view data for facilitating corresponding module to be further processed to connect, It can also be and be compressed the view data after processing for convenience of transmission, then carried out after the view data is received corresponding Decompression, etc..Crowd's quantity that the analysis of the step S200 is mainly based upon to obtain, crowd region, crowd movement Direction and speed are based on to analyze crowd's event residing for the crowd for determining monitor area be what kind of crowd's event The analytical conclusions of crowd's event carry out event type judgement.When carrying out event type judgement, preferred sentencing by formulation Disconnected rule judges crowd's event with the presence or absence of abnormal.The type of crowd's event may be because different system be to crowd's event Classification it is different and different.

In one embodiment, there is provided calculate the preferred computation model of crowd's quantity, i.e.,:Crowd's quantity is obtaining Further it is calculated on the basis of crowd density, the crowd density is by using depth convolutional neural networks (DCNN, Deep Convolutional Neural Network) model estimated.

The learning objective of the depth convolutional neural networks model is mapping F:X → D, wherein X represent image in training set Pixel characteristic, D represent crowd density image.The model has following features:Have on study crowd characteristic with good Effect property and robustness, it is not necessary to extra mark work, and be independent with display foreground segmentation, therefore can more be closed The result of calculation of reason.The model is used when crowd density calculates, reliable data can be provided for follow-up analysis and supported, entered And when abnormal crowd's event occurs, help can be provided for the decision-making of supervisor.

In one embodiment, there is provided a kind of preferred parted pattern of carry out crowd's foreground segmentation, can by the model Acquisition crowd region, i.e.,:The crowd region is by using complete convolutional neural networks (FCNN, Fully Convolutional Neural Network) model is calculated.

The relatively conventional convolutional neural networks of complete convolutional neural networks, the situation of picture dimensional variation can be applied to, Using more flexible and convenient.

In one embodiment, there is provided the preferred method for calculating crowd movement direction and speed, i.e.,:

The crowd movement direction is calculated with speed by using KLT algorithms (Kanade-Lucas-Tomasi).

It is time-consuming that although the problem of carrying out moving object segmentation using optical flow method essentially consists in most of optical flow methods calculating, in real time Property and practicality are all poor, but are that light stream not only carries the movable information of moving object the advantages of optical flow method, but also The abundant information about scenery three-dimensional structure is carried, it can be detected in the case where not knowing any information of scene Moving Objects.

Further, specific crowd density computational methods are provided in one embodiment, i.e.,:The crowd density leads to Crowd density image is crossed to reflect, the expression formula of the crowd density image is:

Wherein:

Di(p) it is the point p around i-th of people in density image D;

Z is normalized parameter;

PiFor the people of i-th of mark;

NhIt is the normalized 2D gaussian kernel functions as head model;

PhFor the position on head;

σhFor NhVariance;

NbIt is the two-variable normal distribution as body model;

PbFor the position of human body;

∑ is NbCovariance matrix.

Above-mentioned mapping can ensure that all density values are equal to crowd's number in artwork on the whole in a density mappings Amount.

In one embodiment, the population characteristic sexual behaviour for producing crowd's event is divided, further clearly made The crowd behaviour of crowd regulation is carried out with method of disclosure, i.e.,:

The population characteristic sexual behaviour includes that the crowd is dense, crowd massing, crowd is detained, crowd runs and driven in the wrong direction with crowd;

It is described the crowd is dense refer to the number average value in past T1 seconds monitor area exceed set parameter threshold;

The crowd massing refers to the number stopped in monitor area in the crowd region to flock together and exceedes setting Threshold value;

The crowd is detained to refer to be tracked to the crowd characteristic point in monitoring image, the motion of the crowd characteristic point away from From the threshold value for being less than setting in time T2;

The crowd, which runs, to be referred to the quantity of people of the movement velocity more than threshold speed V in monitor area and exceedes amount threshold N1;

The quantity of the people for the retrograde direction motion of entry region interior edge that the crowd drives in the wrong direction in finger monitor area exceedes threshold value N2;

Wherein, T1, T2, V, N1 and N2 are preset value or the self-defining value that can be changed.

According to the division of above-mentioned population characteristic sexual behaviour, the disclosure crowd's event to be monitored is including the crowd is dense, crowd Aggregation, crowd are detained, crowd runs and driven in the wrong direction with crowd, and then can be everyone group's event increase to identify crowd's event Event id.

Further, by the embodiment, also it is implied by method of disclosure and how judges crowd's event anomalies.

For the crowd is dense event, by being counted to number average value in T past, inner region second come if more than setting Parameter threshold, then judge that the crowd is dense in region;Wherein T is preset value or the self-defining value that can be changed.In an implementation In example, the T is 10 seconds.

For crowd massing event, by being carried out to stopping the number in the crowd region to flock together in image Statistics, if it exceeds setting threshold value, then judge abnormal aggregation occurs in region.

It is detained event for crowd, by being tracked to the crowd characteristic point in image;Move distance in a period of time The shorter explanation residence time is more long, when the residence time exceeding the threshold value set, then illustrates there is crowd's delay in region.

Run event for crowd, by calculating the movement velocity of tracking characteristics point under scene, statistics movement velocity is more than The number of threshold value T1 characteristic point, when feature point number is more than threshold value T2, show to run there occurs abnormal under scene;Wherein T1, T2 are preset value or the self-defining value that can be changed.

Drive in the wrong direction event for crowd, counted by the drive in the wrong direction number of direction motion characteristics point of entry region interior edge, If it exceeds threshold value is N number of, then judge that region occurs to drive in the wrong direction;Wherein N is preset value or the self-defining value that can change, for example N is 10。

More preferably, methods described further provide in one section of event continuous monitoring to crowd's event whether be same The judgment principle of crowd's event, i.e.,:Certain behavior in the population characteristic sexual behaviour when continuous several times occur, if adjacent two Secondary time interval is more than setting time, then adjacent this kind of behavior occurring twice is considered as event twice, and otherwise adjacent two This kind of behavior of secondary appearance is considered as an event.Such as in the crowd is dense event, if adjacent the crowd is dense twice thing If part interval is more than 60 seconds, it is determined as event twice.In crowd massing event, if the crowd twice in phase overlay area If intensive event interval is more than 60 seconds, it is determined as event twice.In crowd's delay event, if being sent out in identical retention areas If raw delay event was separated by 10 seconds, it is determined as same delay event.In crowd runs event, if adjacent two If secondary crowd runs, event interval is more than 60 seconds, is determined as event twice.In crowd drives in the wrong direction event, if identical retrograde area If the adjacent crowd twice in domain drives in the wrong direction event, interval is more than 60 seconds, is determined as event twice.

In one embodiment, in order to reflect in real time in current point in time, certain crowd in scene corresponding to video all the way Situation, after step S200, methods described also includes:

S300, the dense degree of real-time display crowd:Certain for reflecting current point in time by way of diagram regards all the way Corresponding to frequency in scene crowd situation.

Wherein, the figure of the mode institute foundation of the diagram can be based on the image in monitor video.

Preferably, the figure used in the dense degree of the real-time display crowd includes crowd's stop colored graph and crowd It is distributed thermodynamic chart;The crowd stops colored graph according to residence time length, and Jet dyeing is carried out to the crowd in scene;It is described Crowds Distribute thermodynamic chart shows the real-time distribution situation of crowd in scene according to crowd's density in the form of thermodynamic chart.

Further, the Jet dyeing is maximum to be dyed according to personage's residence time length in scene to target Value shows as red, and minimum value shows as blueness.Wherein:Blueness is personage on the move, and yellow is to have short stay behavior Personage, as personage's residence time is increasingly longer, its color also can increasingly convergence it is red.

This is that is, blueness is personage on the move, and yellow is to have the personage of short stay behavior, with the residence time It is increasingly longer, its color also can increasingly convergence it is red.By this mode, monitoring personnel can find out scene at a glance Crowd's residence time length of interior regional, the crowd for the stop that notes abnormalities in time.

Further, the redder region of the color of the Crowds Distribute thermodynamic chart shows that crowd density herein is higher;Face Color is as the reduction of crowd density is from red to blue gradual change.

In one embodiment, the region held together in order to facilitate positioning scene servant's clustering, after step S200, the side Method also includes:

The region that S400, positioning scene servant's clustering are held together:The area held together by way of diagram come positioning scene servant's clustering Domain.

Wherein, the figure of the mode institute foundation of the diagram can be based on the image in monitor video.Found by trend Module can help the detailed event of a certain passage of monitoring personnel understanding, and a situation arises, finds potential rule and aid decision making.Selecting After fixed a certain passage, i.e. according to time, two, space dimension, to the event on this passage, a situation arises carries out analysis exhibition for system Show.Monitoring personnel can pass through page turning mode Switch Video channel selecting window.

Preferably, the figure used in region that the positioning scene servant clustering is held together includes crowd's quantity line chart and people Group mean is distributed thermodynamic chart;Crowd's quantity line chart represents the number of the crowd within a period of time, including peak value number and Average number;Supervisor can also select in 1 hour as needed, in 1 day, time granularity is checked in 7 days etc..The people Group mean is distributed thermodynamic chart using seclected time section as dimension, the situation that is evenly distributed of crowd is illustrated, according to different face Color region, it can be easy to navigate to the region that easily generation people's clustering is disturbed under scene.

Further, the crowd is evenly distributed the red area of thermodynamic chart and shows to hold together region herein for people's clustering;Color As the reduction of number in a period of time is from red to blue gradual change.

In one embodiment, checked to carry out statistics to various crowd's anomalous events, to find rule, the step After rapid S200, methods described also includes:

S500, statistical phenomeon:Based on the nearest W weeks Frequency of a certain crowd's anomalous event in periodically temporal projection Counted, to check that event occurs in periodically temporal rule;Wherein W is preset value or can changed self-defined Value.For example W is 4.

Preferably, the step S500 is before statistics, in addition to step:

S50001, query event:The event occurred in each monitor video is inquired about., can be to each by inquiry The event occurred in monitor video is traced and positioned, such as by selecting video channel, event type, the time occurred Section, you can inquire the event occurred.Here inquiry, except applied to statistics, when method is implemented as into system, also may be used For being provided out search function, for retrieving all groups' event.

Preferably, the Query Result is shown in the form of the text information of the sectional drawing combination corresponding event of historical events. In the displaying result of one embodiment, the sectional drawing except showing historical events, key message can be also shown, as event occurs Time, aggregation number etc..

Optionally, the Query Result comprises at least key frame sectional drawing, and the key frame passes through to the whole of the event Forming process is tracked to obtain.In one embodiment, can be opened when selection during some event inquired in new window The details of corresponding event.Details are made a summary by key message, key frame sectional drawing forms, and wherein key frame is judged by event Module is tracked to obtain to whole event forming process.Monitoring personnel can understand that event occurs by checking key frame Beginning, climax, the whole generating process terminated, have basic insight to the overall condition of event.

In one embodiment, the step S200 also includes after crowd's event generic is judged:

S201, provide early warning:Corresponding early warning is provided to the crowd's event having determined that.

Optionally, the form of the early warning includes the combining form of following a kind of or any various ways:Static text Word, pattern or dynamic word, dynamic pattern, sound.Such as the crowd is dense event, by calculating crowd's saturation come reaction density feelings Condition.Crowd's saturation therein can be according to real-time scene number N and scene number capacity T ratio r=N/T:It is divided into 5 etc. Level:Sparse (crowd's saturation degree is 0~30%), normal (crowd's saturation degree is 30~60%), it is higher (crowd's saturation degree 60~ 80%), exceeded (crowd's saturation degree is 80~120%), supersaturation (crowd's saturation degree is higher than 120%).When crowd's saturation degree is low When 60%, for green;When crowd's saturation degree 60~80% is yellow, when crowd's saturation degree is shown in red higher than 80%, Real-time display saturation degree simultaneously.Three Estate can be divided into according to the number of current aggregator for crowd massing event:Normally (20 people Below), merit attention (below 50 people), abnormal aggregation (more than 50 people), crowd's event of running extremely can be according to the play run Strong degree is divided into 5 grades, in addition to color tips, while shows grade of running.Delay event can be according to crowd with being detained Time length be divided into three different degree of meriting attention, while real-time display holdup time length.Retrograde event only has two Individual state-event:Occur with not occurring.So contribute to monitoring personnel according to the order of severity of event corresponding measure can be taken to enter Row is intervened.

In one embodiment, before step S100, methods described also includes:

S000, the video image to camera shooting are acquired and decoded.

In one embodiment, monitoring personnel is quick, finds 5 class anomalous events in time to help, and the camera is clapped The video image taken the photograph is showed in the form of real-time panel;The panel shows content in the matrix form, and each of which is classified as one Individual video channel, and the picture of the first behavior camera of matrix shooting, remaining each behavior one kind crowd's event.Passing through will The picture of camera shooting is shown, helps intuitively to see the actual conditions of scene in real time.

In one embodiment, panel is shown according to crowd's event class.Crowd's event class point includes face Plate directly displays saturation degree, panel directly displays crowd's incident duration, the number divided rank according to group of participants's event With without associated ratings.For example in the crowd is dense event, event class is that panel directly displays saturation degree, is shown as more than 80% Orange warning, it is red alarm event more than 100%.In crowd massing event, event class is to show that aggregation is held in panel The continuous time.In crowd's delay event, event class is that Display panel is detained incident duration.In crowd runs event, Event class is the relation according to feature point number on image and threshold value T2, will run and is divided into 5 grades, the T2 is number. Crowd drives in the wrong direction in event, and event class is that, when there is crowd's event, the direct use of panel can distinguish nobody without associated ratings Color when group's event occurs is identified.

Optionally, the panel is automatically by the video channel number progress split screen displaying that display is set in a screen.This is easy to Can effectively it be realized to the extending transversely of video channel number to being monitored while multi-channel video, while by this mode, system.This Outside, optionally, also support to be used with the full frame mode for fully supporting whole monitoring display device, facilitate monitoring personnel quickly to find crucial letter Breath.

Optionally, the video channel can carry out the monitoring parameter setting of scene, and the monitoring parameter of the scene is set Initialize installation including scene, and with the related parameter setting of early warning;The Initialize installation of the scene includes specifying Global response hot-zone in a certain video channel scene, mode are to choose a polygon in schematic picture by mouse to make For the region of population analysis;It is described be included in the related parameter setting of early warning in schematic picture choose it is one or more polygon Shape region is come to be monitored and/or set a direction be response direction or to be not responding to direction.

Optionally, the Initialize installation of the scene also includes demarcating scene, and its mode is:Choose at least two Height 170cm or so adult;If the pedestrian without the condition that meets in scene, need to reselect the image for demarcation.One In individual embodiment, the adult for choosing 2 height 170cm is demarcated.In one embodiment, 2 height 175cm are chosen Adult demarcated.

Optionally, the Initialize installation of the scene also includes demarcating scene, and its mode is:According to people in difference Different scale size under far and near scene, is estimated under same level coordinate, 100cm is mapped in image in actual scene Pixel count.

In one embodiment, rectangle is marked by mouse to select the people chosen in demarcation just frame, and it is on the scene according to people The far and near different scale size of scape, is estimated under same level coordinate, one meter of pixel being mapped in image in actual scene Number.

By demarcation, it is estimated that the parameter such as the real area of scene, gradient, suggestion galleryful.

In one embodiment, generation perspective matrix, the perspective matrix are mainly used to normalize scene after demarcation The size of different far and near crowds.At least two adult is randomly chosen from the single-frame images of scene, if fewer than 2, is then changed One uncalibrated image, and assume that everyone average height is 175cm, estimate one using a linear regression model (LRM) Perspective matrix M.Pixel value M (p) in perspective matrix M represents in pixel p opening position actual scene 1 meter of shared picture in the picture Prime number amount.

In one embodiment, video channel can be increased, and be regarded by inserting monitoring at the interface of addition video channel Frequency address, you can complete the connection to new video passage.

Optionally, all existing video channels are shown with tabular form.

In one embodiment, the step S200 also includes after crowd's event generic is judged:S202, by people The data write into Databasce of group's event.The execution of this step can also can carry before early warning is provided providing early warning After showing, parallel work-flow can also be.

The disclosure is illustrated below in conjunction with the accompanying drawings.

In one embodiment, the schematic network structure of the complete convolutional neural networks used in the process is as schemed Shown in 3, the figure is once to train calculating process or target identification process.In figure, first layer P0 is artwork, and the second layer is original Scheme the sub-regions P0 ' in P0, followed by 3 layers of convolutional layer, parameter 7*7 therein, 5*5 represent the size of convolution kernel.From The binary file blob for the subgraph P0 ' that P0 is obtained dimension is 72 × 72 × 3, after first convolutional layer conv1 is handled Obtained binary file blob dimension is the result 72 × 72 × 32 of a 3-dimensional, is handled by second convolutional layer conv2 To the result 36 × 36 × 32 of individual 3-dimensional, the result 18 × 18 × 64 of a 3-dimensional is obtained by the 3rd convolutional layer conv3 processing, this In pond layer has been lain in convolutional layer, it can be seen that the image size that convolutional layer obtains all be preceding layer half.The layer Enter two layers of full linking layer afterwards, fc4 and fc5 is two layers of full linking layer, and fc4 is 1000 × 1 result, and fc5 is 400 × 1 As a result, fc6 is the output of regression target, and fc7 is the output of classifying type target, last Denisty Map and Gobal number The calibration result of original image, manually provide mark and it is generated, network generation fc6 (324 × 1) result can be by figure As rows of density images arranged as (18 × 18), and with demarcating the density_map counting loss concentrated, for training Network;The result that fc7 is 1 × 1 contrasts with gobal number is lost, and can be used for training network.As needed may be used To be switched between the two of training pattern convergence target, convergence target loss loss measurements, the loss's Computational methods are as follows:

Loss=fc7 result-globa number

Another convergence target loses Eucldiean loss measurements with Euler, and the Euler loses Eucldiean loss Computational methods it is as follows:

Eucldiean loss=fc6 result-density map

When loss is unsatisfactory for requiring, this process of iteration.

The disclosure is described in detail above, used herein principle and embodiment party of the specific case to the disclosure Formula is set forth, and the explanation of above example is only intended to help the system and its core concept for understanding the disclosure;It is meanwhile right In those skilled in the art, according to the thought of the disclosure, there will be changes, comprehensive in specific embodiments and applications Upper described, this specification content should not be construed as the limitation to the disclosure.

Claims (44)

  1. A kind of 1. large-scale crowd video analytic system, it is characterised in that:
    The system includes crowd density computing module, crowd's foreground segmentation module, crowd's tracking module, crowd state analysis mould Block, event determination module;Wherein:
    The crowd density computing module be used for obtain certain two field picture in monitor video view data cover it is within the vision Crowd's quantity;
    Crowd's foreground segmentation module be used for obtain certain two field picture in monitor video view data cover it is within the vision Foreground and background segmentation after, the crowd region in prospect;
    Crowd's tracking module covers crowd within the vision for obtaining certain two field picture in monitor video view data The direction of motion and speed;
    The crowd state analysis module based on obtained crowd's quantity, crowd region, crowd movement direction and speed at Reason analysis, and analysis result is sent into event determination module;
    The event determination module is used to judge whether crowd's event is abnormal;
    Crowd's event refers to that in monitor area certain population characteristic sexual behaviour occurs in the crowd for reaching certain scale;
    The population characteristic sexual behaviour include the crowd is dense, crowd massing, crowd drive in the wrong direction and crowd runs;
    It is described the crowd is dense refer to the number average value in past T1 seconds monitor area exceed set parameter threshold;
    The crowd massing refers to the number stopped in monitor area in the crowd region to flock together and exceedes setting threshold value;
    The quantity of the people for the retrograde direction motion of entry region interior edge that the crowd drives in the wrong direction in finger monitor area exceedes threshold value N2;
    The crowd, which runs, to be referred to the quantity of people of the movement velocity more than threshold speed V in monitor area and exceedes amount threshold N1;
    Wherein, T1, V, N1 and N2 are preset value or the self-defining value that can be changed.
  2. 2. system according to claim 1, it is characterised in that:
    The crowd density computing module uses depth convolutional neural networks (DCNN, Deep Convolutional Neural Network) model carries out crowd density estimation and then obtains crowd's quantity.
  3. 3. system according to claim 1, it is characterised in that:
    Crowd's foreground segmentation module uses complete convolutional neural networks (FCNN, Fully Convolutional Neural Network) model calculates crowd region.
  4. 4. system according to claim 1, it is characterised in that:
    Crowd's tracking module obtains crowd movement direction and speed using KLT algorithms (Kanade-Lucas-Tomasi).
  5. 5. system according to claim 2, it is characterised in that:
    The crowd density computing module includes crowd density image map unit, the table of the crowd density image map unit It is up to formula:
    <mrow> <msub> <mi>D</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>P</mi> <mo>&amp;Element;</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> </mrow> </munder> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <mo>|</mo> <mi>Z</mi> <mo>|</mo> <mo>|</mo> </mrow> </mfrac> <mrow> <mo>(</mo> <mo>-</mo> <msub> <mi>N</mi> <mi>h</mi> </msub> <mo>(</mo> <mrow> <mi>p</mi> <mo>;</mo> <msub> <mi>P</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>&amp;sigma;</mi> <mi>h</mi> </msub> </mrow> <mo>)</mo> <mo>+</mo> <msub> <mi>N</mi> <mi>b</mi> </msub> <mo>(</mo> <mrow> <mi>p</mi> <mo>;</mo> <msub> <mi>P</mi> <mi>b</mi> </msub> <mo>,</mo> <mi>&amp;Sigma;</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> </mrow>
    Wherein:
    Di(p) it is the point p around i-th of people in density image D;
    Z is normalized parameter;
    PiFor the people of i-th of mark;
    NhIt is the normalized 2D gaussian kernel functions as head model;
    PhFor the position on head;
    σhFor NhVariance;
    NbIt is the two-variable normal distribution as body model;
    PbFor the position of human body;
    ∑ is NbCovariance matrix.
  6. 6. system according to claim 1, it is characterised in that:
    The population characteristic sexual behaviour also includes crowd and is detained;
    The crowd is detained to refer to be tracked to the crowd characteristic point in monitoring image, and the move distance of the crowd characteristic point exists Less than the threshold value set in time T2;
    Wherein, T2 is preset value or the self-defining value that can be changed.
  7. 7. according to any described system of claim 1~6, it is characterised in that:
    The system also includes real-time monitoring module, and the real-time monitoring module is used to reflect current point in time, certain video all the way The situation of crowd in corresponding scene, the dense degree of crowd is shown by way of diagram.
  8. 8. according to any described system of claim 1~6, it is characterised in that:
    The system also includes trend discovery module, and the trend discovery module is by way of diagram come crowd under positioning scene The region gathered.
  9. 9. according to any described system of claim 1~6, it is characterised in that:
    The system also includes event statistics module, and the event statistics module is based on a certain crowd's anomalous event and sent out for nearest W weeks Raw frequency is in periodically temporal projection, to check that event occurs in periodically temporal rule;
    Wherein W is preset value or the self-defining value that can be changed.
  10. 10. according to any described system of claim 1~6, it is characterised in that:
    The event determination module provides early warning it is determined that during crowd's event anomalies.
  11. 11. according to any described system of claim 1~6, it is characterised in that:
    The system also includes video acquisition and decoder module, for being acquired reconciliation to the video image that camera is shot Code.
  12. 12. according to any described system of claim 1~6, it is characterised in that:
    The event determination module is additionally operable to the data write into Databasce of crowd's event.
  13. 13. system according to claim 7, it is characterised in that:
    Figure used in the real-time monitoring module includes crowd and stops colored graph and Crowds Distribute thermodynamic chart;
    The crowd stops colored graph according to residence time length, and Jet dyeing is carried out to the crowd in scene;
    The Crowds Distribute thermodynamic chart shows the real-time distribution feelings of crowd in scene according to crowd's density in the form of thermodynamic chart Condition.
  14. 14. system according to claim 8, it is characterised in that:
    Figure used in the trend discovery module includes crowd's quantity line chart and crowd is evenly distributed thermodynamic chart;
    Crowd's quantity line chart represents the number of the crowd within a period of time, including peak value number and average number;
    The crowd is evenly distributed thermodynamic chart using seclected time section as dimension, and the situation that is evenly distributed of crowd is illustrated.
  15. 15. system according to claim 9, it is characterised in that:
    The event statistics module includes event query unit, and the event query unit can be to having occurred in each monitor video Event traced and positioned.
  16. 16. system according to claim 10, it is characterised in that:
    The form of the early warning includes the combining form of following a kind of or any various ways:Static text, pattern or dynamic State word, dynamic pattern, sound.
  17. 17. system according to claim 11, it is characterised in that:
    The video image of the camera shooting is showed in the form of real-time panel;
    The panel shows content in the matrix form, and each of which is classified as a video channel, and the first behavior of matrix is taken the photograph As the picture that head is shot, remaining each behavior one kind crowd's event.
  18. 18. system according to claim 15, it is characterised in that:
    The Query Result of the event query unit comprises at least key frame sectional drawing, and the key frame passes through to the whole of the event Individual forming process is tracked to obtain.
  19. 19. system according to claim 17, it is characterised in that:
    The panel is automatically by the video channel number progress split screen displaying that display is set in a screen.
  20. 20. system according to claim 19, it is characterised in that:
    The monitoring parameter that the video channel can carry out scene is set, and the monitoring parameter of the scene, which is set, includes the first of scene Beginningization set, and with the related parameter setting of early warning;
    The Initialize installation of the scene includes specifying the global response hot-zone in a certain video channel scene, and mode is to pass through Mouse chooses region of the polygon as population analysis in schematic picture;
    It is described to be included in the interior one or more polygonal regions of selection of schematic picture with the related parameter setting of early warning to carry out Monitoring and/or setting a direction are response direction or are to be not responding to direction.
  21. 21. system according to claim 20, it is characterised in that:
    The Initialize installation of the scene also includes demarcating scene, and its mode is:It is left to choose at least two height 170cm Right adult;If the pedestrian without the condition that meets in scene, need to reselect the image for demarcation.
  22. 22. system according to claim 20, it is characterised in that:
    The Initialize installation of the scene also includes demarcating scene, and its mode is:According to people under different far and near scenes Different scale size, estimate under same level coordinate, 100cm is mapped to the pixel count in image in actual scene.
  23. A kind of 23. large-scale crowd video analysis method, it is characterised in that methods described comprises the steps:
    S100, calculating parameter crowd quantity, crowd region, crowd movement direction and speed:
    Calculate certain two field picture in monitor video view data and cover crowd's quantity within the vision;
    Calculate certain two field picture in monitor video view data cover foreground and background within the vision split after, in prospect Crowd region;
    Calculate certain two field picture in monitor video view data and cover crowd movement direction within the vision and speed;
    S200, analysis judgement crowd's event category:Based on obtained crowd's quantity, crowd region, crowd movement direction and speed Treatment Analysis is carried out, and judges crowd's event generic;
    Crowd's event refers to that in monitor area certain population characteristic sexual behaviour occurs in the crowd for reaching certain scale;
    The population characteristic sexual behaviour include the crowd is dense, crowd massing, crowd drive in the wrong direction and crowd runs;
    It is described the crowd is dense refer to the number average value in past T1 seconds monitor area exceed set parameter threshold;
    The crowd massing refers to the number stopped in monitor area in the crowd region to flock together and exceedes setting threshold value;
    The quantity of the people for the retrograde direction motion of entry region interior edge that the crowd drives in the wrong direction in finger monitor area exceedes threshold value N2;
    The crowd, which runs, to be referred to the quantity of people of the movement velocity more than threshold speed V in monitor area and exceedes amount threshold N1;
    Wherein, T1, V, N1 and N2 are preset value or the self-defining value that can be changed.
  24. 24. according to the method for claim 23, it is characterised in that:
    Crowd's quantity is further calculated on the basis of crowd density is obtained, and the crowd density is by using depth Convolutional neural networks (DCNN, Deep Convolutional Neural Network) models is estimated.
  25. 25. according to the method for claim 23, it is characterised in that:
    The crowd region is by using complete convolutional neural networks (FCNN, Fully Convolutional Neural Network) model is calculated.
  26. 26. according to the method for claim 23, it is characterised in that:
    The crowd movement direction is calculated with speed by using KLT algorithms (Kanade-Lucas-Tomasi).
  27. 27. according to the method for claim 24, it is characterised in that:
    The crowd density reflects that the expression formula of the crowd density image is by crowd density image:
    <mrow> <msub> <mi>D</mi> <mi>i</mi> </msub> <mrow> <mo>(</mo> <mi>p</mi> <mo>)</mo> </mrow> <mo>=</mo> <munder> <mo>&amp;Sigma;</mo> <mrow> <mi>P</mi> <mo>&amp;Element;</mo> <msub> <mi>P</mi> <mi>i</mi> </msub> </mrow> </munder> <mfrac> <mn>1</mn> <mrow> <mo>|</mo> <mo>|</mo> <mi>Z</mi> <mo>|</mo> <mo>|</mo> </mrow> </mfrac> <mrow> <mo>(</mo> <mo>-</mo> <msub> <mi>N</mi> <mi>h</mi> </msub> <mo>(</mo> <mrow> <mi>p</mi> <mo>;</mo> <msub> <mi>P</mi> <mi>h</mi> </msub> <mo>,</mo> <msub> <mi>&amp;sigma;</mi> <mi>h</mi> </msub> </mrow> <mo>)</mo> <mo>+</mo> <msub> <mi>N</mi> <mi>b</mi> </msub> <mo>(</mo> <mrow> <mi>p</mi> <mo>;</mo> <msub> <mi>P</mi> <mi>b</mi> </msub> <mo>,</mo> <mi>&amp;Sigma;</mi> </mrow> <mo>)</mo> <mo>)</mo> </mrow> </mrow>
    Wherein:
    Di(p) it is the point p around i-th of people in density image D;
    Z is normalized parameter;
    PiFor the people of i-th of mark;
    NhIt is the normalized 2D gaussian kernel functions as head model;
    PhFor the position on head;
    σhFor NhVariance;
    NbIt is the two-variable normal distribution as body model;
    PbFor the position of human body;
    ∑ is NbCovariance matrix.
  28. 28. according to the method for claim 23, it is characterised in that:
    The population characteristic sexual behaviour also includes crowd and is detained;
    The crowd is detained to refer to be tracked to the crowd characteristic point in monitoring image, and the move distance of the crowd characteristic point exists Less than the threshold value set in time T2;
    Wherein, T2 is preset value or the self-defining value that can be changed.
  29. 29. according to any described method of claim 23~28, it is characterised in that after step S200, methods described is also Including:
    S300, the dense degree of real-time display crowd:Reflect certain video institute all the way of current point in time by way of diagram The situation of crowd in corresponding scene.
  30. 30. according to any described method of claim 23~28, it is characterised in that after step S200, methods described is also Including:
    The region that S400, positioning scene servant's clustering are held together:The region held together by way of diagram come positioning scene servant's clustering.
  31. 31. according to any described method of claim 23~28, it is characterised in that after the step S200, methods described Also include:
    S500, statistical phenomeon:Carried out based on the nearest W weeks Frequency of a certain crowd's anomalous event in periodically temporal projection Statistics, to check that event occurs in periodically temporal rule;
    Wherein W is preset value or the self-defining value that can be changed.
  32. 32. according to any described method of claim 23~28, it is characterised in that the step S200 is judging crowd's event Also include after generic:
    S201, provide early warning:Corresponding early warning is provided to the crowd's event having determined that.
  33. 33. according to any described method of claim 23~28, it is characterised in that before step S100, methods described is also Including:
    S000, the video image to camera shooting are acquired and decoded.
  34. 34. according to any described method of claim 23~28, it is characterised in that the step S200 is judging crowd's event Also include after generic:
    S202, the data write into Databasce by crowd's event.
  35. 35. according to the method for claim 29, it is characterised in that:
    Figure used in the dense degree of the real-time display crowd includes crowd and stops colored graph and Crowds Distribute thermodynamic chart;
    The crowd stops colored graph according to residence time length, and Jet dyeing is carried out to the crowd in scene;
    The Crowds Distribute thermodynamic chart shows the real-time distribution feelings of crowd in scene according to crowd's density in the form of thermodynamic chart Condition.
  36. 36. according to the method for claim 30, it is characterised in that:
    The figure used in region that the positioning scene servant clustering is held together includes crowd's quantity line chart and crowd is evenly distributed Thermodynamic chart;
    Crowd's quantity line chart represents the number of the crowd within a period of time, including peak value number and average number;
    The crowd is evenly distributed thermodynamic chart using seclected time section as dimension, and the situation that is evenly distributed of crowd is illustrated.
  37. 37. according to the method for claim 31, it is characterised in that the step S500 is before statistics, in addition to step:
    S50001, query event:The event occurred in each monitor video is inquired about.
  38. 38. according to the method for claim 32, it is characterised in that:
    The form of the early warning includes the combining form of following a kind of or any various ways:Static text, pattern or dynamic State word, dynamic pattern, sound.
  39. 39. according to the method for claim 33, it is characterised in that:
    The video image of the camera shooting is showed in the form of real-time panel;
    The panel shows content in the matrix form, and each of which is classified as a video channel, and the first behavior of matrix is taken the photograph As the picture that head is shot, remaining each behavior one kind crowd's event.
  40. 40. according to the method for claim 37, it is characterised in that:
    The Query Result of the query event comprises at least key frame sectional drawing, and the key frame passes through the whole shape to the event It is tracked to obtain into process.
  41. 41. according to the method for claim 39, it is characterised in that:
    The panel is automatically by the video channel number progress split screen displaying that display is set in a screen.
  42. 42. according to the method for claim 41, it is characterised in that:
    The monitoring parameter that the video channel can carry out scene is set, and the monitoring parameter of the scene, which is set, includes the first of scene Beginningization set, and with the related parameter setting of early warning;
    The Initialize installation of the scene includes specifying the global response hot-zone in a certain video channel scene, and mode is to pass through Mouse chooses region of the polygon as population analysis in schematic picture;
    It is described to be included in the interior one or more polygonal regions of selection of schematic picture with the related parameter setting of early warning to carry out Monitoring and/or setting a direction are response direction or are to be not responding to direction.
  43. 43. according to the method for claim 42, it is characterised in that:
    The Initialize installation of the scene also includes demarcating scene, and its mode is:It is left to choose at least two height 170cm Right adult;If the pedestrian without the condition that meets in scene, need to reselect the image for demarcation.
  44. 44. according to the method for claim 42, it is characterised in that:
    The Initialize installation of the scene also includes demarcating scene, and its mode is:According to people under different far and near scenes Different scale size, estimate under same level coordinate, 100cm is mapped to the pixel count in image in actual scene.
CN201510791068.0A 2015-11-17 2015-11-17 A kind of large-scale crowd video analytic system and method CN105447458B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201510791068.0A CN105447458B (en) 2015-11-17 2015-11-17 A kind of large-scale crowd video analytic system and method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201510791068.0A CN105447458B (en) 2015-11-17 2015-11-17 A kind of large-scale crowd video analytic system and method

Publications (2)

Publication Number Publication Date
CN105447458A CN105447458A (en) 2016-03-30
CN105447458B true CN105447458B (en) 2018-02-27

Family

ID=55557618

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201510791068.0A CN105447458B (en) 2015-11-17 2015-11-17 A kind of large-scale crowd video analytic system and method

Country Status (1)

Country Link
CN (1) CN105447458B (en)

Families Citing this family (28)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106295502B (en) * 2016-07-25 2019-07-12 厦门中控智慧信息技术有限公司 A kind of method for detecting human face and device
CN106444753B (en) * 2016-09-20 2019-12-03 智易行科技(武汉)有限公司 Human body situation based on artificial neural network judges intelligent follower method
CN109416728A (en) * 2016-09-30 2019-03-01 富士通株式会社 Object detection method, device and computer system
CN107944327A (en) * 2016-10-10 2018-04-20 杭州海康威视数字技术股份有限公司 A kind of demographic method and device
GB2554948A (en) * 2016-10-17 2018-04-18 Calipsa Ltd Video monitoring using machine learning
CN106709453B (en) * 2016-12-24 2020-04-17 北京工业大学 Sports video key posture extraction method based on deep learning
CN106803069A (en) * 2016-12-29 2017-06-06 南京邮电大学 Crowd's level of happiness recognition methods based on deep learning
CN106845389B (en) * 2017-01-18 2019-09-06 山东大学 It is a kind of to be become separated in flight event detecting method and system based on direction-sociability model crowd
CN106845621B (en) * 2017-01-18 2019-04-30 山东大学 Dense population number method of counting and system based on depth convolutional neural networks
CN107122707A (en) * 2017-03-17 2017-09-01 山东大学 Video pedestrian based on macroscopic features compact representation recognition methods and system again
CN108986064A (en) * 2017-05-31 2018-12-11 杭州海康威视数字技术股份有限公司 A kind of people flow rate statistical method, equipment and system
CN107274630A (en) * 2017-06-09 2017-10-20 安徽富煌科技股份有限公司 It is a kind of that early warning system is trampled based on the anti-of astable operation object statistics
CN107481260A (en) * 2017-06-22 2017-12-15 深圳市深网视界科技有限公司 A kind of region crowd is detained detection method, device and storage medium
CN107480452A (en) * 2017-08-17 2017-12-15 深圳先进技术研究院 Multi-user's mood monitoring method, device, equipment and storage medium
CN107679503A (en) * 2017-10-12 2018-02-09 中科视拓(北京)科技有限公司 A kind of crowd's counting algorithm based on deep learning
CN109697392A (en) * 2017-10-23 2019-04-30 北京京东尚科信息技术有限公司 Draw the method and device of target object thermodynamic chart
CN108154113A (en) * 2017-12-22 2018-06-12 重庆邮电大学 Tumble event detecting method based on full convolutional network temperature figure
CN108399618A (en) * 2018-02-28 2018-08-14 清华大学 The position of crowd and number acquisition device
CN108830145A (en) * 2018-05-04 2018-11-16 深圳技术大学(筹) A kind of demographic method and storage medium based on deep neural network
CN108566537A (en) * 2018-05-16 2018-09-21 中国科学院计算技术研究所 Image processing apparatus for carrying out neural network computing to video frame
CN108810485A (en) * 2018-07-02 2018-11-13 重庆中科云丛科技有限公司 A kind of monitoring system working method
CN109341763B (en) * 2018-10-10 2020-02-04 广东长盈科技股份有限公司 Transportation data acquisition system and method based on Internet of things
CN109376689A (en) * 2018-11-20 2019-02-22 图普科技(广州)有限公司 Population analysis method and device
CN109801260A (en) * 2018-12-20 2019-05-24 北京海益同展信息科技有限公司 The recognition methods of livestock number and device
CN109961060A (en) * 2019-04-11 2019-07-02 北京百度网讯科技有限公司 Method and apparatus for generating crowd density information
CN110084156A (en) * 2019-04-12 2019-08-02 中南大学 A kind of gait feature abstracting method and pedestrian's personal identification method based on gait feature
CN110213530A (en) * 2019-04-26 2019-09-06 视联动力信息技术股份有限公司 Method for early warning, device and readable storage medium storing program for executing
CN110502967A (en) * 2019-07-01 2019-11-26 特斯联(北京)科技有限公司 Target scene artificial intelligence matching process and device based on personnel's big data

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933412A (en) * 2015-06-16 2015-09-23 电子科技大学 Abnormal state detection method of medium and high density crowd

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
SG174519A1 (en) * 2009-04-01 2011-10-28 Univ Curtin Tech Systems and methods for detecting anomalies from data

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104933412A (en) * 2015-06-16 2015-09-23 电子科技大学 Abnormal state detection method of medium and high density crowd

Also Published As

Publication number Publication date
CN105447458A (en) 2016-03-30

Similar Documents

Publication Publication Date Title
US20190246073A1 (en) Method for finding paths in video
US9299162B2 (en) Multi-mode video event indexing
Wu et al. A Bayesian model for crowd escape behavior detection
KR101808587B1 (en) Intelligent integration visual surveillance control system by object detection and tracking and detecting abnormal behaviors
Ge et al. Vision-based analysis of small groups in pedestrian crowds
US9111148B2 (en) Unsupervised learning of feature anomalies for a video surveillance system
CN104021570B (en) Directive in a kind of video cross the border and mix line detecting method
EP2310981B1 (en) Apparatus and method of classifying movement of objects in a monitoring zone
CN102436662B (en) Human body target tracking method in nonoverlapping vision field multi-camera network
KR101375583B1 (en) Object Density Estimation in Video
US20160267777A1 (en) Alert volume normalization in a video surveillance system
Loy et al. Multi-camera activity correlation analysis
Junior et al. Crowd analysis using computer vision techniques
JP5224401B2 (en) Monitoring system and method
CN101281596B (en) Method for detecting objects left-behind in a scene
Smith et al. Tracking the visual focus of attention for a varying number of wandering people
CN101303727B (en) Intelligent management method based on video human number Stat. and system thereof
Benezeth et al. Abnormal events detection based on spatio-temporal co-occurences
US8605155B2 (en) Video surveillance system
CN101872418B (en) Detection method based on group environment abnormal behavior
Brulin et al. Posture recognition based on fuzzy logic for home monitoring of the elderly
CN103164858B (en) Adhesion crowd based on super-pixel and graph model is split and tracking
Shah et al. Automated visual surveillance in realistic scenarios
CN103824070B (en) A kind of rapid pedestrian detection method based on computer vision
CN104123544B (en) Anomaly detection method and system based on video analysis

Legal Events

Date Code Title Description
C06 Publication
PB01 Publication
C10 Entry into substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant