CN107481260A - A kind of region crowd is detained detection method, device and storage medium - Google Patents

A kind of region crowd is detained detection method, device and storage medium Download PDF

Info

Publication number
CN107481260A
CN107481260A CN201710482241.8A CN201710482241A CN107481260A CN 107481260 A CN107481260 A CN 107481260A CN 201710482241 A CN201710482241 A CN 201710482241A CN 107481260 A CN107481260 A CN 107481260A
Authority
CN
China
Prior art keywords
key point
mrow
crowd
msub
detained
Prior art date
Application number
CN201710482241.8A
Other languages
Chinese (zh)
Inventor
杨延生
马志国
赵瑞
Original Assignee
深圳市深网视界科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市深网视界科技有限公司 filed Critical 深圳市深网视界科技有限公司
Priority to CN201710482241.8A priority Critical patent/CN107481260A/en
Publication of CN107481260A publication Critical patent/CN107481260A/en

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/194Segmentation; Edge detection involving foreground-background segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/207Analysis of motion for motion estimation over a hierarchy of resolutions
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/215Motion-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING; COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20024Filtering details

Abstract

Being detained detection method, device and storage medium, wherein this method the invention discloses a kind of region crowd includes:Obtain crowd's prospect in scene image;The key point of acquisition crowd's prospect;Key point is tracked, obtains the movement locus of key point;According to the move distance of moving track calculation key point;If the movement velocity of key point is less than predetermined threshold value, by key point labeled as delay key point, and delay dot matrix is generated;It is updated according to dot matrix is detained to being detained statistical matrix.By foreground segmentation, the key point of crowd's prospect is obtained, and by the tracking of the key point to the crowd region in scene, considerably increases the robustness of human body tracking, will not be influenceed by crowd's adhesion;Then the motor pattern in scene using key point as the crowd region of representative is analyzed, the estimation of crowd's holdup time is carried out by being detained statistical matrix, it is achieved thereby that being monitored analysis to crowd's delay situation of whole scene.

Description

A kind of region crowd is detained detection method, device and storage medium

Technical field

The present invention relates to computer vision technique, more particularly to a kind of region crowd to be detained detection method, device and storage Medium.

Background technology

Crowd behaviour analysis in region is that one with image analysis technology in field of video monitoring of computer vision is important Using.General fashion is to gather regarding in real time for the public safety regions such as park, square, large-scale indoor activity place by camera Frequency picture, application image Intellectual Analysis Technology, quantity, movement tendency to the crowd in scene are analyzed and predicted, and root Crowd's accident such as make some decision-makings according to the result of analysis and support, trample to prevent some.

The behavior being detained present invention is generally directed to crowd in scene is tested and analyzed, and counts each region in frame out The holdup time of upper crowd, can effectively detect some sensitizing ranges, as railway station, airport, subway gateway, passage, The crowd in ladder or military secrecy region such as is detained, hovered at the abnormal behaviour.It is simultaneously stagnant by analyzing each region crowd in scene The length of time is stayed, stream of people's evacuation planning when some danger occur can be done with current scene.

Traditional region crowd behaviour analysis of delay in to(for) crowd, often using be based on foreground moving object with Track realizes that these methods typically first pass through background modeling and carry out the segmentation of moving target, then moving target carried out with Track, to count target residence time in designated area.Moving object detection is carried out using the method for background modeling, to motion Target does not have distinction, and the target that cannot be distinguished by motion is people, car or other targets, and some algorithm combination size has carried out letter Single screening, this way are easily disturbed by camera angle, scene size, easy false retrieval or missing inspection.In addition in recent years with The rise of deep learning, the method that many methods improve target detection, replaced using based on the human testing of deep learning Target detection based on background model, solves subproblem, but the human testing based on deep learning do not adapt to it is larger The problem of people is more in scene and scene, and adhesion is serious.Moreover, independence of the existing Moving Target Tracking Algorithm to target It is required that higher, situation about being sticked together to multiple target, often effect performance is poor.Enter additionally by the track to moving target Row analysis judges the delay of target, can only detect the stop or static of crowd, if target there occurs the movement of small range, Then easy leak detection.

To sum up, there is following defect in existing general population's delay detection scheme:On the one hand it is limited to foreground moving object The levels of precision of segmentation, on the other hand also it is easy to be influenceed by crowd's adhesion in terms of tracking, more holds in actual applications It is vulnerable to the interference such as complex background, weather conditions, easily produces wrong report with failing to report.

The content of the invention

For overcome the deficiencies in the prior art, an object of the present invention is that providing a kind of region crowd is detained detection side Method, it can solve existing crowd and be detained on the one hand levels of precision that detection scheme is limited to foreground moving object segmentation, another Aspect is also easy to be influenceed by crowd's adhesion in terms of tracking, is easier in actual applications by complex background, weather Factor etc. is disturbed, the problem of easily producing wrong report and fail to report.

The second object of the present invention is that providing a kind of region crowd is detained detection means, and it is stagnant that it can solve existing crowd Stay detection scheme on the one hand be limited to foreground moving object segmentation levels of precision, on the other hand in terms of tracking also be easy to by To the influence of crowd's adhesion, be easier to be disturbed by complex background, weather conditions etc. in actual applications, easily produce wrong report with The problem of failing to report.

The third object of the present invention is to provide a kind of storage medium, and be stored with computer program, it can solve existing Crowd is detained on the one hand levels of precision that detection scheme is limited to foreground moving object segmentation, on the other hand the also pole in terms of tracking Easily influenceed by crowd's adhesion, be easier to be disturbed by complex background, weather conditions etc. in actual applications, easily produce The problem of reporting by mistake and failing to report.

An object of the present invention is realized using following technical scheme:

A kind of region crowd is detained detection method, comprises the following steps:

Obtain crowd's prospect in the scene image;

Obtain the key point of crowd's prospect;

The key point is tracked, obtains the movement locus of the key point;

According to the move distance of key point described in the moving track calculation;

If the movement velocity of the key point is less than predetermined threshold value, by the key point labeled as delay key point, and Dot matrix is detained in generation;

It is updated according to the delay dot matrix to being detained statistical matrix.

Further, it is further comprising the steps of to be detained detection method by the region crowd:Calculate the saturating of the scene image Depending on matrix;

The predetermined threshold value is specially to be calculated according to the perspective matrix;

It is if the movement velocity of the key point is less than predetermined threshold value, the key point is crucial labeled as being detained Point, and generate and be detained after dot matrix, it is further comprising the steps of:

Gaussian filtering is carried out to the delay dot matrix, the standard deviation of the filtering core of gaussian filtering is according to the perspective matrix Obtain.

Further, the key point for obtaining crowd's prospect, specifically includes following sub-step:

Crowd's prospect is calculated in X and the gradient I of Y-directionxAnd Iy

Calculate gradient IxAnd IyProduct:

Ixy=Ix·Iy

It is rightAnd IxyCarry out Gauss weighting, generation weighting matrix M elements A, B and C:

The Harris that each pixel in crowd's prospect is calculated according to the weighting matrix M is responded;

Non- maximum suppression is carried out to each pixel according to Harris responses, it is institute to choose local maximum point State the key point of crowd's prospect.

Further, it is described that the key point is tracked by sparse optical flow method, obtain the motion of the key point Track, specifically include following sub-step:

The crucial neighborhood of a point is obtained, the constraint equation for generating the ith pixel Ii in the neighborhood is:

Wherein,U, v represents x-component, the y of the key point instantaneous velocity Component;

Object function is solved by least square methodObtain the pass Key point instantaneous velocity U=(u, v);

The movement locus of the key point is calculated according to the instantaneous velocity.

Further, it is further comprising the steps of to be detained detection method by the region crowd:

If the key point for having tracking is lost, the new key point for choosing current scene image is supplemented;

Specifically, including following sub-step:

The distance between new key point and the key point of the tracking sum are calculated respectively;

The new key point maximum apart from sum is chosen to be supplemented.

Further, it is described according to the delay dot matrix to being detained after statistical matrix is updated, it is in addition to following Step:

Crowd is carried out to the scene image according to the delay statistical matrix after renewal and is detained visualization.

The second object of the present invention is realized using following technical scheme:

A kind of region crowd is detained detection means, including:

First acquisition module, for obtaining crowd's prospect in the scene image;

Second acquisition module, for obtaining the key point of crowd's prospect;

Tracking module, for being tracked to the key point, obtain the movement locus of the key point;

First computing module, the move distance for the key point according to the moving track calculation;

Mark module, if the movement velocity for the key point is less than predetermined threshold value, the key point is labeled as It is detained key point, and generates delay dot matrix;

Update module, for being updated according to the delay dot matrix to being detained statistical matrix.

Further, the region crowd is detained detection means and also included:

Second computing module, for calculating the perspective matrix of the scene image;

Filtration module, for carrying out gaussian filtering, the standard deviation root of the filtering core of gaussian filtering to the delay dot matrix Obtained according to the perspective matrix.

Further, the region crowd is detained detection means and also included:

Complementary module, if for there is the key point of tracking loss, the new key point for choosing current scene image is mended Fill.

The third object of the present invention is realized using following technical scheme:

A kind of region crowd is detained detection means, including memory, processor and is stored in the memory and can The computer program run on the processor, foregoing people from region is realized described in the computing device during computer program Group is detained the step of detection method.

Compared with prior art, the beneficial effects of the present invention are:By foreground segmentation, obtain crowd's prospect key point, And the tracking of the key point in the crowd region that sparse optical flow method is used in scene, estimation is carried out, rather than to single mesh Mark is tracked, and considerably increases the robustness of human body tracking, will not be influenceed by crowd's adhesion;Then analyze scene in Key point is the motor pattern in the crowd region of representative, and the estimation of crowd's holdup time is carried out by being detained statistical matrix, can be with The region being detained and corresponding holdup time easily are calculated, it is achieved thereby that being detained feelings to the crowd of whole scene Condition is monitored analysis.

Brief description of the drawings

Fig. 1 is that the region crowd of the embodiment of the present invention one is detained the schematic flow sheet of detection method;

Fig. 2 is a kind of structural representation of the FCN models of crowd's foreground segmentation;

Fig. 3 is the schematic diagram that pedestrian is marked;

Fig. 4 is that scene image crowd is detained visualization schematic diagram;

Fig. 5 is that the region crowd of the embodiment of the present invention two is detained the structural representation of detection means;

Fig. 6 is that the region crowd of the embodiment of the present invention three is detained the structural representation of detection means.

Embodiment

Below, with reference to accompanying drawing and embodiment, the present invention is described further, it is necessary to which explanation is, not Under the premise of afoul, new implementation can be formed between various embodiments described below or between each technical characteristic in any combination Example.

Embodiment one

As Fig. 1 be a kind of region crowd be detained detection method, comprise the following steps:

Step S110, crowd's prospect in the scene image is obtained.

Specifically, the usable crowd monitoring technology based on deep learning of the present invention, improves the essence to crowd's foreground detection Exactness, it can be very good the scene that accommodation is big, people is more.

In this step, the convolutional neural networks of a full convolution can be utilized, the frame of video in monitoring scene is carried out The segmentation of crowd's prospect.The advantage of this method is that the segmentation to crowd's prospect only depends on the information of current frame image, to taking the photograph The erection pattern of camera machine can not required.

Full convolutional neural networks FCN, that is, Fully Convolutional Network, it is one and does not include and connect entirely Connect the convolutional neural networks of layer.The difference of it and convolutional neural networks is that in general convolutional neural networks can be generally divided into Many levels, the corresponding feature for learning different levels in image, shallower convolutional layer perception domain is smaller, and some partial zones are arrived in study The feature in domain;Deeper convolutional layer has larger Perception Area, can learn to more abstract feature, finally utilize full connection Layer deploys to feature, and classifies.And in FCN network structure, full articulamentum is instead of with the convolution kernel that core is 1 × 1, So that the relation spatially of feature is still saved in output layer.

A kind of FCN models for crowd's foreground segmentation are as shown in Figure 2.The input of wherein network is that we need to carry out The frame of video F of the t of foreground segmentationt, output is then the result S of corresponding foreground segmentationt, wherein StMeet:

Step S120, the key point of crowd's prospect is obtained.

In this step, the motion of crowd's prospect will be analyzed, by being clicked through to some keys in crowd's prospect Line trace, to evaluate the motion conditions in crowd region near these points.In the present embodiment, key is detected using Harris Point, and using the segmentation figure picture of crowd's prospect, the point in non-crowd region is filtered.

Further, step S120 obtains the key point of crowd's prospect, specifically includes following sub-step:

Step S121, crowd's prospect F is calculatedt(x, y) is in X and the gradient I of Y-directionxAnd Iy

Step S122, gradient I is calculatedxAnd IyProduct:

Ixy=Ix·Iy

Step S123, using Gaussian function pairAnd IxyGauss weighting is carried out, can use σ=1, generation weighting matrix M's Elements A, B and C:

Step S124, the Harris that each pixel in crowd's prospect is calculated according to the weighting matrix M responds R.And can Threshold value t or the not R zero setting of the pixel in crowd's foreground area further are less than to Harris responses R, i.e.,:

R (i, j)={ R (i, j): detM=α (traceM)2<T or St(i, j)=0 }.

Step S125, responded according to the Harris and non-maximum suppression is carried out to each pixel, choose part most Big value point is the key point of crowd's prospect.

Can carry out non-maximum suppression in the neighborhood of the 5 × 5 of pixel, local maximum point that be angle in image Point, these angle points are key point C1,C2,Cn

Step S130, the key point is tracked, obtains the movement locus of the key point.

Further, the key point is tracked by sparse optical flow method.

What light stream represented is the instantaneous velocity of pixel motion between image sequence, in the present invention and need not calculate image The light stream of middle all pixels, it is only necessary to calculate the key point of crowd's prospect, can so greatly improve light stream Calculating speed.

Further, step S130 is tracked by sparse optical flow method to the key point, obtains the key point Movement locus, specifically include following sub-step:

Step S131, the crucial neighborhood of a point is obtained.

To any one key point (x, y), a n × n small field Ω is established around it, and assume light stream U=(u, v) Keep constant in this neighborhood.To each pixel in neighborhood Ω, we can write out a constraint equation.

Step S132, the constraint equation of the ith pixel Ii in the generation neighborhood is:

Wherein,Represent partial derivative of the gradation of image to space;Represent image Partial derivative of the gray scale to the time;U, v represents x-component, the y-component of the key point instantaneous velocity.

Step S133, object function is solved by least square method Obtain the key point instantaneous velocity U=(u, v).

Step S134, the movement locus of the key point is calculated according to the instantaneous velocity.

In the case of learning key point instantaneous velocity, it is possible to key point is tracked, its movement locus is calculated, obtains Position of the key point in each two field picture.

When we are tracked to the key point in crowd's prospect, in t0At the moment, we first select N (such as 1000) individual pass Key point, is expressed as { P1,P2,P3,…,PN, in ensuing t1,t2,…tnMoment, we can keep carrying out a little key points with Track, each point record its past history Grid Track, and use hereRepresent i-th Key point is from t0Moment is to tnThe historical track at moment, whereinRepresent tiThe coordinate position of moment key point in the picture.

Step S140, the move distance of key point according to the moving track calculation.

For the key point { P in all groups' prospect in tracking mode1,P2,P3,…,PN, we specifically divide Analyse the motion state of each key point.Due to key point be in crowd's zone location, so the motion state of key point, one Determine to reflect the motion state of crowd near zone Ω in degree.

For i-th of key point Pi, from t0To tnThe movement locus at moment is Ti={ p0,p1,…pt…pn, wherein pt Represent t key point PiThe location of in the picture.We are to key point PiIn the past period, such as the motion in 1 second DistanceCalculated, computational methods are as follows:

Wherein r be video frame per second (frame number of video per second), K tnThe sequence number of the frame of previous second at moment.Dist(pt, pt-1) represent p in the plane of delineationt,pt-1Euclidean distance between 2 points.The move distance of key pointActually piPast The average distance moved in one second.

If the movement velocity of step S150, described key point is less than predetermined threshold value, by the key point labeled as delay Key point, and generate delay dot matrix.

Pass through the move distance to key pointAnalysis, to judge key point piWhether it was delay within past one second State.Decision method is, ifγ is predetermined threshold value, then in one second in the past, the key point remains static, As it is detained key point.

It can use and be detained dot matrixTo represent tiAt the moment, the key point of retaining state is in image:

Further, predetermined threshold value can calculate according to the perspective matrix of scene image, such asWherein Pmap is the perspective matrix of scene image.The perspective relation of scene image is introduced, the truth of crowd movement can be reduced, made The judge that must be detained is more accurate.

Therefore, it is further comprising the steps of to be detained detection method by region crowd:

Step S101, the perspective matrix of the scene image is calculated.

Crowd is detained the method detected and needs to carry out video camera photographed scene the estimation of perspective relation in region, with estimation The distance of crowd movement and the relation of actual distance in scene.

Perspective matrix PMap, the size one for the video pictures that the size of perspective matrix arrives with camera acquisition are initialized first Cause;PMap (x, y) represents image (x, y) place, 1 pixel height representative in real scene.It is saturating to scene in the present invention Estimation depending on relation is a kind of approximate computational methods, only considers the proportionate relationship in the vertical aspect of image, and assume to image 1.7m per capita in head picture.

A two field picture is made choice of in the video pictures having been taken, it is desirable to which at least difference of more than 2 is far and near in picture Pedestrian, certain quantity is The more the better.As shown in figure 3, these pedestrians are marked, their head and the vertical direction of pin On y-coordinate record, be denoted asWherein K is the institute of mark The number of someone.

Thus have under nearly different y-coordinates, the height of people high 1.7m shared pixel in the picture, use line Property function h=wy+b represents the mapping relations between the height h of image y-coordinate and people in the picture.And w, b are then us Need the parameter asked.

Note:

Then:

Finally, we can obtain the value for carrying out each element in PMap according to linear mapping function:

PMap (i, j)=wj+b.

Further, if the movement velocity of the step S150 key points is less than predetermined threshold value, by the key point mark It is designated as being detained key point, and generates and be detained after dot matrix, it is further comprising the steps of:

Step S102, gaussian filtering is carried out to the delay dot matrix, the standard deviation of the filtering core of gaussian filtering is according to institute Perspective matrix is stated to obtain.

Delay dot matrix to the i momentGaussian filtering is carried out, the standard deviation sigma of filtering core is saturating with each opening position of image Depending on relation, i.e. the size of the value of corresponding element is relevant in PMap.The standard deviation of position p filtering core in image is represented with σ (p), It can then makeDelay dot matrix after gaussian filteringFor:

It is detained dot matrixIn value represent whether crowd in the band of position picture has resting state.If value is 0, Illustrate that the region does not have crowd or crowd to be kept in motion.

Step S160, it is updated according to the delay dot matrix to being detained statistical matrix.

In t0At the moment, we are detained statistical matrix S by initialization value, and the size of matrix is identical with the size of image, in matrix Element S (i, j) representative image in (i, j) opening position crowd's residence time length, unit is the second.If position nobody Group is detained, then S (i, j)=0.

In the ensuing time, once dot matrix is detained in secondary superposition on statistical matrix S is detainedOr after gaussian filtering It is detained dot matrixTo be superimposed the delay dot matrix after gaussian filteringExemplified by, the renewal to delay statistical matrix is as follows:

Wherein λ is the decline factor, and λ is higher, then the motion to people is more sensitive, is less susceptible to produce and is detained statistics.λ can lead to Cross empirical value formulation.

By above step, for each frame video image, we can go to analyze current time, the motion of key point On being detained influence caused by statistical matrix SThen S is updated again.After having S, any position crowd in image is just easily analyzed The state of delay.

The present invention passes through foreground segmentation, the key point of acquisition crowd's prospect, and the people that sparse optical flow method is used in scene The tracking of the key point in group region, carries out estimation, rather than single target is tracked, and considerably increases human body tracking Robustness, will not be influenceed by crowd's adhesion;Then the motion in scene using key point as the crowd region of representative is analyzed Pattern, the estimation of crowd's holdup time is carried out by being detained statistical matrix, in that context it may be convenient to calculate the region be detained with And the corresponding holdup time, it is achieved thereby that being monitored analysis to crowd's delay situation of whole scene.

As a further improvement on the present invention, step S160 is carried out more according to the delay dot matrix to being detained statistical matrix It is further comprising the steps of after new:

Step S170, crowd is carried out to the scene image according to the delay statistical matrix after renewal to be detained visually Change.As shown in figure 4, the statistic for being detained statistical matrix is visually superimposed on the video images, it is convenient to observe image difference The crowd in region is detained situation contrast.

In the present embodiment, statistical matrix will be detained and is mapped as a kind of color of flame, by indigo plant to red gradual change, with blueness Then represent the delay of no crowd, red then representative delay longer time.

Concordance list is initially set up, 0-255 256 numerical value are established the mapping one by one with RGB24 colors by us respectively, For any value G in statistical matrix, the color-values of a triple channel with following formula, will be mapped as, mapping relations are as follows:

Wherein L is map index table.

Mapping above is applied to being detained all elements in statistical matrix S, then has obtained a colored thermodynamic chart H.

Then thermodynamic chart H and artwork I are overlapped, colored graph R is detained in generation, as shown in Figure 4.To facilitate in former video More intuitively show the degree of different zones crowd delay in frame.Superposition utilizes equation below:

R=I+ (H-B)

Wherein B be with video image size identical, background for blueness coloured image, it is therefore an objective to without crowd delay Region be changed into transparent.

As a further improvement on the present invention, it is further comprising the steps of to be detained detection method by region crowd:

If the key point for step S180, having tracking is lost, the new key point for choosing current scene image is supplemented.

With the progress of tracking, the people in scene may leave scene or produce and block, so some key points It can lose therewith, after a period of time is tracked, the key point of loss can be more and more, and newly enter with the crowd in scene Key point is not within our following ranges.So the new key point that we need to choose current scene image is mended Fill, one is selected in the key point detected from present frame, is supplemented.

Further, if the key point that step S180 has tracking is lost, the new crucial click-through of current scene image is chosen Row supplement, including following sub-step:

Step S181, the distance between new key point and the key point of the tracking sum are calculated respectively;

Step S182, the new key point maximum apart from sum is chosen to be supplemented.

A kind of embodiment of Selection Strategy is as follows:

1st, for the key point newly detected, we use { Q1,Q2,Q3,…QNRepresent, it is new that we calculate each respectively Key point and be currently in the distance between key point of tracking mode, and obtain a Distance matrix DN×N, wherein D (i, j) =Dist (Pi,Qj), wherein Dist (p1,p2) represent p in the plane of delineation1,p2Euclidean distance between 2 points.

2nd, each column element in matrix D is added to obtain the row vector W that a length is N1×N

3rd, the location index k=argmax W of maximum in vectorial W are obtained.

QkIt is the new key point that we choose.

By step S180, we remain and the key point in N number of crowd's prospect are tracked point at any time Analysis, ensure that good crowd is detained Detection results.

Embodiment two

Region crowd as shown in Figure 5 is detained detection means, including:

First acquisition module 110, for obtaining crowd's prospect in the scene image;

Second acquisition module 120, for obtaining the key point of crowd's prospect;

Tracking module 130, for being tracked to the key point, obtain the movement locus of the key point;

First computing module 140, the move distance for the key point according to the moving track calculation;

Mark module 150, if the movement velocity for the key point is less than predetermined threshold value, the key point is marked To be detained key point, and generate delay dot matrix;

Update module 160, for being updated according to the delay dot matrix to being detained statistical matrix.

Further, region crowd is detained detection means and also included:

Second computing module 101, for calculating the perspective matrix of the scene image;

Filtration module 102, for carrying out gaussian filtering, the standard deviation of the filtering core of gaussian filtering to the delay dot matrix Obtained according to the perspective matrix.

Further, region crowd is detained detection means and also included:

Visualization model 170, for carrying out crowd to the scene image according to the delay statistical matrix after renewal It is detained visualization.

Further, region crowd is detained detection means and also included:

Complementary module 180, if for there is the key point of tracking loss, the new key point for choosing current scene image is carried out Supplement.

Specifically, complementary module 180 includes:

The distance between computing unit 181, the key point for calculating new key point and the tracking respectively sum;

Unit 182 is chosen, is supplemented for choosing the new key point maximum apart from sum.

The method in device and previous embodiment in the present embodiment be based on two under same inventive concept aspects, Above method implementation process is described in detail, thus those skilled in the art can according to it is described above clearly The structure and implementation process of the system in this implementation are solved, it is succinct for specification, just repeat no more herein.

For convenience of description, it is divided into various modules during description apparatus above with function to describe respectively.Certainly, this is being implemented The function of each module can be realized in same or multiple softwares and/or hardware during invention.

As seen through the above description of the embodiments, those skilled in the art can be understood that the present invention can Realized by the mode of software plus required general hardware platform.Based on such understanding, technical scheme essence On the part that is contributed in other words to prior art can be embodied in the form of software product, the computer software product It can be stored in storage medium, such as ROM/RAM, magnetic disc, CD, including some instructions are causing a computer equipment (can be personal computer, server, either network equipment etc.) performs some of each embodiment of the present invention or embodiment Method described in part, such as:

A kind of storage medium, the storage medium are stored with computer program, and the computer program is executed by processor Shi Shixian aforementioned areas crowd is detained the step of detection method.

The device embodiment of description is only schematical, wherein the module illustrated as separating component or unit can To be or may not be physically separate, the part illustrated as module or unit can be or may not be thing Module is managed, can both be located at a place, or can also be distributed on multiple mixed-media network modules mixed-medias.It can select according to the actual needs Some or all of unit therein is selected to realize the purpose of this embodiment scheme.Those of ordinary skill in the art are not paying wound In the case that the property made is worked, you can to understand and implement.

The present invention can be used in numerous general or special purpose computing system environments or configuration.Such as:Personal computer, service Device computer, handheld device or portable set, laptop device, multicomputer system, the system based on microprocessor, machine top Box, programmable consumer-elcetronics devices, network PC, minicom, mainframe computer including any of the above system or equipment DCE etc., such as example IV.

Embodiment three

Region crowd as shown in Figure 6 is detained detection means, including memory 200, processor 300 and is stored in memory In 200 and the computer program that can be run on processor 300, processor 300 realize above-mentioned zone when performing computer program Crowd is detained the step of detection method.

The method in device and previous embodiment in the present embodiment be based on two under same inventive concept aspects, Above method implementation process is described in detail, thus those skilled in the art can according to it is described above clearly The structure and implementation process of the system in this implementation are solved, it is succinct for specification, just repeat no more herein.

Region crowd provided in an embodiment of the present invention is detained detection means, can pass through foreground segmentation, acquisition crowd's prospect Key point, and by sparse optical flow method be used for scene in crowd region key point tracking, carry out estimation, rather than Single target is tracked, considerably increases the robustness of human body tracking, will not be influenceed by crowd's adhesion;Then analyze Motor pattern in scene using key point as the crowd region of representative, estimating for crowd's holdup time is carried out by being detained statistical matrix Meter, in that context it may be convenient to the region being detained and corresponding holdup time are calculated, it is achieved thereby that the people to whole scene Group's delay situation is monitored analysis.

Above-mentioned embodiment is only the preferred embodiment of the present invention, it is impossible to the scope of protection of the invention is limited with this, The change and replacement for any unsubstantiality that those skilled in the art is done on the basis of the present invention belong to institute of the present invention Claimed scope.

Claims (10)

1. a kind of region crowd is detained detection method, it is characterised in that comprises the following steps:
Obtain crowd's prospect in the scene image;
Obtain the key point of crowd's prospect;
The key point is tracked, obtains the movement locus of the key point;
According to the move distance of key point described in the moving track calculation;
If the movement velocity of the key point is less than predetermined threshold value, by the key point labeled as delay key point, and generate It is detained dot matrix;
It is updated according to the delay dot matrix to being detained statistical matrix.
2. region crowd as claimed in claim 1 is detained detection method, it is characterised in that further comprising the steps of:Calculate institute State the perspective matrix of scene image;
The predetermined threshold value is specially to be calculated according to the perspective matrix;
If the movement velocity of the key point is less than predetermined threshold value, by the key point labeled as delay key point, and Generation is detained after dot matrix, further comprising the steps of:
Gaussian filtering is carried out to the delay dot matrix, the standard deviation of the filtering core of gaussian filtering obtains according to the perspective matrix Arrive.
3. region crowd as claimed in claim 1 is detained detection method, it is characterised in that acquisition crowd's prospect Key point, specifically include following sub-step:
Crowd's prospect is calculated in X and the gradient I of Y-directionxAnd Iy
<mrow> <msub> <mi>I</mi> <mi>x</mi> </msub> <mo>=</mo> <mfrac> <mrow> <mo>&amp;part;</mo> <mi>F</mi> </mrow> <mrow> <mo>&amp;part;</mo> <mi>x</mi> </mrow> </mfrac> <mo>=</mo> <mi>F</mi> <mo>&amp;CircleTimes;</mo> <mfenced open = "(" close = ")"> <mtable> <mtr> <mtd> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </mtd> <mtd> <mn>0</mn> </mtd> <mtd> <mn>1</mn> </mtd> </mtr> </mtable> </mfenced> </mrow>
<mrow> <msub> <mi>I</mi> <mi>y</mi> </msub> <mo>=</mo> <mfrac> <mrow> <mo>&amp;part;</mo> <mi>F</mi> </mrow> <mrow> <mo>&amp;part;</mo> <mi>y</mi> </mrow> </mfrac> <mo>=</mo> <mi>F</mi> <mo>&amp;CircleTimes;</mo> <msup> <mfenced open = "(" close = ")"> <mtable> <mtr> <mtd> <mrow> <mo>-</mo> <mn>1</mn> </mrow> </mtd> <mtd> <mn>0</mn> </mtd> <mtd> <mn>1</mn> </mtd> </mtr> </mtable> </mfenced> <mi>T</mi> </msup> <mo>;</mo> </mrow>
Calculate gradient IxAnd IyProduct:
<mrow> <msubsup> <mi>I</mi> <mi>x</mi> <mn>2</mn> </msubsup> <mo>=</mo> <msub> <mi>I</mi> <mi>x</mi> </msub> <mo>&amp;CenterDot;</mo> <msub> <mi>I</mi> <mi>x</mi> </msub> </mrow>
<mrow> <msubsup> <mi>I</mi> <mi>y</mi> <mn>2</mn> </msubsup> <mo>=</mo> <msub> <mi>I</mi> <mi>y</mi> </msub> <mo>&amp;CenterDot;</mo> <msub> <mi>I</mi> <mi>y</mi> </msub> </mrow>
<mrow> <msub> <mi>I</mi> <mrow> <mi>x</mi> <mi>y</mi> </mrow> </msub> <mo>=</mo> <msub> <mi>I</mi> <mi>x</mi> </msub> <mo>&amp;CenterDot;</mo> <msub> <mi>I</mi> <mi>y</mi> </msub> <mo>;</mo> </mrow>
It is rightAnd IxyCarry out Gauss weighting, generation weighting matrix M elements A, B and C:
<mrow> <mi>A</mi> <mo>=</mo> <mi>g</mi> <mrow> <mo>(</mo> <msubsup> <mi>I</mi> <mi>x</mi> <mn>2</mn> </msubsup> <mo>)</mo> </mrow> <mo>=</mo> <msubsup> <mi>I</mi> <mi>x</mi> <mn>2</mn> </msubsup> <mo>&amp;CircleTimes;</mo> <mi>w</mi> </mrow>
<mrow> <mi>B</mi> <mo>=</mo> <mi>g</mi> <mrow> <mo>(</mo> <msubsup> <mi>I</mi> <mi>y</mi> <mn>2</mn> </msubsup> <mo>)</mo> </mrow> <mo>=</mo> <msubsup> <mi>I</mi> <mi>y</mi> <mn>2</mn> </msubsup> <mo>&amp;CircleTimes;</mo> <mi>w</mi> </mrow>
<mrow> <mi>C</mi> <mo>=</mo> <mi>g</mi> <mrow> <mo>(</mo> <msub> <mi>I</mi> <mrow> <mi>x</mi> <mi>y</mi> </mrow> </msub> <mo>)</mo> </mrow> <mo>=</mo> <msub> <mi>I</mi> <mrow> <mi>x</mi> <mi>y</mi> </mrow> </msub> <mo>&amp;CircleTimes;</mo> <mi>w</mi> <mo>;</mo> </mrow>
The Harris that each pixel in crowd's prospect is calculated according to the weighting matrix M is responded;
Non- maximum suppression is carried out to each pixel according to Harris responses, selection local maximum point is the people The key point of group's prospect.
4. region crowd as claimed in claim 1 is detained detection method, it is characterised in that it is described by sparse optical flow method to institute State key point to be tracked, obtain the movement locus of the key point, specifically include following sub-step:
The crucial neighborhood of a point is obtained, the constraint equation for generating the ith pixel Ii in the neighborhood is:
<mrow> <msub> <mi>I</mi> <msub> <mi>x</mi> <mi>i</mi> </msub> </msub> <mo>&amp;CenterDot;</mo> <mi>u</mi> <mo>+</mo> <msub> <mi>I</mi> <msub> <mi>y</mi> <mi>i</mi> </msub> </msub> <mo>&amp;CenterDot;</mo> <mi>v</mi> <mo>+</mo> <msub> <mi>I</mi> <msub> <mi>t</mi> <mi>i</mi> </msub> </msub> <mo>=</mo> <mn>0</mn> </mrow>
Wherein,U, v represents x-component, the y-component of the key point instantaneous velocity;
Object function is solved by least square methodObtain the key point Instantaneous velocity U=(u, v);
The movement locus of the key point is calculated according to the instantaneous velocity.
5. region crowd as described in any of claims 4 is detained detection method, it is characterised in that also including following step Suddenly:
If the key point for having tracking is lost, the new key point for choosing current scene image is supplemented;
Specifically, including following sub-step:
The distance between new key point and the key point of the tracking sum are calculated respectively;
The new key point maximum apart from sum is chosen to be supplemented.
6. the region crowd as any one of claim 1-5 is detained detection method, it is characterised in that described in the basis It is further comprising the steps of after delay dot matrix is updated to delay statistical matrix:
Crowd is carried out to the scene image according to the delay statistical matrix after renewal and is detained visualization.
7. a kind of region crowd is detained detection means, it is characterised in that including:
First acquisition module, for obtaining crowd's prospect in the scene image;
Second acquisition module, for obtaining the key point of crowd's prospect;
Tracking module, for being tracked to the key point, obtain the movement locus of the key point;
First computing module, the move distance for the key point according to the moving track calculation;
Mark module, if the movement velocity for the key point is less than predetermined threshold value, by the key point labeled as delay Key point, and generate delay dot matrix;
Update module, for being updated according to the delay dot matrix to being detained statistical matrix.
8. region crowd as claimed in claim 7 is detained detection means, it is characterised in that also includes:
Second computing module, for calculating the perspective matrix of the scene image;
Filtration module, for carrying out gaussian filtering to the delay dot matrix, the standard deviation of the filtering core of gaussian filtering is according to institute Perspective matrix is stated to obtain.
9. region crowd as claimed in claim 7 or 8 is detained detection means, it is characterised in that also includes:
Complementary module, if for there is the key point of tracking loss, the new key point for choosing current scene image is supplemented.
10. a kind of storage medium, the storage medium is stored with computer program, it is characterised in that the computer program quilt The step of region crowd as any one of claim 1-6 is detained detection method is realized during computing device.
CN201710482241.8A 2017-06-22 2017-06-22 A kind of region crowd is detained detection method, device and storage medium CN107481260A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201710482241.8A CN107481260A (en) 2017-06-22 2017-06-22 A kind of region crowd is detained detection method, device and storage medium

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201710482241.8A CN107481260A (en) 2017-06-22 2017-06-22 A kind of region crowd is detained detection method, device and storage medium

Publications (1)

Publication Number Publication Date
CN107481260A true CN107481260A (en) 2017-12-15

Family

ID=60594803

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201710482241.8A CN107481260A (en) 2017-06-22 2017-06-22 A kind of region crowd is detained detection method, device and storage medium

Country Status (1)

Country Link
CN (1) CN107481260A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263619A (en) * 2019-04-30 2019-09-20 上海商汤智能科技有限公司 Image processing method and device, electronic equipment and computer storage medium

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101325690A (en) * 2007-06-12 2008-12-17 上海正电科技发展有限公司 Method and system for detecting human flow analysis and crowd accumulation process of monitoring video flow
CN102156880A (en) * 2011-04-11 2011-08-17 上海交通大学 Method for detecting abnormal crowd behavior based on improved social force model
CN102799863A (en) * 2012-07-02 2012-11-28 中国计量学院 Method for detecting group crowd abnormal behaviors in video monitoring
CN102867311A (en) * 2011-07-07 2013-01-09 株式会社理光 Target tracking method and target tracking device
CN102968802A (en) * 2012-11-28 2013-03-13 无锡港湾网络科技有限公司 Moving target analyzing and tracking method and system based on video monitoring
CN103679149A (en) * 2013-12-11 2014-03-26 哈尔滨工业大学深圳研究生院 Method and device for detecting crowd gathering expressed in convex hull based on angular points
CN104933710A (en) * 2015-06-10 2015-09-23 华南理工大学 Intelligent analysis method of store people stream track on the basis of surveillance video
CN105447458A (en) * 2015-11-17 2016-03-30 深圳市商汤科技有限公司 Large scale crowd video analysis system and method thereof
CN106023262A (en) * 2016-06-06 2016-10-12 深圳市深网视界科技有限公司 Crowd flowing main direction estimating method and device

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101325690A (en) * 2007-06-12 2008-12-17 上海正电科技发展有限公司 Method and system for detecting human flow analysis and crowd accumulation process of monitoring video flow
CN102156880A (en) * 2011-04-11 2011-08-17 上海交通大学 Method for detecting abnormal crowd behavior based on improved social force model
CN102867311A (en) * 2011-07-07 2013-01-09 株式会社理光 Target tracking method and target tracking device
CN102799863A (en) * 2012-07-02 2012-11-28 中国计量学院 Method for detecting group crowd abnormal behaviors in video monitoring
CN102968802A (en) * 2012-11-28 2013-03-13 无锡港湾网络科技有限公司 Moving target analyzing and tracking method and system based on video monitoring
CN103679149A (en) * 2013-12-11 2014-03-26 哈尔滨工业大学深圳研究生院 Method and device for detecting crowd gathering expressed in convex hull based on angular points
CN104933710A (en) * 2015-06-10 2015-09-23 华南理工大学 Intelligent analysis method of store people stream track on the basis of surveillance video
CN105447458A (en) * 2015-11-17 2016-03-30 深圳市商汤科技有限公司 Large scale crowd video analysis system and method thereof
CN106023262A (en) * 2016-06-06 2016-10-12 深圳市深网视界科技有限公司 Crowd flowing main direction estimating method and device

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
LIXIN CHEN 等: "Detecting Anomaly Based on Time Dependence for Large Scenes", 《PROCEEDINGS OF THE IEEE INTERNATIONAL CONFERENCE ON INFORMATION AND AUTOMATION》 *
MD. HAIDAR SHARIF 等: "Crowd Behavior Surveillance Using Bhattacharyya Distance Metric", 《COMPIMAGE》 *
兰红 等: "动态背景下的稀疏光流目标提取与跟踪", 《中国图象图形学报》 *
曹志通 等: "改进的基于角点检测的视频人数统计方法", 《计算机应用》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110263619A (en) * 2019-04-30 2019-09-20 上海商汤智能科技有限公司 Image processing method and device, electronic equipment and computer storage medium

Similar Documents

Publication Publication Date Title
JP5224401B2 (en) Monitoring system and method
CN105447458B (en) A kind of large-scale crowd video analytic system and method
Luber et al. Place-dependent people tracking
CN105956268A (en) Construction method and device applied to test scene of pilotless automobile
US8854469B2 (en) Method and apparatus for tracking persons and locations using multiple cameras
CN103164711B (en) The method of region based on pixel and support vector machine artificial abortion&#39;s density Estimation
CN101577812B (en) Method and system for post monitoring
Zaurin et al. Integration of computer imaging and sensor data for structural health monitoring of bridges
CN102629384B (en) Method for detecting abnormal behavior during video monitoring
Moore et al. Visual crowd surveillance through a hydrodynamics lens
CN106204638B (en) It is a kind of based on dimension self-adaption and the method for tracking target of taking photo by plane for blocking processing
CN106127204A (en) A kind of multi-direction meter reading Region detection algorithms of full convolutional neural networks
CN103093212B (en) The method and apparatus of facial image is intercepted based on Face detection and tracking
CN107145908A (en) A kind of small target detecting method based on R FCN
CN101141633B (en) Moving object detecting and tracing method in complex scene
CN101464944B (en) Crowd density analysis method based on statistical characteristics
CN103903008B (en) A kind of method and system of the mist grade based on image recognition transmission line of electricity
CN106803071A (en) Object detecting method and device in a kind of image
CN108038409A (en) A kind of pedestrian detection method
CN104680555B (en) Cross the border detection method and out-of-range monitoring system based on video monitoring
Venetianer et al. Performance evaluation of an intelligent video surveillance system–A case study
CN101883261B (en) Method and system for abnormal target detection and relay tracking under large-range monitoring scene
CN106682697A (en) End-to-end object detection method based on convolutional neural network
CN103093198B (en) A kind of crowd density monitoring method and device
CN109117794A (en) A kind of moving target behavior tracking method, apparatus, equipment and readable storage medium storing program for executing

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination