CN106407903A - Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method - Google Patents
Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method Download PDFInfo
- Publication number
- CN106407903A CN106407903A CN201610790306.0A CN201610790306A CN106407903A CN 106407903 A CN106407903 A CN 106407903A CN 201610790306 A CN201610790306 A CN 201610790306A CN 106407903 A CN106407903 A CN 106407903A
- Authority
- CN
- China
- Prior art keywords
- layer
- video
- neural networks
- convolutional neural
- multiple dimensioned
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Bioinformatics & Computational Biology (AREA)
- Software Systems (AREA)
- Evolutionary Biology (AREA)
- General Engineering & Computer Science (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Artificial Intelligence (AREA)
- Computational Linguistics (AREA)
- Evolutionary Computation (AREA)
- Life Sciences & Earth Sciences (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses a multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method. A convolution neural network is used for replacing a conventional feature extraction algorithm, and the convolution neural network is improved so as to satisfy requirements for human body behavior classification; specifically, three dimensional convolution, three dimensional down-sampling, NIN, three dimensional pyramid structures are added; human body abnormal behavior feature extraction capability of the convolution neural network is enabled to be increased; training operation is performed in a specific video set, features with great classification capacity can be obtained, robustness and accuracy of a whole identification algorithm can be improved, GPU speed is increased so as to satisfy requirements for practical application, and therefore multi-channel videos can be monitored in real time.
Description
Technical field
The present invention relates to computer vision and machine learning field, particularly to the detection skill of human body abnormal behaviour in video
Art.
Background technology
The occasion higher to security requirement such as railway station, bank, airport has huge demand, such as to Human bodys' response
Fruit system can identify human body behavior it is possible to automatically judging abnormal case and reporting to the police, thus substantially reducing human cost, and carries
Rise control and monitoring, accomplish on-line monitoring, Realtime Alerts.
The scheme of conventional body's Activity recognition be substantially based on background modeling and and characteristic matching, such scheme contains three
Individual step：The first step is main extraction space-time characteristic point, that is, possess the pixel of time and spatial character, using background difference or
Person's optical flow method carries out the modeling of background and the extraction of prospect；Second step is according to the feature selected, using regarding that the first step obtains
The video of its characteristic point peripheral region of the feature point pairs of frequency carries out particular conversion and process, to obtain the spy of description specific behavior
Levy, wherein majority is based on static scene and object recognition technique；3rd step is to be trained these features input grader,
Obtain grader and be applied in identification.
Patent No. 101719216A proposes a kind of movement human abnormal behaviour recognition methods based on template matches, should
Having main steps that of scheme, is first pre-processed to video and denoising with gaussian filtering method, then will be pretreated after image
It is divided into the little subregion of W*W, Gaussian Background modeling is carried out to each subregion；By moving target recognition out after empty in HSV color
Between lower remove shade, this moving target is carried out coupling with the master pattern storehouse of foundation and compares, the class of behavior is judged with this
Not.
But all there is problems with said method：
1st, Gaussian Background modeling method in if the color of prospect and background is same or like, judgement prospect with look for
Seek during connected region it is easy to be background by the judgement of part prospect mistake, lead to extract the disappearance in region；
2nd, the method compared again using the feature extracting picture, this to a certain extent can not be well time-domain
Characteristic use；
3rd, using the mode of template matches on Activity recognition, due to feature distribution and the true monitoring of environmental of training set video
Difference, the discrepant situation of sample during some can not be identified well with Sample Storehouse, the change of such as video angle, behavior
, easily wrong report and fails to report in the change of action.
Content of the invention
The present invention is to solve above-mentioned technical problem it is proposed that a kind of real-time body based on multiple dimensioned convolutional neural networks is different
Often Activity recognition method, employs convolutional neural networks and replaces traditional feature extraction algorithm, and convolutional neural networks are entered
Row improves, and increased Three dimensional convolution, three-dimensional down-sampling, NIN, three-dimensional pyramid structure so that convolutional neural networks can be to people
Body abnormal behaviour has higher ability in feature extraction；In specific video concentration training, obtain the feature with more classification capacity,
Strengthen robustness and the accuracy of whole recognizer；The real-time prison of multi-channel video in addition, having carried out GPU acceleration, can be met
Survey.
The technical solution used in the present invention is：Real-time body's abnormal behaviour identification side based on multiple dimensioned convolutional neural networks
Method, including：
S1, determine the structure of multiple dimensioned convolutional neural networks；Including ground floor, the second layer, third layer, the 4th layer, the 5th
Layer, layer 6 and layer 7；Described ground floor is input layer, comprises three passages, and this three passages accept gray scale respectively and turn
Two passage Ox, Oy of the dense optical flow that the video image information of current time upper one second after changing and this video calculate；The
Two layers is Three dimensional convolution layer, be n with quantity, yardstick be that the video inputting and light stream are carried out to ground floor for the convolution kernel of cw*ch*cl
Convolution algorithm；Third layer is three-dimensional down-sampling layer, is pw with yardstick, and the convolution kernel of ph, pl carries out maximum to the output of the second layer
Chi Hua；4th layer is Three dimensional convolution layer, for carrying out convolution algorithm to the output of third layer；Layer 5 is NIN layer, adopts by two
The network composition of layer perceptron convolutional layer, for extracting the nonlinear characteristic of human body behavior according to the 4th layer of output；Layer 6
It is pyramid down-sampling layer, be made up of different size of three-dimensional down-sampling layer, non-for human body behavior that layer 5 is exported
Linear character carries out down-sampling process；Layer 7 is full articulamentum, according to the output of layer 6 be fixed the feature of dimension to
Amount；
Wherein, Ox represents component on x axis of orientation for the light stream, and Oy represents component on y axis of orientation for the light stream；Cw represents volume
The width of long-pending core, ch represents convolution kernel height, and cl represents convolution kernel length on a timeline；
S2, the off-line training of multiple dimensioned convolutional neural networks；By being learnt in abnormal human body behavior storehouse, obtain net
Network parameter model combines the structure of the multiple dimensioned convolutional neural networks that step S1 determines as model file during ONLINE RECOGNITION；
S3, the ONLINE RECOGNITION of convolutional neural networks；By video input model file being obtained the spy as basis of characterization
Levy vector.
Further, the three-dimensional down-sampling described in third layer adopts below equation：
Wherein, x ' represents input vector, and y ' represents the output that obtains after sampling, and s, t and r are picture traverse, highly respectively
Sampling step length with three directions of length of the video time；In Three dimensional convolution, the characteristic pattern of last layer output is the matrix of two dimension,
S1、S2With total columns, m, n then represent m row in this matrix, n row, and 0≤m ＜ S to the total line number being respectively this two-dimensional matrix1,
0≤n ＜ S2；S3Represent the time span of video, that is, video long S altogether3Frame, l represents l frame, and 0≤l ＜ S3；I represents this
Two-dimensional matrix line number sequence number；J represents this two-dimensional matrix columns sequence number；K represents video frame number.
Further, described down-sampling processes and is specially：The nonlinear characteristic of the human body behavior of layer 5 output is carried out
Then each unit spliced of the characteristic pattern obtaining is become a vector by multiple window sizes and the overlapping down-sampling of step-length, makees
Output for layer 6.
Further, the network paramter models described in step S2；Specifically comprise following step by step：
The Sample Storehouse that S21, loading are collected and marked, described Sample Storehouse comprises positive sample and negative sample, described positive sample
For abnormal human body behavior video, negative sample is normal human body behavior video；
The multiple dimensioned convolutional neural networks that S22, loading are determined by step S1；
S23, video is pre-processed after input multiple dimensioned convolutional neural networks；
S24, judge that whether the error of multiple dimensioned convolutional neural networks is less than threshold value, if then going to step S25；Otherwise will
The output result of multiple dimensioned convolutional neural networks enters multiple dimensioned convolutional neural networks with the difference reverse conduction of true tag, and
Adjustment network parameter, execution step S23 re -training；
S25, the network parameter of the multiple dimensioned convolutional neural networks of preservation.
Further, described pretreatment is：Be converted to gray scale color space, and deduct average, extract half-tone information and
Optic flow information.
Further, the ONLINE RECOGNITION of the convolutional neural networks described in step S3；Specifically include following step by step：
S31, obtain the video V of a second from camera or video file；
S32, the video obtaining step S31 zoom to fixed resolution according to demand；
S33, step S32 is scaled after video first carry out gray processing process, and calculate dense optical flow and obtain two light streams
Passage Ox, Oy；
S34, gray processing in step S33 is processed after the image that obtains carry out whitening processing；
S35, the O that will be obtained by the image obtaining after step S34 whitening processing and step S33x, OyIt is separately input to many
In yardstick convolutional neural networks, output characteristic vector F after network calculations；
S36, video features F being input in grader C, judging the behavior species of this video, if belonging to abnormal behaviour
Then process abnormal.
Beneficial effects of the present invention：The application instead of traditional feature extraction algorithm using convolutional neural networks, and will
Convolutional neural networks improve, and adapt to the demand of human body behavior classification；Specifically increased Three dimensional convolution, three-dimensional down-sampling, NIN,
Three-dimensional pyramid structure is so that convolutional neural networks can have higher ability in feature extraction to human body abnormal behaviour；Specific
Video concentration training, obtain the feature with more classification capacity, strengthen the robustness of whole recognizer and accuracy；Separately
Outward, in order to the demand meeting practical application has carried out GPU acceleration, the real-time monitoring of multi-channel video can be met；
The present processes have advantages below：
1st, instead of the method for traditional feature extraction so that convolutional neural networks can be to human body with convolutional neural networks
Abnormal behaviour has higher ability in feature extraction；
2nd, the information that Three dimensional convolution is extracted time-domain, preferably capture movement information are employed；
3rd, amount of calculation is not only greatly reduced using three-dimensional down-sampling technology, and introduce algorithm in time-domain when
Between consistency, improve the stability of identification and the discrimination of Geng Gao；
4th, adopt NIN so that the more complicated people of structure extraction of multiple dimensioned convolutional neural networks that proposes of the application
Body behavior nonlinear characteristic；
5th, the flexibility of system is improved so that the video segment of different resolution and duration is permissible using pyramid structure
This system can be used without any changes, improve flexibility and the range of application of system；
6th, increased light circulation road in input, allow whole algorithm that higher recognition capability is had on time-domain；
7th, network employs specifically abnormal Human bodys' response storehouse and is learnt, by increase as far as possible sample size and
Increase the scene species of sample, being capable of preferably training pattern；
8th, accelerate to allow whole recognizer to meet many videos real-time detection using gpu.
Brief description
The multiple dimensioned convolutional neural networks configuration diagram that Fig. 1 provides for the present invention.
The comparison diagram of the Three dimensional convolution that Fig. 2 provides for the present invention and two-dimensional convolution；
Wherein, a figure is the schematic diagram of Three dimensional convolution, and b figure is the schematic diagram of two-dimensional convolution.
Linear convolution and MLP convolution schematic diagram that Fig. 3 provides for the present invention.
The pyramid structure schematic diagram that Fig. 4 provides for the present invention.
The parameter model training flow chart that Fig. 5 provides for the present invention.
The ONLINE RECOGNITION flow chart that Fig. 6 provides for the present invention.
Specific embodiment
For ease of skilled artisan understands that the technology contents of the present invention, below in conjunction with the accompanying drawings one being entered to present invention
Step explaination.
The technical scheme is that：Based on real-time body's abnormal behaviour recognition methods of multiple dimensioned convolutional neural networks,
Including：
S1, determine the structure of multiple dimensioned convolutional neural networks；As shown in figure 1, include ground floor, the second layer, third layer, the
Four layers, layer 5, layer 6 and layer 7；If each layer all comprises dry contact, interstitial content determines extraction feature species
Number, data is more, and the characteristic information of extraction is more, but amount of calculation is also bigger.
Ground floor is input layer input, comprises three passages, this three passages accept after gradation conversion respectively current when
Engrave the video image information of a second and two passage Ox, Oy of dense optical flow that this video calculates；In original video input
Add two light circulation roads on the basis of passage and can largely strengthen the sensitiveness to behavior act, in Activity recognition
There is higher discrimination.
Second layer conv1 is Three dimensional convolution layer, with quantity be n, a size of the convolution kernel of cw*ch*cl to ground floor input
Video and light stream carry out convolution algorithm；The application adds time domain information using Three dimensional convolution computing, can preferably capture
Movable information.
Particular content is：In the task of processing video, need the information from multiple continuous frame-grab campaigns, the application
Different from other convolutional neural networks, employ the convolutional layer of three-dimensional.The formula of Three dimensional convolution such as formula (1)：
Wherein, w is the weight of convolution kernel, and u is input vector, represents the image intensity value of three passages, the level of light stream
Component and vertical component, yefgIt is output, subscript e, f, g represent the element value referring to relevant position, and that is, g frame e row f arranges
Element, P, Q, R are the size of three dimensions respectively, and in Three dimensional convolution, the characteristic pattern feature map of last layer output is two
The matrix of dimension, P, Q be respectively total line number of this two-dimensional matrix with total columns, and small letter p, q then represent pth row in this matrix, q
Row, and 0≤p ＜ P, 0≤q ＜ Q；R represents the length of video, that is, video long R frame altogether, and small letter r represents r frame, and 0≤r
＜ R.
Here Three dimensional convolution computing is considered as the cube being got up with three-dimensional core come the accumulation of convolution multiple frame, such as schemes
2a show the schematic diagram of Three dimensional convolution, and reference axis indicates three dimensions：The time of the width of image, height and video,
The cube of lower section represents the input of convolution, and the cube of top represents the output of convolution.Fig. 2 b is the schematic diagram of two-dimensional convolution,
Its input and output are all the rectangles of two dimension, only include width, the elevation information of image, the no information of time-domain.With three-dimensional
Convolution, the feature map of each convolutional layer is the multiple successive frames being associated with preceding layer, thus captures motion letter
Breath.
Third layer pool1 is three-dimensional down-sampling layer, is pw with yardstick, and the convolution kernel of ph, pl carries out maximum pond；The application
Using three-dimensional down-sampling technology not only greatly reduce amount of calculation, and it is constant to introduce the time in time-domain for the algorithm
Property, improve the stability of identification and the discrimination of Geng Gao.
Particular content is：The same with three-dimensional convolution, when convolutional neural networks process video, down-sampling layer also needs
Three-dimensional to be extended to.Process the down-sampling layer of the convolutional neural networks of picture, so that after data volume strongly reduces, accelerates
Calculating, also make network have certain consistency, consistency here is the consistency in spatial domain simultaneously.And at place
When reason video, certain consistency is also required on time-domain, and the data processing of video is than single frames
Picture is much bigger, therefore it is necessary to down-sampling is also extended to three-dimensional.The formula such as (2) of three-dimensional overlapping maximum down-sampling：
Wherein, x ' is three-dimensional input vector, represents the feature (feature map) that previous convolutional layer extracts, that is, second
The feature that layer Three dimensional convolution extracts, y ' is the output that obtains after sampling, and s, t and r be picture traverse respectively, highly with video time
The sampling step length in three directions of length；In Three dimensional convolution, the characteristic pattern feature map of last layer output is the matrix of two dimension,
I.e. the characteristic pattern of the second layer output in the application is two-dimensional matrix, S1、S2The total line number being respectively this two-dimensional matrix arranges with total
Number, m, n then represent m row in this matrix, n row, and 0≤m ＜ S1, 0≤n ＜ S2；S3Represent the length of video, that is, regard
Frequency long S altogether3Frame, small letter l represents l frame, and 0≤l ＜ S3；I represents this two-dimensional matrix line number sequence number, i=1,2, S1；
J represents this two-dimensional matrix columns sequence number, j=1,2, S2；K represents video frame number, k=1,2, S3；Although
S in the Three dimensional convolution computing of third layer1、S2、S3, m, n, l and the second layer Three dimensional convolution computing in P, Q, R, p, q, r institute right
Identical implication should be expressed, but respective value differing in the second layer and third layer, adopt different letters here
In order to distinguish；Feature map data volume after sampling reduces at double, and amount of calculation also can greatly reduce, meanwhile, network to when
Between change more robust on domain.
4th layer of conv2 is Three dimensional convolution layer, similar with the second layer, for doing convolution algorithm to the output of third layer.
Layer 5 is NIN (Network in Network) layer, and the network using the perceptron convolutional layer by two-layer forms,
For extracting the nonlinear characteristic of human body behavior according to the 4th layer of output；Using NIN so that the system can be extracted more
Plus the human body behavior nonlinear characteristic of complexity.
Particular content is：Employ NIN structure in the application and improve whole network structure.Volume in convolutional neural networks
Long-pending is a generalized linear model (generalized linear model, GLM).GLM is only when sample is linear separability
It is abstract that time has had.That the convolution of convolutional neural networks is implied by this linear separability it is assumed that but human body behavior model being discontented with
This hypothesis of foot.So, GLM is substituted for and there is the model of non-linear sign ability is capable of the abstracting power of boosting algorithm.
Feedforward neural network or multi-layer perception (MLP) (MLP) are the strong nonlinear models of an abstracting power, if use
Common linear convolution core is substituted for and does nonlinear convolution operation with MLP, will necessarily increase the abstracting power of model.In network
It is referred to herein as MLP convolutional layer with the layer that MLP does convolution, be only referred to as linear convolution layer with the convolutional layer that linear kernel does convolution.Make
It is referred to herein as NIN (Networks in Networks) with the network of this MLP convolutional layer.
Classical convolution operation and MLP convolution operation are as shown in Figure 3.Classical convolution is that line is made in the input to one piece of region
Property weighted sum, and exported by a nonlinear activation function, fit simple nonlinear model.MLP convolution uses
One multi-layer perception (MLP) being made up of multiple full articulamentums calculates in the enterprising line slip of feature map of last layer, then passes through
One nonlinear activation function (adopting ReLU here), has thus obtained the feature map of current layer.The meter of MLP convolution
Calculation mode such as (3)：
Wherein, (i, j) is the pixel index of the feature map of current layer, xijIt is the input block at (i, j) for the center, kn
It is the index of the feature map of current layer, n is the number of plies of MPL.Because being ReLU (Y=max (0, X)) the activation letter using
Number, so using the maximum comparing with 0 in formula.
From the point of view of another kind of viewpoint, MLP convolutional layer is equivalent to multiple linear convolution layers, if a MLP has n-layer, then one
Individual MLP convolutional layer can regard n linear convolution layer as, be this n convolutional layer rear n-1 layer be 1 × 1 core, and often
The part feature map of individual map convolution last layer of feature, using NIN so that the system can be extracted more
Complicated human body behavior nonlinear characteristic.
Layer 6 pyramid is three-dimensional pyramid down-sampling layer, is made up of different size of three-dimensional down-sampling layer, in order to right
The output of layer 5 carries out down-sampling process, obtains the output feature map of different resolution；Three Vygens that the application adopts
The flexibility of word tower down-sampling this skill upgrading of layer system is so that the video segment of different resolution and duration can not do
Any change can use this system, improves flexibility and the range of application of system.
Particular content is：The three-dimensional pyramid down-sampling layer of the application is made up of the feature map of multiple resolution ratio,
Conventional down-sampling layer is all with identical sampling scale and the feature map that inputs has identical size, so obtaining
Feature map have identical resolution ratio.And pyramid down-sampling layer obtained using multiple sampling scale a series of solid
The feature map of fixed different resolution.
The sample of Activity recognition is all some video segments, may have different resolution ratio it is also possible to different videos is long
Degree.These othernesses so that traditional convolutional neural networks have no idea process because traditional convolutional neural networks each
Feature map is fixed size.Cause traditional convolutional neural networks cannot process different resolution and length video
Reason does not lie in convolutional layer and down-sampling layer, but at full articulamentum (the FC layer in Fig. 1), because the framework of full articulamentum is solid
Fixed, have no idea to change, the size that result in the feature map of this layer of input also must be fixing.And in convolutional layer,
The size of the feature map of input does not interfere with the structure of network, and the size of the feature map simply exporting can be with this
The change of feature map size of layer input and change, because convolution kernel simply slides on input feature map.?
Down-sampling layer, simply to reduce size in a manner input feature map, also not to interfere with the structure of network.By
This is apparently necessary for carrying out the process of suitable feature map size before full articulamentum so that various sizes of input
Feature map obtains identical size.
This process can be realized using the overlapping down-sampling of different windows size and different step-length.With two-dimensional case it is
Example, the extension that three-dimensional situation can be natural.The size of the feature map of hypothesis input is a × a, needs to be down sampled to
Size n × n, currently uses window size：
Slide step-length be：
In formulaFor the operation that rounds up,For downward floor operation.Such as now with a=13, n=3, then according to public affairs
Formula (4) and (5) we obtain win=5 and str=4.
Determine that using above formula window size and sliding step can be effectively from different input sizes
Feature map obtains Output Size identical feature map.But different resolution ratio uses identical size meeting
Make high-resolution feature map lose too many information, and the feature map of low resolution may sample very little and not have
There is consistency, so, adopt pyramidal mode here.As shown in Figure 4.Pyramid is by some different resolutions
Feature map forming, existing comparatively bigger resolution ratio, also have little resolution ratio, also there are some mistakes centre
Cross the resolution ratio of size.(as 16*256-d in Fig. 3,4*256-d, 256-d) wants to obtain difference from the feature map of input
The output feature map of resolution ratio needs to carry out the overlapping down-sampling of multiple window sizes and step-length.Obtain these
Each unit of feature map is spliced into a vector again, the such as L-Fix layer of Fig. 4, then, then accesses full articulamentum.In figure
Example in 4 is 3 grades of pyramids, and fixing resolution ratio is respectively 3 × 3,2 × 2,1 × 1, and last layer has 256
feature map.
The output unit total number of pyramidal layer is all fixing, it is possible to fixing output feature map even
Connect full articulamentum, and, the introducing of pyramid model, the feature map of multiple resolution ratio can be formed, it also avoid regular not
The impact being brought with the input of resolution ratio.
In the present invention, that use is all three-dimensional feature map, pyramid model is expanded into three-dimensional also very square
Just, often one-dimensional window size and step-length, all according to formula (4) and (5), input the ratio E with the length of side of output by calculating,
Again by the acquisition window size that rounds up, round downwards acquisition step-length.
Layer 7 FC is full articulamentum, for exporting the characteristic vector of fixed dimension, is supplied to grader (softmax) and makees
For identifying the characteristic of division of human body behavior.
S2, the off-line training of multiple dimensioned convolutional neural networks；By being learnt in abnormal human body behavior storehouse, obtain net
Network parameter model combines the structure of the multiple dimensioned convolutional neural networks that step S1 determines as model file during ONLINE RECOGNITION；As
Described parameter model shown in Fig. 5 specifically comprise following step by step：
The Sample Storehouse that S21, loading have marked, described Sample Storehouse comprises positive sample and negative sample, and described positive sample is abnormal
Human body behavior video, negative sample be normal human body behavior video；
The multiple dimensioned convolutional neural networks that S22, loading are determined by step S1；
S23, video is pre-processed after input multiple dimensioned convolutional neural networks；Described pretreatment be：Be converted to gray scale
Color space, and deduct average, extract half-tone information and Optic flow information；
Whether the difference of S24, the output result judging multiple dimensioned convolutional neural networks and true tag is less than threshold value, if
Then go to step S25；Otherwise the difference reverse conduction of the output result of multiple dimensioned convolutional neural networks and true tag is entered
Multiple dimensioned convolutional neural networks, and adjust network parameter, execution step S23；Here network parameter includes convolutional neural networks
Weight coefficient and grader connection weight, learning to suitable parameter is the off-line training of multiple dimensioned convolutional neural networks
Purpose.
S25, the network parameter of the multiple dimensioned convolutional neural networks of preservation；These network parameters will be used for behavioral value.
S3, the ONLINE RECOGNITION of convolutional neural networks；By video input model file being obtained the spy as basis of characterization
Levy vector.Specifically include as shown in Figure 6 following step by step：
S31, obtain the video V of a second from camera or video file, if N two field picture composition, then just have every
One two field picture Vi, i=[1, N]；The video being directed to one second length every time carries out the identification of human body behavior.
S32, the video obtaining step S31 zoom to fixed resolution, Vi=resize (Vi) according to demand.
S33, step S32 is scaled after video first carry out gray processing process, and calculate dense optical flow and obtain two light streams
Passage Ox, Oy；Dense optical flow is a kind of method for registering images carrying out pointwise coupling for the adjacent image in video, is different from
Just for several characteristic points on image, dense optical flow calculates the side-play amount of all of point on image to sparse optical flow, thus being formed
One dense optical flow field.By this dense optical flow field, the image registration of pixel scale can be carried out, so after its registration
Effect be also significantly better than sparse optical flow registration effect.
S34, the image H after gray processing obtaining in step S33 is carried out whitening processing；Deduct each frame figure
Average H of picturei(mean), Hi:=Hi-Hi(mean), then by H normalize, H:=(H-0)/(256-0)；Albefaction and normalized
Purpose is that the information of input is carried out a certain degree of denoising, the computing of convolutional neural networks after also allowing for.
S35, the O that image through whitening processing will be obtained by step S34 and step S33 obtainsx, OyIt is separately input to
In multiple dimensioned convolutional neural networks, output characteristic vector F after network calculations, F are exactly the use that goes out of video extraction of this second
In the feature that video human behavior is classified；
S36, video features F being input in grader C, judging the behavior species of this video, if belonging to abnormal behaviour
Then process abnormal.
So it is recycled to, after record result, the video that step S31 obtains next second.The ONLINE RECOGNITION of convolutional neural networks
During be directed to every time the video of one second length and carry out the identification of human body behavior；The video scaling of each second is to fixing resolution ratio
Afterwards, using dense optical flow algorithm, calculate Optical-flow Feature；Then whitening processing will be carried out after video gradation together with Optical-flow Feature
And normalized；The abstract characteristics of higher-dimension can be obtained afterwards by convolutional neural networks.The feature of these outputs is exactly last
The input of human body behavior grader.
The convolutional neural networks of the use in the application have very strong ability in feature extraction it is possible to train identification
The grader of the outstanding multiple difference behavior of ability.In order to meet what convolutional neural networks can be exported by different application scenarios
In the different grader of characteristic vector input, (classification is beaten for such as C1 grader (classification is fought and normal behaviour), C2 grader
Struggle against, run, normal behaviour).Such combination will have higher adaptive capacity to environment and using value.
Consider the convolutional neural networks using in the present invention, it is big that number of parameters is big, Three dimensional convolution computing calculates consumption
And the feature of the complicated network structure, employ the technology of GPU acceleration.Employ CUDA storehouse and cuDNN storehouse in the present invention excellent
Change the recognition speed of whole algorithm.CUDA storehouse is mainly used in convolution algorithm in whole network computing, first according to matrix size
Distribute corresponding video memory, afterwards parallel distribution multiple tasks in multiple GPU cores, actual is exactly to be whole matrix-split
Multiple minor matrixs, and cuDNN storehouse mainly optimizes computational efficiency during three-dimensional computing further.Table 1 is the video of a minute
Recognition speed.
Table 1 parallel detection speedometer
From table it is seen that, when this algorithm detects multiple video at the same time, using GPU speed technology, can meet and connect
The real-time detection of nearly 20 videos, the algorithm comparing conventional body's Activity recognition is advantageous in recognition speed on the contrary.Such
Recognition speed can carry out real-time detection completely in various real application scenarios.
Those of ordinary skill in the art will be appreciated that, embodiment described here is to aid in reader and understands this
Bright principle is it should be understood that protection scope of the present invention is not limited to such special statement and embodiment.For ability
For the technical staff in domain, the present invention can have various modifications and variations.All within the spirit and principles in the present invention, made
Any modification, equivalent substitution and improvement etc., should be included within scope of the presently claimed invention.
Claims (6)
1. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks is it is characterised in that include：
S1, determine the structure of multiple dimensioned convolutional neural networks；Including ground floor, the second layer, third layer, the 4th layer, layer 5,
Six layers and layer 7；Described ground floor is input layer, comprises three passages, after this three passages accept gradation conversion respectively
Two passage Ox, Oy of the dense optical flow that the video image information of current time upper one second and this video calculate；The second layer is
Three dimensional convolution layer is n with quantity, yardstick is that the video inputting and light stream carry out convolution fortune to ground floor for the convolution kernel of cw*ch*cl
Calculate；Third layer is three-dimensional down-sampling layer, is pw with yardstick, and the convolution kernel of ph, pl carries out maximum pond to the output of the second layer；The
Four layers is Three dimensional convolution layer, for carrying out convolution algorithm to the output of third layer；Layer 5 is NIN layer, perceives using by two-layer
The network composition of machine convolutional layer, for extracting the nonlinear characteristic of human body behavior according to the 4th layer of output；Layer 6 is golden word
Tower down-sampling layer, is made up of different size of three-dimensional down-sampling layer, the non-linear spy of the human body behavior for exporting to layer 5
Levy and carry out down-sampling process；Layer 7 is full articulamentum, is fixed the characteristic vector of dimension according to the output of layer 6；
Wherein, Ox represents component on x axis of orientation for the light stream, and Oy represents component on y axis of orientation for the light stream；Cw represents convolution kernel
Width, ch represents convolution kernel height, and cl represents convolution kernel length on a timeline；
S2, the off-line training of multiple dimensioned convolutional neural networks；By being learnt in abnormal human body behavior storehouse, obtain network ginseng
Exponential model combines the structure of the multiple dimensioned convolutional neural networks that step S1 determines as model file during ONLINE RECOGNITION；
S3, the ONLINE RECOGNITION of convolutional neural networks；By video input model file is obtained feature as basis of characterization to
Amount.
2. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks according to claim 1, its
It is characterised by, the three-dimensional down-sampling described in third layer adopts below equation：
Wherein, x ' represents input vector, and y ' represents the output that obtains after sampling, and s, t and r be picture traverse respectively, highly with regarding
The sampling step length in three directions of frequency time span；In Three dimensional convolution, the characteristic pattern of last layer output is the matrix of two dimension, S1、S2
With total columns, m, n then represent m row in this matrix, n row, and 0≤m ＜ S to the total line number being respectively this two-dimensional matrix1, 0≤n
＜ S2；S3Represent the time span of video, that is, video long S altogether3Frame, l represents l frame, and 0≤l ＜ S3；I represents this two dimension
Row matrix number sequence number；J represents this two-dimensional matrix columns sequence number；K represents video frame number.
3. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks according to claim 1, its
It is characterised by, down-sampling described in layer 6 processes and is specially：The nonlinear characteristic of the human body behavior of layer 5 output is carried out
Then each unit spliced of the characteristic pattern obtaining is become a vector by multiple window sizes and the overlapping down-sampling of step-length, makees
Output for layer 6.
4. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks according to claim 1, its
It is characterised by, the network paramter models described in step S2；Specifically comprise following step by step：
The Sample Storehouse that S21, loading are collected and marked, described Sample Storehouse comprises positive sample and negative sample, and described positive sample is different
Normal human body behavior video, negative sample is normal human body behavior video；
The multiple dimensioned convolutional neural networks that S22, loading are determined by step S1；
S23, video is pre-processed after input multiple dimensioned convolutional neural networks；
S24, judge that whether the error of multiple dimensioned convolutional neural networks is less than threshold value, if then going to step S25；Otherwise by many chis
The output result of degree convolutional neural networks enters multiple dimensioned convolutional neural networks with the difference reverse conduction of true tag, and adjusts
Network parameter, execution step S23 re -training；
S25, the network parameter of the multiple dimensioned convolutional neural networks of preservation, obtain network paramter models.
5. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks according to claim 4, its
It is characterised by, described pretreatment is：Be converted to gray scale color space, and deduct average, extract half-tone information and Optic flow information.
6. the real-time body's abnormal behaviour recognition methods based on multiple dimensioned convolutional neural networks according to claim 1, its
It is characterised by, the ONLINE RECOGNITION of the convolutional neural networks described in step S3；Specifically include following step by step：
S31, obtain the video V of a second from camera or video file；
S32, the video obtaining step S31 zoom to fixed resolution according to demand；
S33, step S32 is scaled after video first carry out gray processing process, and calculate dense optical flow and obtain two light circulation roads
Ox, Oy；
S34, gray processing in step S33 is processed after the image that obtains carry out whitening processing；
S35, the O that will be obtained by the image obtaining after step S34 whitening processing and step S33x, OyIt is separately input to multiple dimensioned
In convolutional neural networks, output characteristic vector F after network calculations；
S36, video features F is input in grader C, judging the behavior species of this video, if belonging to abnormal behaviour, locating
Reason is abnormal.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610790306.0A CN106407903A (en) | 2016-08-31 | 2016-08-31 | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610790306.0A CN106407903A (en) | 2016-08-31 | 2016-08-31 | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106407903A true CN106407903A (en) | 2017-02-15 |
Family
ID=58001242
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610790306.0A Pending CN106407903A (en) | 2016-08-31 | 2016-08-31 | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106407903A (en) |
Cited By (37)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980826A (en) * | 2017-03-16 | 2017-07-25 | 天津大学 | A kind of action identification method based on neutral net |
| CN107016415A (en) * | 2017-04-12 | 2017-08-04 | 合肥工业大学 | A kind of coloured image Color Semantic sorting technique based on full convolutional network |
| CN107194559A (en) * | 2017-05-12 | 2017-09-22 | 杭州电子科技大学 | A kind of work stream recognition method based on Three dimensional convolution neutral net |
| CN107256386A (en) * | 2017-05-23 | 2017-10-17 | 东南大学 | Human behavior analysis method based on deep learning |
| CN107341452A (en) * | 2017-06-20 | 2017-11-10 | 东北电力大学 | Human bodys' response method based on quaternary number space-time convolutional neural networks |
| CN107510452A (en) * | 2017-09-30 | 2017-12-26 | 扬美慧普（北京）科技有限公司 | A kind of ECG detecting method based on multiple dimensioned deep learning neutral net |
| CN107862275A (en) * | 2017-11-01 | 2018-03-30 | 电子科技大学 | Human bodys' response model and its construction method and Human bodys' response method |
| CN107992894A (en) * | 2017-12-12 | 2018-05-04 | 北京小米移动软件有限公司 | Image-recognizing method, device and computer-readable recording medium |
| CN108124485A (en) * | 2017-12-28 | 2018-06-05 | 深圳市锐明技术股份有限公司 | For the alarm method of limbs conflict behavior, device, storage medium and server |
| CN108805083A (en) * | 2018-06-13 | 2018-11-13 | 中国科学技术大学 | The video behavior detection method of single phase |
| CN108830327A (en) * | 2018-06-21 | 2018-11-16 | 中国科学技术大学 | A kind of crowd density estimation method |
| CN108830204A (en) * | 2018-06-01 | 2018-11-16 | 中国科学技术大学 | The method for detecting abnormality in the monitor video of target |
| CN108898042A (en) * | 2017-12-27 | 2018-11-27 | 浩云科技股份有限公司 | A kind of detection method applied to user's abnormal behaviour in ATM machine cabin |
| CN109145874A (en) * | 2018-09-28 | 2019-01-04 | 大连民族大学 | Measure application of the difference in the detection of obstacles of Autonomous Vehicle visual response part between video successive frame and its convolution characteristic pattern |
| CN109359519A (en) * | 2018-09-04 | 2019-02-19 | 杭州电子科技大学 | A kind of video anomaly detection method based on deep learning |
| CN109492755A (en) * | 2018-11-07 | 2019-03-19 | 北京旷视科技有限公司 | Image processing method, image processing apparatus and computer readable storage medium |
| CN109636802A (en) * | 2019-01-18 | 2019-04-16 | 天津工业大学 | Pulmonary parenchyma based on depth convolutional neural networks is through CT image partition method |
| CN110019793A (en) * | 2017-10-27 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of text semantic coding method and device |
| CN110059545A (en) * | 2019-03-08 | 2019-07-26 | 佛山市云米电器科技有限公司 | A kind of smart home user behavior recognition method based on convolutional neural networks |
| CN110120020A (en) * | 2019-04-30 | 2019-08-13 | 西北工业大学 | A kind of SAR image denoising method based on multiple dimensioned empty residual error attention network |
| CN110245696A (en) * | 2019-05-30 | 2019-09-17 | 武汉智云集思技术有限公司 | Illegal incidents monitoring method, equipment and readable storage medium storing program for executing based on video |
| CN110298210A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | A kind of method and apparatus that view-based access control model is judged extremely |
| CN110462637A (en) * | 2017-03-24 | 2019-11-15 | 华为技术有限公司 | Neural Network Data processing unit and method |
| CN110533053A (en) * | 2018-05-23 | 2019-12-03 | 杭州海康威视数字技术股份有限公司 | A kind of event detecting method, device and electronic equipment |
| CN110574077A (en) * | 2017-03-24 | 2019-12-13 | 株式会社Jlk英思陪胜 | Image analysis device and method using virtual three-dimensional deep neural network |
| CN110634557A (en) * | 2019-08-23 | 2019-12-31 | 广东省智能制造研究所 | Medical care resource auxiliary allocation method and system based on deep neural network |
| CN110866526A (en) * | 2018-08-28 | 2020-03-06 | 北京三星通信技术研究有限公司 | Image segmentation method, electronic device and computer-readable storage medium |
| CN111126115A (en) * | 2018-11-01 | 2020-05-08 | 顺丰科技有限公司 | Violence sorting behavior identification method and device |
| CN111178344A (en) * | 2020-04-15 | 2020-05-19 | 中国人民解放军国防科技大学 | Multi-scale time sequence behavior identification method |
| CN111507297A (en) * | 2020-04-24 | 2020-08-07 | 中国科学院空天信息创新研究院 | Radar signal identification method and system based on measurement information matrix |
| CN111832336A (en) * | 2019-04-16 | 2020-10-27 | 四川大学 | Improved C3D video behavior detection method |
| CN111898418A (en) * | 2020-06-17 | 2020-11-06 | 北京航空航天大学 | Human body abnormal behavior detection method based on T-TINY-YOLO network |
| CN112986210A (en) * | 2021-02-10 | 2021-06-18 | 四川大学 | Scale-adaptive microbial Raman spectrum detection method and system |
| CN113221689A (en) * | 2021-04-27 | 2021-08-06 | 苏州工业职业技术学院 | Video multi-target emotion prediction method and system |
| CN114973020A (en) * | 2022-06-15 | 2022-08-30 | 北京鹏鹄物宇科技发展有限公司 | Abnormal behavior analysis method based on satellite monitoring video |
| CN116402691A (en) * | 2023-06-05 | 2023-07-07 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
| CN116433755A (en) * | 2023-03-31 | 2023-07-14 | 哈尔滨工业大学 | Structure dense displacement recognition method and system based on deformable three-dimensional model and optical flow representation learning |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281853A (en) * | 2014-09-02 | 2015-01-14 | 电子科技大学 | Behavior identification method based on 3D convolution neural network |
-
2016
- 2016-08-31 CN CN201610790306.0A patent/CN106407903A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104281853A (en) * | 2014-09-02 | 2015-01-14 | 电子科技大学 | Behavior identification method based on 3D convolution neural network |
Non-Patent Citations (2)
| Title |
|---|
| MIN LIN等: "Network In Network", 《NEURAL AND EVOLUTIONARY COMPUTING (CS.NE)》 * |
| 吴杰: "基于卷积神经网络的行为识别研究", 《中国优秀硕士学位论文全文数据库-信息科技辑》 * |
Cited By (53)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106980826A (en) * | 2017-03-16 | 2017-07-25 | 天津大学 | A kind of action identification method based on neutral net |
| CN110574077B (en) * | 2017-03-24 | 2023-08-01 | 株式会社Jlk英思陪胜 | Image analysis device and method using virtual three-dimensional deep neural network |
| CN110462637A (en) * | 2017-03-24 | 2019-11-15 | 华为技术有限公司 | Neural Network Data processing unit and method |
| CN110574077A (en) * | 2017-03-24 | 2019-12-13 | 株式会社Jlk英思陪胜 | Image analysis device and method using virtual three-dimensional deep neural network |
| CN110462637B (en) * | 2017-03-24 | 2022-07-19 | 华为技术有限公司 | Neural network data processing device and method |
| CN107016415A (en) * | 2017-04-12 | 2017-08-04 | 合肥工业大学 | A kind of coloured image Color Semantic sorting technique based on full convolutional network |
| CN107016415B (en) * | 2017-04-12 | 2019-07-19 | 合肥工业大学 | A kind of color image Color Semantic classification method based on full convolutional network |
| CN107194559B (en) * | 2017-05-12 | 2020-06-05 | 杭州电子科技大学 | Workflow identification method based on three-dimensional convolutional neural network |
| CN107194559A (en) * | 2017-05-12 | 2017-09-22 | 杭州电子科技大学 | A kind of work stream recognition method based on Three dimensional convolution neutral net |
| CN107256386A (en) * | 2017-05-23 | 2017-10-17 | 东南大学 | Human behavior analysis method based on deep learning |
| CN107341452B (en) * | 2017-06-20 | 2020-07-14 | 东北电力大学 | Human behavior identification method based on quaternion space-time convolution neural network |
| CN107341452A (en) * | 2017-06-20 | 2017-11-10 | 东北电力大学 | Human bodys' response method based on quaternary number space-time convolutional neural networks |
| CN107510452B (en) * | 2017-09-30 | 2019-10-08 | 扬美慧普（北京）科技有限公司 | A kind of ECG detecting method based on multiple dimensioned deep learning neural network |
| CN107510452A (en) * | 2017-09-30 | 2017-12-26 | 扬美慧普（北京）科技有限公司 | A kind of ECG detecting method based on multiple dimensioned deep learning neutral net |
| CN110019793A (en) * | 2017-10-27 | 2019-07-16 | 阿里巴巴集团控股有限公司 | A kind of text semantic coding method and device |
| CN107862275A (en) * | 2017-11-01 | 2018-03-30 | 电子科技大学 | Human bodys' response model and its construction method and Human bodys' response method |
| CN107992894A (en) * | 2017-12-12 | 2018-05-04 | 北京小米移动软件有限公司 | Image-recognizing method, device and computer-readable recording medium |
| CN108898042B (en) * | 2017-12-27 | 2021-10-22 | 浩云科技股份有限公司 | Method for detecting abnormal user behavior in ATM cabin |
| CN108898042A (en) * | 2017-12-27 | 2018-11-27 | 浩云科技股份有限公司 | A kind of detection method applied to user's abnormal behaviour in ATM machine cabin |
| WO2019127271A1 (en) * | 2017-12-28 | 2019-07-04 | 深圳市锐明技术股份有限公司 | Warning method, device, storage medium and server regarding physical conflict behavior |
| CN108124485A (en) * | 2017-12-28 | 2018-06-05 | 深圳市锐明技术股份有限公司 | For the alarm method of limbs conflict behavior, device, storage medium and server |
| CN110298210A (en) * | 2018-03-21 | 2019-10-01 | 北京猎户星空科技有限公司 | A kind of method and apparatus that view-based access control model is judged extremely |
| CN110533053A (en) * | 2018-05-23 | 2019-12-03 | 杭州海康威视数字技术股份有限公司 | A kind of event detecting method, device and electronic equipment |
| CN108830204B (en) * | 2018-06-01 | 2021-10-19 | 中国科学技术大学 | Method for detecting abnormality in target-oriented surveillance video |
| CN108830204A (en) * | 2018-06-01 | 2018-11-16 | 中国科学技术大学 | The method for detecting abnormality in the monitor video of target |
| CN108805083A (en) * | 2018-06-13 | 2018-11-13 | 中国科学技术大学 | The video behavior detection method of single phase |
| CN108830327A (en) * | 2018-06-21 | 2018-11-16 | 中国科学技术大学 | A kind of crowd density estimation method |
| CN108830327B (en) * | 2018-06-21 | 2022-03-01 | 中国科学技术大学 | Crowd density estimation method |
| CN110866526A (en) * | 2018-08-28 | 2020-03-06 | 北京三星通信技术研究有限公司 | Image segmentation method, electronic device and computer-readable storage medium |
| CN109359519A (en) * | 2018-09-04 | 2019-02-19 | 杭州电子科技大学 | A kind of video anomaly detection method based on deep learning |
| CN109145874A (en) * | 2018-09-28 | 2019-01-04 | 大连民族大学 | Measure application of the difference in the detection of obstacles of Autonomous Vehicle visual response part between video successive frame and its convolution characteristic pattern |
| CN111126115A (en) * | 2018-11-01 | 2020-05-08 | 顺丰科技有限公司 | Violence sorting behavior identification method and device |
| CN111126115B (en) * | 2018-11-01 | 2024-06-07 | 顺丰科技有限公司 | Violent sorting behavior identification method and device |
| CN109492755A (en) * | 2018-11-07 | 2019-03-19 | 北京旷视科技有限公司 | Image processing method, image processing apparatus and computer readable storage medium |
| CN109492755B (en) * | 2018-11-07 | 2022-03-01 | 北京旷视科技有限公司 | Image processing method, image processing apparatus, and computer-readable storage medium |
| CN109636802A (en) * | 2019-01-18 | 2019-04-16 | 天津工业大学 | Pulmonary parenchyma based on depth convolutional neural networks is through CT image partition method |
| CN110059545A (en) * | 2019-03-08 | 2019-07-26 | 佛山市云米电器科技有限公司 | A kind of smart home user behavior recognition method based on convolutional neural networks |
| CN111832336B (en) * | 2019-04-16 | 2022-09-02 | 四川大学 | Improved C3D video behavior detection method |
| CN111832336A (en) * | 2019-04-16 | 2020-10-27 | 四川大学 | Improved C3D video behavior detection method |
| CN110120020A (en) * | 2019-04-30 | 2019-08-13 | 西北工业大学 | A kind of SAR image denoising method based on multiple dimensioned empty residual error attention network |
| CN110245696A (en) * | 2019-05-30 | 2019-09-17 | 武汉智云集思技术有限公司 | Illegal incidents monitoring method, equipment and readable storage medium storing program for executing based on video |
| CN110634557A (en) * | 2019-08-23 | 2019-12-31 | 广东省智能制造研究所 | Medical care resource auxiliary allocation method and system based on deep neural network |
| CN110634557B (en) * | 2019-08-23 | 2022-08-23 | 广东省智能制造研究所 | Medical care resource auxiliary allocation method and system based on deep neural network |
| CN111178344A (en) * | 2020-04-15 | 2020-05-19 | 中国人民解放军国防科技大学 | Multi-scale time sequence behavior identification method |
| CN111507297A (en) * | 2020-04-24 | 2020-08-07 | 中国科学院空天信息创新研究院 | Radar signal identification method and system based on measurement information matrix |
| CN111898418A (en) * | 2020-06-17 | 2020-11-06 | 北京航空航天大学 | Human body abnormal behavior detection method based on T-TINY-YOLO network |
| CN112986210A (en) * | 2021-02-10 | 2021-06-18 | 四川大学 | Scale-adaptive microbial Raman spectrum detection method and system |
| CN113221689A (en) * | 2021-04-27 | 2021-08-06 | 苏州工业职业技术学院 | Video multi-target emotion prediction method and system |
| CN114973020A (en) * | 2022-06-15 | 2022-08-30 | 北京鹏鹄物宇科技发展有限公司 | Abnormal behavior analysis method based on satellite monitoring video |
| CN116433755A (en) * | 2023-03-31 | 2023-07-14 | 哈尔滨工业大学 | Structure dense displacement recognition method and system based on deformable three-dimensional model and optical flow representation learning |
| CN116433755B (en) * | 2023-03-31 | 2023-11-14 | 哈尔滨工业大学 | Structure dense displacement recognition method and system based on deformable three-dimensional model and optical flow representation learning |
| CN116402691A (en) * | 2023-06-05 | 2023-07-07 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
| CN116402691B (en) * | 2023-06-05 | 2023-08-04 | 四川轻化工大学 | Image super-resolution method and system based on rapid image feature stitching |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN106407903A (en) | Multiple dimensioned convolution neural network-based real time human body abnormal behavior identification method | |
| Tao et al. | Smoke detection based on deep convolutional neural networks | |
| CN104978580B (en) | A kind of insulator recognition methods for unmanned plane inspection transmission line of electricity | |
| CN104809443B (en) | Detection method of license plate and system based on convolutional neural networks | |
| CN107016357B (en) | Video pedestrian detection method based on time domain convolutional neural network | |
| CN107341452A (en) | Human bodys' response method based on quaternary number space-time convolutional neural networks | |
| CN109635744A (en) | A kind of method for detecting lane lines based on depth segmentation network | |
| CN107229929A (en) | A kind of license plate locating method based on R CNN | |
| CN106920243A (en) | The ceramic material part method for sequence image segmentation of improved full convolutional neural networks | |
| CN107945153A (en) | A kind of road surface crack detection method based on deep learning | |
| CN106682569A (en) | Fast traffic signboard recognition method based on convolution neural network | |
| CN110378288A (en) | A kind of multistage spatiotemporal motion object detection method based on deep learning | |
| CN104281853A (en) | Behavior identification method based on 3D convolution neural network | |
| CN105160310A (en) | 3D (three-dimensional) convolutional neural network based human body behavior recognition method | |
| CN104992223A (en) | Intensive population estimation method based on deep learning | |
| CN104182772A (en) | Gesture recognition method based on deep learning | |
| CN112464911A (en) | Improved YOLOv 3-tiny-based traffic sign detection and identification method | |
| CN110163077A (en) | A kind of lane recognition method based on full convolutional neural networks | |
| CN104299006A (en) | Vehicle license plate recognition method based on deep neural network | |
| CN112950780B (en) | Intelligent network map generation method and system based on remote sensing image | |
| CN103049763A (en) | Context-constraint-based target identification method | |
| CN109002752A (en) | A kind of complicated common scene rapid pedestrian detection method based on deep learning | |
| CN107273870A (en) | The pedestrian position detection method of integrating context information under a kind of monitoring scene | |
| CN101236608A (en) | Human face detection method based on picture geometry | |
| CN109948607A (en) | Candidate frame based on deep learning deconvolution network generates and object detection method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170215 |