CN105787458A - Infrared behavior identification method based on adaptive fusion of artificial design feature and depth learning feature - Google Patents
Infrared behavior identification method based on adaptive fusion of artificial design feature and depth learning feature Download PDFInfo
- Publication number
- CN105787458A CN105787458A CN201610139450.8A CN201610139450A CN105787458A CN 105787458 A CN105787458 A CN 105787458A CN 201610139450 A CN201610139450 A CN 201610139450A CN 105787458 A CN105787458 A CN 105787458A
- Authority
- CN
- China
- Prior art keywords
- feature
- engineer
- weight
- cnn
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/25—Fusion techniques
- G06F18/254—Fusion techniques of classification results, e.g. of results related to same input data
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
- G06N3/084—Backpropagation, e.g. using gradient descent
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V20/00—Scenes; Scene-specific elements
- G06V20/40—Scenes; Scene-specific elements in video content
- G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Artificial Intelligence (AREA)
- Multimedia (AREA)
- Life Sciences & Earth Sciences (AREA)
- Evolutionary Computation (AREA)
- Computer Vision & Pattern Recognition (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Biology (AREA)
- General Health & Medical Sciences (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Biophysics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- Computational Linguistics (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Biomedical Technology (AREA)
- Image Analysis (AREA)
Abstract
The invention relates to an infrared behavior identification method based on adaptive fusion of an artificial design feature and a depth learning feature. The method comprises: S1, improved dense track feature extraction is carried out on an original video by using an artificial design feature module; S2, feature coding is carried out on the extracted artificial design feature; S3, with a CNN feature module, optic flow information extraction is carried out on an original video image sequence by using a variation optic flow algorithm, thereby obtaining a corresponding optic flow image sequence; S4, CNN feature extraction is carried out on the optic flow sequence obtained at the S3 by using a convolutional neural network; and S5, a data set is divided into a training set and a testing set; and weight learning is carried out on the training set data by using a weight optimization network, weight fusion is carried out on probability outputs of a CNN feature classification network and an artificial design feature classification network by using the learned weight, an optimal weight is obtained based on a comparison identification result, and then the optimal weight is applied to testing set data classification. According to the method, a novel feature fusion way is provided; and reliability of behavior identification in an infrared video is improved. Therefore, the method has the great significance in a follow-up video analysis.
Description
Technical field
The invention belongs to image procossing and technical field of computer vision, relate to a kind of infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation.
Background technology
In recent years, image and the Activity recognition in video become a vital task of computer vision field, and the work such as video monitoring, Video Information Retrieval Techniques, man-machine interaction are all of great significance by the Activity recognition in video.Along with various actions recognizer constantly refreshes the recognition accuracy of each public data collection, the Activity recognition task in video has made great progress.But current most data set is all based on visible light video, the Activity recognition based on infrared video works relatively fewer.
The Activity recognition algorithm of current main flow relates generally to two kinds of description: engineer's Feature Descriptor and description obtained by degree of depth study.
Engineer's feature and typical local description, such as space-time interest points feature (Spatial-TemporalInterestPoint, STIP), histograms of oriented gradients (HistogramofOrientedGradient, HOG), light stream direction histogram (HistogramofOpticalFlow, HOF), intensive track characteristic (DenseTrajectory, DT) etc., it is based on movable information between the texture information of image, vision shape and different frame etc. different behaviors is classified and identifies；Owing to intensive track characteristic comprises the abundant sub-information of description such as HOG, HOF, MBH (MotionBoundaryHistory), become engineer's feature that current recognition accuracy is higher；Lifting along with computing power, the CNN feature extracted by convolutional neural networks is used as image, Activity recognition in video becomes popular research direction in recent years, convolutional neural networks achieves the profound excavation to pictorial information, effective extraction has the information of identification, the 3D convolutional neural networks having been proposed that at present, degree of depth convolutional neural networks, in the models such as dual pathways convolutional neural networks, effect is a dual pathways convolutional neural networks being made up of temporal information passage and spatial information passage most preferably, each more challenging data set all achieves good experiment effect.
But the Activity recognition research work being currently based on infrared video is relatively fewer, in video monitoring works, if running into the relatively low vile weather of the visibility such as rain, mist or nighttime conditions descends, it is seen that light video monitoring just loses meaning.Therefore, infrared video Activity recognition has critically important practical value, is badly in need of a kind of effective infrared video Activity recognition algorithm at present.
Summary of the invention
In view of this, it is an object of the invention to provide a kind of infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation, the method takes full advantage of feature and the advantage of infrared image, engineer's feature is improved, and by two sorter networks and a weight learning function, it is weighted the probability output of engineer's tagsort network and degree of deep learning characteristic sorter network merging, the accuracy rate being effectively improved in infrared video Activity recognition.
For reaching above-mentioned purpose, the present invention provides following technical scheme:
A kind of infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation, the method comprises the following steps:
S1: by engineer's characteristic module, the intensive track characteristic that original video is improved extracts；
S2: the engineer's feature extracted in step S1 is carried out feature coding；
S3: by CNN characteristic module, to original video image sequence, utilizes a kind of variation optical flow algorithm to extract Optic flow information, obtains corresponding light flow image sequences, as the input of convolutional neural networks；
S4: utilize convolutional neural networks, extracts CNN feature to the light stream graphic sequence obtained in step S3；
S5: data set is divided into training set and test set；To training set data, by weight optimization e-learning weight, utilize study to weight be weighted the probability output of CNN tagsort network and engineer's tagsort network merging, draw optimal weights by contrasting recognition result, and be applied to the classification of test set data.
Further, in step sl, described to the engineer's feature after original video extraction improvement, specifically include: first each frame infrared image is carried out point of interest intensive sampling, then between consecutive image sequence, follow the trail of point of interest, describe addition track correspondence Pictures location gray value weight in son at the track obtained, the track after weighting is described the son intensive track characteristic as improvement.
Further, in step s 2, described is encoded intensive track characteristic after the improvement extracted, and coding carries out in the following ways:
1): Bag-of-Words (BOW)
It is step by step rapid that this feature coding method includes two:
11) visual dictionary is generated: in training set data, extract feature, by clustering algorithm, the feature extracted is clustered；Cluster centre can be regarded as the visual vocabulary in dictionary, and all visual vocabularies form a visual dictionary；
12) with the lexical representation image in dictionary: each feature in image is mapped on certain word of visual dictionary, then add up each visual word occurrence number on an image, image table can be shown as the histogram vectors that a dimension is fixing；
2): FisherVector (FV)
FisherVector coding is a kind of coded system based on Fisher core principle, gauss hybrid models (GMM) is trained by the method for maximal possibility estimation first with training sample, then utilizing gauss hybrid models that the primitive character (such as Dense-Traj) extracted from sample is modeled, the primitive character of sample is encoded into the Fisher vector being easy to study and tolerance by the model parameter that recycling generates；
3): VectorofLocallyAggregatedDescriptors (VLAD)
First the feature extracted in training set is clustered by VLAD coding, obtain code book, then calculate the gap of each word in primitive character and code book, and these differences are added up, finally the accumulative difference of all words is carried out being spliced to form a new vector to represent image.
Further, in step s3, described utilization one variation optical flow algorithm extracts Optic flow information, obtains corresponding light flow image sequences, specifically includes:
S31: when the constant hypothesis of brightness, constant gradient hypothesis and space-time smoothness constraint continuously, it is proposed to the energy functional of improvement；
S32: derive the Euler-Lagrange equation of its correspondence according to energy function, then utilizes Gauss-Seidel or SOR method to obtain light stream vector, obtains corresponding light flow graph.
Further, in step s 4, to the light flow image sequences obtained, utilize convolutional neural networks, extract the output of full articulamentum as CNN feature；CNN adopts multitiered network structure, convolutional layer and sampling layer to be alternately present, and network output layer is the full connected mode of feedforward network.
Further, in step s 5, described data set is divided into training set and test set；Utilize training set data to pass through the study of self adaptation Fusion Module and obtain optimal weights, and optimal weights is applied to test set data classifies, specifically include:
S51: utilize training set data to learn optimal weights:
Self adaptation Fusion Module includes two sorter networks comprising full articulamentum and softmax grader layer and a single node logical function；The input of two sorter networks is engineer's feature and the CNN feature of training set data respectively, and two kinds of features obtain respective probability output P1 and P2 after being input to map network, utilizes single node logical function to calculate the weight Q of correspondence simultaneously；The final probability output of two sorter networks being weighted according to specified weight is merged, and utilizes error backpropagation algorithm to judge identification error, update gradient, it is judged that and export optimal weights；
S52: optimal weights is applied to test set data and classifies:
Engineer's feature test set data extracted and CNN feature are input to corresponding sorter network, obtain corresponding probability output, utilize the probability output of two sorter networks of optimal weights Weighted Fusion obtained in S51, obtain the recognition result of test set data.
The beneficial effects of the present invention is: the method for the invention highlights the infrared video advantage compared with visible light video, Traditional Man design feature and the degree of deep learning characteristic recently quickly grown are conjointly employed in Activity recognition task, innovate Feature Fusion mode, improve the reliability of Activity recognition in infrared video, subsequent video analysis work is of great importance.
Accompanying drawing explanation
In order to make the purpose of the present invention, technical scheme and beneficial effect clearly, the present invention provides drawings described below to illustrate:
Fig. 1 is the dual pathways infrared behavior recognizer schematic diagram of the present invention；
Fig. 2 is that the intensive track characteristic improved extracts schematic diagram；
Fig. 3 is light stream convolutional neural networks feature extraction schematic diagram；
Fig. 4 is self-adaptive features Fusion Model schematic diagram.
Detailed description of the invention
Below in conjunction with accompanying drawing, the preferred embodiments of the present invention are described in detail.
In the present invention, engineer's characteristic module, the intensive track characteristic after original video is improved extracts, and the feature extracted is encoded；The intensive track characteristic improved is add gray value weight information in description of original intensive track, the space time information of sequence of video images of the intensive track characteristic major embodiment after improvement, highlights the foreground moving information of sequence of pictures；CNN characteristic module, original Infrared video image sequence utilize a kind of variation optical flow algorithm extract Optic flow information, form light flow image sequences, using the light flow graph that extracts as the input of convolutional neural networks, take the feature of the full articulamentum of convolutional neural networks as video CNN Feature Descriptor；First data set is divided into training set and test set by self adaptation Fusion Module；To training set data, weight is learnt by weight learning network, utilize study to weight be weighted the probability output of CNN tagsort network and engineer's tagsort network merging, draw optimal weights by contrasting recognition result, and optimal weights be applied to the classification of test set data.
Fig. 1 is the dual pathways infrared behavior recognizer schematic diagram of the present invention, as it can be seen, the method for the invention specifically includes following steps:
Step S1: to original video extract improve after intensive track characteristic, its particular flow sheet as shown in Figure 2:
S101: each frame infrared image of original video is carried out point of interest intensive sampling；
S102: following the trail of point of interest between consecutive image sequence, the pixel that between two continuous frames, displacement is only small and excessive is all rejected, and finally gives the point of interest track that can characterize action；
S103: describe addition track correspondence Pictures location gray value weight in son at the track obtained, describes the son intensive track characteristic as improvement using the track after weighting.
Original video is resolution be 293 × 256 infrared data collection, totally 12 type of action, each type of action has 50 video samples.Intensive track characteristic is engineer's feature that in current image Classification and Identification task, accuracy rate is higher, but the data set being mostly used in test is all visible data collection.Owing to infrared image can reflect the infrared emanation of each target in image, heat radiation is more strong, gray value is more big, making there is obvious contrast between target and background, based on this feature of infrared image, we add gray value weight on the basis of original intensive track characteristic, the track making gray value high has bigger weight, the track that gray value is less distributes less weight, improves original intensive track characteristic, highlights the advantage of infrared image Activity recognition.
Step S2: the engineer's feature extracted is carried out FisherVector coding.
Step S3: utilize a kind of variation optical flow algorithm that original video image sequence extracts Optic flow information, form light flow image sequences.The data item of this algorithm energy functional is made up of the constant hypothesis of brightness and the constant hypothesis of gradient, adds discontinuous space-time smoothness constraint, has continuous preferably and rotational invariance, calculates speed fast, and precision is high.
Step S4: the picture of the light flow image sequences that step S3 is obtained carries out pretreatment, it is 227 × 227 that dimension of picture adjusts resolution, and it can be used as the input of convolutional neural networks, this convolutional neural networks is made up of five convolutional layers and three full articulamentums, finally taking the feature of second convolutional layer as feature representation, flow chart is as shown in Figure 3.
Convolutional neural networks (ConvolutionalNeuralNet, CNN) it is a kind of Multi-layered Feedforward Networks, convolutional layer and sampling layer as network intermediate layer are alternately present, and network is output as the full connected mode of feedforward network, and the dimension of output layer is the classification number in classification task.Convolutional neural networks directly can automatically learn individual features from original input data, saves the characteristic Design process that general algorithm needs, it is possible to learn and find more effective feature.
Step S5: training set data utilizes self adaptation Fusion Module to learn optimal weights, and is applied to the classification of test set data.The schematic diagram of self adaptation Fusion Model is as shown in Figure 4:
S501: data set is divided into training set and test set；
S502: utilize training set sample data to learn optimal weights: self adaptation Fusion Model includes two sorter networks comprising full articulamentum and softmax grader layer and a single node logical function；The input of two sorter networks is engineer's feature and CNN feature that training set data is extracted respectively, and two kinds of features obtain respective probability output P1 and P2 after being input to map network, single node logical function calculates corresponding weight Q simultaneously；The final probability output of two sorter networks being weighted according to specified weight is merged, and utilizes error backpropagation algorithm to judge identification error, update gradient, it is judged that and export optimal weights；
S503: optimal weights is applied to test set data and classifies: engineer's feature test set sample data extracted and CNN feature are input to corresponding sorter network, obtain corresponding probability output, utilize optimal weights proportion weighted to merge the probability output of two sorter networks, obtain test set data recognition result.
The main purpose of error back propagation (ErrorBackpropagation, BP) algorithm is by by output error anti-pass, giving all unit of each layer by error distribution, thus obtaining the error signal of each layer unit, and then revises the weights of each unit.The learning process of BP algorithm is made up of two processes of back propagation of the forward-propagating of signal Yu error.During forward-propagating, input sample is incoming from input layer, after each hidden layer successively processes, is transmitted to output layer.If the actual output of output layer and desired output are not inconsistent, then proceed to the back-propagation phase of error.Error-duration model be by output error with some form by each hidden layer to input layer successively anti-pass, and error distribution is given all unit of each layer, thus obtaining the error signal of each layer unit, this error signal is namely as the foundation of correction unit weights.In this module, the process that this weights gone round and begun again constantly adjust just makes to finally give optimal weights.
What finally illustrate is, preferred embodiment above is only in order to illustrate technical scheme and unrestricted, although the present invention being described in detail by above preferred embodiment, but skilled artisan would appreciate that, in the form and details it can be made various change, without departing from claims of the present invention limited range.
Claims (6)
1. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation, it is characterised in that: the method comprises the following steps:
S1: by engineer's characteristic module, the intensive track characteristic that original video is improved extracts；
S2: the engineer's feature extracted in step S1 is carried out feature coding；
S3: by CNN characteristic module, to original video image sequence, utilizes a kind of variation optical flow algorithm to extract Optic flow information, obtains corresponding light flow image sequences, as the input of convolutional neural networks；
S4: utilize convolutional neural networks, extracts CNN feature to the light stream graphic sequence obtained in step S3；
S5: data set is divided into training set and test set；To training set data, by weight optimization e-learning weight, utilize study to weight be weighted the probability output of CNN tagsort network and engineer's tagsort network merging, draw optimal weights by contrasting recognition result, and be applied to the classification of test set data.
2. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation according to claim 1, it is characterized in that: in step sl, described to the engineer's feature after original video extraction improvement, specifically include: first each frame infrared image is carried out point of interest intensive sampling, then between consecutive image sequence, follow the trail of point of interest, describe addition track correspondence Pictures location gray value weight in son at the track obtained, the track after weighting is described the son intensive track characteristic as improvement.
3. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation according to claim 1, it is characterized in that: in step s 2, described is encoded intensive track characteristic after the improvement extracted, and coding carries out in the following ways:
1): Bag-of-Words (BOW)
It is step by step rapid that this feature coding method includes two:
11) visual dictionary is generated: in training set data, extract feature, by clustering algorithm, the feature extracted is clustered；Cluster centre can be regarded as the visual vocabulary in dictionary, and all visual vocabularies form a visual dictionary；
12) with the lexical representation image in dictionary: each feature in image is mapped on certain word of visual dictionary, then add up each visual word occurrence number on an image, image table can be shown as the histogram vectors that a dimension is fixing；
2): FisherVector (FV)
FisherVector coding is a kind of coded system based on Fisher core principle, gauss hybrid models (GMM) is trained by the method for maximal possibility estimation first with training sample, then utilizing gauss hybrid models that the primitive character (such as Dense-Traj) extracted from sample is modeled, the primitive character of sample is encoded into the Fisher vector being easy to study and tolerance by the model parameter that recycling generates；
3): VectorofLocallyAggregatedDescriptors (VLAD)
First the feature extracted in training set is clustered by VLAD coding, obtain code book, then calculate the gap of each word in primitive character and code book, and these differences are added up, finally the accumulative difference of all words is carried out being spliced to form a new vector to represent image.
4. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation according to claim 1, it is characterized in that: in step s3, described utilization one variation optical flow algorithm extracts Optic flow information, obtains corresponding light flow image sequences, specifically includes:
S31: when the constant hypothesis of brightness, constant gradient hypothesis and space-time smoothness constraint continuously, it is proposed to the energy functional of improvement；
S32: derive the Euler-Lagrange equation of its correspondence according to energy function, then utilizes Gauss-Seidel or SOR method to obtain light stream vector, obtains corresponding light flow graph.
5. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation according to claim 1, it is characterized in that: in step s 4, to the light flow image sequences obtained, utilize convolutional neural networks, extract the output of full articulamentum as CNN feature；CNN adopts multitiered network structure, convolutional layer and sampling layer to be alternately present, and network output layer is the full connected mode of feedforward network.
6. the infrared behavior recognition methods merged based on engineer's feature and degree of deep learning characteristic self adaptation according to claim 1, it is characterised in that: in step s 5, described data set is divided into training set and test set；Utilize training set data to pass through the study of self adaptation Fusion Module and obtain optimal weights, and optimal weights is applied to test set data classifies, specifically include:
S51: utilize training set data to learn optimal weights:
Self adaptation Fusion Module includes two sorter networks comprising full articulamentum and softmax grader layer and a single node logical function；The input of two sorter networks is engineer's feature and the CNN feature of training set data respectively, and two kinds of features obtain respective probability output P1 and P2 after being input to map network, utilizes single node logical function to calculate the weight Q of correspondence simultaneously；The final probability output of two sorter networks being weighted according to specified weight is merged, and utilizes error backpropagation algorithm to judge identification error, update gradient, it is judged that and export optimal weights；
S52: optimal weights is applied to test set data and classifies:
Engineer's feature test set data extracted and CNN feature are input to corresponding sorter network, obtain corresponding probability output, utilize the probability output of two sorter networks of optimal weights Weighted Fusion obtained in S51, obtain the recognition result of test set data.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610139450.8A CN105787458B (en) | 2016-03-11 | 2016-03-11 | The infrared behavior recognition methods adaptively merged based on artificial design features and deep learning feature |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201610139450.8A CN105787458B (en) | 2016-03-11 | 2016-03-11 | The infrared behavior recognition methods adaptively merged based on artificial design features and deep learning feature |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN105787458A true CN105787458A (en) | 2016-07-20 |
| CN105787458B CN105787458B (en) | 2019-01-04 |
Family
ID=56392545
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201610139450.8A Active CN105787458B (en) | 2016-03-11 | 2016-03-11 | The infrared behavior recognition methods adaptively merged based on artificial design features and deep learning feature |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN105787458B (en) |
Cited By (50)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106355210A (en) * | 2016-09-14 | 2017-01-25 | 华北电力大学（保定） | Method for expressing infrared image features of insulators on basis of depth neuron response modes |
| CN106446937A (en) * | 2016-09-08 | 2017-02-22 | 天津大学 | Multi-convolution identifying system for AER image sensor |
| CN106709447A (en) * | 2016-12-21 | 2017-05-24 | 华南理工大学 | Abnormal behavior detection method in video based on target positioning and characteristic fusion |
| CN106979980A (en) * | 2017-04-28 | 2017-07-25 | 武汉理工大学 | A kind of new multiple spectra acoustics camera and recognition methods |
| CN107122712A (en) * | 2017-03-27 | 2017-09-01 | 大连大学 | It polymerize the palmprint image recognition methods of description vectors based on convolutional neural networks and two-way local feature |
| CN107133354A (en) * | 2017-05-25 | 2017-09-05 | 北京小米移动软件有限公司 | The acquisition methods and device of description information of image |
| CN107273835A (en) * | 2017-06-07 | 2017-10-20 | 南京航空航天大学 | Act of violence intelligent detecting method based on video analysis |
| CN107679522A (en) * | 2017-10-31 | 2018-02-09 | 内江师范学院 | Action identification method based on multithread LSTM |
| CN108009486A (en) * | 2017-11-28 | 2018-05-08 | 中科亿海微电子科技（苏州）有限公司 | Video frequency identifying method and system based on deep learning training reconstruct image |
| CN108021889A (en) * | 2017-12-05 | 2018-05-11 | 重庆邮电大学 | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information |
| WO2018086513A1 (en) * | 2016-11-08 | 2018-05-17 | 杭州海康威视数字技术股份有限公司 | Target detection method and device |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| CN108229580A (en) * | 2018-01-26 | 2018-06-29 | 浙江大学 | Sugared net ranking of features device in a kind of eyeground figure based on attention mechanism and Fusion Features |
| CN108288035A (en) * | 2018-01-11 | 2018-07-17 | 华南理工大学 | The human motion recognition method of multichannel image Fusion Features based on deep learning |
| CN108335402A (en) * | 2017-01-18 | 2018-07-27 | 武汉卓目科技有限公司 | A kind of cash inspecting machine infrared tube false distinguishing method based on deep learning |
| CN108596082A (en) * | 2018-04-20 | 2018-09-28 | 重庆邮电大学 | Human face in-vivo detection method based on image diffusion velocity model and color character |
| CN108664922A (en) * | 2018-05-10 | 2018-10-16 | 东华大学 | A kind of infrared video Human bodys' response method based on personal safety |
| CN108830308A (en) * | 2018-05-31 | 2018-11-16 | 西安电子科技大学 | A kind of Modulation Identification method that traditional characteristic signal-based is merged with depth characteristic |
| CN108830330A (en) * | 2018-06-22 | 2018-11-16 | 西安电子科技大学 | Classification of Multispectral Images method based on self-adaptive features fusion residual error net |
| CN108875756A (en) * | 2017-05-08 | 2018-11-23 | 上海荆虹电子科技有限公司 | A kind of the behavior classification acquisition methods and device of video |
| CN109684933A (en) * | 2018-11-30 | 2019-04-26 | 广州大学 | A kind of front pedestrian scurries out the method for early warning of road |
| CN109815967A (en) * | 2019-02-28 | 2019-05-28 | 北京环境特性研究所 | CNN ship seakeeping system and method based on Fusion Features |
| CN109977846A (en) * | 2019-03-22 | 2019-07-05 | 中国科学院重庆绿色智能技术研究院 | A kind of in-vivo detection method and system based on the camera shooting of near-infrared monocular |
| CN110135352A (en) * | 2019-05-16 | 2019-08-16 | 南京砺剑光电技术研究院有限公司 | A kind of tactical operation appraisal procedure based on deep learning |
| CN110135386A (en) * | 2019-05-24 | 2019-08-16 | 长沙学院 | A kind of human motion recognition method and system based on deep learning |
| CN110236560A (en) * | 2019-06-06 | 2019-09-17 | 深圳市联邦佳维工业装备有限公司 | Six axis attitude detecting methods of intelligent wearable device, system |
| CN110378254A (en) * | 2019-07-03 | 2019-10-25 | 中科软科技股份有限公司 | Recognition methods, system, electronic equipment and the storage medium of vehicle damage amending image trace |
| CN110555387A (en) * | 2019-08-02 | 2019-12-10 | 华侨大学 | Behavior identification method based on local joint point track space-time volume in skeleton sequence |
| CN110674884A (en) * | 2019-09-30 | 2020-01-10 | 山东浪潮人工智能研究院有限公司 | Image identification method based on feature fusion |
| US10600208B2 (en) | 2017-12-21 | 2020-03-24 | Industrial Technology Research Institute | Object detecting device, object detecting method and non-transitory computer-readable medium |
| CN110959160A (en) * | 2017-08-01 | 2020-04-03 | 华为技术有限公司 | Gesture recognition method, device and equipment |
| CN111079655A (en) * | 2019-12-18 | 2020-04-28 | 航天物联网技术有限公司 | Method for recognizing human body behaviors in video based on fusion neural network |
| CN111145222A (en) * | 2019-12-30 | 2020-05-12 | 浙江中创天成科技有限公司 | Fire detection method combining smoke movement trend and textural features |
| CN111310519A (en) * | 2018-12-11 | 2020-06-19 | 成都智叟智能科技有限公司 | Goods deep learning training method based on machine vision and data sampling |
| CN111400440A (en) * | 2020-02-28 | 2020-07-10 | 深圳市华海同创科技有限公司 | Intention identification method and device |
| CN111680543A (en) * | 2020-04-23 | 2020-09-18 | 北京迈格威科技有限公司 | Action recognition method and device and electronic equipment |
| CN111860086A (en) * | 2019-06-26 | 2020-10-30 | 广州凡拓数字创意科技股份有限公司 | Gesture recognition method, device and system based on deep neural network |
| CN112329815A (en) * | 2020-09-30 | 2021-02-05 | 华南师范大学 | Model training method, device and medium for detecting travel track abnormality |
| CN112597975A (en) * | 2021-02-26 | 2021-04-02 | 上海闪马智能科技有限公司 | Fire smoke and projectile detection method and system based on video |
| CN112669320A (en) * | 2021-03-22 | 2021-04-16 | 四川大学 | SPECT thyroid imaging intelligent identification method based on deep neural network |
| CN112749672A (en) * | 2021-01-19 | 2021-05-04 | 携程旅游网络技术（上海）有限公司 | Photo album video identification method, system, equipment and storage medium |
| CN112801042A (en) * | 2021-03-08 | 2021-05-14 | 南京大学 | Student class-taking behavior identification method based on deep learning and video tracks |
| CN113139518A (en) * | 2021-05-14 | 2021-07-20 | 杭州旭颜科技有限公司 | Section bar cutting state monitoring method based on industrial internet |
| CN113395138A (en) * | 2021-06-15 | 2021-09-14 | 重庆邮电大学 | PC-SCMA joint iterative detection decoding method based on deep learning |
| CN109740670B (en) * | 2019-01-02 | 2022-01-11 | 京东方科技集团股份有限公司 | Video classification method and device |
| CN115147864A (en) * | 2022-05-31 | 2022-10-04 | 合肥学院 | Infrared human body behavior identification method based on collaborative heterogeneous deep learning network |
| CN115965553A (en) * | 2023-01-03 | 2023-04-14 | 无锡学院 | Face image local enhancement method based on multi-feature fusion |
| CN117528085A (en) * | 2024-01-08 | 2024-02-06 | 中国矿业大学 | Video compression coding method based on intelligent feature clustering |
| CN118587635A (en) * | 2024-08-05 | 2024-09-03 | 深圳市旭景数字技术有限公司 | Self-adaptive video content safety monitoring method and system based on deep learning |
| CN118587635B (en) * | 2024-08-05 | 2024-10-29 | 深圳市旭景数字技术有限公司 | Self-adaptive video content safety monitoring method and system based on deep learning |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106407889B (en) * | 2016-08-26 | 2020-08-04 | 上海交通大学 | Method for recognizing human body interaction in video based on optical flow graph deep learning model |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8345984B2 (en) * | 2010-01-28 | 2013-01-01 | Nec Laboratories America, Inc. | 3D convolutional neural networks for automatic human action recognition |
| CN104281853A (en) * | 2014-09-02 | 2015-01-14 | 电子科技大学 | Behavior identification method based on 3D convolution neural network |
| US20150117760A1 (en) * | 2013-10-30 | 2015-04-30 | Nec Laboratories America, Inc. | Regionlets with Shift Invariant Neural Patterns for Object Detection |
| CN104933417A (en) * | 2015-06-26 | 2015-09-23 | 苏州大学 | Behavior recognition method based on sparse spatial-temporal characteristics |
-
2016
- 2016-03-11 CN CN201610139450.8A patent/CN105787458B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US8345984B2 (en) * | 2010-01-28 | 2013-01-01 | Nec Laboratories America, Inc. | 3D convolutional neural networks for automatic human action recognition |
| US20150117760A1 (en) * | 2013-10-30 | 2015-04-30 | Nec Laboratories America, Inc. | Regionlets with Shift Invariant Neural Patterns for Object Detection |
| CN104281853A (en) * | 2014-09-02 | 2015-01-14 | 电子科技大学 | Behavior identification method based on 3D convolution neural network |
| CN104933417A (en) * | 2015-06-26 | 2015-09-23 | 苏州大学 | Behavior recognition method based on sparse spatial-temporal characteristics |
Non-Patent Citations (1)
| Title |
|---|
| 雷庆等: "复杂场景下的人体行为识别研究新进展", 《计算机科学》 * |
Cited By (71)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106446937A (en) * | 2016-09-08 | 2017-02-22 | 天津大学 | Multi-convolution identifying system for AER image sensor |
| CN106355210A (en) * | 2016-09-14 | 2017-01-25 | 华北电力大学（保定） | Method for expressing infrared image features of insulators on basis of depth neuron response modes |
| CN106355210B (en) * | 2016-09-14 | 2019-03-19 | 华北电力大学（保定） | Insulator Infrared Image feature representation method based on depth neuron response modes |
| WO2018086513A1 (en) * | 2016-11-08 | 2018-05-17 | 杭州海康威视数字技术股份有限公司 | Target detection method and device |
| US10949673B2 (en) * | 2016-11-08 | 2021-03-16 | Hangzhou Hikvision Digital Technology Co., Ltd. | Target detection method and device |
| CN106709447A (en) * | 2016-12-21 | 2017-05-24 | 华南理工大学 | Abnormal behavior detection method in video based on target positioning and characteristic fusion |
| CN108335402A (en) * | 2017-01-18 | 2018-07-27 | 武汉卓目科技有限公司 | A kind of cash inspecting machine infrared tube false distinguishing method based on deep learning |
| CN107122712A (en) * | 2017-03-27 | 2017-09-01 | 大连大学 | It polymerize the palmprint image recognition methods of description vectors based on convolutional neural networks and two-way local feature |
| CN106979980B (en) * | 2017-04-28 | 2020-06-09 | 武汉理工大学 | Multi-spectrum acoustic camera and identification method |
| CN106979980A (en) * | 2017-04-28 | 2017-07-25 | 武汉理工大学 | A kind of new multiple spectra acoustics camera and recognition methods |
| CN108875756A (en) * | 2017-05-08 | 2018-11-23 | 上海荆虹电子科技有限公司 | A kind of the behavior classification acquisition methods and device of video |
| CN107133354A (en) * | 2017-05-25 | 2017-09-05 | 北京小米移动软件有限公司 | The acquisition methods and device of description information of image |
| CN107273835A (en) * | 2017-06-07 | 2017-10-20 | 南京航空航天大学 | Act of violence intelligent detecting method based on video analysis |
| CN110959160A (en) * | 2017-08-01 | 2020-04-03 | 华为技术有限公司 | Gesture recognition method, device and equipment |
| CN107679522B (en) * | 2017-10-31 | 2020-10-13 | 内江师范学院 | Multi-stream LSTM-based action identification method |
| CN107679522A (en) * | 2017-10-31 | 2018-02-09 | 内江师范学院 | Action identification method based on multithread LSTM |
| CN108009486B (en) * | 2017-11-28 | 2021-06-08 | 中科亿海微电子科技（苏州）有限公司 | Video identification method and system based on deep learning training reconfiguration picture |
| CN108009486A (en) * | 2017-11-28 | 2018-05-08 | 中科亿海微电子科技（苏州）有限公司 | Video frequency identifying method and system based on deep learning training reconstruct image |
| CN108021889A (en) * | 2017-12-05 | 2018-05-11 | 重庆邮电大学 | A kind of binary channels infrared behavior recognition methods based on posture shape and movable information |
| US10600208B2 (en) | 2017-12-21 | 2020-03-24 | Industrial Technology Research Institute | Object detecting device, object detecting method and non-transitory computer-readable medium |
| CN108288035A (en) * | 2018-01-11 | 2018-07-17 | 华南理工大学 | The human motion recognition method of multichannel image Fusion Features based on deep learning |
| CN108229580A (en) * | 2018-01-26 | 2018-06-29 | 浙江大学 | Sugared net ranking of features device in a kind of eyeground figure based on attention mechanism and Fusion Features |
| CN108229580B (en) * | 2018-01-26 | 2020-12-11 | 浙江大学 | Sugar net feature grading device in fundus map based on attention mechanism and feature fusion |
| CN108229440A (en) * | 2018-02-06 | 2018-06-29 | 北京奥开信息科技有限公司 | One kind is based on Multi-sensor Fusion indoor human body gesture recognition method |
| CN108596082A (en) * | 2018-04-20 | 2018-09-28 | 重庆邮电大学 | Human face in-vivo detection method based on image diffusion velocity model and color character |
| CN108664922A (en) * | 2018-05-10 | 2018-10-16 | 东华大学 | A kind of infrared video Human bodys' response method based on personal safety |
| CN108830308B (en) * | 2018-05-31 | 2021-12-14 | 西安电子科技大学 | Signal-based traditional feature and depth feature fusion modulation identification method |
| CN108830308A (en) * | 2018-05-31 | 2018-11-16 | 西安电子科技大学 | A kind of Modulation Identification method that traditional characteristic signal-based is merged with depth characteristic |
| CN108830330B (en) * | 2018-06-22 | 2021-11-02 | 西安电子科技大学 | Multispectral image classification method based on self-adaptive feature fusion residual error network |
| CN108830330A (en) * | 2018-06-22 | 2018-11-16 | 西安电子科技大学 | Classification of Multispectral Images method based on self-adaptive features fusion residual error net |
| CN109684933A (en) * | 2018-11-30 | 2019-04-26 | 广州大学 | A kind of front pedestrian scurries out the method for early warning of road |
| CN111310519B (en) * | 2018-12-11 | 2024-01-05 | 成都智叟智能科技有限公司 | Goods deep learning training method based on machine vision and data sampling |
| CN111310519A (en) * | 2018-12-11 | 2020-06-19 | 成都智叟智能科技有限公司 | Goods deep learning training method based on machine vision and data sampling |
| CN109740670B (en) * | 2019-01-02 | 2022-01-11 | 京东方科技集团股份有限公司 | Video classification method and device |
| CN109815967A (en) * | 2019-02-28 | 2019-05-28 | 北京环境特性研究所 | CNN ship seakeeping system and method based on Fusion Features |
| CN109977846A (en) * | 2019-03-22 | 2019-07-05 | 中国科学院重庆绿色智能技术研究院 | A kind of in-vivo detection method and system based on the camera shooting of near-infrared monocular |
| CN110135352A (en) * | 2019-05-16 | 2019-08-16 | 南京砺剑光电技术研究院有限公司 | A kind of tactical operation appraisal procedure based on deep learning |
| CN110135352B (en) * | 2019-05-16 | 2023-05-12 | 南京砺剑光电技术研究院有限公司 | Tactical action evaluation method based on deep learning |
| CN110135386A (en) * | 2019-05-24 | 2019-08-16 | 长沙学院 | A kind of human motion recognition method and system based on deep learning |
| CN110236560A (en) * | 2019-06-06 | 2019-09-17 | 深圳市联邦佳维工业装备有限公司 | Six axis attitude detecting methods of intelligent wearable device, system |
| CN111860086A (en) * | 2019-06-26 | 2020-10-30 | 广州凡拓数字创意科技股份有限公司 | Gesture recognition method, device and system based on deep neural network |
| CN110378254B (en) * | 2019-07-03 | 2022-04-19 | 中科软科技股份有限公司 | Method and system for identifying vehicle damage image modification trace, electronic device and storage medium |
| CN110378254A (en) * | 2019-07-03 | 2019-10-25 | 中科软科技股份有限公司 | Recognition methods, system, electronic equipment and the storage medium of vehicle damage amending image trace |
| CN110555387B (en) * | 2019-08-02 | 2022-07-19 | 华侨大学 | Behavior identification method based on space-time volume of local joint point track in skeleton sequence |
| CN110555387A (en) * | 2019-08-02 | 2019-12-10 | 华侨大学 | Behavior identification method based on local joint point track space-time volume in skeleton sequence |
| CN110674884A (en) * | 2019-09-30 | 2020-01-10 | 山东浪潮人工智能研究院有限公司 | Image identification method based on feature fusion |
| CN111079655A (en) * | 2019-12-18 | 2020-04-28 | 航天物联网技术有限公司 | Method for recognizing human body behaviors in video based on fusion neural network |
| CN111145222A (en) * | 2019-12-30 | 2020-05-12 | 浙江中创天成科技有限公司 | Fire detection method combining smoke movement trend and textural features |
| CN111400440A (en) * | 2020-02-28 | 2020-07-10 | 深圳市华海同创科技有限公司 | Intention identification method and device |
| CN111680543B (en) * | 2020-04-23 | 2023-08-29 | 北京迈格威科技有限公司 | Action recognition method and device and electronic equipment |
| CN111680543A (en) * | 2020-04-23 | 2020-09-18 | 北京迈格威科技有限公司 | Action recognition method and device and electronic equipment |
| CN112329815B (en) * | 2020-09-30 | 2022-07-22 | 华南师范大学 | Model training method, device and medium for detecting travel track abnormality |
| CN112329815A (en) * | 2020-09-30 | 2021-02-05 | 华南师范大学 | Model training method, device and medium for detecting travel track abnormality |
| CN112749672A (en) * | 2021-01-19 | 2021-05-04 | 携程旅游网络技术（上海）有限公司 | Photo album video identification method, system, equipment and storage medium |
| CN112597975A (en) * | 2021-02-26 | 2021-04-02 | 上海闪马智能科技有限公司 | Fire smoke and projectile detection method and system based on video |
| CN112801042B (en) * | 2021-03-08 | 2023-12-15 | 南京大学 | Student lesson behavior recognition method based on deep learning and video track |
| CN112801042A (en) * | 2021-03-08 | 2021-05-14 | 南京大学 | Student class-taking behavior identification method based on deep learning and video tracks |
| CN112669320B (en) * | 2021-03-22 | 2021-08-13 | 四川大学 | SPECT thyroid imaging intelligent identification method based on deep neural network |
| CN112669320A (en) * | 2021-03-22 | 2021-04-16 | 四川大学 | SPECT thyroid imaging intelligent identification method based on deep neural network |
| CN113139518A (en) * | 2021-05-14 | 2021-07-20 | 杭州旭颜科技有限公司 | Section bar cutting state monitoring method based on industrial internet |
| CN113139518B (en) * | 2021-05-14 | 2022-07-29 | 江苏中天互联科技有限公司 | Section bar cutting state monitoring method based on industrial internet |
| CN113395138B (en) * | 2021-06-15 | 2022-05-03 | 重庆邮电大学 | PC-SCMA joint iterative detection decoding method based on deep learning |
| CN113395138A (en) * | 2021-06-15 | 2021-09-14 | 重庆邮电大学 | PC-SCMA joint iterative detection decoding method based on deep learning |
| CN115147864A (en) * | 2022-05-31 | 2022-10-04 | 合肥学院 | Infrared human body behavior identification method based on collaborative heterogeneous deep learning network |
| CN115147864B (en) * | 2022-05-31 | 2024-08-30 | 合肥学院 | Infrared human behavior recognition method based on cooperative heterogeneous deep learning network |
| CN115965553B (en) * | 2023-01-03 | 2023-12-19 | 无锡学院 | Multi-feature fusion-based face image local enhancement method |
| CN115965553A (en) * | 2023-01-03 | 2023-04-14 | 无锡学院 | Face image local enhancement method based on multi-feature fusion |
| CN117528085A (en) * | 2024-01-08 | 2024-02-06 | 中国矿业大学 | Video compression coding method based on intelligent feature clustering |
| CN117528085B (en) * | 2024-01-08 | 2024-03-19 | 中国矿业大学 | Video compression coding method based on intelligent feature clustering |
| CN118587635A (en) * | 2024-08-05 | 2024-09-03 | 深圳市旭景数字技术有限公司 | Self-adaptive video content safety monitoring method and system based on deep learning |
| CN118587635B (en) * | 2024-08-05 | 2024-10-29 | 深圳市旭景数字技术有限公司 | Self-adaptive video content safety monitoring method and system based on deep learning |
Also Published As
| Publication number | Publication date |
|---|---|
| CN105787458B (en) | 2019-01-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN105787458A (en) | Infrared behavior identification method based on adaptive fusion of artificial design feature and depth learning feature | |
| CN110111335B (en) | Urban traffic scene semantic segmentation method and system for adaptive countermeasure learning | |
| CN110163187B (en) | F-RCNN-based remote traffic sign detection and identification method | |
| CN112819065B (en) | Unsupervised pedestrian sample mining method and unsupervised pedestrian sample mining system based on multi-clustering information | |
| CN110674483B (en) | Identity recognition method based on multi-mode information | |
| CN112434723B (en) | Day/night image classification and object detection method based on attention network | |
| CN111339849A (en) | Pedestrian re-identification method integrating pedestrian attributes | |
| CN111882620A (en) | Road drivable area segmentation method based on multi-scale information | |
| CN108108688B (en) | Limb conflict behavior detection method based on low-dimensional space-time feature extraction and topic modeling | |
| CN113569882A (en) | Knowledge distillation-based rapid pedestrian detection method | |
| CN116110022B (en) | Lightweight traffic sign detection method and system based on response knowledge distillation | |
| CN112036276A (en) | Artificial intelligent video question-answering method | |
| KR20210151773A (en) | Target re-recognition method and apparatus, terminal and storage medium | |
| CN113298817A (en) | High-accuracy semantic segmentation method for remote sensing image | |
| CN116994176A (en) | Video key data extraction method based on multidimensional semantic information | |
| CN115147459A (en) | Unmanned aerial vehicle target tracking method based on Swin transducer | |
| CN113870160A (en) | Point cloud data processing method based on converter neural network | |
| Qin et al. | Application of video scene semantic recognition technology in smart video | |
| Yin | Object Detection Based on Deep Learning: A Brief Review | |
| CN111523361A (en) | Human behavior recognition method | |
| Zheng et al. | Action recognition based on the modified twostream CNN | |
| CN117710841A (en) | Small target detection method and device for aerial image of unmanned aerial vehicle | |
| CN117576149A (en) | Single-target tracking method based on attention mechanism | |
| CN116185182B (en) | Controllable image description generation system and method for fusing eye movement attention | |
| Salam et al. | You Only Look Once (YOLOv3): Object Detection and Recognition for Indoor Environment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |