CN108986143A - Target detection tracking method in a kind of video - Google Patents
Target detection tracking method in a kind of video Download PDFInfo
- Publication number
- CN108986143A CN108986143A CN201810940035.1A CN201810940035A CN108986143A CN 108986143 A CN108986143 A CN 108986143A CN 201810940035 A CN201810940035 A CN 201810940035A CN 108986143 A CN108986143 A CN 108986143A
- Authority
- CN
- China
- Prior art keywords
- video
- target
- sequence
- image frame
- frame sequence
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000001514 detection method Methods 0.000 title claims abstract description 32
- 238000000034 method Methods 0.000 title claims abstract description 28
- 238000003062 neural network model Methods 0.000 claims abstract description 22
- 238000012360 testing method Methods 0.000 claims abstract description 13
- 238000005070 sampling Methods 0.000 claims abstract description 10
- 230000011218 segmentation Effects 0.000 claims abstract description 10
- 238000000605 extraction Methods 0.000 claims abstract description 5
- 239000011159 matrix material Substances 0.000 claims abstract description 5
- 238000012549 training Methods 0.000 claims description 17
- 238000013135 deep learning Methods 0.000 claims description 7
- 238000005352 clarification Methods 0.000 claims description 5
- 230000008034 disappearance Effects 0.000 claims description 2
- 239000012141 concentrate Substances 0.000 claims 1
- 238000004364 calculation method Methods 0.000 abstract description 7
- 230000000694 effects Effects 0.000 description 2
- 238000005457 optimization Methods 0.000 description 2
- 238000011160 research Methods 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 230000004069 differentiation Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000012806 monitoring device Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Abstract
The invention discloses target detection tracking methods in a kind of video.The present invention carries out segmentation sampling to video first, obtains several segments video image frame sequence.Then neural network model is usedTarget detection and feature extraction are carried out to every section of video image frame sequence.The correlation matrix of the corresponding target feature vector of all testing results exported in video sequence is calculated again, and then obtains tracking result of all targets detected in frequency sequence in video sequence.Finally temporally axis is ranked up the video image frame sequence of segmentation sampling, and the target detection pursuit path and eigenmatrix of video image frame sequence are input to neural network model, the tracking characteristics of each target in each video image frame sequence are obtained, the correlation of all targets between two adjacent video image frame sequence are calculated using this tracking characteristics, to complete the tracking of target in entire video-frequency band.The present invention, which completes the calculation amount that target detection tracing task needs in video, can be effectively reduced.
Description
Technical field
The invention belongs to technical field of computer vision, it is related to target detection tracking method in a kind of video.
Background technique
The monitoring devices such as bayonet, public security and disparate networks video camera are largely installed and are used, these equipment
Video data collected traffic offence, in terms of play the role of it is very big, but with these equipment pacify
Loading amount is increasing, and the data volume of production also increasingly increases, and is stored and is faced huge challenge, video knot using these data
Structure has become a research hotspot of scientific research and industry.
The underlying issue that all can't steer clear of in all kinds of video structural schemes be exactly accurately and efficiently detection and tracking view
Common-denominator target in frequency.A kind of " target following optimization method based on tracking study detection " 107967692A, a kind of " real-time nothing
Man-machine video object detection and tracking " 108108697A, " multiple target pedestrian detection and track side based on deep learning
Method " it is all to complete target detection using single-frame images in the patents such as 107563313A, calculate object detection results relevant range
Then feature relies on matching and tracking that these features complete close interframe target.Target detection relies on all in these methods
It is the information of single-frame images, the standard of testing result cannot be led to using the relevant information between close picture frame in time series
True rate will receive limitation;The feature used during matched jamming simultaneously is also extracting on single-frame images, and the spy
Sign wants that the different target individual of multiclass can be distinguished, and the similar purpose for encountering colleague is very easy to matching error, and tracking is caused to be lost
It loses;Finally, the accuracy rate in order to guarantee detecting and tracking, limited every the interval of frame sampling, cause calculation amount bigger, efficiency compares
It is low.
Summary of the invention
In view of the deficiencies of the prior art, the present invention provides target detection tracking methods in a kind of video.
The technical solution adopted for solving the technical problem of the present invention are as follows:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence.
Step 2, using neural network model M1Target detection and feature extraction are carried out to every section of video image frame sequence, it is defeated
Information out includes: the number in the sequence of image where target, rectangle frame, the clarification of objective vector of target in the picture.
Step 3, the correlation matrix for calculating the corresponding target feature vector of all testing results exported in video sequence, into
And obtain tracking result of all targets detected in frequency sequence in video sequence.
Step 4, temporally axis, by inside video image frame sequence target detection pursuit path and eigenmatrix be input to
Neural network model M2, the tracking characteristics of each target in each video image frame sequence are obtained, are calculated using this tracking characteristics
The correlation of all targets between two adjacent video image frame sequence, to complete the tracking of target in entire video-frequency band.
Beneficial effects of the present invention:
1, the accuracy rate of detector is improved using the inter-frame information of time-series image.
2, the space time information of time-series image is made full use of to improve the tracking effect of target.
3, the calculation amount of detecting and tracking can be effectively reduced, operational efficiency is improved.
4, detection and tracking effectively merges, and improves detecting and tracking overall effect.
Detailed description of the invention
Fig. 1 the method for the present invention flow chart.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only
It is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiment of the present invention, ordinary skill people
Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
As shown in Figure 1, the present invention the following steps are included:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence.
Step 2 includes: target to every section of video image frame sequence progress target detection and feature extraction, the information of output
Place image number in the sequence, rectangle frame, the clarification of objective vector of target in the picture.
The correlation matrix of the corresponding target feature vector of all testing results exported in video sequence in step 3, calculating,
And then obtain the tracking result in frequency sequence of all targets detected in video sequence.
Step 4, temporally axis, using the target detection pursuit path inside video image frame sequence, (including target is in sequence
The rectangle frame of number, target in the picture in column) and eigenmatrix (the serial splicing of feature vector), to front and back adjacent video
Target in sequence carries out matched jamming.
Wherein the target detection of every section of video image frame sequence and the calculation method of feature extraction are: execution has trained
Neural network model M1Reasoning process, directly obtain the number in the sequence of image where target, target in the picture
Rectangle frame, clarification of objective vector.
Wherein neural network model M1Training method is:
Collect mark video data;
Sample Video section is cut, the volume of image in the sequence where obtaining video image frame sequence and the target that marked
Number, the number information of target rectangle frame in the picture, target;
Pass through the detection to target in sequence of video images and multitask training optimization network model of classifying.
It is a kind of embodiment of target detection tracking method in video below, the specific steps are as follows:
The neural network model M that target detection and matching characteristic calculate in training video image frame sequence1, specific step
It is as follows:
1, multitude of video section V is collected;Target position and each target in artificial mark sequence of video images from occur to
The id information of disappearance obtains original mark sample set A={ V1,V2,…,VL}。
2, using deep learning theory and method, to each video-frequency band V in original mark sample set Ai, segmentation sampling life
At several video image frame sequences Pi,Pi+1,…,Pi+k∈Vi, obtain training test sample collection B={ P1,P2,…,Pi,
Pi+1,…,Pi+k…,Pn-k,…,Pn-1,Pn}。
3, using deep learning theory and method, combined training test sample collection B, training is obtained in the way of multitask
It can detecte target and calculate the neural network model M of target signature1。
The neural network model M that object matching tracking characteristics calculate between training video image frame sequence2, specific step
It is as follows:
1, neural network model M is utilized1, obtain every section of sequence of video images P in training test sample collection BiIn each target
Pursuit path (number in the sequence of image where target, the rectangle frame of target in the picture) and eigenmatrix (feature to
The serial splicing of amount).
2, every section of video V is utilizediThe target information of middle mark and each video image frame sequence Pi+jBy neural network mould
Type M1The pursuit path and eigenmatrix of obtained target, obtain video-frequency band ViIn each target in different video image frame sequence
In feature samples collection: O={ q1,q2,…,qk, wherein qiBy M1In PiMiddle generation, to generate mesh between sequence of video images
Mark the training dataset C={ O of matched jamming feature1,O2,…,Os}
3, using deep learning theory and method, combined training test sample collection C is trained to obtain for calculating video image
The neural network model M of object matching tracking characteristics between sequence2。
Utilize neural network model M1, M2, target in detecting and tracking video, the specific steps are as follows:
1, it samples to the video segmentation that needs are analyzed, generates several video image frame sequences
2, to each video image frame sequence, neural network model M is executed1Reasoning process, where obtaining each target
Image number in the sequence, rectangle frame, the clarification of objective vector of target in the picture
3, the correlation matrix of the corresponding target feature vector of all testing results exported in video image frame sequence is calculated,
Wherein the calculating of correlation can use Euclidean distance, mahalanobis distance etc., and then obtain all in video image frame sequence detect
Target the tracking result in video image frame sequence.
4, temporally axis information is ranked up the video image frame sequence of segmentation sampling, according to pursuit path and feature square
Battle array executes neural network model M2Reasoning process, obtain the tracking characteristics of each target in each video image frame sequence
Using the correlations of all targets between this feature calculation two adjacent video image frame sequence, (the wherein calculating of correlation can be with
With Euclidean distance, mahalanobis distance etc.), to complete the tracking of target in entire video-frequency band.
To sum up, the present invention is based on video image frame sequence data, in conjunction with the information and video image frame sequence of single-frame images
Between frame-to-frame correlation, realize target detection tracking method in a kind of video.Compared to the target detection side based on single-frame images
Method combines relevant information between image sequence in the present invention, and target detection performance has promotion.Using machine learning method from
It being calculated in single-frame images, is used for the matched feature of target following, this feature needs to meet the differentiation of similar similar purpose,
Very big or feature the separating capacity of calculation amount for obtaining this feature is poor, is easy matching error, tracking is caused to be lost
It loses.Tracking and matching of the invention is divided into two stages thus, in short time inner video image frame sequence the matched jamming of target and
Object matching tracking between different images frame frequency sequence: the matched jamming feature inside video image frame sequence relies on video figure
As the spininess image information in frame sequence and the correlation between sequence, the separating capacity of feature is also only limitted to video image frame sequence
Between internal target;Object matching between video image frame sequence mainly utilizes of video image frame sequence internal object
Feature with tracking result and target in video image frame sequence, can very effective raising tracking accuracy rate.Same phase
Than other methods, the present invention, which completes the calculation amount that target detection tracing task needs in video, be can be effectively reduced.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention, should
Understand, the present invention is not limited to implementation as described herein, the purpose of these implementations description is to help this field
In technical staff practice the present invention.
Claims (3)
1. target detection tracking method in a kind of video, it is characterised in that method includes the following steps:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence;
Step 2, using neural network modelTarget detection and feature extraction, output are carried out to every section of video image frame sequence
Information include: the number in the sequence of image where target, rectangle frame, the clarification of objective vector of target in the picture;
Step 3, the correlation matrix for calculating the corresponding target feature vector of all testing results exported in video sequence, and then
Tracking result of all targets detected in frequency sequence into video sequence;
Step 4, temporally axis are ranked up the video image frame sequence of segmentation sampling, and the target of video image frame sequence is examined
It surveys pursuit path and eigenmatrix is input to neural network model, obtain each target in each video image frame sequence
Tracking characteristics calculate the correlation of all targets between two adjacent video image frame sequence using this tracking characteristics, thus complete
At the tracking of target in entire video-frequency band.
2. target detection tracking method in a kind of video according to claim 1, it is characterised in that: the neural network
ModelIt establishes in the following ways:
Multitude of video section, the artificial target position marked in sequence of video images and each target are collected from the ID occurred to disappearance
Information obtains original mark sample set;
Using deep learning method, to each video-frequency band in original mark sample set, segmentation sampling generates several video figures
As frame sequence, obtain training test sample collection;
Using deep learning method, combined training test sample collection and in the way of multitask training obtain neural network model。
3. target detection tracking method in a kind of video according to claim 2, it is characterised in that: the neural network
ModelIt establishes in the following ways:
Utilize neural network model, obtain the tracking rail that training test sample concentrates each target in every section of sequence of video images
Mark and eigenmatrix;
Pass through neural network model using the target information and each video image frame sequence marked in every section of videoObtained mesh
Target pursuit path and eigenmatrix obtain feature samples of each target in different video image frame sequence in every section of video
Collection, to generate the training dataset of object matching tracking characteristics between sequence of video images;
Using deep learning method, combined training data set, training obtains neural network model。
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810940035.1A CN108986143B (en) | 2018-08-17 | 2018-08-17 | Target detection tracking method in video |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201810940035.1A CN108986143B (en) | 2018-08-17 | 2018-08-17 | Target detection tracking method in video |
Publications (2)
Publication Number | Publication Date |
---|---|
CN108986143A true CN108986143A (en) | 2018-12-11 |
CN108986143B CN108986143B (en) | 2022-05-03 |
Family
ID=64553984
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
CN201810940035.1A Active CN108986143B (en) | 2018-08-17 | 2018-08-17 | Target detection tracking method in video |
Country Status (1)
Country | Link |
---|---|
CN (1) | CN108986143B (en) |
Cited By (5)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109711332A (en) * | 2018-12-26 | 2019-05-03 | 浙江捷尚视觉科技股份有限公司 | A kind of face tracking method and application based on regression algorithm |
CN109934096A (en) * | 2019-01-22 | 2019-06-25 | 浙江零跑科技有限公司 | Automatic Pilot visual perception optimization method based on feature timing dependence |
CN110503663A (en) * | 2019-07-22 | 2019-11-26 | 电子科技大学 | A kind of random multi-target automatic detection tracking based on pumping frame detection |
CN111862145A (en) * | 2019-04-24 | 2020-10-30 | 四川大学 | Target tracking method based on multi-scale pedestrian detection |
CN113033582A (en) * | 2019-12-09 | 2021-06-25 | 杭州海康威视数字技术股份有限公司 | Model training method, feature extraction method and device |
Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
BRPI0806019A2 (en) * | 2008-07-15 | 2010-08-31 | Invisys Sist S De Visao Comput Ltda | counting and tracking of people on the move based on computer vision |
CN102004920A (en) * | 2010-11-12 | 2011-04-06 | 浙江工商大学 | Method for splitting and indexing surveillance videos |
CN102750527A (en) * | 2012-06-26 | 2012-10-24 | 浙江捷尚视觉科技有限公司 | Long-time stable human face detection and tracking method in bank scene and long-time stable human face detection and tracking device in bank scene |
CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
CN104954743A (en) * | 2015-06-12 | 2015-09-30 | 西安理工大学 | Multi-camera semantic association target tracking method |
CN105574505A (en) * | 2015-12-16 | 2016-05-11 | 深圳大学 | Human body target re-identification method and system among multiple cameras |
CN106920248A (en) * | 2017-01-19 | 2017-07-04 | 博康智能信息技术有限公司上海分公司 | A kind of method for tracking target and device |
US20180183650A1 (en) * | 2012-12-05 | 2018-06-28 | Origin Wireless, Inc. | Method, apparatus, and system for object tracking and navigation |
-
2018
- 2018-08-17 CN CN201810940035.1A patent/CN108986143B/en active Active
Patent Citations (8)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
BRPI0806019A2 (en) * | 2008-07-15 | 2010-08-31 | Invisys Sist S De Visao Comput Ltda | counting and tracking of people on the move based on computer vision |
CN102004920A (en) * | 2010-11-12 | 2011-04-06 | 浙江工商大学 | Method for splitting and indexing surveillance videos |
CN102750527A (en) * | 2012-06-26 | 2012-10-24 | 浙江捷尚视觉科技有限公司 | Long-time stable human face detection and tracking method in bank scene and long-time stable human face detection and tracking device in bank scene |
US20180183650A1 (en) * | 2012-12-05 | 2018-06-28 | Origin Wireless, Inc. | Method, apparatus, and system for object tracking and navigation |
CN104094279A (en) * | 2014-04-30 | 2014-10-08 | 中国科学院自动化研究所 | Large-range-first cross-camera visual target re-identification method |
CN104954743A (en) * | 2015-06-12 | 2015-09-30 | 西安理工大学 | Multi-camera semantic association target tracking method |
CN105574505A (en) * | 2015-12-16 | 2016-05-11 | 深圳大学 | Human body target re-identification method and system among multiple cameras |
CN106920248A (en) * | 2017-01-19 | 2017-07-04 | 博康智能信息技术有限公司上海分公司 | A kind of method for tracking target and device |
Non-Patent Citations (4)
Title |
---|
YINGHAO CAI 等: "Matching tracking sequences across widely separated cameras", 《2008 15TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING》 * |
吴尔杰: "监控视频中多目标检测与跟踪研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 * |
杨秋英: "相关序列小目标图像运动跟踪与仿真研究", 《系统仿真学报》 * |
王乐东 等: "序列帧间双重匹配的红外点目标跟踪算法", 《光电子·激光》 * |
Cited By (9)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
CN109711332A (en) * | 2018-12-26 | 2019-05-03 | 浙江捷尚视觉科技股份有限公司 | A kind of face tracking method and application based on regression algorithm |
CN109711332B (en) * | 2018-12-26 | 2021-03-26 | 浙江捷尚视觉科技股份有限公司 | Regression algorithm-based face tracking method and application |
CN109934096A (en) * | 2019-01-22 | 2019-06-25 | 浙江零跑科技有限公司 | Automatic Pilot visual perception optimization method based on feature timing dependence |
CN109934096B (en) * | 2019-01-22 | 2020-12-11 | 浙江零跑科技有限公司 | Automatic driving visual perception optimization method based on characteristic time sequence correlation |
CN111862145A (en) * | 2019-04-24 | 2020-10-30 | 四川大学 | Target tracking method based on multi-scale pedestrian detection |
CN110503663A (en) * | 2019-07-22 | 2019-11-26 | 电子科技大学 | A kind of random multi-target automatic detection tracking based on pumping frame detection |
CN110503663B (en) * | 2019-07-22 | 2022-10-14 | 电子科技大学 | Random multi-target automatic detection tracking method based on frame extraction detection |
CN113033582A (en) * | 2019-12-09 | 2021-06-25 | 杭州海康威视数字技术股份有限公司 | Model training method, feature extraction method and device |
CN113033582B (en) * | 2019-12-09 | 2023-09-26 | 杭州海康威视数字技术股份有限公司 | Model training method, feature extraction method and device |
Also Published As
Publication number | Publication date |
---|---|
CN108986143B (en) | 2022-05-03 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
Xiao et al. | End-to-end deep learning for person search | |
CN108986143A (en) | Target detection tracking method in a kind of video | |
Chen et al. | Real-time multiple people tracking with deeply learned candidate selection and person re-identification | |
Zheng et al. | Person re-identification in the wild | |
CN106096577B (en) | A kind of target tracking method in camera distribution map | |
Yu et al. | Fast action proposals for human action detection and search | |
CN108986064B (en) | People flow statistical method, equipment and system | |
Ding et al. | Violence detection in video by using 3D convolutional neural networks | |
Ullah et al. | AI-assisted edge vision for violence detection in IoT-based industrial surveillance networks | |
US10679067B2 (en) | Method for detecting violent incident in video based on hypergraph transition | |
CN102890781B (en) | A kind of Highlight recognition methods for badminton game video | |
Long et al. | Stand-alone inter-frame attention in video models | |
CN107145862A (en) | A kind of multiple features matching multi-object tracking method based on Hough forest | |
CN113591968A (en) | Infrared weak and small target detection method based on asymmetric attention feature fusion | |
CN111968152B (en) | Dynamic identity recognition method and device | |
CN103902966A (en) | Video interaction event analysis method and device base on sequence space-time cube characteristics | |
CN109344842A (en) | A kind of pedestrian's recognition methods again based on semantic region expression | |
Su et al. | PCG-TAL: Progressive cross-granularity cooperation for temporal action localization | |
Gao et al. | Osmo: Online specific models for occlusion in multiple object tracking under surveillance scene | |
US20190171899A1 (en) | Automatic extraction of attributes of an object within a set of digital images | |
Fang et al. | Traffic police gesture recognition by pose graph convolutional networks | |
Tsai et al. | Swin-JDE: Joint detection and embedding multi-object tracking in crowded scenes based on swin-transformer | |
Wang et al. | Fast and accurate action detection in videos with motion-centric attention model | |
Yang et al. | C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks | |
Wu et al. | Track-clustering error evaluation for track-based multi-camera tracking system employing human re-identification |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PB01 | Publication | ||
PB01 | Publication | ||
SE01 | Entry into force of request for substantive examination | ||
SE01 | Entry into force of request for substantive examination | ||
GR01 | Patent grant | ||
GR01 | Patent grant | ||
TR01 | Transfer of patent right | ||
TR01 | Transfer of patent right |
Effective date of registration: 20231030 Address after: Room 319-2, 3rd Floor, Building 2, No. 262 Wantang Road, Xihu District, Hangzhou City, Zhejiang Province, 310012 Patentee after: Zhejiang Jiehuixin Digital Technology Co.,Ltd. Address before: 311121 East Building, building 7, No. 998, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province Patentee before: ZHEJIANG ICARE VISION TECHNOLOGY Co.,Ltd. |