CN108986143A - Target detection tracking method in a kind of video - Google Patents

Target detection tracking method in a kind of video Download PDF

Info

Publication number
CN108986143A
CN108986143A CN201810940035.1A CN201810940035A CN108986143A CN 108986143 A CN108986143 A CN 108986143A CN 201810940035 A CN201810940035 A CN 201810940035A CN 108986143 A CN108986143 A CN 108986143A
Authority
CN
China
Prior art keywords
video
target
sequence
image frame
frame sequence
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN201810940035.1A
Other languages
Chinese (zh)
Other versions
CN108986143B (en
Inventor
尚凌辉
张兆生
王弘玥
郑永宏
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Jiehuixin Digital Technology Co.,Ltd.
Original Assignee
ZHEJIANG ICARE VISION TECHNOLOGY Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by ZHEJIANG ICARE VISION TECHNOLOGY Co Ltd filed Critical ZHEJIANG ICARE VISION TECHNOLOGY Co Ltd
Priority to CN201810940035.1A priority Critical patent/CN108986143B/en
Publication of CN108986143A publication Critical patent/CN108986143A/en
Application granted granted Critical
Publication of CN108986143B publication Critical patent/CN108986143B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/246Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence

Abstract

The invention discloses target detection tracking methods in a kind of video.The present invention carries out segmentation sampling to video first, obtains several segments video image frame sequence.Then neural network model is usedTarget detection and feature extraction are carried out to every section of video image frame sequence.The correlation matrix of the corresponding target feature vector of all testing results exported in video sequence is calculated again, and then obtains tracking result of all targets detected in frequency sequence in video sequence.Finally temporally axis is ranked up the video image frame sequence of segmentation sampling, and the target detection pursuit path and eigenmatrix of video image frame sequence are input to neural network model, the tracking characteristics of each target in each video image frame sequence are obtained, the correlation of all targets between two adjacent video image frame sequence are calculated using this tracking characteristics, to complete the tracking of target in entire video-frequency band.The present invention, which completes the calculation amount that target detection tracing task needs in video, can be effectively reduced.

Description

Target detection tracking method in a kind of video
Technical field
The invention belongs to technical field of computer vision, it is related to target detection tracking method in a kind of video.
Background technique
The monitoring devices such as bayonet, public security and disparate networks video camera are largely installed and are used, these equipment Video data collected traffic offence, in terms of play the role of it is very big, but with these equipment pacify Loading amount is increasing, and the data volume of production also increasingly increases, and is stored and is faced huge challenge, video knot using these data Structure has become a research hotspot of scientific research and industry.
The underlying issue that all can't steer clear of in all kinds of video structural schemes be exactly accurately and efficiently detection and tracking view Common-denominator target in frequency.A kind of " target following optimization method based on tracking study detection " 107967692A, a kind of " real-time nothing Man-machine video object detection and tracking " 108108697A, " multiple target pedestrian detection and track side based on deep learning Method " it is all to complete target detection using single-frame images in the patents such as 107563313A, calculate object detection results relevant range Then feature relies on matching and tracking that these features complete close interframe target.Target detection relies on all in these methods It is the information of single-frame images, the standard of testing result cannot be led to using the relevant information between close picture frame in time series True rate will receive limitation;The feature used during matched jamming simultaneously is also extracting on single-frame images, and the spy Sign wants that the different target individual of multiclass can be distinguished, and the similar purpose for encountering colleague is very easy to matching error, and tracking is caused to be lost It loses;Finally, the accuracy rate in order to guarantee detecting and tracking, limited every the interval of frame sampling, cause calculation amount bigger, efficiency compares It is low.
Summary of the invention
In view of the deficiencies of the prior art, the present invention provides target detection tracking methods in a kind of video.
The technical solution adopted for solving the technical problem of the present invention are as follows:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence.
Step 2, using neural network model M1Target detection and feature extraction are carried out to every section of video image frame sequence, it is defeated Information out includes: the number in the sequence of image where target, rectangle frame, the clarification of objective vector of target in the picture.
Step 3, the correlation matrix for calculating the corresponding target feature vector of all testing results exported in video sequence, into And obtain tracking result of all targets detected in frequency sequence in video sequence.
Step 4, temporally axis, by inside video image frame sequence target detection pursuit path and eigenmatrix be input to Neural network model M2, the tracking characteristics of each target in each video image frame sequence are obtained, are calculated using this tracking characteristics The correlation of all targets between two adjacent video image frame sequence, to complete the tracking of target in entire video-frequency band.
Beneficial effects of the present invention:
1, the accuracy rate of detector is improved using the inter-frame information of time-series image.
2, the space time information of time-series image is made full use of to improve the tracking effect of target.
3, the calculation amount of detecting and tracking can be effectively reduced, operational efficiency is improved.
4, detection and tracking effectively merges, and improves detecting and tracking overall effect.
Detailed description of the invention
Fig. 1 the method for the present invention flow chart.
Specific embodiment
In order to make the object, technical scheme and advantages of the embodiment of the invention clearer, below in conjunction with the embodiment of the present invention In attached drawing, the technical scheme in the embodiment of the invention is clearly and completely described, it is clear that described embodiment is only It is only a part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiment of the present invention, ordinary skill people Member's every other embodiment obtained without making creative work, shall fall within the protection scope of the present invention.
As shown in Figure 1, the present invention the following steps are included:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence.
Step 2 includes: target to every section of video image frame sequence progress target detection and feature extraction, the information of output Place image number in the sequence, rectangle frame, the clarification of objective vector of target in the picture.
The correlation matrix of the corresponding target feature vector of all testing results exported in video sequence in step 3, calculating, And then obtain the tracking result in frequency sequence of all targets detected in video sequence.
Step 4, temporally axis, using the target detection pursuit path inside video image frame sequence, (including target is in sequence The rectangle frame of number, target in the picture in column) and eigenmatrix (the serial splicing of feature vector), to front and back adjacent video Target in sequence carries out matched jamming.
Wherein the target detection of every section of video image frame sequence and the calculation method of feature extraction are: execution has trained Neural network model M1Reasoning process, directly obtain the number in the sequence of image where target, target in the picture Rectangle frame, clarification of objective vector.
Wherein neural network model M1Training method is:
Collect mark video data;
Sample Video section is cut, the volume of image in the sequence where obtaining video image frame sequence and the target that marked Number, the number information of target rectangle frame in the picture, target;
Pass through the detection to target in sequence of video images and multitask training optimization network model of classifying.
It is a kind of embodiment of target detection tracking method in video below, the specific steps are as follows:
The neural network model M that target detection and matching characteristic calculate in training video image frame sequence1, specific step It is as follows:
1, multitude of video section V is collected;Target position and each target in artificial mark sequence of video images from occur to The id information of disappearance obtains original mark sample set A={ V1,V2,…,VL}。
2, using deep learning theory and method, to each video-frequency band V in original mark sample set Ai, segmentation sampling life At several video image frame sequences Pi,Pi+1,…,Pi+k∈Vi, obtain training test sample collection B={ P1,P2,…,Pi, Pi+1,…,Pi+k…,Pn-k,…,Pn-1,Pn}。
3, using deep learning theory and method, combined training test sample collection B, training is obtained in the way of multitask It can detecte target and calculate the neural network model M of target signature1
The neural network model M that object matching tracking characteristics calculate between training video image frame sequence2, specific step It is as follows:
1, neural network model M is utilized1, obtain every section of sequence of video images P in training test sample collection BiIn each target Pursuit path (number in the sequence of image where target, the rectangle frame of target in the picture) and eigenmatrix (feature to The serial splicing of amount).
2, every section of video V is utilizediThe target information of middle mark and each video image frame sequence Pi+jBy neural network mould Type M1The pursuit path and eigenmatrix of obtained target, obtain video-frequency band ViIn each target in different video image frame sequence In feature samples collection: O={ q1,q2,…,qk, wherein qiBy M1In PiMiddle generation, to generate mesh between sequence of video images Mark the training dataset C={ O of matched jamming feature1,O2,…,Os}
3, using deep learning theory and method, combined training test sample collection C is trained to obtain for calculating video image The neural network model M of object matching tracking characteristics between sequence2
Utilize neural network model M1, M2, target in detecting and tracking video, the specific steps are as follows:
1, it samples to the video segmentation that needs are analyzed, generates several video image frame sequences
2, to each video image frame sequence, neural network model M is executed1Reasoning process, where obtaining each target Image number in the sequence, rectangle frame, the clarification of objective vector of target in the picture
3, the correlation matrix of the corresponding target feature vector of all testing results exported in video image frame sequence is calculated, Wherein the calculating of correlation can use Euclidean distance, mahalanobis distance etc., and then obtain all in video image frame sequence detect Target the tracking result in video image frame sequence.
4, temporally axis information is ranked up the video image frame sequence of segmentation sampling, according to pursuit path and feature square Battle array executes neural network model M2Reasoning process, obtain the tracking characteristics of each target in each video image frame sequence Using the correlations of all targets between this feature calculation two adjacent video image frame sequence, (the wherein calculating of correlation can be with With Euclidean distance, mahalanobis distance etc.), to complete the tracking of target in entire video-frequency band.
To sum up, the present invention is based on video image frame sequence data, in conjunction with the information and video image frame sequence of single-frame images Between frame-to-frame correlation, realize target detection tracking method in a kind of video.Compared to the target detection side based on single-frame images Method combines relevant information between image sequence in the present invention, and target detection performance has promotion.Using machine learning method from It being calculated in single-frame images, is used for the matched feature of target following, this feature needs to meet the differentiation of similar similar purpose, Very big or feature the separating capacity of calculation amount for obtaining this feature is poor, is easy matching error, tracking is caused to be lost It loses.Tracking and matching of the invention is divided into two stages thus, in short time inner video image frame sequence the matched jamming of target and Object matching tracking between different images frame frequency sequence: the matched jamming feature inside video image frame sequence relies on video figure As the spininess image information in frame sequence and the correlation between sequence, the separating capacity of feature is also only limitted to video image frame sequence Between internal target;Object matching between video image frame sequence mainly utilizes of video image frame sequence internal object Feature with tracking result and target in video image frame sequence, can very effective raising tracking accuracy rate.Same phase Than other methods, the present invention, which completes the calculation amount that target detection tracing task needs in video, be can be effectively reduced.
The foregoing is only a preferred embodiment of the present invention, is not intended to limit the scope of the present invention, should Understand, the present invention is not limited to implementation as described herein, the purpose of these implementations description is to help this field In technical staff practice the present invention.

Claims (3)

1. target detection tracking method in a kind of video, it is characterised in that method includes the following steps:
Step 1 carries out segmentation sampling to video, obtains several segments video image frame sequence;
Step 2, using neural network modelTarget detection and feature extraction, output are carried out to every section of video image frame sequence Information include: the number in the sequence of image where target, rectangle frame, the clarification of objective vector of target in the picture;
Step 3, the correlation matrix for calculating the corresponding target feature vector of all testing results exported in video sequence, and then Tracking result of all targets detected in frequency sequence into video sequence;
Step 4, temporally axis are ranked up the video image frame sequence of segmentation sampling, and the target of video image frame sequence is examined It surveys pursuit path and eigenmatrix is input to neural network model, obtain each target in each video image frame sequence Tracking characteristics calculate the correlation of all targets between two adjacent video image frame sequence using this tracking characteristics, thus complete At the tracking of target in entire video-frequency band.
2. target detection tracking method in a kind of video according to claim 1, it is characterised in that: the neural network ModelIt establishes in the following ways:
Multitude of video section, the artificial target position marked in sequence of video images and each target are collected from the ID occurred to disappearance Information obtains original mark sample set;
Using deep learning method, to each video-frequency band in original mark sample set, segmentation sampling generates several video figures As frame sequence, obtain training test sample collection;
Using deep learning method, combined training test sample collection and in the way of multitask training obtain neural network model
3. target detection tracking method in a kind of video according to claim 2, it is characterised in that: the neural network ModelIt establishes in the following ways:
Utilize neural network model, obtain the tracking rail that training test sample concentrates each target in every section of sequence of video images Mark and eigenmatrix;
Pass through neural network model using the target information and each video image frame sequence marked in every section of videoObtained mesh Target pursuit path and eigenmatrix obtain feature samples of each target in different video image frame sequence in every section of video Collection, to generate the training dataset of object matching tracking characteristics between sequence of video images;
Using deep learning method, combined training data set, training obtains neural network model
CN201810940035.1A 2018-08-17 2018-08-17 Target detection tracking method in video Active CN108986143B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201810940035.1A CN108986143B (en) 2018-08-17 2018-08-17 Target detection tracking method in video

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201810940035.1A CN108986143B (en) 2018-08-17 2018-08-17 Target detection tracking method in video

Publications (2)

Publication Number Publication Date
CN108986143A true CN108986143A (en) 2018-12-11
CN108986143B CN108986143B (en) 2022-05-03

Family

ID=64553984

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201810940035.1A Active CN108986143B (en) 2018-08-17 2018-08-17 Target detection tracking method in video

Country Status (1)

Country Link
CN (1) CN108986143B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711332A (en) * 2018-12-26 2019-05-03 浙江捷尚视觉科技股份有限公司 A kind of face tracking method and application based on regression algorithm
CN109934096A (en) * 2019-01-22 2019-06-25 浙江零跑科技有限公司 Automatic Pilot visual perception optimization method based on feature timing dependence
CN110503663A (en) * 2019-07-22 2019-11-26 电子科技大学 A kind of random multi-target automatic detection tracking based on pumping frame detection
CN111862145A (en) * 2019-04-24 2020-10-30 四川大学 Target tracking method based on multi-scale pedestrian detection
CN113033582A (en) * 2019-12-09 2021-06-25 杭州海康威视数字技术股份有限公司 Model training method, feature extraction method and device

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BRPI0806019A2 (en) * 2008-07-15 2010-08-31 Invisys Sist S De Visao Comput Ltda counting and tracking of people on the move based on computer vision
CN102004920A (en) * 2010-11-12 2011-04-06 浙江工商大学 Method for splitting and indexing surveillance videos
CN102750527A (en) * 2012-06-26 2012-10-24 浙江捷尚视觉科技有限公司 Long-time stable human face detection and tracking method in bank scene and long-time stable human face detection and tracking device in bank scene
CN104094279A (en) * 2014-04-30 2014-10-08 中国科学院自动化研究所 Large-range-first cross-camera visual target re-identification method
CN104954743A (en) * 2015-06-12 2015-09-30 西安理工大学 Multi-camera semantic association target tracking method
CN105574505A (en) * 2015-12-16 2016-05-11 深圳大学 Human body target re-identification method and system among multiple cameras
CN106920248A (en) * 2017-01-19 2017-07-04 博康智能信息技术有限公司上海分公司 A kind of method for tracking target and device
US20180183650A1 (en) * 2012-12-05 2018-06-28 Origin Wireless, Inc. Method, apparatus, and system for object tracking and navigation

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
BRPI0806019A2 (en) * 2008-07-15 2010-08-31 Invisys Sist S De Visao Comput Ltda counting and tracking of people on the move based on computer vision
CN102004920A (en) * 2010-11-12 2011-04-06 浙江工商大学 Method for splitting and indexing surveillance videos
CN102750527A (en) * 2012-06-26 2012-10-24 浙江捷尚视觉科技有限公司 Long-time stable human face detection and tracking method in bank scene and long-time stable human face detection and tracking device in bank scene
US20180183650A1 (en) * 2012-12-05 2018-06-28 Origin Wireless, Inc. Method, apparatus, and system for object tracking and navigation
CN104094279A (en) * 2014-04-30 2014-10-08 中国科学院自动化研究所 Large-range-first cross-camera visual target re-identification method
CN104954743A (en) * 2015-06-12 2015-09-30 西安理工大学 Multi-camera semantic association target tracking method
CN105574505A (en) * 2015-12-16 2016-05-11 深圳大学 Human body target re-identification method and system among multiple cameras
CN106920248A (en) * 2017-01-19 2017-07-04 博康智能信息技术有限公司上海分公司 A kind of method for tracking target and device

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
YINGHAO CAI 等: "Matching tracking sequences across widely separated cameras", 《2008 15TH IEEE INTERNATIONAL CONFERENCE ON IMAGE PROCESSING》 *
吴尔杰: "监控视频中多目标检测与跟踪研究", 《中国优秀博硕士学位论文全文数据库(硕士)信息科技辑》 *
杨秋英: "相关序列小目标图像运动跟踪与仿真研究", 《系统仿真学报》 *
王乐东 等: "序列帧间双重匹配的红外点目标跟踪算法", 《光电子·激光》 *

Cited By (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711332A (en) * 2018-12-26 2019-05-03 浙江捷尚视觉科技股份有限公司 A kind of face tracking method and application based on regression algorithm
CN109711332B (en) * 2018-12-26 2021-03-26 浙江捷尚视觉科技股份有限公司 Regression algorithm-based face tracking method and application
CN109934096A (en) * 2019-01-22 2019-06-25 浙江零跑科技有限公司 Automatic Pilot visual perception optimization method based on feature timing dependence
CN109934096B (en) * 2019-01-22 2020-12-11 浙江零跑科技有限公司 Automatic driving visual perception optimization method based on characteristic time sequence correlation
CN111862145A (en) * 2019-04-24 2020-10-30 四川大学 Target tracking method based on multi-scale pedestrian detection
CN110503663A (en) * 2019-07-22 2019-11-26 电子科技大学 A kind of random multi-target automatic detection tracking based on pumping frame detection
CN110503663B (en) * 2019-07-22 2022-10-14 电子科技大学 Random multi-target automatic detection tracking method based on frame extraction detection
CN113033582A (en) * 2019-12-09 2021-06-25 杭州海康威视数字技术股份有限公司 Model training method, feature extraction method and device
CN113033582B (en) * 2019-12-09 2023-09-26 杭州海康威视数字技术股份有限公司 Model training method, feature extraction method and device

Also Published As

Publication number Publication date
CN108986143B (en) 2022-05-03

Similar Documents

Publication Publication Date Title
Xiao et al. End-to-end deep learning for person search
CN108986143A (en) Target detection tracking method in a kind of video
Chen et al. Real-time multiple people tracking with deeply learned candidate selection and person re-identification
Zheng et al. Person re-identification in the wild
CN106096577B (en) A kind of target tracking method in camera distribution map
Yu et al. Fast action proposals for human action detection and search
CN108986064B (en) People flow statistical method, equipment and system
Ding et al. Violence detection in video by using 3D convolutional neural networks
Ullah et al. AI-assisted edge vision for violence detection in IoT-based industrial surveillance networks
US10679067B2 (en) Method for detecting violent incident in video based on hypergraph transition
CN102890781B (en) A kind of Highlight recognition methods for badminton game video
Long et al. Stand-alone inter-frame attention in video models
CN107145862A (en) A kind of multiple features matching multi-object tracking method based on Hough forest
CN113591968A (en) Infrared weak and small target detection method based on asymmetric attention feature fusion
CN111968152B (en) Dynamic identity recognition method and device
CN103902966A (en) Video interaction event analysis method and device base on sequence space-time cube characteristics
CN109344842A (en) A kind of pedestrian's recognition methods again based on semantic region expression
Su et al. PCG-TAL: Progressive cross-granularity cooperation for temporal action localization
Gao et al. Osmo: Online specific models for occlusion in multiple object tracking under surveillance scene
US20190171899A1 (en) Automatic extraction of attributes of an object within a set of digital images
Fang et al. Traffic police gesture recognition by pose graph convolutional networks
Tsai et al. Swin-JDE: Joint detection and embedding multi-object tracking in crowded scenes based on swin-transformer
Wang et al. Fast and accurate action detection in videos with motion-centric attention model
Yang et al. C-RPNs: Promoting object detection in real world via a cascade structure of Region Proposal Networks
Wu et al. Track-clustering error evaluation for track-based multi-camera tracking system employing human re-identification

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20231030

Address after: Room 319-2, 3rd Floor, Building 2, No. 262 Wantang Road, Xihu District, Hangzhou City, Zhejiang Province, 310012

Patentee after: Zhejiang Jiehuixin Digital Technology Co.,Ltd.

Address before: 311121 East Building, building 7, No. 998, Wenyi West Road, Wuchang Street, Yuhang District, Hangzhou City, Zhejiang Province

Patentee before: ZHEJIANG ICARE VISION TECHNOLOGY Co.,Ltd.