CN111508006A - Moving target synchronous detection, identification and tracking method based on deep learning - Google Patents

Moving target synchronous detection, identification and tracking method based on deep learning Download PDF

Info

Publication number
CN111508006A
CN111508006A CN202010325315.9A CN202010325315A CN111508006A CN 111508006 A CN111508006 A CN 111508006A CN 202010325315 A CN202010325315 A CN 202010325315A CN 111508006 A CN111508006 A CN 111508006A
Authority
CN
China
Prior art keywords
target
tracking
identification
detection
frame
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010325315.9A
Other languages
Chinese (zh)
Inventor
王鸿鹏
代婉
宋玉琳
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nankai University
Original Assignee
Nankai University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nankai University filed Critical Nankai University
Priority to CN202010325315.9A priority Critical patent/CN111508006A/en
Publication of CN111508006A publication Critical patent/CN111508006A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/20Analysis of motion
    • G06T7/277Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/22Matching criteria, e.g. proximity measures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/11Region-based segmentation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/10Segmentation; Edge detection
    • G06T7/136Segmentation; Edge detection involving thresholding
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00Arrangements for image or video recognition or understanding
    • G06V10/20Image preprocessing
    • G06V10/25Determination of region of interest [ROI] or a volume of interest [VOI]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/10Image acquisition modality
    • G06T2207/10016Video; Image sequence
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20081Training; Learning
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/20Special algorithmic details
    • G06T2207/20084Artificial neural networks [ANN]
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

The invention provides a moving target synchronous detection, identification and tracking method based on deep learning, which comprises the following steps: acquiring video data to be detected collected by a camera, and inputting a video sequence; performing video capture on the video sequence; detecting an interested area of a target by utilizing an SSD target detection model; segmenting a target area by using a significance detection method; carrying out target identification by adopting a weighted cross entropy loss function; predicting the position of the tracking object in the next key frame by using a Kalman equation; performing feature matching on the position predicted by using a Kalman equation and the position calculated by a target detection algorithm; and calculating the ratio of the side length of the segmentation frame to the side length of the identification frame to perform scale updating judgment. The invention ensures the real-time performance of the algorithm, can be applied to various fields, not single field, and can solve the problem of real-time detection, identification and tracking under various dynamic complex environments, and the final design aim is to form a practical and generalized technical framework.

Description

Moving target synchronous detection, identification and tracking method based on deep learning
Technical Field
The invention belongs to the technical field of computer vision, and particularly relates to a moving target synchronous detection, identification and tracking method based on deep learning.
Background
With the rapid development of computer technology, the synchronous target detection, identification and tracking has become a research hotspot in the field of computer vision due to the high application value in the aspects of video monitoring, scene understanding, human-computer interaction and the like, and attracts more and more overseas and overseas scholars and scientific research institutions to participate in the research in the field. In brief, the target detection, identification and tracking is to find and locate an interested target object from a specific image or video sequence frame by using computer knowledge such as machine learning, natural language processing, image processing and the like, and identify, segment and extract a moving target from background information of an image sequence, so as to deduce the position condition of the maximum probability of the target, thereby providing a reliable basis for people to further identify and understand the behavior of the target.
In recent years, a great number of improved algorithms are proposed in the target tracking field and the target detection and recognition field, but the distance enables a computer to perceive the real world, reasonable decisions are made in a complex and dynamic environment, and a plurality of scientific problems need to be overcome. Based on the above, the invention provides a method for synchronously detecting, identifying and tracking a moving target based on deep learning, and aims to solve the problem of synchronously detecting, identifying and tracking the moving target in a complex and dynamic environment (such as shielding between the target and a background, deformation and scale change of the target, illumination change and the like).
Disclosure of Invention
In order to solve the technical problem, the invention provides a moving target synchronous detection, identification and tracking method based on deep learning, which comprises the following steps:
s1: acquiring video data to be detected collected by a camera, wherein the video data to be detected comprises a plurality of frames of images to be detected, and inputting a video sequence;
s2: performing video capture on the video sequence;
s3: target detection: firstly, acquiring first sample data, wherein the first sample data is image data containing a target, manually labeling the first sample data, dividing the labeled first sample data into a first training set and a first testing set, training a neural network by using the first training set, and testing the trained neural network by using the first testing set to establish the target detection model based on the SSD; detecting the target by using an SSD target detection model to detect an interested area;
s4: target segmentation: after video capture is carried out, extracting target features and target edge features by using a saliency detection method, and segmenting a target based on the target features, wherein the segmented salient region is a target region;
s5: target identification: target identification is carried out on the basis of target detection, firstly, the ID of a target is determined, and if the problem of unbalanced proportion of training samples occurs, a weighted cross entropy loss function can be adopted, so that the identification accuracy is improved;
s6: target tracking: for the I-k key frames, performing target detection on videos acquired by a camera by adopting an SSD target detection model, obtaining tracking objects according to target detection results, maintaining a Kalman equation for each tracking object, and predicting the positions of the tracking objects in the I- (k +1) key frames by utilizing the Kalman equation;
s7: and (3) judging feature matching: matching the position predicted by using the Kalman equation with the position calculated by a target detection algorithm in a calculation and comparison mode, updating the Kalman equation if the feature matching is successful, and continuing to detect the next frame without any processing if the feature matching is failed;
s8: and (3) scale updating judgment: and calculating the ratio of the side length of the segmentation frame to the side length of the identification frame, and when the ratio of the two is greater than the threshold, making lambda equal to the ratio, namely, the identification frame is increased to the size of the segmentation frame in proportion, and if the ratio of the two is less than the threshold, continuing to segment the next frame without any processing.
Preferably, in the target segmentation, the target feature and the target edge feature are extracted by using a saliency detection method, the higher the saliency is, the more likely the saliency is to belong to the target, and the regions with higher and lower saliency are represented by 0 and 1 in the binary image respectively, that is, the target segmentation is completed.
Preferably, the feature matching is performed on the subject position calculated by the target detection algorithm and the tracked subject position in the feature matching determination, and when the features cannot be matched, a feature matching mechanism with the target detection as a reference is started.
Preferably, the size of the body of the segmented frame after the target is segmented in the scale updating determination and the size of the body of the identified frame after the target is identified are subjected to scale updating, and when the ratio of the two is higher than a preset threshold, a scale coordination mechanism taking the target segmentation as a reference is started, and iterative tracking is continued.
Compared with the prior art, the invention has the beneficial effects that:
1. according to the method, an image segmentation technology is fully utilized, a target segmentation result is used as a target reference size aiming at different scales in the target segmentation and target identification processes, the size of a region to be identified is adjusted timely, a target identification position is updated once when scale updating judgment meets conditions, and the inference process does not need to wait for a judgment result, so that the algorithm real-time performance is guaranteed;
2. the design of the invention is intended to be applied to various fields, but not to a single field, and can solve the problems of real-time detection, identification and tracking of environments such as shielding between a target and a background, deformation and scale change of the target, illumination change and the like under various dynamic complex environments, and the final target of the design is a technical framework forming practicality and generalization.
Drawings
FIG. 1 is an overall framework flow diagram of the present invention;
FIG. 2 is a flow diagram of the object detection and tracking framework of the present invention;
FIG. 3 is a flow diagram of an object segmentation and recognition framework of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
The invention is further described below:
example (b):
as shown in fig. 1, a method for synchronously detecting, identifying and tracking a moving object based on deep learning is characterized in that the method for synchronously detecting, identifying and tracking a moving object comprises the following steps:
(1) acquiring video data to be detected collected by a camera, wherein the video data to be detected comprises a plurality of frames of images to be detected, and inputting a video sequence;
(2) performing video capture on a video sequence, wherein the video capture mainly comprises video frame capture;
(3) target detection and tracking: the traditional target detection algorithm usually needs to occupy a large amount of computing resources and is difficult to operate efficiently under the condition of limited computing resources, so that the target is detected by using an SSD target detection model: as shown in fig. 2, first sample data is obtained, wherein the first sample data is image data including a target, the first sample data is manually subjected to image annotation, the annotated first sample data is divided into a first training set and a first test set, a neural network is trained by using the first training set, the trained neural network is tested by using the first test set, so as to establish a target detection model based on SSD, the target is detected by using the SSD target detection model, coordinates of four positions of a rectangular detection frame are obtained, region division is performed according to the position coordinates, and the position of a first frame region of interest is determined; for the I-k key frames, performing target detection on a video acquired by a camera by adopting an SSD target detection model, introducing a Kalman filtering mechanism into a tracking area obtained by using a correlation matrix of target characteristic vectors corresponding to all detection results output in a video sequence, and performing filtering processing on observation data of a target in an image to be detected of the current frame to obtain filtered target observation data in the current observation image, namely predicting the position of a tracking object in the I- (k +1) key frames by utilizing a Kalman equation;
(4) target identification and segmentation: as shown in fig. 3, after video capture is performed, a saliency detection method is used to extract target features and target edge features, the higher the saliency is, the more likely the saliency is to belong to a target, areas with higher and lower saliency are represented by 0 and 1 in a binary image respectively, that is, target segmentation is completed, and the segmented salient areas are target areas; in order to reduce the use of computing resources, target identification is carried out on the basis of target detection, firstly, the ID of a target is determined, and if the problem of unbalanced proportion of training samples occurs, a weighted cross entropy loss function can be adopted, so that the identification accuracy is improved;
(5) and (3) judging feature matching: carrying out feature matching on the main body position calculated by the target detection algorithm and the tracked main body position, if the feature matching is successful, updating the Kalman equation, if the feature matching is failed, carrying out no processing, continuously detecting the next frame, and when the features cannot be matched, starting a feature matching mechanism taking target detection as a reference;
(6) and (3) scale updating judgment: and carrying out scale updating on the main body size of the segmentation frame after the target is segmented and the main body size of the identification frame after the target is identified, calculating the ratio of the side length of the segmentation frame to the side length of the identification frame, starting a scale coordination mechanism taking the target as a reference when the ratio of the two is greater than a threshold value, continuing iterative tracking to enable lambda to be equal to the ratio, namely, the identification frame is increased to the size of the segmentation frame in proportion, and if the ratio of the two is less than the threshold value, continuing segmenting the next frame without any treatment.
Specifically, the method can be applied to various fields, not a single field, and can solve the problems of real-time detection, identification and tracking of various dynamic complex environments, such as shielding between a target and a background, deformation and scale change of the target, illumination change and the like, and the designed final target is a technical framework for forming practicality and generalization.
It should be noted that, in this document, moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus.
Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that changes, modifications, substitutions and alterations can be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the appended claims and their equivalents.

Claims (4)

1. A method for synchronously detecting, identifying and tracking a moving target based on deep learning is characterized in that the method for synchronously detecting, identifying and tracking the moving target comprises the following steps:
s1: acquiring video data to be detected collected by a camera, wherein the video data to be detected comprises a plurality of frames of images to be detected, and inputting a video sequence;
s2: performing video capture on the video sequence;
s3: target detection: firstly, acquiring first sample data, wherein the first sample data is image data containing a target, manually labeling the first sample data, dividing the labeled first sample data into a first training set and a first testing set, training a neural network by using the first training set, and testing the trained neural network by using the first testing set to establish the target detection model based on the SSD; detecting the target by using an SSD target detection model to detect an interested area;
s4: target segmentation: after video capture is carried out, extracting target features and target edge features by using a saliency detection method, and segmenting a target based on the target features, wherein the segmented salient region is a target region;
s5: target identification: target identification is carried out on the basis of target detection, firstly, the ID of a target is determined, and if the problem of unbalanced proportion of training samples occurs, a weighted cross entropy loss function can be adopted, so that the identification accuracy is improved;
s6: target tracking: for the I-k key frames, performing target detection on videos acquired by a camera by adopting an SSD target detection model, obtaining tracking objects according to target detection results, maintaining a Kalman equation for each tracking object, and predicting the positions of the tracking objects in the I- (k +1) key frames by utilizing the Kalman equation;
s7: and (3) judging feature matching: matching the position predicted by using the Kalman equation with the position calculated by a target detection algorithm in a calculation and comparison mode, updating the Kalman equation if the feature matching is successful, and continuing to detect the next frame without any processing if the feature matching is failed;
s8: and (3) scale updating judgment: and calculating the ratio of the side length of the segmentation frame to the side length of the identification frame, and when the ratio of the two is greater than the threshold, making lambda equal to the ratio, namely, the identification frame is increased to the size of the segmentation frame in proportion, and if the ratio of the two is less than the threshold, continuing to segment the next frame without any processing.
2. The method for synchronously detecting, identifying and tracking the moving target based on the deep learning as claimed in claim 1, wherein the method using saliency detection in the target segmentation extracts target features and target edge features, the higher the saliency is, the more likely the region belongs to the target, and the regions with higher and lower saliency are represented by 0 and 1 in a binary image respectively, i.e. the target segmentation is completed.
3. The method for synchronously detecting, identifying and tracking the moving target based on the deep learning as claimed in claim 1, wherein the feature matching is performed on the subject position calculated by the target detection algorithm and the tracked subject position in the feature matching determination, and when the features cannot be matched, a feature matching mechanism based on the target detection is started.
4. The method for synchronously detecting, identifying and tracking the moving target based on the deep learning as claimed in claim 1, wherein the body size of the segmentation frame after the target segmentation in the scale update determination and the body size of the identification frame after the target identification are subjected to scale update, and when a ratio of the two is higher than a preset threshold, a scale coordination mechanism based on the target segmentation is started to continue iterative tracking.
CN202010325315.9A 2020-04-23 2020-04-23 Moving target synchronous detection, identification and tracking method based on deep learning Pending CN111508006A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010325315.9A CN111508006A (en) 2020-04-23 2020-04-23 Moving target synchronous detection, identification and tracking method based on deep learning

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010325315.9A CN111508006A (en) 2020-04-23 2020-04-23 Moving target synchronous detection, identification and tracking method based on deep learning

Publications (1)

Publication Number Publication Date
CN111508006A true CN111508006A (en) 2020-08-07

Family

ID=71877898

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010325315.9A Pending CN111508006A (en) 2020-04-23 2020-04-23 Moving target synchronous detection, identification and tracking method based on deep learning

Country Status (1)

Country Link
CN (1) CN111508006A (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112132136A (en) * 2020-09-11 2020-12-25 华为技术有限公司 Target tracking method and device
CN113256637A (en) * 2021-07-15 2021-08-13 北京小蝇科技有限责任公司 Urine visible component detection method based on deep learning and context correlation
CN113810146A (en) * 2021-08-05 2021-12-17 西安理工大学 Data synchronous transmission method without external reference clock
CN114066936A (en) * 2021-11-06 2022-02-18 中国电子科技集团公司第五十四研究所 Target reliability tracking method in small target capturing process
CN115542362A (en) * 2022-12-01 2022-12-30 成都信息工程大学 High-precision space positioning method, system, equipment and medium for electric power operation site

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875425A (en) * 2017-01-22 2017-06-20 北京飞搜科技有限公司 A kind of multi-target tracking system and implementation method based on deep learning
CN110188596A (en) * 2019-01-04 2019-08-30 北京大学 Monitor video pedestrian real-time detection, Attribute Recognition and tracking and system based on deep learning
CN110211150A (en) * 2019-04-25 2019-09-06 南开大学 A kind of real-time vision target identification method with scale coordination mechanism
CN110399808A (en) * 2019-07-05 2019-11-01 桂林安维科技有限公司 A kind of Human bodys' response method and system based on multiple target tracking

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106875425A (en) * 2017-01-22 2017-06-20 北京飞搜科技有限公司 A kind of multi-target tracking system and implementation method based on deep learning
CN110188596A (en) * 2019-01-04 2019-08-30 北京大学 Monitor video pedestrian real-time detection, Attribute Recognition and tracking and system based on deep learning
CN110211150A (en) * 2019-04-25 2019-09-06 南开大学 A kind of real-time vision target identification method with scale coordination mechanism
CN110399808A (en) * 2019-07-05 2019-11-01 桂林安维科技有限公司 A kind of Human bodys' response method and system based on multiple target tracking

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ALEX BEWLEY ET AL.: "SIMPLE ONLINE AND REALTIME TRACKING", 《ARXIV:1602.00763.V2》 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112132136A (en) * 2020-09-11 2020-12-25 华为技术有限公司 Target tracking method and device
CN113256637A (en) * 2021-07-15 2021-08-13 北京小蝇科技有限责任公司 Urine visible component detection method based on deep learning and context correlation
CN113256637B (en) * 2021-07-15 2021-11-05 北京小蝇科技有限责任公司 Urine visible component detection method based on deep learning and context correlation
CN113810146A (en) * 2021-08-05 2021-12-17 西安理工大学 Data synchronous transmission method without external reference clock
CN113810146B (en) * 2021-08-05 2023-09-22 西安理工大学 Data synchronous transmission method without external reference clock
CN114066936A (en) * 2021-11-06 2022-02-18 中国电子科技集团公司第五十四研究所 Target reliability tracking method in small target capturing process
CN114066936B (en) * 2021-11-06 2023-09-12 中国电子科技集团公司第五十四研究所 Target reliability tracking method in small target capturing process
CN115542362A (en) * 2022-12-01 2022-12-30 成都信息工程大学 High-precision space positioning method, system, equipment and medium for electric power operation site

Similar Documents

Publication Publication Date Title
CN111259786B (en) Pedestrian re-identification method based on synchronous enhancement of appearance and motion information of video
CN111508006A (en) Moving target synchronous detection, identification and tracking method based on deep learning
CN107123131B (en) Moving target detection method based on deep learning
CN111783576B (en) Pedestrian re-identification method based on improved YOLOv3 network and feature fusion
CN109919977B (en) Video motion person tracking and identity recognition method based on time characteristics
CN108520226B (en) Pedestrian re-identification method based on body decomposition and significance detection
CN107133569B (en) Monitoring video multi-granularity labeling method based on generalized multi-label learning
Lin et al. Face gender recognition based on face recognition feature vectors
CN111275010A (en) Pedestrian re-identification method based on computer vision
CN113487610B (en) Herpes image recognition method and device, computer equipment and storage medium
CN115527269B (en) Intelligent human body posture image recognition method and system
CN113706481A (en) Sperm quality detection method, sperm quality detection device, computer equipment and storage medium
CN113780145A (en) Sperm morphology detection method, sperm morphology detection device, computer equipment and storage medium
Xie et al. Feature consistency-based prototype network for open-set hyperspectral image classification
Zhu et al. Scene text relocation with guidance
CN108985216B (en) Pedestrian head detection method based on multivariate logistic regression feature fusion
Wang et al. Self-supervised learning for high-resolution remote sensing images change detection with variational information bottleneck
CN116912670A (en) Deep sea fish identification method based on improved YOLO model
CN115100681A (en) Clothes identification method, system, medium and equipment
CN110909678B (en) Face recognition method and system based on width learning network feature extraction
CN111191575B (en) Naked flame detection method and system based on flame jumping modeling
CN114743257A (en) Method for detecting and identifying image target behaviors
CN113761987A (en) Pedestrian re-identification method and device, computer equipment and storage medium
CN112651996A (en) Target detection tracking method and device, electronic equipment and storage medium
Zhang et al. Robust visual tracking using discriminative stable regions and K-means clustering

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200807