CN110910428B - Real-time multi-target tracking method based on neural network - Google Patents

Real-time multi-target tracking method based on neural network Download PDF

Info

Publication number
CN110910428B
CN110910428B CN201911236736.8A CN201911236736A CN110910428B CN 110910428 B CN110910428 B CN 110910428B CN 201911236736 A CN201911236736 A CN 201911236736A CN 110910428 B CN110910428 B CN 110910428B
Authority
CN
China
Prior art keywords
target
tracking
tracked
matched
position coordinate
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN201911236736.8A
Other languages
Chinese (zh)
Other versions
CN110910428A (en
Inventor
吴圣杰
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jiangsu Zhongyun Smart Data Technology Co ltd
Original Assignee
Jiangsu Zhongyun Smart Data Technology Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jiangsu Zhongyun Smart Data Technology Co ltd filed Critical Jiangsu Zhongyun Smart Data Technology Co ltd
Priority to CN201911236736.8A priority Critical patent/CN110910428B/en
Publication of CN110910428A publication Critical patent/CN110910428A/en
Application granted granted Critical
Publication of CN110910428B publication Critical patent/CN110910428B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Computational Linguistics (AREA)
  • Multimedia (AREA)
  • Evolutionary Computation (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Biophysics (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a real-time multi-target tracking method based on a neural network, which comprises the following steps: s1: establishing a pre-training target detection model; s2: initializing a global tracker; s3: performing target detection on the video stream to be tracked by using the pre-training target detection model; s4: acquiring the actual position coordinates and the predicted position coordinates of each tracked target in a target set with an established tracking state in the global tracker; s5: respectively carrying out cross calculation on the position coordinate of each target to be matched, the actual position coordinate and the predicted position coordinate of each tracked target; s6: analyzing the maximum matching coefficient in S5 and updating the robust state of each tracked target to obtain tracking information; s7: s3 through S6 are repeated. The method can effectively improve the detection efficiency of the tracked object under special conditions such as shielding, deformation, environmental change, rapid movement and the like.

Description

Real-time multi-target tracking method based on neural network
Technical Field
The invention relates to the field of deep learning, in particular to a real-time multi-target tracking method based on a neural network.
Background
For video target tracking, some excellent traditional methods such as KCF, TLD, CSRT and other tracking algorithms based on optical flow method, Mean-Shift and template matching and other basic algorithms emerge for many years. However, the traditional method needs manual work to participate in tracking target calibration, and target tracking of a specific category cannot be performed fully automatically. When the conditions of shielding, deformation, environmental change, rapid movement and the like are met, the tracking target is easily lost by the traditional method.
In order to solve these disadvantages of the conventional method, some neural network-based tracking algorithms have been proposed in recent years and achieve better tracking effect than the conventional algorithms, but they still have certain disadvantages. For example, chinese patent application No. 201611271054.7 discloses a target tracking algorithm based on a neural network, which uses a neural network self-encoder to train and extract target features for object detection in each frame of image, and has a drawback in that when a plurality of similar targets exist in a scene, upper and lower frame target matching cannot be performed well, and thus a tracking target is lost. Meanwhile, under the condition that the time of the target existing in the video sequence is short, enough training samples cannot be obtained to train the self-encoder to learn the features on line. In order to solve the above problems, chinese patent application No. 201811532811.0 proposes a tracking method similar to the SORT algorithm, which matches and tracks targets according to detected IOU values between targets, but this method too depends on the IOU values of the target detected by the current frame and the target to be tracked, and when the detection interval is too long, detection is missed or the object moves too fast each time, the target cannot be matched.
Disclosure of Invention
In view of the above, in order to solve the defects of the prior art, the invention provides a real-time multi-target tracking algorithm based on a neural network, which can perform robust tracking on a target in a video stream in real time, and can effectively solve the problem that the prior method has poor performance in special situations such as occlusion, deformation, environmental change, rapid movement and the like, and has good target matching capability. The specific contents are as follows: a real-time multi-target tracking method based on a neural network comprises the following steps:
s1: establishing a pre-training target detection model, wherein the pre-training target detection model is used for detecting a tracking target in a video stream to be tracked;
s2: initializing a global tracker;
s3: performing target detection on the video stream to be tracked by using the pre-training target detection model to obtain a target set to be matched and a corresponding position coordinate set;
s4: acquiring the actual position coordinates and the predicted position coordinates of each tracked target in a target set with an established tracking state in the global tracker;
s5: respectively carrying out cross calculation on the position coordinate of each target to be matched in the target set to be matched in S3, the actual position coordinate and the predicted position coordinate of each tracked target in S4 to obtain a Jaccard coefficient, and comparing to obtain a maximum matching coefficient of each target to be matched and each tracked target;
s6: analyzing the maximum matching coefficient in S5, updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of a target to be matched which is not matched with the tracked target;
s7: repeating S3-S6 until the video stream ends.
Further, the step S1: establishing a pre-training target detection model, comprising: and pre-training the object type to be detected by using a deep learning neural network target detection model to obtain a pre-training target detection model.
Further, the detection time of the target detection model is less than the time of generating each frame of image by the video stream.
Further, the step S2: the global tracker includes: the actual tracking target set and the buffer tracking target set.
Further, when the global tracker is initialized, the actual tracking target set and the buffered tracking target set are both empty sets.
Further, the set of actual tracked objects and the set of buffered tracked objects will store tracking information for the tracked objects, the tracking information including: and the tracked target is one or more of the current position coordinate, the historical position coordinate, the number of frames which are accumulated and disappeared in the disappearance state and the number of frames which are continuously in the continuous tracking state in the image.
Further, the obtaining of the predicted position coordinate in S4 includes: and calculating the actual position coordinates by using an elastic offset updating method to obtain the corresponding predicted position coordinates in the current frame image.
Further, the step S6: analyzing S5 the maximum matching coefficient, updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, including:
a: when the maximum matching coefficient is smaller than a threshold value n, updating the tracked target to be in a disappearing state, and then judging whether the number of frames accumulated by the tracked target to disappear is larger than or equal to a threshold value L:
a1: if the number of the frames is larger than or equal to a threshold value L, deleting the tracked target in the global tracker;
a2: and if the number of frames is less than the threshold L, adding 1 to the number of frames counted by the number of lost tracked targets.
Further, the step S6: analyzing S5 the maximum matching coefficient and updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, further comprising:
b: when the maximum matching coefficient is larger than or equal to a threshold value n, judging whether the tracked target is located in an actual tracking target set or a buffer tracking target set in a global tracker:
b1: if the tracked target is located in the actual tracked target set, updating the current position coordinate and the historical position coordinate of the tracked target;
b2: if the tracked target is located in the buffered tracking target set, judging whether the frame number of the tracked target in the continuous tracking state currently is greater than or equal to a threshold K frame:
b21: if the frame number is larger than or equal to the threshold K frame, updating the current position coordinates and the historical position coordinates of the tracked target, and moving the tracked target from the buffer tracking target set to the actual tracking target set;
b22: and if the current position coordinate and the historical position coordinate of the tracked target are less than the threshold K frames, updating the current position coordinate and the historical position coordinate of the tracked target, and adding 1 to the count of the number of frames which are continuously in the continuous tracking state at present.
Further, the step S6: analyzing S5 the maximum matching coefficient and updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, further comprising:
c: if the target to be matched is not matched with any tracked target in the global tracker, establishing a new tracked target by using the current position coordinate information of the target to be matched in the buffer tracked target set in the global tracker.
The invention has the beneficial effects that:
the detection efficiency of the tracked object under special conditions such as shielding, deformation, environmental change, rapid movement and the like can be effectively improved, meanwhile, after a soft-shift elastic offset updating algorithm and a tracked object robust state updating algorithm are added, the phenomena of false detection and missing detection are restrained, and good target matching and tracking capability is obtained.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and for those skilled in the art, other drawings can be obtained according to these drawings without creative efforts.
Fig. 1 shows a flowchart of a multi-target tracking method according to an embodiment of the present invention.
Detailed Description
Exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be embodied in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.
As shown in fig. 1, this embodiment shows an implementation of tracking a pedestrian target based on the present algorithm.
S1: establishing a pre-training target detection model, wherein the pre-training target detection model is used for detecting a tracking target in a video stream to be tracked.
The video stream is composed of images in a time sequence, and a video of one second can be decomposed into a plurality of images, wherein each image after decomposition is called a frame image. In this embodiment, the recording standard of the video stream on the hardware platform is to generate 30 frames of images in one second, so that the time consumed for generating each frame of image can be calculated to be about 1000ms/30 ≈ 33ms, and when the time consumed for detecting each frame of image by the pre-training target detection model is less than the calculated 33ms, the image generated by each frame can be processed in time, so as to achieve a real-time effect.
Firstly, collecting a pedestrian labeling data set which comprises 10000 picture training sets and 1000 picture testing sets, and labeling a human body on each picture. And then, a deep learning neural network target detection model is set up to train on a training set of the data set, and data augmentation methods such as mirroring, scaling, rotation, contrast enhancement and the like are used in the training process to increase the generalization of the pre-training target detection model to pedestrian detection.
After a lightweight target detection model H-SSD (Half-Shot multi box Detector) designed according to this embodiment is trained, 97.6% of mapp (mean Average Precision value, i.e., Average value of APs of each category) is achieved on a test set after an SSD detection algorithm framework is used as a basis and an improved target detection model of depth separable convolution and Half-Precision quantization technology is combined, the Average detection time on each frame image of the model after deployment on a Jetson TX2 hardware platform of NVIDIA corporation is about 22ms, which is less than 33ms, and thus, the real-time requirement can be satisfied. After obtaining the H-SSD pre-trained pedestrian detection model, the pedestrians in the video stream are tracked by the following steps.
S2: the global tracker is initialized.
Before each frame of image is input into a video stream, a global pedestrian tracker is initialized, the global pedestrian tracker comprises an actual pedestrian tracking target set and a buffering pedestrian tracking target set, and both sets are empty sets during initialization.
S3: and performing target detection on the current frame image in the video stream to be tracked by using the pre-training target detection model to obtain a target set to be matched and a corresponding position coordinate set detected in the current frame image.
Carrying out pedestrian detection on the current frame image in the video stream to be tracked by using an H-SSD pre-trained pedestrian detection model to obtain all detected pedestrian target sets { x } in the current frame imageiSet of position coordinates { Loc (x) }i) Where the symbols represent sets, { x { } symbolsiI.e. the set of all detected pedestrian objects, where x isiRepresents { xiThe ith detected pedestrian target in the set. Similarly, { Loc (x)i) Position coordinate set for storing all detected pedestrian targets, Loc (x)i) Is xiThe position coordinates of (a). { xiThe set needs to be matched with the pedestrian target set with the tracking state established in the global tracker in the step S2 in the following step, so the set is called { x }iAnd the set is a set of pedestrian targets to be matched.
S4: and acquiring the actual position coordinates of each tracked target in the target set with the established tracking state in the global tracker in the previous frame image, and calculating the actual position coordinates by using a soft-shift elastic offset updating algorithm to obtain the corresponding predicted position coordinates in the current frame image.
Acquiring a pedestrian target set { y) with tracking states established in the global trackeriEach tracked pedestrian target y in the fouriActual position coordinates Loc (y) in the previous frame imagei) Stored in { Loc (y) }i) And the tracked pedestrian target actual position coordinate set. Then pair { Loc (y)i) Each actual position coordinate Loc (y) in the seti) Calculating and obtaining corresponding predicted position coordinates Loc (Sy) in the current frame image by using soft-shift elastic offset updating algorithmi) Stored in { Loc (Sy) }i) In the tracked pedestrian target predicted position coordinate set, the calculation formula of the soft-shift elastic offset updating algorithm is as follows: loc (Sy)i)= Loc(yi)+ Loc(Δyi)，Loc(Δyi) Is yiEstimated position deviationAmount of displacement, by yiThe historical position coordinate information of the mobile terminal is obtained through calculation.
S5: and respectively performing cross calculation on the position coordinates of each target to be matched in the target set to be matched in the current frame image in the S3, the actual position coordinates of each tracked target in the target set with the established tracking state in the global tracker in the last frame image and the corresponding predicted position coordinates in the current frame image in the S4 to obtain the maximum matching coefficient of each target to be matched and each tracked target.
Will { Loc (x)i) Set of coordinates of pedestrian target positions to be matched with { Loc (y) }i) Set of tracked pedestrian target actual position coordinates and { Loc (Sy)i) Respectively carrying out sub-element cross calculation on Jaccard coefficients of the coordinate set of the predicted position of the tracked pedestrian target, and obtaining x of each pedestrian target to be matched through calculation and comparisoniWith each tracked pedestrian target yiMaximum matching coefficient of (Jmax) (x)i，yi) Maximum matching coefficient Jmax (x)i，yi) Is Jmax (x)i，yi)=Max(J(Loc(xi), Loc(Syi))，J(Loc(xi), Loc(yi))). Here, J () is a Jaccard coefficient used to quantify the degree of overlap of the positions on the image between two objects, and is calculated by the formula: j (a, B) = | a ≦ B |/| a ≦ B |, | a ≦ B | represents the location intersection region of the a target and the B target, and | a ≦ B | represents the location union region of the a target and the B target. The interval of the calculation result of the Jaccard coefficient is 0,1]The larger the calculation result, the more overlapping the positions of the two objects on the image is represented.
S6: and analyzing S5 the maximum matching coefficient, updating the robust state of each tracked target in the global tracker to obtain the tracking information of the tracked target in the current frame image, and establishing the tracking state of the target to be matched which is not matched with the tracked target.
Analysis of the maximum matching coefficient J calculated in S5max(xi，yi) There are several cases:
a: when J ismax(xi，yi) Small valueWhen the threshold value n =0.1, updating the tracked pedestrian target yiIs in a vanishing state, then y is judgediWhether or not the number of frames that have disappeared in totalization is equal to or greater than a threshold L =2, a 1: if the tracked pedestrian target y is larger than or equal to the threshold L, deleting the tracked pedestrian target y in the global pedestrian trackeri(ii) a A2: if less than threshold L, then the tracked pedestrian target yiThe count of accumulated number of frames that have disappeared is incremented by 1.
B: when J ismax(xi，yi) When the value is equal to or greater than the threshold value n =0.1, the tracked pedestrian target y is determinediWith corresponding pedestrian objects x to be matchediOn match, updating the tracked pedestrian target yiTo continue the tracking state, the tracked pedestrian target y is then judgediWhether an actual set of pedestrian tracking targets or a buffered set of pedestrian tracking targets located in the global pedestrian tracker:
b1: if y isiIf the target is located in the actual pedestrian tracking target set, y is updatediCurrent location coordinates and historical location coordinates.
B2: if y isiIf the pedestrian tracking target is located in the buffering pedestrian tracking target set, y is judgediWhether the number of frames currently continuously in the continuous tracking state is greater than or equal to a threshold value K =4 frames. B21: if the value is larger than or equal to the threshold value K frame, y is updatediCurrent position coordinates and historical position coordinates, and y isiMoving from the buffered pedestrian tracking target set to the actual pedestrian tracking target set; b22: if less than the threshold K frames, update yiAnd 1 is added to the count of the number of frames currently continuously in the continuous tracking state.
C: if there is a pedestrian target x to be matchediHas not yet tracked pedestrian target y with any of the global pedestrian trackersiIn matching, the target x to be matched is used in the buffering pedestrian tracking target setiAnd establishing and adding a new tracking target according to the current position coordinate information.
After the analysis is completed through S6, the tracked pedestrian target in the global pedestrian tracker will obtain its new position coordinates in the current frame image, and the newly detected pedestrian target will establish the tracking state.
S7: s3 through S6 are repeated until the video stream ends.
And collecting a pedestrian tracking test set, wherein the test set consists of 1000 test videos. The performance of the experiment on the test set and other methods of this example is compared as shown in the following table:
Figure DEST_PATH_IMAGE002
wherein, the MOTA (Multi-Object Tracking Accuracy) represents the Accuracy after the missing detection rate and the false alarm rate are combined, the MOTP (Multi-Object Tracking Precision) represents the average frame overlapping rate of all the tracked targets, and the two evaluation indexes are both higher and better.
The experimental result shows that compared with the traditional method, the method has huge improvement on the tracking effect, and compared with the target tracking algorithm SORT based on deep learning, due to the introduction of the elastic optimization strategy (soft-shift elastic offset updating algorithm and tracker robust state updating algorithm), the method has obvious improvement on two evaluation indexes.
It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, elements recited by the phrase "comprising a" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element.
Finally, it is to be noted that: the above description is only a preferred embodiment of the present invention, and is only used to illustrate the technical solutions of the present invention, and not to limit the protection scope of the present invention. Any modification, equivalent replacement, or improvement made within the spirit and principle of the present invention shall fall within the protection scope of the present invention.

Claims (10)

1. A real-time multi-target tracking method based on a neural network is characterized by comprising the following steps:
s1: establishing a pre-training target detection model, wherein the pre-training target detection model is used for detecting a tracking target in a video stream to be tracked;
s2: initializing a global tracker;
s3: performing target detection on the video stream to be tracked by using the pre-training target detection model to obtain a target set to be matched and a corresponding position coordinate set;
s4: acquiring the actual position coordinates and the predicted position coordinates of each tracked target in a target set with an established tracking state in the global tracker;
s5: respectively carrying out cross calculation on the position coordinate of each target to be matched in the target set to be matched in S3, the actual position coordinate and the predicted position coordinate of each tracked target in S4 to obtain a Jaccard coefficient, and comparing to obtain a maximum matching coefficient of each target to be matched and each tracked target;
s6: analyzing the maximum matching coefficient in S5, updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of a target to be matched which is not matched with the tracked target;
s7: repeating S3-S6 until the video stream ends.
2. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein said S1: establishing a pre-training target detection model, comprising: and pre-training the object type to be detected by using a deep learning neural network target detection model to obtain a pre-training target detection model.
3. The real-time multi-target tracking method based on the neural network as claimed in claim 2, wherein the time consumption for detecting the target detection model is less than the time consumption for generating each frame of image by the video stream.
4. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein said S2: the global tracker includes: the actual tracking target set and the buffer tracking target set.
5. The real-time multi-target tracking method based on the neural network as claimed in claim 4, wherein the actual tracking target set and the buffered tracking target set are both empty sets when the global tracker is initialized.
6. The real-time multi-target tracking method based on neural network as claimed in claim 5, wherein the actual tracking target set and the buffered tracking target set will store tracking information of the tracked targets, the tracking information comprising: and the tracked target is one or more of the current position coordinate, the historical position coordinate, the number of frames which are accumulated and disappeared in the disappearance state and the number of frames which are continuously in the continuous tracking state in the image.
7. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein the obtaining of the predicted position coordinates in S4 includes: and calculating the actual position coordinates by using an elastic offset updating method to obtain the corresponding predicted position coordinates in the current frame image.
8. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein said S6: analyzing S5 the maximum matching coefficient, updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, including:
a: when the maximum matching coefficient is smaller than a threshold value n, updating the tracked target to be in a disappearing state, and then judging whether the number of frames accumulated by the tracked target to disappear is larger than or equal to a threshold value L:
a1: if the number of the frames is larger than or equal to a threshold value L, deleting the tracked target in the global tracker;
a2: and if the number of frames is less than the threshold L, adding 1 to the number of frames counted by the number of lost tracked targets.
9. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein said S6: analyzing S5 the maximum matching coefficient and updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, further comprising:
b: when the maximum matching coefficient is larger than or equal to a threshold value n, judging whether the tracked target is located in an actual tracking target set or a buffer tracking target set in a global tracker:
b1: if the tracked target is located in the actual tracked target set, updating the current position coordinate and the historical position coordinate of the tracked target;
b2: if the tracked target is located in the buffered tracking target set, judging whether the frame number of the tracked target in the continuous tracking state currently is greater than or equal to a threshold K frame:
b21: if the frame number is larger than or equal to the threshold K frame, updating the current position coordinates and the historical position coordinates of the tracked target, and moving the tracked target from the buffer tracking target set to the actual tracking target set;
b22: and if the current position coordinate and the historical position coordinate of the tracked target are less than the threshold K frames, updating the current position coordinate and the historical position coordinate of the tracked target, and adding 1 to the count of the number of frames which are continuously in the continuous tracking state at present.
10. The real-time multi-target tracking method based on neural network as claimed in claim 1, wherein said S6: analyzing S5 the maximum matching coefficient and updating the robust state of each tracked target in the global tracker to obtain tracking information, and establishing a tracking state of the target to be matched which is not matched with the tracked target, further comprising:
c: and if the target to be matched is not matched with any tracked target in the global tracker, establishing a new tracked target by using the current position coordinate information of the target to be matched in a buffer tracked target set in the global tracker.
CN201911236736.8A 2019-12-05 2019-12-05 Real-time multi-target tracking method based on neural network Active CN110910428B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201911236736.8A CN110910428B (en) 2019-12-05 2019-12-05 Real-time multi-target tracking method based on neural network

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201911236736.8A CN110910428B (en) 2019-12-05 2019-12-05 Real-time multi-target tracking method based on neural network

Publications (2)

Publication Number Publication Date
CN110910428A CN110910428A (en) 2020-03-24
CN110910428B true CN110910428B (en) 2022-04-01

Family

ID=69823202

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201911236736.8A Active CN110910428B (en) 2019-12-05 2019-12-05 Real-time multi-target tracking method based on neural network

Country Status (1)

Country Link
CN (1) CN110910428B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111445501B (en) * 2020-03-25 2023-03-24 苏州科达科技股份有限公司 Multi-target tracking method, device and storage medium
CN114067564B (en) * 2021-11-15 2023-08-29 武汉理工大学 Traffic condition comprehensive monitoring method based on YOLO

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109446942A (en) * 2018-10-12 2019-03-08 北京旷视科技有限公司 Method for tracking target, device and system
CN109784173A (en) * 2018-12-14 2019-05-21 合肥阿巴赛信息科技有限公司 A kind of shop guest's on-line tracking of single camera
CN110751674A (en) * 2018-07-24 2020-02-04 北京深鉴智能科技有限公司 Multi-target tracking method and corresponding video analysis system
CN111598066A (en) * 2020-07-24 2020-08-28 之江实验室 Helmet wearing identification method based on cascade prediction

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11216954B2 (en) * 2018-04-18 2022-01-04 Tg-17, Inc. Systems and methods for real-time adjustment of neural networks for autonomous tracking and localization of moving subject

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110751674A (en) * 2018-07-24 2020-02-04 北京深鉴智能科技有限公司 Multi-target tracking method and corresponding video analysis system
CN109446942A (en) * 2018-10-12 2019-03-08 北京旷视科技有限公司 Method for tracking target, device and system
CN109784173A (en) * 2018-12-14 2019-05-21 合肥阿巴赛信息科技有限公司 A kind of shop guest's on-line tracking of single camera
CN111598066A (en) * 2020-07-24 2020-08-28 之江实验室 Helmet wearing identification method based on cascade prediction

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Learning-Based Hand Motion Capture and Understanding in Assembly Process;Liang Liu et al;《IEEE TRANSACTIONS ON INDUSTRIAL ELECTRONICS》;20181207;全文 *
基于Kalman预测器改进的CamShift目标跟踪;龙涛;《中国优秀硕士学位论文全文数据库信息科技辑》;20170315;全文 *

Also Published As

Publication number Publication date
CN110910428A (en) 2020-03-24

Similar Documents

Publication Publication Date Title
WO2020173226A1 (en) Spatial-temporal behavior detection method
WO2022111506A1 (en) Video action recognition method and apparatus, electronic device and storage medium
CN109344725B (en) Multi-pedestrian online tracking method based on space-time attention mechanism
CN110378288B (en) Deep learning-based multi-stage space-time moving target detection method
US7982774B2 (en) Image processing apparatus and image processing method
CN109146912B (en) Visual target tracking method based on target analysis
CN110120064B (en) Depth-related target tracking algorithm based on mutual reinforcement and multi-attention mechanism learning
CN109544592B (en) Moving object detection algorithm for camera movement
JP7272024B2 (en) Object tracking device, monitoring system and object tracking method
CN109902601B (en) Video target detection method combining convolutional network and recursive network
CN110910428B (en) Real-time multi-target tracking method based on neural network
CN110555868A (en) method for detecting small moving target under complex ground background
CN105374049B (en) Multi-corner point tracking method and device based on sparse optical flow method
CN110853074A (en) Video target detection network system for enhancing target by utilizing optical flow
CN113608663B (en) Fingertip tracking method based on deep learning and K-curvature method
CN112541424A (en) Real-time detection method for pedestrian falling under complex environment
CN111368770B (en) Gesture recognition method based on skeleton point detection and tracking
CN111931571B (en) Video character target tracking method based on online enhanced detection and electronic equipment
CN111461011A (en) Weak and small target detection method based on probabilistic pipeline filtering
CN111191535A (en) Pedestrian detection model construction method based on deep learning and pedestrian detection method
CN112233145A (en) Multi-target shielding tracking method based on RGB-D space-time context model
CN115049954A (en) Target identification method, device, electronic equipment and medium
CN113327269A (en) Unmarked cervical vertebra movement detection method
CN111161323B (en) Complex scene target tracking method and system based on correlation filtering
CN117315547A (en) Visual SLAM method for solving large duty ratio of dynamic object

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant