WO2021018141A1 - 用于多目标的行人跟踪方法、装置及设备 - Google Patents

用于多目标的行人跟踪方法、装置及设备 Download PDF

Info

Publication number
WO2021018141A1
WO2021018141A1 PCT/CN2020/105196 CN2020105196W WO2021018141A1 WO 2021018141 A1 WO2021018141 A1 WO 2021018141A1 CN 2020105196 W CN2020105196 W CN 2020105196W WO 2021018141 A1 WO2021018141 A1 WO 2021018141A1
Authority
WO
WIPO (PCT)
Prior art keywords
tracking
frame
pedestrian detection
detection frame
candidate
Prior art date
Application number
PCT/CN2020/105196
Other languages
English (en)
French (fr)
Inventor
杨静林
Original Assignee
京东方科技集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 京东方科技集团股份有限公司 filed Critical 京东方科技集团股份有限公司
Priority to US17/280,821 priority Critical patent/US11830273B2/en
Publication of WO2021018141A1 publication Critical patent/WO2021018141A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/761—Proximity, similarity or dissimilarity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/41—Higher-level, semantic clustering, classification or understanding of video scenes, e.g. detection, labelling or Markovian modelling of sport events or news items
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/103—Static body considered as a whole, e.g. static pedestrian or occupant recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/07—Target detection

Definitions

  • the present disclosure relates to the field of image processing technology, in particular to a pedestrian tracking method, device and equipment.
  • Target tracking such as pedestrian tracking
  • AI artificial intelligence
  • video surveillance human-computer interaction
  • robotics robotics
  • military guidance military guidance
  • the multi-target tracking algorithm in the related technology is mainly based on the tracking-by-detection mode, which achieves the effect of multi-target tracking by associating pedestrians detected in adjacent video frames.
  • the problems of pedestrian tracking loss and mismatch are prone to exist.
  • the embodiments of the present disclosure provide a pedestrian tracking method for multiple targets, including:
  • the updating of the tracking counter is stopped, and the temporary tracking indicator is converted into a confirming tracking indicator.
  • the value of the tracking counter is increased by 1; wherein, the tracking counter is established after the candidate pedestrian detection frame is detected for the first time, and the tracking The initial value of the counter is 0.
  • the value of the tracking counter is increased by 1, and when the candidate pedestrian detection frame does not match the existing tracking frame, the value of the tracking counter Subtract 1; wherein, the tracking counter is established after detecting the candidate pedestrian detection frame for the first time, and the initial value of the tracking counter is an integer greater than zero.
  • the candidate pedestrian detection frame is deleted, and the preset second threshold is less than the preset first threshold.
  • the judging whether the candidate pedestrian detection frame matches an existing tracking frame includes:
  • the calculating the characteristic distance between the candidate pedestrian detection frame and the tracking frame in the previous N frames of the current frame includes:
  • n is an integer greater than or equal to 1 and less than or equal to N;
  • the method further includes:
  • the candidate is not saved The feature of the pedestrian detection frame; when all the values of the partial intersection ratio are less than or equal to the preset fourth threshold, the feature of the candidate pedestrian detection frame is saved as the feature of the tracking frame in the current frame image.
  • the partial intersection union ratio is Wherein, A is the candidate pedestrian detection frame and B is any other candidate pedestrian detection frame.
  • the embodiment of the present disclosure also provides a pedestrian tracking device for multiple targets, including:
  • the detection module is used to detect multiple candidate pedestrian detection frames in the image to be detected in the current frame, wherein a temporary tracking mark and a tracking counter are set for each candidate pedestrian detection frame;
  • the judgment module is used to judge whether each candidate pedestrian detection frame matches an existing tracking frame, update the value of the tracking counter according to the judgment result, and continue to detect the next frame to be detected;
  • the processing module is configured to stop updating the tracking counter when the value of the tracking counter reaches a preset first threshold, and convert the temporary tracking indicator into a confirmed tracking indicator.
  • the judgment module is configured to add 1 to the value of the tracking counter when the candidate pedestrian detection frame matches an existing tracking frame; wherein, the tracking counter is the first detection of the candidate pedestrian After the detection frame is established, the initial value of the tracking counter is 0.
  • the judgment module is configured to increase the value of the tracking counter by 1 when the candidate pedestrian detection frame matches an existing tracking frame, and when the candidate pedestrian detection frame does not match an existing tracking frame , The value of the tracking counter is reduced by 1; wherein the tracking counter is established after the candidate pedestrian detection frame is detected for the first time, and the initial value of the tracking counter is an integer greater than zero.
  • the pedestrian tracking device further includes:
  • the deleting module is configured to delete the candidate pedestrian detection frame when the value of the tracking counter is less than a preset second threshold, and the preset second threshold is less than the preset first threshold.
  • the judgment module includes:
  • the calculation unit is used to calculate the feature distance between the candidate pedestrian detection frame and the tracking frame in the first N frames of the current frame, and when the feature distance is less than a preset third threshold, determine whether the candidate pedestrian detection frame is There is tracking frame matching.
  • a preset third threshold determines whether the candidate pedestrian detection frame is There is tracking frame matching.
  • the calculation unit is specifically configured to calculate the feature of the candidate pedestrian detection frame; calculate the distance dist between the feature of the candidate pedestrian detection frame and the feature of the tracking frame in the previous nth frame of the current frame (n), n is an integer greater than or equal to 1 and less than or equal to N; the characteristic distance D mean is calculated by the following formula:
  • the device further includes:
  • the calculation module is used to calculate the partial intersection and union ratio of each candidate pedestrian detection frame in the image to be detected in the current frame with other candidate pedestrian detection frames.
  • the value of any partial intersection and union ratio is greater than the preset fourth threshold, The feature of the candidate pedestrian detection frame is not saved; when all the values of the partial intersection ratio are less than or equal to the preset fourth threshold, the feature of the candidate pedestrian detection frame is saved as the feature of the tracking frame in the current frame image.
  • the partial intersection union ratio is Wherein, A is the candidate pedestrian detection frame and B is any other candidate pedestrian detection frame.
  • the embodiment of the present disclosure also provides a pedestrian tracking device for multiple targets, including: a memory, a processor, and a computer program stored in the memory and running on the processor, the computer program being used by the processor.
  • a pedestrian tracking device for multiple targets, including: a memory, a processor, and a computer program stored in the memory and running on the processor, the computer program being used by the processor.
  • the embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, the above-mentioned multi-target pedestrian tracking is implemented Steps in the method.
  • FIG. 1 is a schematic flowchart of a pedestrian tracking method according to an embodiment of the disclosure
  • Figure 2 is a structural block diagram of an embodiment of the pedestrian tracking device of the present disclosure
  • FIG. 3 is a schematic flowchart of a pedestrian tracking method according to a specific embodiment of the present disclosure
  • Figure 4-6 is a schematic diagram of calculating the intersection ratio.
  • the multi-pedestrian tracking in the video has the characteristics of multiple occlusion and crossing of pedestrians, and the related technology uses the tracking-by-detection framework for tracking.
  • the first step is to perform pedestrian target detection on the video frame.
  • the detected targets are distinguished by IOU (Intersection-over-Union) and pedestrian re-identification features, and then the Hungarian algorithm is used to compare the detection results and The tracking results are matched and Kalman filtering is used for tracking, but this algorithm has the following shortcomings in practical applications:
  • a target pedestrian detection frame will often contain some other target pedestrians, so the re-identification features of the two target pedestrian detection frames will be relatively close, which will bring about feature matching. Difficulty and error.
  • embodiments of the present disclosure provide a pedestrian tracking method, device, and equipment, which can improve the accuracy of pedestrian tracking.
  • the embodiment of the present disclosure provides a pedestrian tracking method, as shown in FIG. 1, including:
  • the updating of the tracking counter is stopped, and the temporary tracking indicator is converted into a confirming tracking indicator.
  • the value of the tracking counter is increased by 1; wherein, the tracking counter is established after the candidate pedestrian detection frame is detected for the first time, and the tracking The initial value of the counter is 0.
  • the value of the tracking counter is increased by 1, and when the candidate pedestrian detection frame does not match the existing tracking frame, the tracking counter Minus 1; wherein the tracking counter is established after the candidate pedestrian detection frame is detected for the first time, and the initial value of the tracking counter is an integer greater than zero.
  • the candidate pedestrian detection frame matches the existing tracking frame.
  • the temporary tracking identifier of the candidate pedestrian detection frame is updated, and the value of the tracking counter is updated according to the judgment result.
  • this temporary tracking mark is converted into a confirmed tracking mark, and the candidate pedestrian detection frame is determined as the target pedestrian detection frame, which can improve the robustness of tracking mark switching and avoid
  • a new tracking mark is created for the pedestrian detection frame, which can improve the accuracy of pedestrian tracking.
  • the value of the first threshold can be set according to actual conditions.
  • the difference between the first threshold and the initial value of the tracking counter should be greater than 1, so as to prevent a pedestrian detection frame from being mistakenly detected in a certain frame of image, and then determining the pedestrian detection frame as the target pedestrian detection frame, and The pedestrian detection frame creates a new tracking logo.
  • the method further includes:
  • the candidate pedestrian detection frame is deleted, and the preset second threshold is less than the preset first threshold.
  • the value of the tracking counter is judged, and the candidate pedestrian detection frame is deleted only after the value of the tracking counter is less than the preset second threshold.
  • the value of the second threshold it can be avoided in a certain frame of image When the candidate pedestrian detection frame is missed, the candidate pedestrian detection frame is deleted.
  • the judging whether the candidate pedestrian detection frame matches an existing tracking frame includes:
  • feature accumulation is performed on the first N frames of the current frame, and the first N frames of the current frame are used to calculate the feature distance to reduce feature matching. This can improve the accuracy of pedestrian tracking.
  • the calculating the characteristic distance between the candidate pedestrian detection frame and the tracking frame in the previous N frames of the current frame includes:
  • n is an integer greater than or equal to 1 and less than or equal to N;
  • the method further includes:
  • the candidate is not saved The feature of the pedestrian detection frame; when all the values of the partial intersection ratio are less than or equal to the preset fourth threshold, the feature of the candidate pedestrian detection frame is saved as the feature of the tracking frame in the current frame image.
  • a target pedestrian detection frame will often contain some other target pedestrians, so the re-identification features of the two target pedestrian detection frames will be relatively close, which will bring difficulty and error to feature matching and affect pedestrian tracking
  • This embodiment calculates the partial intersection ratio between each candidate pedestrian detection frame and other candidate pedestrian detection frames. When the partial intersection ratio is larger, the feature of the candidate pedestrian detection frame is not saved, so the feature is being performed When matching, the features of unsaved candidate pedestrian detection frames will not be used for feature matching, which can reduce the difficulty and error of feature matching, thereby helping to improve the accuracy of pedestrian tracking.
  • the partial intersection ratio is Wherein, A is the candidate pedestrian detection frame, and B is any other candidate pedestrian detection frame.
  • A is the candidate pedestrian detection frame
  • B is any other candidate pedestrian detection frame.
  • the partial IOU of A is larger, and vice versa, it is smaller.
  • the partial IOU between the candidate pedestrian detection frame A and the candidate pedestrian detection frame B is large, it means that the candidate pedestrian detection frame B contains most of the area of the candidate pedestrian detection frame A, and the reid feature of the candidate pedestrian detection frame A does not have a significant distinction. The effect, therefore, is not used to calculate the feature distance.
  • the embodiment of the present disclosure also provides a pedestrian tracking device, as shown in FIG. 2, including:
  • the detection module 21 is configured to detect a plurality of candidate pedestrian detection frames in the image to be detected in the current frame, wherein a temporary tracking mark and a tracking counter are set for each candidate pedestrian detection frame;
  • the judgment module 22 is configured to judge whether each candidate pedestrian detection frame matches an existing tracking frame, update the value of the tracking counter according to the judgment result, and continue to detect the next frame to be detected;
  • the processing module 23 is configured to stop updating the tracking counter when the value of the tracking counter reaches a preset first threshold, and convert the temporary tracking indicator into a confirmation tracking indicator.
  • the judging module 22 may be configured to add 1 to the value of the tracking counter when the candidate pedestrian detection frame matches an existing tracking frame; wherein, the tracking counter is the first detection of the candidate pedestrian After the pedestrian detection frame is established, the initial value of the tracking counter is 0.
  • the judging module 22 may be configured to add 1 to the value of the tracking counter when the candidate pedestrian detection frame matches an existing tracking frame, and when the candidate pedestrian detection frame does not match the existing tracking frame.
  • the value of the tracking counter is decreased by 1; wherein the tracking counter is established after the candidate pedestrian detection frame is detected for the first time, and the initial value of the tracking counter is an integer greater than zero.
  • the candidate pedestrian detection frame after detecting the candidate pedestrian detection frame in the image to be detected in the current frame, it is judged whether the candidate pedestrian detection frame matches the existing tracking frame.
  • the value of the tracking counter is increased by 1; when there is no match , The value of the tracking counter is subtracted by 1, and when the value of the tracking counter is greater than the preset first threshold, the candidate pedestrian detection frame is determined as the target pedestrian detection frame.
  • the value of the first threshold can be set according to actual conditions.
  • the difference between the first threshold and the initial value of the tracking counter should be greater than 1, so as to prevent a pedestrian detection frame from being mistakenly detected in a certain frame of image, and then determining the pedestrian detection frame as the target pedestrian detection frame, and The pedestrian detection frame creates a new tracking logo.
  • the device further includes:
  • the deleting module is configured to delete the candidate pedestrian detection frame when the value of the tracking counter is less than a preset second threshold, and the preset second threshold is less than the preset first threshold.
  • the value of the tracking counter is judged, and the candidate pedestrian detection frame is deleted only after the value of the tracking counter is less than the preset second threshold.
  • the value of the second threshold it can be avoided in a certain frame of image When the candidate pedestrian detection frame is missed, the candidate pedestrian detection frame is deleted.
  • the judgment module includes:
  • the calculation unit is used to calculate the feature distance between the candidate pedestrian detection frame and the tracking frame in the first N frames of the current frame, and when the feature distance is less than a preset third threshold, determine whether the candidate pedestrian detection frame is There is tracking frame matching.
  • a preset third threshold determines whether the candidate pedestrian detection frame is There is tracking frame matching.
  • feature accumulation is performed on the first N frames of the current frame, and the first N frames of the current frame are used to calculate the feature distance to reduce feature matching. This can improve the accuracy of pedestrian tracking.
  • the calculation unit is specifically configured to calculate the feature of the candidate pedestrian detection frame; calculate the difference between the feature of the candidate pedestrian detection frame and the feature of the tracking frame in the previous nth frame of the current frame
  • the distance dist(n), n is an integer greater than or equal to 1 and less than or equal to N
  • the characteristic distance D mean is calculated by the following formula:
  • the device further includes:
  • the calculation module is used to calculate the partial intersection and union ratio of each candidate pedestrian detection frame in the image to be detected in the current frame with other candidate pedestrian detection frames.
  • the value of any partial intersection and union ratio is greater than the preset fourth threshold, The feature of the candidate pedestrian detection frame is not saved; when all the values of the partial intersection ratio are less than or equal to the preset fourth threshold, the feature of the candidate pedestrian detection frame is saved as the feature of the tracking frame in the current frame image.
  • a target pedestrian detection frame will often contain some other target pedestrians, so the re-identification features of the two target pedestrian detection frames will be relatively close, which will bring difficulty and error to feature matching and affect pedestrian tracking
  • This embodiment calculates the partial intersection ratio between each candidate pedestrian detection frame and other candidate pedestrian detection frames. When the partial intersection ratio is larger, the feature of the candidate pedestrian detection frame is not saved, so the feature is being performed When matching, the features of unsaved candidate pedestrian detection frames will not be used for feature matching, which can reduce the difficulty and error of feature matching, thereby helping to improve the accuracy of pedestrian tracking.
  • the partial intersection ratio is Wherein, A is the candidate pedestrian detection frame, and B is any other candidate pedestrian detection frame.
  • A is the candidate pedestrian detection frame
  • B is any other candidate pedestrian detection frame.
  • the partial IOU of A is larger, and vice versa, it is smaller.
  • the partial IOU between the candidate pedestrian detection frame A and the candidate pedestrian detection frame B is large, it means that the candidate pedestrian detection frame B contains most of the area of the candidate pedestrian detection frame A, and the reid feature of the candidate pedestrian detection frame A does not have a significant distinction. The effect, therefore, is not used to calculate the feature distance.
  • the pedestrian tracking method includes the following steps:
  • Step 301 Input the current frame to be detected image
  • the video to be detected includes multiple frames of images to be detected, and a pedestrian tracking operation needs to be performed separately for each frame of the image to be detected.
  • Step 302 Detect a candidate pedestrian detection frame in the image to be detected in the current frame, and the candidate pedestrian detection frame corresponds to a counter, and then go to step 303 and step 307;
  • a temporary tracking mark is established, and two tracking counters a and b are set.
  • the initial value of counter a can be set to 0, and the value of counter a +1, if the value of counter a
  • this temporary tracking mark is converted into a confirmed tracking mark, the candidate pedestrian detection frame is determined as the target pedestrian detection frame, and the tracking is continued; the initial value of the counter b can be set to a constant greater than zero.
  • the temporary tracking flag of the candidate pedestrian detection frame is updated, and the value of counter b is +1. If the candidate pedestrian detection frame does not match the existing tracking frame , Set the value of counter b -1.
  • the value of counter b When the value of counter b reaches the counting threshold thr2 or the temporary tracking indicator is converted into a confirmed tracking indicator, the value of counter b stops updating, and when the value of counter b decreases to 0, the candidate pedestrian detection frame is deleted.
  • the counter a and the counter b can use the same counter, and at this time, thr1 is equal to thr2, and both are equal to the above-mentioned first threshold.
  • Step 303 Calculate the characteristic distance between the candidate pedestrian detection frame and the tracking frame in the previous N frames of the current frame;
  • a 128-dimensional reid feature vector is used to match the candidate pedestrian detection frame and the tracking frame through the Deepsort algorithm.
  • the deepsort algorithm weights and accumulates the distance between the current frame and the previous N frames of images, and performs matching between features.
  • the characteristic distance can be a cosine distance.
  • other characteristic distance measurement methods can also be used.
  • Step 304 Determine whether the feature distance is less than the preset third threshold, if the feature distance is less than the preset third threshold, judge that the candidate pedestrian detection frame matches the existing tracking frame, if the feature distance is greater than or equal to the preset third threshold, judge all The candidate pedestrian detection frame does not match the existing tracking frame, and the value of the counter is updated according to the matching result;
  • Step 305 Determine whether the value of the counter is greater than the preset first threshold, if yes, go to step 306, if not, obtain the next frame to be detected, and go to step 301;
  • Step 306 Determine the candidate pedestrian detection frame as the target pedestrian detection frame, and create a new tracking mark for the target pedestrian detection frame;
  • Step 307 For each candidate pedestrian detection frame in the to-be-detected image in the current frame, calculate the partial intersection and ratio with other candidate pedestrian detection frames;
  • IOU can assist pedestrian tracking.
  • the traditional IOU has certain drawbacks.
  • the candidate pedestrian detection frame A and the candidate pedestrian detection frame B have overlapping areas.
  • the union of the candidate pedestrian detection frame A and the candidate pedestrian detection frame B is the part filled with horizontal lines in Figure 5.
  • the intersection of the detection frame A and the candidate pedestrian detection frame B is the part filled with vertical lines in FIG. 6.
  • the traditional IOU is to calculate the ratio of the intersection of A and B (A ⁇ B) and the union of A and B (A ⁇ B).
  • the candidate pedestrian detection frame A may contain part of the candidate pedestrian detection frame B, especially when the candidate pedestrian detection frame B contains most of the candidate pedestrian detection frame A and the candidate pedestrian detection frame A.
  • the reid feature of the pedestrian detection frame B may appear the feature distance is relatively close, making the feature matching fail and tracking error.
  • this embodiment proposes a method for calculating the partial IOU. For each candidate pedestrian detection frame in the current frame, the partial IOU with other candidate pedestrian detection frames is calculated separately, namely
  • the partial IOU (partial_iou) of the candidate pedestrian detection frame A is larger, and vice versa, it is smaller.
  • the partial IOU between the candidate pedestrian detection frame A and the candidate pedestrian detection frame B is large, it means that the candidate pedestrian detection frame B contains most of the area of the candidate pedestrian detection frame A, and the reid feature of the candidate pedestrian detection frame A does not have a significant distinction. Therefore, the feature of the candidate pedestrian detection frame A is not used for the calculation of the feature distance. Otherwise, the feature of the candidate pedestrian detection frame A is included in the statistics and applied to the calculation of the feature distance of the previous N frames.
  • Step 308 Determine whether the value of the partial intersection ratio is greater than the preset fourth threshold, and when the value of any partial intersection ratio is greater than the preset fourth threshold, the feature of the candidate pedestrian detection frame is not saved; When the values of are all less than or equal to the preset fourth threshold, the feature of the candidate pedestrian detection frame is saved as the feature of the tracking frame in the current frame image and used in the calculation of the feature distance in step 303.
  • the candidate pedestrian detection frame matches the existing tracking frame.
  • the temporary tracking identifier of the candidate pedestrian detection frame is updated, and the value of the tracking counter is updated according to the judgment result.
  • this temporary tracking mark is converted into a confirmed tracking mark, and the candidate pedestrian detection frame is determined as the target pedestrian detection frame, which can improve the robustness of tracking mark switching and avoid
  • a new tracking mark is created for the pedestrian detection frame, which can improve the accuracy of pedestrian tracking.
  • the embodiments of the present disclosure also provide a pedestrian tracking device, including: a memory, a processor, and a computer program stored in the memory and capable of running on the processor.
  • a computer program stored in the memory and capable of running on the processor.
  • the embodiments of the present disclosure also provide a computer-readable storage medium having a computer program stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps in the pedestrian tracking method described above are implemented.
  • the embodiments described herein can be implemented by hardware, software, firmware, middleware, microcode, or a combination thereof.
  • the processing unit can be implemented in one or more Application Specific Integrated Circuits (ASIC), Digital Signal Processing (DSP), Digital Signal Processing Equipment (DSP Device, DSPD), programmable Logic Device (Programmable Logic Device, PLD), Field-Programmable Gate Array (Field-Programmable Gate Array, FPGA), general-purpose processors, controllers, microcontrollers, microprocessors, and others for performing the functions described in this application Electronic unit or its combination.
  • ASIC Application Specific Integrated Circuits
  • DSP Digital Signal Processing
  • DSP Device Digital Signal Processing Equipment
  • PLD programmable Logic Device
  • Field-Programmable Gate Array Field-Programmable Gate Array
  • FPGA Field-Programmable Gate Array
  • the technology described herein can be implemented through modules (such as procedures, functions, etc.) that perform the functions described herein.
  • the software codes can be stored in the memory and executed by the processor.
  • the memory can be implemented in the processor or external to the processor.
  • the embodiments of the embodiments of the present disclosure may be provided as methods, devices, or computer program products. Therefore, the embodiments of the present disclosure may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the embodiments of the present disclosure may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.
  • computer-usable storage media including but not limited to disk storage, CD-ROM, optical storage, etc.
  • These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing user terminal equipment to produce a machine, which can be executed by the processor of the computer or other programmable data processing user terminal equipment
  • the instructions generate means for implementing the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing user terminal equipment to work in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including the instruction device,
  • the instruction device implements the functions specified in one process or multiple processes in the flowchart and/or one block or multiple blocks in the block diagram.
  • These computer program instructions can also be loaded on a computer or other programmable data processing user terminal equipment, so that a series of operation steps are executed on the computer or other programmable user terminal equipment to produce computer-implemented processing, so that the computer or other programmable data processing
  • the instructions executed on the user terminal device provide steps for implementing functions specified in one or more processes in the flowchart and/or one block or more in the block diagram.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Data Mining & Analysis (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Human Computer Interaction (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Evolutionary Biology (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

本公开提供了一种用于多目标的行人跟踪方法、装置及设备，属于图像处理技术领域。该行人跟踪方法包括：检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像，其中，当所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。

Description

用于多目标的行人跟踪方法、装置及设备
相关申请的交叉引用
本申请主张在2019年7月31日在中国提交的中国专利申请号No.201910703259.5的优先权，其全部内容通过引用包含于此。
技术领域
本公开涉及图像处理技术领域，特别是指一种行人跟踪方法、装置及设备。
背景技术
目标跟踪，如行人跟踪是计算机视觉领域的一个重要方面，在人工智能(Artificial Intelligence，AI)、视频监控、人机交互、机器人、军事制导等领域都有广泛的应用前景。
相关技术中的多目标跟踪算法，主要是基于tracking-by-detection的模式，通过对相邻视频帧中检测出来的行人进行关联，达到多目标跟踪的效果。但是，由于行人之间的遮挡和交叉较多，容易存在行人跟踪丢失和误匹配的问题。
发明内容
本公开的实施例提供了一种用于多目标的行人跟踪方法，包括：
检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像，
其中，当所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
可选地，当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立， 所述跟踪计数器的初始值为0。
可选地，当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，所述跟踪计数器的值减1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
可选地，当所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
可选地，所述判断所述候选行人检测框是否与已有跟踪框匹配包括：
计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
可选地，所述计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离包括：
计算所述候选行人检测框的特征；
计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；
通过以下公式计算所述特征距离D mean：
Figure PCTCN2020105196-appb-000001
可选地，所述检测当前帧待检测图像中的候选行人检测框之后，所述方法还包括：
对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
可选地，所述偏交并比为
Figure PCTCN2020105196-appb-000002
其中，A为所述候选行人检测框，B为任一其他候选行人检测框。
本公开的实施例还提供了一种用于多目标的行人跟踪装置，包括：
检测模块，用于检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
判断模块，用于判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像；
处理模块，用于在所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
可选地，所述判断模块构造为，当所述候选行人检测框与已有跟踪框匹配时，将所述跟踪计数器的值加1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为0。
可选地，所述判断模块构造为，当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，所述跟踪计数器的值减1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
可选地，所述行人跟踪装置还包括：
删除模块，用于在所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
可选地，所述判断模块包括：
计算单元，用于计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
可选地，所述计算单元具体用于计算所述候选行人检测框的特征；计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；通过以下公式计算所述特征距离D mean：
Figure PCTCN2020105196-appb-000003
可选地，所述装置还包括：
计算模块，用于对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈 值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
可选地，所述偏交并比为
Figure PCTCN2020105196-appb-000004
其中，A为所述候选行人检测框，B为任一其他候选行人检测框。
本公开的实施例还提供了一种用于多目标的行人跟踪设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如上所述的行人跟踪方法中的步骤。
本公开的实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的用于多目标的行人跟踪方法中的步骤。
附图说明
为了更清楚地说明本公开实施例的技术方案，下面将对本公开实施例的描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术人员来讲，在不付出创造性劳动性的前提下，还可以根据这些附图获得其他的附图。
图1为本公开实施例行人跟踪方法的流程示意图；
图2为本公开实施例行人跟踪装置的结构框图；
图3为本公开具体实施例行人跟踪方法的流程示意图；
图4-图6为计算交并比的示意图。
具体实施方式
为使本公开的实施例要解决的技术问题、技术方案和优点更加清楚，下面将结合附图及具体实施例进行详细描述。
下面将参照附图更详细地描述本公开的示例性实施例。虽然附图中显示了本公开的示例性实施例，然而应当理解，可以以各种形式实现本公开而不应被这里阐述的实施例所限制。相反，提供这些实施例是为了能够更透彻地理解本公开，并且能够将本公开的范围完整的传达给本领域的技术人员。
视频中的多行人跟踪具有行人多遮挡和交叉的特点，相关技术采用tracking-by-detection的框架进行跟踪。其中，首先要对视频帧进行行人的目标检测，对检测到的目标通过IOU(Intersection-over-Union，交并比)和行人重识别特征进行不同目标的区分，再通过匈牙利算法对检测结果和跟踪结果进行匹配，利用卡尔曼滤波进行跟踪，但该算法在实际应用中，存在如下不足：
(1)跟踪标识切换的鲁棒性不足：当某一帧图像中误检出一个行人检测框时，就会为该行人检测框创建一个新的跟踪标识，同样地，当某一帧图像中漏检一个行人检测框时，就会删除该行人检测框的跟踪标识。
(2)在多个行人的场景中，一个目标行人检测框中往往会包含部分其他目标行人，这样两个目标行人检测框的重识别(re-identification)特征会相对接近，对特征匹配带来难度和误差。
(3)另外，在进行目标行人检测框与跟踪框的匹配时，仅计算特征间的余弦距离，选取距离最小的跟踪框与当前的目标行人检测框进行匹配，利用的信息少，特征匹配有可能存在误差。
为了解决上述问题，本公开实施例提供一种行人跟踪方法、装置及设备，能够提高行人跟踪的准确率。
本公开的实施例提供了一种行人跟踪方法，如图1所示，包括：
检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像，
其中，当所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
可选地，当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为0。
或者，可选地，当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，所述跟踪 计数器的值减1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
本实施例中，在检测出当前帧待检测图像中的候选行人检测框后，判断候选行人检测框是否与已有跟踪框匹配，在每一帧待检测图像中，当候选行人检测框与已有跟踪框匹配时，更新候选行人检测框的临时跟踪标识，并根据判断结果更新跟踪计数器的值。在跟踪计数器的值大于预设第一阈值时，才将此临时跟踪标识转化为确认的跟踪标识，将候选行人检测框确定为目标行人检测框，这样可以提高跟踪标识切换的鲁棒性，避免在某一帧图像中误检出一个行人检测框后就为该行人检测框创建一个新的跟踪标识，能够提高行人跟踪的准确率。
其中，第一阈值的取值可以根据实际情况进行设置。第一阈值与跟踪计数器的初始值之间的差值应大于1，这样可以避免在某一帧图像中误检出一个行人检测框后就将该行人检测框确定为目标行人检测框，并为该行人检测框创建一个新的跟踪标识。
一具体实施例中，所述方法还包括：
在所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
本实施例中，对跟踪计数器的值进行判断，在跟踪计数器的值小于预设第二阈值后，才删除候选行人检测框，通过对第二阈值的值进行设置，可以避免在某一帧图像中漏检候选行人检测框时，就删除候选行人检测框。
一具体实施例中，所述判断所述候选行人检测框是否与已有跟踪框匹配包括：
计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
本实施例中，为了能更好地判断候选行人检测框与已有跟踪框是否匹配，对当前帧的前N帧图像进行特征累计，利用当前帧的前N帧图像计算特征距离，降低特征匹配的误差，从而可以提高行人跟踪的准确率。
一具体示例中，所述计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离包括：
计算所述候选行人检测框的特征；
计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；
通过以下公式计算所述特征距离D mean：
Figure PCTCN2020105196-appb-000005
一具体实施例中，所述检测当前帧待检测图像中的候选行人检测框之后，所述方法还包括：
对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
在多个行人的场景中，一个目标行人检测框中往往会包含部分其他目标行人，这样两个目标行人检测框的重识别特征会相对接近，对特征匹配带来难度和误差，会影响行人跟踪的准确率，本实施例计算每个候选行人检测框与其他候选行人检测框的偏交并比，在偏交并比的值较大时，不保存候选行人检测框的特征，这样在进行特征匹配时，就不会利用未保存的候选行人检测框的特征进行特征匹配，从而能够降低特征匹配的难度和误差，进而有助于提高行人跟踪的准确率。
一具体示例中，所述偏交并比为
Figure PCTCN2020105196-appb-000006
其中，A为所述候选行人检测框，B为任一其他候选行人检测框。这样在候选行人检测框A被候选行人检测框B包含的时候，A的偏IOU较大，反之，则较小。在候选行人检测框A与候选行人检测框B的偏IOU较大时，说明候选行人检测框B包含了候选行人检测框A的大部分区域，候选行人检测框A的reid特征不具有显著的区分效果，因此，不用于计算特征距离。
本公开的实施例还提供了一种行人跟踪装置，如图2所示，包括：
检测模块21，用于检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
判断模块22，用于判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像；
处理模块23，用于在所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
可选地，该判断模块22可以构造为，当所述候选行人检测框与已有跟踪框匹配时，将所述跟踪计数器的值加1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为0。
或者，可选地，该判断模块22可以构造为，当所述候选行人检测框与已有跟踪框匹配时，将所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，所述跟踪计数器的值减1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
本实施例中，在检测出当前帧待检测图像中的候选行人检测框后，判断候选行人检测框是否与已有跟踪框匹配，在匹配时，将跟踪计数器的值加1；在不匹配时，将跟踪计数器的值减1，在跟踪计数器的值大于预设第一阈值时，才将候选行人检测框确定为目标行人检测框，这样可以提高跟踪标识切换的鲁棒性，避免在某一帧图像中误检出一个行人检测框后就为该行人检测框创建一个新的跟踪标识，能够提高行人跟踪的准确率。
其中，第一阈值的取值可以根据实际情况进行设置。第一阈值与跟踪计数器的初始值之间的差值应大于1，这样可以避免在某一帧图像中误检出一个行人检测框后就将该行人检测框确定为目标行人检测框，并为该行人检测框创建一个新的跟踪标识。
一具体实施例中，所述装置还包括：
删除模块，用于在所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
本实施例中，对跟踪计数器的值进行判断，在跟踪计数器的值小于预设第二阈值后，才删除候选行人检测框，通过对第二阈值的值进行设置，可以避免在某一帧图像中漏检候选行人检测框时，就删除候选行人检测框。
一具体实施例中，所述判断模块包括：
计算单元，用于计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
本实施例中，为了能更好地判断候选行人检测框与已有跟踪框是否匹配，对当前帧的前N帧图像进行特征累计，利用当前帧的前N帧图像计算特征距离，降低特征匹配的误差，从而可以提高行人跟踪的准确率。
一具体实施例中，所述计算单元具体用于计算所述候选行人检测框的特征；计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；通过以下公式计算所述特征距离D mean：
Figure PCTCN2020105196-appb-000007
一具体实施例中，所述装置还包括：
计算模块，用于对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
在多个行人的场景中，一个目标行人检测框中往往会包含部分其他目标行人，这样两个目标行人检测框的重识别特征会相对接近，对特征匹配带来难度和误差，会影响行人跟踪的准确率，本实施例计算每个候选行人检测框与其他候选行人检测框的偏交并比，在偏交并比的值较大时，不保存候选行人检测框的特征，这样在进行特征匹配时，就不会利用未保存的候选行人检测框的特征进行特征匹配，从而能够降低特征匹配的难度和误差，进而有助于提高行人跟踪的准确率。
一具体示例中，所述偏交并比为
Figure PCTCN2020105196-appb-000008
其中，A为所述候选行人检测框，B为任一其他候选行人检测框。这样在候选行人检测框A被候选行人检测框B包含的时候，A的偏IOU较大，反之，则较小。在候选行人检测框A与候选行人检测框B的偏IOU较大时，说明候选行人检测框B包 含了候选行人检测框A的大部分区域，候选行人检测框A的reid特征不具有显著的区分效果，因此，不用于计算特征距离。
一具体实施例中，如图3所示，行人跟踪方法包括以下步骤：
步骤301：输入当前帧待检测图像；
其中，待检测视频包括有多帧待检测图像，对每一帧待检测图像都需要分别执行行人跟踪操作。
步骤302：检测当前帧待检测图像中的候选行人检测框，该候选行人检测框对应有计数器，转向步骤303和步骤307；
对每一个新的候选行人检测框，建立一个临时的跟踪标识，并设置两个跟踪计数器a和b，其中计数器a初始值可以设置为0，且将计数器a的值+1，如果计数器a的值达到计数阈值thr1，此临时跟踪标识转化为确认的跟踪标识，将候选行人检测框确定为目标行人检测框，并继续跟踪；计数器b的初始值可以设置为一个大于零的常数，在每一帧待检测图像中，当候选行人检测框与已有跟踪框匹配时，更新候选行人检测框的临时跟踪标识，且将计数器b的值+1，如果候选行人检测框与已有跟踪框不匹配，将计数器b的值-1。当计数器b的值达到计数阈值thr2或者临时跟踪标识转化为确认的跟踪标识时，计数器b的值停止更新，当计数器b的值减至0时，删除该候选行人检测框。其中，计数器a和计数器b可以采用同一个计数器，此时thr1等于thr2，均等于上述第一阈值。
步骤303：计算该候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离；
本实施例中，通过Deepsort算法采用128维的reid特征向量进行候选行人检测框与跟踪框的匹配。为了能更好的进行匹配，deepsort算法对当前帧与前N帧图像的距离进行加权累计，进行特征间的匹配。
首先可以计算候选行人检测框与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数，n的取值从1到N，可以得到N个距离计算结果。其中，特征距离可以采用余弦距离，除了余弦距离，也可以采用其他的特征距离度量方法。
以每一帧的特征距离作为权值，进行累加，得到特征距离的累加值：
Figure PCTCN2020105196-appb-000009
对此加权累加值取平均值，作为最后的特征距离：
Figure PCTCN2020105196-appb-000010
步骤304：判断特征距离是否小于预设第三阈值，如果特征距离小于预设第三阈值，判断候选行人检测框与已有跟踪框匹配，如果特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，根据匹配结果更新计数器的值；
步骤305：判断计数器的值是否大于预设第一阈值，如果是，转向步骤306，如果否，获取下一帧待检测图像，转向步骤301；
步骤306：将候选行人检测框确定为目标行人检测框，并为该目标行人检测框创建一个新的跟踪标识；
之后，获取下一帧待检测图像，转向步骤301。
步骤307：对当前帧待检测图像中的每一候选行人检测框，计算与其他候选行人检测框的偏交并比；
IOU作为reid的补充，可以对行人跟踪起到辅助作用。但是，在多行人的场景下，传统的IOU具有一定的缺陷。如图4-图6所示，候选行人检测框A和候选行人检测框B存在重合区域，候选行人检测框A和候选行人检测框B的并集为图5中填充横线的部分，候选行人检测框A和候选行人检测框B的交集为图6中填充竖线的部分。传统的IOU是计算A与B的交集(A∩B)与A与B的并集(A∪B)的比值。但是，在多行人的情况下，候选行人检测框A中，有可能包含了候选行人检测框B的部分区域，尤其在包含了候选行人检测框B的大部分时，候选行人检测框A与候选行人检测框B的reid特征可能会出现特征距离较为接近的情况，使得特征匹配失败，进而跟踪错误。针对这一问题，本实施例提出偏IOU的计算方法，对当前帧的每个候选行人检测框，分别计算与其他候选行人检测框的偏IOU，即
Figure PCTCN2020105196-appb-000011
在候选行人检测框A被候选行人检测框B包含的时候，候选行人检测框 A的偏IOU(partial_iou)较大，反之，则较小。在候选行人检测框A与候选行人检测框B的偏IOU较大时，说明候选行人检测框B包含了候选行人检测框A的大部分区域，候选行人检测框A的reid特征不具有显著的区分效果，因此，不将候选行人检测框A的特征用于特征距离的计算，否则，将候选行人检测框A的特征计入统计，应用于前述的前N帧特征距离的计算中。
步骤308：判断偏交并比的值是否大于预设第四阈值，在任一偏交并比的值大于预设第四阈值时，不保存该候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存该候选行人检测框的特征作为当前帧图像中的跟踪框的特征，用于步骤303的特征距离的计算中。
本实施例中，在检测出当前帧待检测图像中的候选行人检测框后，判断候选行人检测框是否与已有跟踪框匹配，在每一帧待检测图像中，当候选行人检测框与已有跟踪框匹配时，更新候选行人检测框的临时跟踪标识，并根据判断结果更新跟踪计数器的值。在跟踪计数器的值大于预设第一阈值时，才将此临时跟踪标识转化为确认的跟踪标识，将候选行人检测框确定为目标行人检测框，这样可以提高跟踪标识切换的鲁棒性，避免在某一帧图像中误检出一个行人检测框后就为该行人检测框创建一个新的跟踪标识，能够提高行人跟踪的准确率。
本公开的实施例还提供了一种行人跟踪设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如上所述的行人跟踪方法中的步骤。
本公开的实施例还提供了一种计算机可读存储介质，所述计算机可读存储介质上存储有计算机程序，所述计算机程序被处理器执行时实现如上所述的行人跟踪方法中的步骤。
可以理解的是，本文描述的这些实施例可以用硬件、软件、固件、中间件、微码或其组合来实现。对于硬件实现，处理单元可以实现在一个或多个专用集成电路(Application Specific Integrated Circuits，ASIC)、数字信号处理器(Digital Signal Processing，DSP)、数字信号处理设备(DSP Device，DSPD)、可编程逻辑设备(Programmable Logic Device，PLD)、现场可编程门阵列(Field-Programmable Gate Array，FPGA)、通用处理器、控制器、微控制器、 微处理器、用于执行本申请所述功能的其它电子单元或其组合中。
对于软件实现，可通过执行本文所述功能的模块(例如过程、函数等)来实现本文所述的技术。软件代码可存储在存储器中并通过处理器执行。存储器可以在处理器中或在处理器外部实现。
本说明书中的各个实施例均采用递进的方式描述，每个实施例重点说明的都是与其他实施例的不同之处，各个实施例之间相同相似的部分互相参见即可。
本领域内的技术人员应明白，本公开实施例的实施例可提供为方法、装置、或计算机程序产品。因此，本公开实施例可采用完全硬件实施例、完全软件实施例、或结合软件和硬件方面的实施例的形式。而且，本公开实施例可采用在一个或多个其中包含有计算机可用程序代码的计算机可用存储介质(包括但不限于磁盘存储器、CD-ROM、光学存储器等)上实施的计算机程序产品的形式。
本公开实施例是参照根据本公开实施例的方法、用户终端设备(系统)、和计算机程序产品的流程图和/或方框图来描述的。应理解可由计算机程序指令实现流程图和/或方框图中的每一流程和/或方框、以及流程图和/或方框图中的流程和/或方框的结合。可提供这些计算机程序指令到通用计算机、专用计算机、嵌入式处理机或其他可编程数据处理用户终端设备的处理器以产生一个机器，使得通过计算机或其他可编程数据处理用户终端设备的处理器执行的指令产生用于实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能的装置。
这些计算机程序指令也可存储在能引导计算机或其他可编程数据处理用户终端设备以特定方式工作的计算机可读存储器中，使得存储在该计算机可读存储器中的指令产生包括指令装置的制造品，该指令装置实现在流程图一个流程或多个流程和/或方框图一个方框或多个方框中指定的功能。
这些计算机程序指令也可装载到计算机或其他可编程数据处理用户终端设备上，使得在计算机或其他可编程用户终端设备上执行一系列操作步骤以产生计算机实现的处理，从而在计算机或其他可编程用户终端设备上执行的指令提供用于实现在流程图一个流程或多个流程和/或方框图一个方框或多 个方框中指定的功能的步骤。
尽管已描述了本公开实施例的可选实施例，但本领域内的技术人员一旦得知了基本创造性概念，则可对这些实施例做出另外的变更和修改。所以，所附权利要求意欲解释为包括可选实施例以及落入本公开实施例范围的所有变更和修改。
还需要说明的是，在本文中，诸如第一和第二等之类的关系术语仅仅用来将一个实体或者操作与另一个实体或操作区分开来，而不一定要求或者暗示这些实体或操作之间存在任何这种实际的关系或者顺序。而且，术语“包括”、“包含”或者其任何其他变体意在涵盖非排他性的包含，从而使得包括一系列要素的过程、方法、物品或者用户终端设备不仅包括那些要素，而且还包括没有明确列出的其他要素，或者是还包括为这种过程、方法、物品或者用户终端设备所固有的要素。在没有更多限制的情况下，由语句“包括一个……”限定的要素，并不排除在包括所述要素的过程、方法、物品或者用户终端设备中还存在另外的相同要素。
以上所述的是本公开的可选实施方式，应当指出对于本技术领域的普通人员来说，在不脱离本公开所述的原理前提下还可以作出若干改进和润饰，这些改进和润饰也在本公开的保护范围内。

Claims (18)

  1. 一种用于多目标的行人跟踪方法，包括：
    检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
    判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像，
    其中，当所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
  2. 根据权利要求1所述的行人跟踪方法，其中所述根据判断结果更新所述跟踪计数器的值，包括：
    当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1；其中，
    所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为0。
  3. 根据权利要求1所述的行人跟踪方法，其中所述根据判断结果更新所述跟踪计数器的值，包括：
    当所述候选行人检测框与已有跟踪框匹配时，所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，所述跟踪计数器的值减1；其中，
    所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
  4. 根据权利要求3所述的行人跟踪方法，其中当所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
  5. 根据权利要求1所述的行人跟踪方法，其中，所述判断所述候选行人检测框是否与已有跟踪框匹配包括：
    计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪 框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
  6. 根据权利要求5所述的行人跟踪方法，其中，所述计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离包括：
    计算所述候选行人检测框的特征；
    计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；
    通过以下公式计算所述特征距离D mean：
    Figure PCTCN2020105196-appb-100001
  7. 根据权利要求6所述的行人跟踪方法，其中，所述检测当前帧待检测图像中的候选行人检测框之后，所述方法还包括：
    对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
  8. 根据权利要求7所述的行人跟踪方法，其中，
    所述偏交并比为
    Figure PCTCN2020105196-appb-100002
    其中，A为所述候选行人检测框，B为任一其他候选行人检测框。
  9. 一种用于多目标的行人跟踪装置，包括：
    检测模块，用于检测当前帧待检测图像中的多个候选行人检测框，其中针对每个所述候选行人检测框设置有临时跟踪标识和跟踪计数器；
    判断模块，用于判断每个所述候选行人检测框是否与已有跟踪框匹配，根据判断结果更新所述跟踪计数器的值，并继续检测下一帧待检测图像；
    处理模块，用于在所述跟踪计数器的值达到预设第一阈值时，停止更新所述跟踪计数器，并将所述临时跟踪标识转变为确认跟踪标识。
  10. 根据权利要求9所述的行人跟踪装置，其中所述判断模块构造为，当所述候选行人检测框与已有跟踪框匹配时，将所述跟踪计数器的值加1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪 计数器的初始值为0。
  11. 根据权利要求9所述的行人跟踪装置，其中所述判断模块构造为，当所述候选行人检测框与已有跟踪框匹配时，将所述跟踪计数器的值加1，当所述候选行人检测框不与已有跟踪框匹配时，将所述跟踪计数器的值减1；其中，所述跟踪计数器为首次检测到所述候选行人检测框后建立，所述跟踪计数器的初始值为大于零的整数。
  12. 根据权利要求11所述的行人跟踪装置，还包括：
    删除模块，用于在所述跟踪计数器的值小于预设第二阈值时，删除所述候选行人检测框，所述预设第二阈值小于所述预设第一阈值。
  13. 根据权利要求9所述的行人跟踪装置，其中，所述判断模块包括：
    计算单元，用于计算所述候选行人检测框与当前帧的前N帧图像中的跟踪框的特征距离，在所述特征距离小于预设第三阈值时，判断所述候选行人检测框与已有跟踪框匹配，在所述特征距离大于等于预设第三阈值时，判断所述候选行人检测框与已有跟踪框不匹配，N为大于1的整数。
  14. 根据权利要求13所述的行人跟踪装置，其中，
    所述计算单元具体用于计算所述候选行人检测框的特征；计算所述候选行人检测框的特征与当前帧的前第n帧图像中的跟踪框的特征之间的距离dist(n)，n为大于等于1小于等于N的整数；通过以下公式计算所述特征距离D mean：
    Figure PCTCN2020105196-appb-100003
  15. 根据权利要求14所述的行人跟踪装置，还包括：
    计算模块，用于对当前帧待检测图像中的每个候选行人检测框，分别计算与其他候选行人检测框的偏交并比，在任一偏交并比的值大于预设第四阈值时，不保存所述候选行人检测框的特征；在所有偏交并比的值均小于等于预设第四阈值时，保存所述候选行人检测框的特征作为当前帧图像中的跟踪框的特征。
  16. 根据权利要求15所述的行人跟踪装置，其中，
    所述偏交并比为
    Figure PCTCN2020105196-appb-100004
    其中，A为所述候选行人检测框，B 为任一其他候选行人检测框。
  17. 一种用于多目标的行人跟踪设备，包括：存储器、处理器及存储在存储器上并可在处理器上运行的计算机程序，所述计算机程序被所述处理器执行时实现如权利要求1至8中任一项所述的用于多目标的行人跟踪方法中的步骤。
  18. 一种计算机可读存储介质，其上存储有计算机程序，所述计算机程序被处理器执行时实现如权利要求1至8中任一项所述的用于多目标的行人跟踪方法中的步骤。
PCT/CN2020/105196 2019-07-31 2020-07-28 用于多目标的行人跟踪方法、装置及设备 WO2021018141A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/280,821 US11830273B2 (en) 2019-07-31 2020-07-28 Multi-target pedestrian tracking method, multi-target pedestrian tracking apparatus and multi-target pedestrian tracking device

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910703259.5 2019-07-31
CN201910703259.5A CN110414447B (zh) 2019-07-31 2019-07-31 行人跟踪方法、装置及设备

Publications (1)

Publication Number Publication Date
WO2021018141A1 true WO2021018141A1 (zh) 2021-02-04

Family

ID=68364755

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/105196 WO2021018141A1 (zh) 2019-07-31 2020-07-28 用于多目标的行人跟踪方法、装置及设备

Country Status (3)

Country Link
US (1) US11830273B2 (zh)
CN (1) CN110414447B (zh)
WO (1) WO2021018141A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114757972A (zh) * 2022-04-15 2022-07-15 中国电信股份有限公司 目标跟踪方法、装置、电子设备及计算机可读存储介质

Families Citing this family (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110414447B (zh) 2019-07-31 2022-04-15 京东方科技集团股份有限公司 行人跟踪方法、装置及设备
CN113255411A (zh) * 2020-02-13 2021-08-13 北京百度网讯科技有限公司 目标检测方法、装置、电子设备及存储介质
CN111784224A (zh) * 2020-03-26 2020-10-16 北京京东乾石科技有限公司 物体跟踪方法和装置、控制平台和存储介质
CN112069879B (zh) * 2020-07-22 2024-06-07 深圳市优必选科技股份有限公司 一种目标人物跟随方法、计算机可读存储介质及机器人
CN112800841B (zh) * 2020-12-28 2024-05-17 深圳市捷顺科技实业股份有限公司 一种行人计数方法、装置、系统及计算机可读存储介质
CN112784725B (zh) * 2021-01-15 2024-06-07 北京航天自动控制研究所 行人防撞预警方法、设备、存储介质及堆高机
CN114782495B (zh) * 2022-06-16 2022-10-18 西安中科立德红外科技有限公司 一种多目标跟踪方法、系统及计算机存储介质
CN115546192B (zh) * 2022-11-03 2023-03-21 中国平安财产保险股份有限公司 牲畜数量识别方法、装置、设备及存储介质
CN116563769B (zh) * 2023-07-07 2023-10-20 南昌工程学院 一种视频目标识别追踪方法、系统、计算机及存储介质
CN117611636B (zh) * 2024-01-23 2024-04-26 中国水产科学研究院黄海水产研究所 一种鱼类跟踪方法和系统

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180046865A1 (en) * 2016-08-15 2018-02-15 Qualcomm Incorporated Multi-to-multi tracking in video analytics
CN108022258A (zh) * 2017-10-20 2018-05-11 南京邮电大学 基于单次多框检测器与卡尔曼滤波的实时多目标跟踪方法
CN110414447A (zh) * 2019-07-31 2019-11-05 京东方科技集团股份有限公司 行人跟踪方法、装置及设备

Family Cites Families (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP5279635B2 (ja) * 2008-08-20 2013-09-04 キヤノン株式会社 画像処理装置、画像処理方法、および、プログラム
US8237847B2 (en) * 2008-10-16 2012-08-07 Fujinon Corporation Auto focus system having AF frame auto-tracking function
US10963893B1 (en) * 2016-02-23 2021-03-30 Videomining Corporation Personalized decision tree based on in-store behavior analysis
CN105844669B (zh) * 2016-03-28 2018-11-13 华中科技大学 一种基于局部哈希特征的视频目标实时跟踪方法
CN109101859A (zh) * 2017-06-21 2018-12-28 北京大学深圳研究生院 使用高斯惩罚检测图像中行人的方法
CN108053427B (zh) * 2017-10-31 2021-12-14 深圳大学 一种基于KCF与Kalman的改进型多目标跟踪方法、系统及装置
US11055555B2 (en) * 2018-04-20 2021-07-06 Sri International Zero-shot object detection
CN108734107B (zh) * 2018-04-24 2021-11-05 武汉幻视智能科技有限公司 一种基于人脸的多目标跟踪方法及系统
CN109063593A (zh) * 2018-07-13 2018-12-21 北京智芯原动科技有限公司 一种人脸跟踪方法及装置
US10943204B2 (en) * 2019-01-16 2021-03-09 International Business Machines Corporation Realtime video monitoring applied to reduce customer wait times
AU2019200976A1 (en) * 2019-02-12 2020-08-27 Canon Kabushiki Kaisha Method, system and apparatus for generating training samples for matching objects in a sequence of images
AU2019100806A4 (en) * 2019-07-24 2019-08-29 Dynamic Crowd Measurement Pty Ltd Real-Time Crowd Measurement And Management Systems And Methods Thereof
CN111242977B (zh) * 2020-01-09 2023-04-25 影石创新科技股份有限公司 全景视频的目标跟踪方法、可读存储介质及计算机设备

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180046865A1 (en) * 2016-08-15 2018-02-15 Qualcomm Incorporated Multi-to-multi tracking in video analytics
CN108022258A (zh) * 2017-10-20 2018-05-11 南京邮电大学 基于单次多框检测器与卡尔曼滤波的实时多目标跟踪方法
CN110414447A (zh) * 2019-07-31 2019-11-05 京东方科技集团股份有限公司 行人跟踪方法、装置及设备

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
CDKNIGHT_HAPPY: "Study on DeepSort", 31 March 2018 (2018-03-31), pages 1 - 7, XP009525845, Retrieved from the Internet <URL:https://blog.csdn.net/cdknight_happy/article/details/79731981> *
NICOLAI WOJKE; ALEX BEWLEY; DIETRICH PAULUS: "Simple Online and Realtime Tracking with a Deep Association Metric", ARXIV.ORG, 21 March 2017 (2017-03-21), pages 1 - 5, XP080758706, DOI: 10.1109/ICIP.2017.8296962 *
PPRP: "Deep SORT multi-target tracking algorithm code analysis", 20 April 2020 (2020-04-20), pages 1 - 26, XP055777519, Retrieved from the Internet <URL:HTTPS://WWW.CNBLOGS.COM/PPRP/ARTICLES/12736831.HTML>> *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114757972A (zh) * 2022-04-15 2022-07-15 中国电信股份有限公司 目标跟踪方法、装置、电子设备及计算机可读存储介质
CN114757972B (zh) * 2022-04-15 2023-10-10 中国电信股份有限公司 目标跟踪方法、装置、电子设备及计算机可读存储介质

Also Published As

Publication number Publication date
US20220004747A1 (en) 2022-01-06
CN110414447B (zh) 2022-04-15
CN110414447A (zh) 2019-11-05
US11830273B2 (en) 2023-11-28

Similar Documents

Publication Publication Date Title
WO2021018141A1 (zh) 用于多目标的行人跟踪方法、装置及设备
CN109522854B (zh) 一种基于深度学习和多目标跟踪的行人流量统计方法
CN111709975B (zh) 多目标跟踪方法、装置、电子设备及存储介质
JP5385105B2 (ja) 画像検索方法およびシステム
WO2020042426A1 (zh) 机器人检测直边的方法和清洁机器人筛选参考墙边的方法
Jodoin et al. Tracking all road users at multimodal urban traffic intersections
JP2011008507A (ja) 画像検索方法およびシステム
CN111383246B (zh) 条幅检测方法、装置及设备
CN114049383B (zh) 一种多目标跟踪方法、设备及可读存储介质
CN105096299A (zh) 多边形检测方法和多边形检测装置
CN113762272B (zh) 道路信息的确定方法、装置和电子设备
WO2022028383A1 (zh) 车道线标注、检测模型确定、车道线检测方法及相关设备
WO2017199840A1 (ja) オブジェクト追跡装置、オブジェクト追跡方法および記録媒体
WO2021051887A1 (zh) 一种困难样本筛选方法及装置
Jung et al. Object Detection and Tracking‐Based Camera Calibration for Normalized Human Height Estimation
WO2020143499A1 (zh) 一种基于动态视觉传感器的角点检测方法
WO2022142416A1 (zh) 目标跟踪方法及相关设备
CN102855473B (zh) 一种基于相似性度量的图像多目标检测方法
CN106950527B (zh) 一种多基线干涉仪测向体制下的脉冲信号分选方法
CN102789645A (zh) 一种用于周界防范的多目标快速跟踪方法
CN110163029B (zh) 一种图像识别方法、电子设备以及计算机可读存储介质
WO2022252482A1 (zh) 机器人及其环境地图构建方法和装置
WO2023005020A1 (zh) 反光板定位方法、机器人及计算机可读存储介质
CN107067411B (zh) 一种结合密集特征的Mean-shift跟踪方法
CN111161225B (zh) 一种图像差异检测方法、装置、电子设备和存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20846623

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20846623

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 20846623

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13.02.2023)

122 Ep: pct application non-entry in european phase

Ref document number: 20846623

Country of ref document: EP

Kind code of ref document: A1