WO2021147826A1 - 一种面向边缘端行人跟踪和人数精确统计的方法 - Google Patents

一种面向边缘端行人跟踪和人数精确统计的方法 Download PDF

Info

Publication number
WO2021147826A1
WO2021147826A1 PCT/CN2021/072538 CN2021072538W WO2021147826A1 WO 2021147826 A1 WO2021147826 A1 WO 2021147826A1 CN 2021072538 W CN2021072538 W CN 2021072538W WO 2021147826 A1 WO2021147826 A1 WO 2021147826A1
Authority
WO
WIPO (PCT)
Prior art keywords
pedestrian
pedestrians
tracking
list
matching
Prior art date
Application number
PCT/CN2021/072538
Other languages
English (en)
French (fr)
Inventor
黄樟钦
盛梦雪
张硕
李洪亮
Original Assignee
北京工业大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京工业大学 filed Critical 北京工业大学
Publication of WO2021147826A1 publication Critical patent/WO2021147826A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/277—Analysis of motion involving stochastic approaches, e.g. using Kalman filters
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/60—Analysis of geometric attributes
    • G06T7/62—Analysis of geometric attributes of area, perimeter, diameter or volume
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30242—Counting objects in image

Definitions

  • the invention relates to the field of pedestrian tracking and people counting based on artificial intelligence components in edge computing scenarios.
  • Pedestrian tracking strategies include the use of special chip analysis of pedestrian color, posture and other characteristic information for similarity matching, or judging the overlap rate of the two frames before and after the detection frame. People counting is mainly triggered by crossing the detection line.
  • the purpose of this algorithm is to design a general pedestrian tracking and accurate counting method of people based on the pedestrian result information obtained after the intelligent analysis of real-time video and related information by artificial intelligence processing components, so that it can be applied to edge computing scenarios. , It can still accurately track pedestrians and count the number of people under resource constraints such as insufficient processing capabilities of artificial intelligence processing components, limited computing capabilities of edge-end processing platforms, and product cost pricing.
  • Step S1 Establish and maintain a pedestrian dynamic tracking list in real time, which is used to dynamically record pedestrian information in the tracking area.
  • Step S2 Obtain pedestrian result information of the artificial intelligence processing component at the edge.
  • Step S3 Extract important pedestrian result information to establish a pedestrian static detection list.
  • Step S4 Match the pedestrians in the pedestrian static detection list and the pedestrian dynamic tracking list.
  • Step S5 Update the pedestrian dynamic tracking list according to the matching result.
  • Step S6 Accurately count the number of pedestrians in the statistical area according to the pedestrian dynamic tracking list.
  • the information recorded in the pedestrian dynamic tracking list in step S1 should include the pedestrian's tracking id, direction, moving radius, and frame loss information.
  • the information recorded in the pedestrian static detection list in step S3 should include the pedestrian's tracking id, location coordinates, and time information.
  • the algorithm is also characterized by: resource limitations at the edge, including the limitation of the processing capabilities of artificial intelligence processing components, insufficient computing power of the edge processing platform, product cost limitations, etc., will cause frame loss in the intelligent analysis process of real-time video.
  • resource limitations at the edge including the limitation of the processing capabilities of artificial intelligence processing components, insufficient computing power of the edge processing platform, product cost limitations, etc., will cause frame loss in the intelligent analysis process of real-time video.
  • the problem leads to deviations in pedestrian result information. Therefore, the matching process described in step S4 first performs exact matching, and if the exact matching fails, intelligent fuzzy matching is performed on pedestrians whose exact matching fails.
  • Step S41 Exact matching.
  • the tracking id in the pedestrian static detection list and the dynamic tracking list it can be judged whether the pedestrians in the two lists are the same person. If the tracking id is the same, it is the same person, and the exact match succeeds, otherwise the exact match fails.
  • Step S42 Perform intelligent fuzzy matching on pedestrians whose exact matching fails.
  • MATCH_THRESHOID For pedestrians who have no frame loss problem and have a clear direction, predict the possible position range in the next frame in the direction of their motion, and select the pedestrian with the greatest similarity to the predicted result from the static detection list and exceeds the matching threshold MATCH_THRESHOID to match it. If MATCH_THRESHOID is a percentage, then 0 ⁇ MATCH_THRESHOID ⁇ 100%. The closer the MATCH_THRESHOID is to 100%, the more stringent the matching conditions are. Generally, set it to ⁇ 80%.
  • LOST_THRESHOID select the pedestrian with the greatest similarity to the predicted result and more than MATCH_THRESHOID in the static detection list to match.
  • step S5 the updated content of the pedestrian dynamic tracking list includes:
  • Pedestrians who are not matched successfully in the static detection list are recorded as newcomers, and the direction is judged and added to the dynamic tracking list.
  • the algorithm is also characterized in that the direction is used as an intelligent fuzzy matching index to better improve the accuracy of people counting.
  • a pedestrian in the static detection list is judged as a newcomer, its direction is judged according to its initial position coordinates. If the pedestrian's initial position coordinates are located in the upper part of the tracking area, it is generally considered to be the upper 1/3 part of the tracking area, and the direction is judged to be in. If the pedestrian's initial position coordinates are located in the lower part of the tracking area, it is generally considered to be the lower 1/3 part of the tracking area, and the judgment direction is out. If the pedestrian's initial position coordinates are located in other parts of the tracking area, it is considered that its direction cannot be judged. According to the processing method of step S42 for pedestrians with no frame loss but unclear direction, the direction can be confirmed with a delay to make a correct judgment on the direction. .
  • the process of accurately counting the number of people in step S6 includes:
  • Step S61 Establish and maintain a pedestrian dynamic statistics column, and the table is used for accurate statistics of the number of people.
  • Step S62 Accurately count the number of pedestrians in the statistical area, specifically as follows:
  • Step S63 When the pedestrian is normally counted and leaves the statistical area or is in the statistical area but the number of dropped frames is greater than LOST_THRESHOID, delete it from the pedestrian dynamic statistical list.
  • the algorithm performs intelligent delay counting processing for this type of pedestrians, specifically:
  • the pedestrian has not been counted normally but has walked out of the statistical area, it will not be deleted. It will not be deleted until the direction has been determined through the above-mentioned pedestrian tracking process and the pedestrian has been counted normally.
  • LOST_THRESHOID If the pedestrian has not been counted normally but the number of lost frames is greater than LOST_THRESHOID, delete it. Considering the actual situation of the set statistical area and LOST_THRESHOID, if it is considered that the pedestrian will not be detected again in the statistical area, the pedestrian will be counted into the total number, otherwise it will be ignored.
  • the algorithm only counts pedestrians when they enter the statistical area for the first time. Although pedestrians stay in the statistical area, they are still in the pedestrian dynamic statistical list, but they are not counted repeatedly.
  • the algorithm is also characterized by a method of delimiting the range of positions where pedestrians may appear in the next frame.
  • the delineation process is detailed as follows:
  • One of the roughest methods is to delineate the prediction range within a full circle with the pedestrian's center of mass as the center and the moving radius as the radius.
  • the method uses the direction as an important matching index, and the limitation of the direction helps us to reduce the prediction range from a full circle to a semicircle in the direction of its movement.
  • the semicircle is specifically: taking the pedestrian's center of mass as the end point, making a ray parallel to the moving direction, and a straight line passing through the end point and perpendicular to the ray can divide the whole circle into semicircles, and just take the semicircle containing the ray.
  • the time difference between the two video frames before and after the intelligent analysis must be within 1 second, and the pedestrian moving direction will not change greatly within 1 second. Specifically, the pedestrian will move to the left or right in the direction of its movement.
  • the angle will not be greater than 45°, so the prediction range is reduced to the angle between the radius of the semicircle and the ray ⁇ 45°. Within a sector of -45°.
  • a universal pedestrian tracking and accurate counting method of the number of people is designed. This method can be used in various detection models, such as face detection, pedestrian detection, head and shoulder detection, etc., accurate
  • the matching conditions can be changed according to the actual situation, and one-way or two-way people counting can be performed, which has good scalability and versatility.
  • Figure 2 is a schematic diagram of the rough forecast range
  • Figure 3 is a schematic diagram of the directional prediction range
  • Figure 4 is a schematic diagram of the angled prediction range
  • Figure 5 is a schematic diagram of the prediction range with matching distance coefficients
  • Figure 6 is a schematic diagram of the two-way matching prediction range
  • Figure 7 is a schematic diagram of the prediction range of reverse matching
  • the pedestrian result information provided by the artificial intelligence component includes the pedestrian's tracking id, position coordinates, current frame number Seq curr , and facial feature vectors.
  • the performance of artificial intelligence components is good, and the processing rate is 12 frames per second, but the edge processing platform has limited preprocessing capabilities for video frames, which can only reach 3 frames per second, so every fixed frame number framestep (we call it the detection frame ofps) for analysis.
  • framestep 7, and the same video frame is used as the input data of the artificial intelligence component between groups, which will inevitably lead to the degradation of the processing performance of the artificial intelligence component and the inaccurate pedestrian tracking id.
  • the present invention provides a method for edge-oriented pedestrian tracking and accurate counting of the number of people.
  • the specific process is as follows:
  • Step S1 Establish and maintain a pedestrian dynamic tracking list and a pedestrian dynamic statistical list in real time.
  • the pedestrian dynamic tracking list is used to dynamically record pedestrian information in the tracking area. When a pedestrian enters the statistical area, it is added to the pedestrian dynamic statistical list.
  • the two lists should include The following 4 characteristics:
  • Tracking id included in the pedestrian result information provided by the artificial intelligence component.
  • Moving radius r Defined as the average distance traveled by pedestrians in each detection frame from entering the tracking area to the present.
  • the initial centroid coordinates of the pedestrian entering the tracking area are (x orgi , y orgi )
  • the centroid coordinates of the current frame are (x curr , y curr )
  • the initial frame number is Seq orgi
  • the current frame number is Seq curr
  • Step S2 Obtain the pedestrian result information of the current video frame from the artificial intelligence component, build a pedestrian static detection list on this basis, and match the pedestrians in the static detection list with the pedestrians in the dynamic tracking list.
  • the basic principle of the algorithm matching is: first perform exact matching, that is, if the tracking id is the same, the exact matching is successful, otherwise according to the intelligent fuzzy matching method, if the pedestrian direction in the dynamic tracking list is known, then select the location in the static detection list In its moving direction, the pedestrian within the prediction range and the closest distance matches it.
  • One of the roughest methods is to delineate the prediction range in a circle with the pedestrian's center of mass (x curr , y curr ) as the center and the moving radius r as the radius, as shown in Figure 2.
  • adding the restriction of the moving direction d can help us reduce the prediction range from Figure 2 to Figure 3, which is within the semicircle in the moving direction.
  • this algorithm can also perform two-way people counting, so there may be two pedestrians walking towards each other, getting closer and closer before passing by (as shown by the two pedestrians id2 and id1 in Figure 4). Matching errors. Therefore, it is not enough to narrow the prediction range to Figure 3. We believe that two pedestrians will try to avoid collision.
  • the algorithm uses a matching table for analysis.
  • the horizontal direction represents the pedestrian in the dynamic tracking list, and the vertical direction represents the pedestrian in the static detection list.
  • the possible values and their meanings in the table are as follows:
  • the filling process is as follows:
  • Step S21 If the two tracking IDs are the same, it means that the artificial intelligence component analyzes that the two are the same person, and the exact match ends, fill in 0
  • Step S22 If the pedestrian in the static detection list is not within the prediction range of the pedestrian in the dynamic tracking list, fill in -2.
  • Step S23 Otherwise, calculate the distance and fill in the distance.
  • Step S3 Take out the minimum distance min other than -2 from the table in turn, if the following relationship is also satisfied:
  • MIN_COEFFICIENT is the minimum coefficient of matching distance
  • MAX_COEFFICIENT is the maximum coefficient of matching distance
  • MIN_COEFFICIENT ⁇ r ⁇ distancemin ⁇ MAX_COEFFICIENT ⁇ r is the prediction range.
  • step S4 the frame loss caused by the performance degradation of artificial intelligence components will affect the accuracy of pedestrian tracking and accurate counting of people. Therefore, after the above three steps, we will then perform intelligent fuzzy matching on pedestrians who have not successfully matched. Specifically as shown in step S4:
  • the delay confirmation strategy of the moving direction d described in step S1 is specifically:
  • Step S5 Update the pedestrian dynamic tracking list according to the exact matching and intelligent fuzzy matching results.
  • Pedestrians who are not matched successfully in the static detection list are recorded as new persons, the movement direction d is judged, and the dynamic tracking list is added.
  • Step S6 Accurately count the number of pedestrians in the statistical area, specifically:
  • Step S62 When pedestrians are normally counted and leave the statistical area or the number of dropped frames in the statistical area is greater than LOST_THRESHOID, delete them from the dynamic statistical list.
  • the smart delay counting strategy in step S61 is specifically:
  • the pedestrian If the pedestrian has not counted normally but has walked out of the statistical area, it will not be deleted. It will not be deleted until the direction is determined after the pedestrian tracking process mentioned above and the pedestrian counts normally.
  • LOST_THRESHOID If the pedestrian has not counted normally but the number of lost frames is greater than LOST_THRESHOID, delete it. Considering the actual situation of the set statistical area and LOST_THRESHOID, if the pedestrian will not be detected again in the statistical area, the pedestrian will be included in the total number, otherwise it will be ignored.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Geometry (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)
  • Traffic Control Systems (AREA)

Abstract

一种面向边缘端行人跟踪和人数精确统计的方法，涉及边缘计算场景下基于人工智能部件的行人跟踪和人数统计领域。该方法初始时建立并实时维护行人动态跟踪列表。从边缘端的人工智能部件获取行人结果信息，并在其基础上建立行人静态检测列表。将行人静态检测列表和行人动态跟踪列表进行精确匹配。边缘端的成本及性能限制使得边缘端计算能力有限会导致行人精确匹配失败。此时，该方法将根据行人动态跟踪信息对行人进行智能模糊匹配。根据匹配后的结果实时更新行人动态跟踪列表，并对统计区域内的行人进行人数精确统计，若精确统计失败，所述方法对该行人使用延迟计数策略。

Description

一种面向边缘端行人跟踪和人数精确统计的方法 技术领域
本发明涉及边缘计算场景下基于人工智能部件的行人跟踪和人数统计领域。
背景技术
随着人们安全意识的普遍提高，监控摄像头已经遍布大街小巷，图像处理、机器视觉等技术的融入使计算机可以代替人脑分析视频中的信息，判断视频中的情况，使监控走向智能化。近些年边缘计算兴起，嵌入式系统以其功耗低、功能专一的特性成为边缘计算的最主要处理平台，使智能监控技术得以大量应用于实际生活中。
嵌入式专用芯片的效率远远高于PC，而成本和功耗又大大低于CPU，很多学者设计了嵌入式图像处理的专用芯片，并针对性地设计了行人跟踪和人数统计方法。行人跟踪策略包括利用专用芯片分析的行人颜色，姿态等特征信息进行相似性匹配，或是判断前后两帧检测框的重叠率。人数统计主要是通过跨越检测线触发。
现在人工智能技术正在快速落地，同样也在向边缘和终端领域扩展。近年来各大设计厂家针对人工智能算法专门设计了人工智能部件，然而人工智能部件对于部署平台的计算能力要求较高，边缘计算平台的处理能力可能会限制人工智能部件的处理性能，同时各厂大商也倾向于降低人工智能部件的处理性能以满足实际应用中成本和售价的平衡。
发明内容
鉴于此，本算法的目的是基于人工智能处理部件对实时视频及其相关信息进行智能分析后得出的行人结果信息，设计通用的行人跟踪和人数精确统计方法，使其应用于边缘计算场景时，在人工智能处理部件自身处理能力不足、边缘端处理平台计算能力有限和产品成本定价等资源限制下依旧能精确地进行行人跟踪和人数统计。
一种面向边缘端行人跟踪和人数精确统计的方法，其特征在于包括如下步骤：
步骤S1：建立并实时维护行人动态跟踪列表，用于动态记录跟踪区域内的行人信息。
步骤S2：获取边缘端人工智能处理部件的行人结果信息。
步骤S3：提取重要行人结果信息建立行人静态检测列表。
步骤S4：将行人静态检测列表和行人动态跟踪列表中的行人进行匹配。
步骤S5：根据匹配结果更新行人动态跟踪列表。
步骤S6：根据行人动态跟踪列表对统计区域的行人进行人数精确统计。
步骤S1中行人动态跟踪列表记录的信息应包括行人的跟踪id、方向、移动半径、丢帧信息。
步骤S3中行人静态检测列表记录的信息应包括行人的跟踪id、位置坐标、时间信息。
所述算法其特征还在于：边缘端的资源限制，包括人工智能处理部件的自身处理能力受限，边缘端处理平台的计算能力不足，产品成本限制等，都会使实时视频的智能分析过程出现丢帧问题从而导致行人结果信息产生偏差。故步骤S4中所述的匹配过程先进行精确匹配，若精确匹配失败则对精确匹配失败的行人进行智能模糊匹配。
步骤S41：精确匹配。
根据行人静态检测列表和动态跟踪列表中的跟踪id可判断两个列表中的行人是否为同一人，若跟踪id相同则为同一个人，精确匹配成功，否则精确匹配失败。
步骤S42：对精确匹配失败的行人进行智能模糊匹配。
对未出现丢帧问题且方向明确的行人，在其运动方向上预测下一帧中可能出现的位置范围，选取静态检测列表中与预测结果相似性最大且超过匹配阈值MATCH_THRESHOID的行人与之匹配。若MATCH_THRESHOID为百分数，则0≤MATCH_THRESHOID≤100％,MATCH_THRESHOID越接近 100％匹配条件越严苛，一般设置在≥80％即可。
对未出现丢帧问题但方向不明确的行人，对其在下一帧中可能出现的位置范围进行双向的预测，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配。匹配成功后根据其位置坐标的移动情况可对行人方向做出判断。
对出现丢帧问题的行人，计算其丢失帧数n，并将预测范围扩大α×n倍，α为扩大系数，一般情况下匹配范围是以移动半径为半径的圆的一部分，则α＝πn 2。当m≤LOST_THRESHOID时，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配。LOST_THRESHOID为丢帧阈值，一般可置LOST_THRESHOID＝3。
对于方向发生更改的行人，经过上述步骤无法成功匹配，此时对其在下一帧中可能出现的位置范围进行反向的预测，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配。若匹配成功后记录其方向产生过突变。
步骤S5对行人动态跟踪列表的更新内容包括：
动态跟踪列表中没有匹配成功的行人，记录丢帧信息。
静态检测列表中没有匹配成功的行人，记为新人，判断方向并加入动态跟踪列表。
走出跟踪区域或丢帧数大于LOST_THRESHOID的行人，删除动态跟踪列表。
所述算法其特征还在于：将方向作为智能模糊匹配指标更好地提升了人数统计的精确性。静态检测列表中的行人被判断为新人时，根据其初始位置坐标判断其方向。若行人初始位置坐标位于跟踪区域上部，一般认为是跟踪区域的上1/3部分，判断方向为入。若行人初始位置坐标位于跟踪区域下部，一般认为是跟踪区域的下1/3部分，判断方向为出。若行人初始位置坐标位于 跟踪区域其他部位，认为无法判断其方向，根据步骤S42中对未出现丢帧问题但方向不明确的行人的处理方式可对其方向进行延迟确认从而对方向做出正确判断。
步骤S6的人数精确统计过程包括：
步骤S61：建立并维护行人动态统计列，表用于人数精确统计。
步骤S62：对统计区域内的行人进行人数精确统计，具体如下：
若行人方向已知且第一次进入统计区域，记录总人数和其方向上的总人数。
若行人方向未知且第一次进入统计区域，只记录总人数，对其方向上的总人数采用智能延迟计数策略。
若行人方向突变且第一次进入统计区域，总人数增加，不进行带方向的计数。
步骤S63：当行人被正常计数并离开统计区域或在统计区域内但丢帧数大于LOST_THRESHOID，将其从行人动态统计列表中删除。
行人动态跟踪信息不明确时无法进行人数精确统计，所述算法对该类行人做智能延迟计数处理，具体为：
若行人进入统计区域但未确定方向，不予计数，直至经过上述行人跟踪过程确定了方向后才进行计数。
若行人还未被正常计数但已走出统计区域，不予删除，直至经过上述行人跟踪过程确定了方向并正常计数后才予以删除。
若行人还未被正常计数但丢帧数大于LOST_THRESHOID，予以删除。考虑设置的统计区域和LOST_THRESHOID的实际情况，若认为该行人不会在统计区域被再次检测到则将该行人计入总数，否则予以忽略。
所述算法中除智能延迟计数策略外，都只有行人在第一次进入统计区域时才进行计数，行人在统计区域中逗留时虽然还在行人动态统计列表中，但不重复计数。
根据权力要求1所述，本算法特征还在于：对行人在下一帧中可能出现 的位置范围的划定方法。划定过程详述如下：
一种最粗略的方法是将预测范围划定在以行人质心为圆心，移动半径为半径的整圆内。
进一步，所述方法将方向也作为重要匹配指标，方向的限制帮我们把预测范围由整圆缩小至在其移动方向上的半圆内。该半圆具体为：以行人质心为端点，做平行于移动方向的射线，穿过端点且垂直于该射线的直线可将整圆划分为半圆，取包含该射线的半圆即可。
更进一步，受智能分析的前后两个视频帧的时间差一定在1秒之内，而行人在1秒内移动方向不会产生巨大的改变，具体来说行人在其移动方向上偏向左或右的角度不会大于45°，故预测范围缩小至上述半圆中半径与该射线夹角<45°和。-45°的扇形内。
所述算法其优点在于：
(1)为边缘端人工智能部件提供的行人结果信息设计了通用的行人跟踪和人数精确统计方法，该方法可通用于各种检测模型，如人脸检测，行人检测，头肩检测等，精确匹配条件可以根据实际情况改变，可以进行单向或双向的人数统计，具有很好的可扩展性和通用性。
(2)兼顾了人工智能处理部件自身处理能力有限、边缘端处理平台的计算能力不足和产品成本定价等资源限制问题导致的行人结果信息出错使人数统计精度下降的问题，在精确匹配的基础上加入了智能模糊匹配，使行人跟踪和人数统计更加精确。
附图说明：
图1算法流程图
图2为粗略预测范围示意图
图3为带方向预测范围示意图
图4为带角度的预测范围示意图
图5为带匹配距离系数的预测范围示意图
图6为双向匹配预测范围示意图
图7为反向匹配预测范围示意图
具体实施方式：
下面将结合本发明的实施例，对本发明的技术方案进行清楚、完整地描述，所举实例仅用于解释本发明，并非限定本发明的应用。
在本实施例中，人工智能部件为我们提供的行人结果信息包括行人的跟踪id、位置坐标、当前帧号Seq curr、人脸特征向量等。人工智能部件性能良好，处理速率为每秒12帧，但边缘端处理平台对于视频帧的预处理能力有限，只能达到每秒3帧，故每隔固定帧数framestep(我们称之为检测帧ofps)进行分析，本实施例中framestep＝7，组间以相同的视频帧作为人工智能部件的输入数据，这必然导致人工智能部件处理性能下降，行人跟踪id不准确。
本发明提供一种面向边缘端行人跟踪和人数精确统计的方法，具体过程如下：
步骤S1：建立并实时维护行人动态跟踪列表和行人动态统计列表，行人动态跟踪列表用于动态记录跟踪区域内的行人信息，当行人进入统计区域时将其加入行人动态统计列表，两列表应包括以下4个特征：
1)跟踪id：包含在人工智能部件提供的行人结果信息中。
2)移动方向d：如果行人的初始位置坐标位于跟踪区域的上1/3部分，我们认为行人是入方向，使其携带入方向属性，记d＝in。如果行人的初始位置坐标位于跟踪区域的下1/3部分，我们认为行人是出方向，使其携带出方向属性，记d＝out。如果行人的初始位置坐标位于跟踪区域的其他部分，无法判断方向，不携带方向属性，对其方向进行延迟确认。
3)移动半径r：定义为行人从进入跟踪区域开始到当前为止每个检测帧移动的平均距离。设行人进入跟踪区域的初始质心坐标为(x orgi，y orgi),当前帧质心坐标为(x curr，y curr),初始帧号为Seq orgi,当前帧号为Seq curr,则
Figure PCTCN2021072538-appb-000001
4)丢帧号Seq lost：未丢帧时Seq lost＝-1，如果经过一轮算法后未将其和行人静态检测列表中任一行人成功匹配，记录丢帧，Swq lost＝Seq curr。
步骤S2：从人工智能部件获取当前视频帧的行人结果信息，在此基础上建立行人静态检测列表，将静态检测列表的行人和动态跟踪列表的行人进行匹配。
所述算法匹配的基本原则为：首先进行精确匹配，即若跟踪id相同则精确匹配成功，否则根据智能模糊匹配方法，如果动态跟踪列表中的行人方向已知，那么在静态检测列表中选取位于其移动方向上的，在预测范围内且距离最近的行人与之匹配。
所述算法的预测范围我们做出如下分析：
一种最粗略的做法可以将预测范围划定在是以行人质心(x curr，y curr)为圆心，以移动半径r为半径的圆内，如图2所示。由算法基本匹配原则可知加上移动方向d的限制可帮我们把预测范围由图2缩小至图3，为在其移动方向上的半圆内。但此算法还可以进行双向的人数统计，故可能会出现两个相向而行的行人，在还没有擦肩而过之前越走越近(如图4中id2和id1两个行人所示)导致匹配出错的情况。故只将预测范围缩小至图3还不够，我们认为两个行人会尽量避免相撞，故当其迎面而走但过于相近时会相互避让进而产生如图4中的角度，并且受智能分析的前后两个视频帧的时间差一定在1秒之内，而行人在1秒内移动方向不会产生巨大的改变，具体来说行人在其移动方向上偏向左或右的角度不会大于45°，因此我们将预测范围进一步缩小至图4，为半径与该射线夹角<45°和>-45°的扇形内。综上本算法认为对于位于图4阴影之外的部分也不属于预测范围(故图4中我们认为id3是id1更好的智能模糊匹配对象)。
所述算法使用匹配表格进行分析，横向代表动态跟踪列表中的行人，纵向代表静态检测列表中的行人。表格中可能出现的值及其含义如下表：
数值 含义
-2 不在预测范围内
-1 已经匹配过
0 跟踪id相同
distance(其他值) 两个行人质心相对距离
填表过程如下：
步骤S21：如果两者跟踪id相同，说明人工智能部件分析得出两人是同一人，精确匹配结束，填写0
步骤S22：如果静态检测列表中的行人不在动态跟踪列表中行人的预测范围内，填写-2.
步骤S23：否则计算距离填入distance。
步骤S3：依次从表格中取出除-2外的最小值distance min，若同时满足如下关系：
1)其对应的动态跟踪列表的行人和静态检测列表的行人均没有被匹配过。
2)设MIN_COEFFICIENT为匹配距离最小系数，MAX_COEFFICIENT为匹配距离最大系数，则MIN_COEFFICIENT×r≤distancemin≤MAX_COEFFICIENT×r，此步骤中将预测范围做了进一步精确，如图5所示。MIN_COEFFICIENT和MAX_COEFFICIENT可以根据实际情况在0-1范围内选取最佳值。如果MATCH_THRESHOID＝80％，可置MIN_COEFFICIENT＝0.9，MAX_COEFFICIENT＝1.1。
我们认为两个行人成功匹配，并将对应项置-1，表示两个行人已经被匹配。
由于人工智能部件性能下降导致的丢帧问题对行人跟踪和人数精确统计的精确性都会造成影响，故经过上述三个步骤后我们对没有匹配成功的行人接着进行智能模糊匹配。具体如步骤S4所示：
步骤S1中所述的移动方向d的延迟确认策略具体为：
我们对动态跟踪列表中方向不明确的行人进行双向范围的匹配，即将预测范围由图5扩充至图6，选取在此范围内距离最近的行人与之匹配，匹配成功后，根据其质心的移动方向即可确认该行人的移动方向。
丢帧补偿策略：
通过计算我们可以得到该行人丢失了几个检测帧ofps，计算公式为
Figure PCTCN2021072538-appb-000002
相应地我们将匹配的移动半径r也扩大n倍。即MIN_COEFFICIENT×r×n≤distance min≤MAX_COEFFICIENT×r×n。当n＞LOST_THRESHOID时将该行人从行人动态跟踪列表删除。若framestap＝7，LOST_THRESHOID＝3。
行人方向的突变：
若经过上述判断后，仍有未被成功匹配的行人，我们考虑是否其移动方向发生了变更，故对其进行如图7的反向匹配，若匹配成功，标记其方向突变。在人数精确统计时我们对其进行总人数的计算，但是不进行带方向的统计。
步骤S5：根据精确匹配和智能模糊匹配结果对行人动态跟踪列表进行更新。
动态跟踪列表中没有匹配成功的行人，记录Seq lost＝Seq curr。
静态检测列表中没有匹配成功的行人，记为新人，判断移动方向d，加入动态跟踪列表。
走出跟踪区域或丢帧数大于LOST_THRESHOID的行人，删除动态跟踪 列表。
步骤S6：对统计区域的行人进行人数精确统计，具体为：
步骤S61：若行人第一次进入统计区域且d＝in，总人数增加，in方向人数增加。
若行人第一次进入统计区域且d＝out，总人数增加，out方向人数增加。
若行人第一次进入统计区域且方向未知，只记录总人数，对其方向上的总人数采用智能延迟计数策略。
若行人第一次进入统计区域且携带方向突变属性，总人数增加，不进行带方向的计数。
步骤S62：当行人被正常计数并离开统计区域或在统计区域的丢帧数大于LOST_THRESHOID后，将其从动态统计列表中删除。
步骤S61中所述智能延迟计数策略具体为：
若行人进入统计区域但未确定方向，不予计数，直至经过上述行人跟踪过程确定了方向后才进行计数。
若行人还未正常计数但已走出统计区域，不予删除，直至经过上述行人跟踪过程确定了方向并正常计数后才予以删除。
若行人还未正常计数但丢帧数大于LOST_THRESHOID，予以删除。考虑设置的统计区域和LOST_THRESHOID的实际情况，若该行人不会在统计区域被再次检测到则将该行人计入总数，否则予以忽略。
所述算法中除延迟计数策略外，都只有行人在第一次进入统计区域时才进行计数，行人在动态统计区域中逗留时虽然还在统计列表中，但不重复计数。

Claims (5)

  1. 一种面向边缘端行人跟踪和人数精确统计的方法，其特征在于，包括如下步骤：
    步骤S1：建立并实时维护行人动态跟踪列表，用于动态记录跟踪区域内的行人信息；
    步骤S2：获取边缘端人工智能处理部件的行人结果信息；
    步骤S3：提取重要行人结果信息建立行人静态检测列表；
    步骤S4：将行人静态检测列表和行人动态跟踪列表中的行人进行匹配；
    步骤S5：根据匹配结果更新行人动态跟踪列表；
    步骤S6：根据行人动态跟踪列表对统计区域的行人进行人数精确统计；
    步骤S1中行人动态跟踪列表记录的信息应包括行人的跟踪id、方向、移动半径、丢帧信息；
    步骤S3中行人静态检测列表记录的信息应包括行人的跟踪id、位置坐标、时间信息；
    故步骤S4中所述的匹配过程先进行精确匹配，若精确匹配失败则对精确匹配失败的行人进行智能模糊匹配；
    步骤S41：精确匹配；
    根据行人静态检测列表和动态跟踪列表中的跟踪id可判断两个列表中的行人是否为同一人，若跟踪id相同则为同一个人，精确匹配成功，否则精确匹配失败；
    步骤S42：对精确匹配失败的行人进行智能模糊匹配；
    对未出现丢帧问题且方向明确的行人，在其运动方向上预测下一帧中可能出现的位置范围，选取静态检测列表中与预测结果相似性最大且超过匹配阈值MATCH_THRESHOID的行人与之匹配；若MATCH_THRESHOID为百分数，则0≤MATCH_THRESHOID≤100％,MATCH_THRESHOID设置在≥80％即可；
    对未出现丢帧问题但方向不明确的行人，对其在下一帧中可能出现的位 置范围进行双向的预测，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配；匹配成功后根据其位置坐标的移动情况对行人方向做出判断；
    对出现丢帧问题的行人，计算其丢失帧数n，并将预测范围扩大α×n倍，α为扩大系数，匹配范围是以移动半径为半径的圆的一部分，则α＝πn 2；当n≤LOST_THRESHOID时，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配；LOST_THRESHOID为丢帧阈值，LOST_THRESHOID＝3；
    对于方向发生更改的行人，经过上述步骤无法成功匹配，此时对其在下一帧中可能出现的位置范围进行反向的预测，选取静态检测列表中与预测结果相似性最大且超过MATCH_THRESHOID的行人与之匹配；若匹配成功记录其方向产生突变；
    步骤S5对行人动态跟踪列表的更新内容包括：
    动态跟踪列表中没有匹配成功的行人，记录丢帧信息；
    静态检测列表中没有匹配成功的行人，记为新人，判断方向并加入动态跟踪列表；
    走出跟踪区域或丢帧数大于LOST_THRESHOID的行人，删除动态跟踪列表。
  2. 根据权利要求1所述的方法，其特征在于：静态检测列表中的行人被判断为新人时，根据其初始位置坐标判断其方向；若行人初始位置坐标位于跟踪区域上部，一般认为是跟踪区域的上1/3部分，判断方向为入；若行人初始位置坐标位于跟踪区域下部，一般认为是跟踪区域的下1/3部分，判断方向为出；若行人初始位置坐标位于跟踪区域其他部位，认为无法判断其方向，根据步骤S42中对未出现丢帧问题但方向不明确的行人的处理方式可对其方向进行延迟确认从而对方向做出正确判断。
  3. 根据权利要求1所述的方法，其特征在于：步骤S6的人数精确统计过 程包括：
    步骤S61：建立并维护行人动态统计列，表用于人数精确统计；
    步骤S62：对统计区域内的行人进行人数精确统计，具体如下：
    若行人方向已知且第一次进入统计区域，记录总人数和其方向上的总人数；
    若行人方向未知且第一次进入统计区域，只记录总人数，对其方向上的总人数采用智能延迟计数策略；
    若行人方向突变且第一次进入统计区域，总人数增加，不进行带方向的计数；
    步骤S63：当行人被正常计数并离开统计区域或在统计区域内但丢帧数大于LOST_THRESHOID，将其从行人动态统计列表中删除。
  4. 根据权利要求1所述的方法，其特征在于：行人动态跟踪信息不明确时无法进行人数精确统计，所述算法对该类行人做智能延迟计数处理，具体为：
    若行人进入统计区域但未确定方向，不予计数，直至经过上述行人跟踪过程确定了方向后才进行计数；
    若行人还未被正常计数但已走出统计区域，不予删除，直至经过上述行人跟踪过程确定了方向并正常计数后才予以删除；
    若行人还未被正常计数但丢帧数大于LOST_THRESHOID，予以删除；考虑设置的统计区域和LOST_THRESHOID的实际情况，若认为该行人不会在统计区域被再次检测到则将该行人计入总数，否则予以忽略；
    所述算法中除智能延迟计数策略外，都只有行人在第一次进入统计区域时才进行计数，行人在统计区域中逗留时虽然还在行人动态统计列表中，但不重复计数。
  5. 根据权利要求1所述的方法，其特征在于：对行人在下一帧中可能出现的位置范围的划定方法，具体如下：
    将预测范围划定在以行人质心为圆心，移动半径为半径的整圆内；
    方向的限制把预测范围由整圆缩小至在其移动方向上的半圆内；该半圆 具体为：以行人质心为端点，做平行于移动方向的射线，穿过端点且垂直于该射线的直线可将整圆划分为半圆，取包含该射线的半圆即可；
    前后两个视频帧的时间差一定在1秒之内，其移动方向上偏向左或右的角度不会大于45°，故预测范围缩小至上述半圆中半径与该射线夹角＜45°和＞-45°的扇形内。
PCT/CN2021/072538 2020-01-21 2021-01-18 一种面向边缘端行人跟踪和人数精确统计的方法 WO2021147826A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202010068506.1A CN111260696B (zh) 2020-01-21 2020-01-21 一种面向边缘端行人跟踪和人数精确统计的方法
CN202010068506.1 2020-01-21

Publications (1)

Publication Number Publication Date
WO2021147826A1 true WO2021147826A1 (zh) 2021-07-29

Family

ID=70954344

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/072538 WO2021147826A1 (zh) 2020-01-21 2021-01-18 一种面向边缘端行人跟踪和人数精确统计的方法

Country Status (2)

Country Link
CN (1) CN111260696B (zh)
WO (1) WO2021147826A1 (zh)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111260696B (zh) * 2020-01-21 2023-04-18 北京工业大学 一种面向边缘端行人跟踪和人数精确统计的方法
CN113361456B (zh) * 2021-06-28 2024-05-07 北京影谱科技股份有限公司 一种人脸识别方法和系统
CN115546192B (zh) * 2022-11-03 2023-03-21 中国平安财产保险股份有限公司 牲畜数量识别方法、装置、设备及存储介质

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049787A (zh) * 2011-10-11 2013-04-17 汉王科技股份有限公司 一种基于头肩特征的人数统计方法和系统
US20150310275A1 (en) * 2012-11-28 2015-10-29 Zte Corporation Method and device for calculating number and moving direction of pedestrians
CN106778570A (zh) * 2016-12-05 2017-05-31 清华大学深圳研究生院 一种行人实时检测与跟踪方法
CN109255802A (zh) * 2018-08-21 2019-01-22 平安科技（深圳）有限公司 行人跟踪方法、装置、计算机设备及存储介质
CN109754409A (zh) * 2017-11-06 2019-05-14 北京航天长峰科技工业集团有限公司 一种监控视频行人目标匹配跟踪系统与方法
CN110443210A (zh) * 2019-08-08 2019-11-12 北京百度网讯科技有限公司 一种行人跟踪方法、装置以及终端
CN111260696A (zh) * 2020-01-21 2020-06-09 北京工业大学 一种面向边缘端行人跟踪和人数精确统计的方法

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101231755B (zh) * 2007-01-25 2013-03-06 上海遥薇(集团)有限公司 运动目标跟踪及数量统计方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103049787A (zh) * 2011-10-11 2013-04-17 汉王科技股份有限公司 一种基于头肩特征的人数统计方法和系统
US20150310275A1 (en) * 2012-11-28 2015-10-29 Zte Corporation Method and device for calculating number and moving direction of pedestrians
CN106778570A (zh) * 2016-12-05 2017-05-31 清华大学深圳研究生院 一种行人实时检测与跟踪方法
CN109754409A (zh) * 2017-11-06 2019-05-14 北京航天长峰科技工业集团有限公司 一种监控视频行人目标匹配跟踪系统与方法
CN109255802A (zh) * 2018-08-21 2019-01-22 平安科技（深圳）有限公司 行人跟踪方法、装置、计算机设备及存储介质
CN110443210A (zh) * 2019-08-08 2019-11-12 北京百度网讯科技有限公司 一种行人跟踪方法、装置以及终端
CN111260696A (zh) * 2020-01-21 2020-06-09 北京工业大学 一种面向边缘端行人跟踪和人数精确统计的方法

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHANG, YAN: "Research on Pedestian Detection and Tracking Technology in Indoor Scene", CHINESE MASTER'S THESES FULL-TEXT DATABASE, 1 May 2017 (2017-05-01), pages 1 - 78, XP055831183, ISSN: 1674-0246 *

Also Published As

Publication number Publication date
CN111260696A (zh) 2020-06-09
CN111260696B (zh) 2023-04-18

Similar Documents

Publication Publication Date Title
WO2021147826A1 (zh) 一种面向边缘端行人跟踪和人数精确统计的方法
US10223597B2 (en) Method and system for calculating passenger crowdedness degree
WO2020215492A1 (zh) 基于YOLOv3多伯努利视频多目标检测跟踪方法
US7200266B2 (en) Method and apparatus for automated video activity analysis
CN111209892A (zh) 一种基于卷积神经网络的人群密度及数量估计方法
CN106778655B (zh) 一种基于人体骨架的入口尾随进入检测方法
CN107909044B (zh) 一种结合卷积神经网络和轨迹预测的人数统计方法
CN107491749B (zh) 一种人群场景中全局和局部异常行为检测方法
Gao et al. Dining activity analysis using a hidden markov model
CN102254151A (zh) 一种基于面部视频分析的驾驶人疲劳检测方法
CN101196991A (zh) 密集客流计数和行人步行速度自动检测方法及系统
CN111639602B (zh) 一种行人遮挡及朝向检测方法
US8339454B1 (en) Vision-based car counting for multi-story carparks
CN113763427B (zh) 一种基于从粗到精遮挡处理的多目标跟踪方法
Valencia et al. Vision-based crowd counting and social distancing monitoring using Tiny-YOLOv4 and DeepSORT
CN115423735A (zh) 一种客流量统计方法及系统
CN115917589A (zh) 攀爬行为预警方法和装置、电子设备、存储介质
Ma et al. Event detection using local binary pattern based dynamic textures
Dever et al. Automatic visual recognition of armed robbery
Xie et al. A multi-object tracking system for surveillance video analysis
CN116342645A (zh) 一种针对游泳馆场景下的多目标跟踪方法
WO2022126668A1 (zh) 一种公共场所行人识别方法及人流统计系统
CN103902990A (zh) 一种基于Haar-like特征的司机面部特征快速检测方法
Ukyo et al. Pedestrian Tracking Using 3D LiDARs–Case for Proximity Scenario
CN114627339A (zh) 茂密丛林区域对越境人员的智能识别跟踪方法及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21744859

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21744859

Country of ref document: EP

Kind code of ref document: A1

122 Ep: pct application non-entry in european phase

Ref document number: 21744859

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 13.02.2023)