WO2022247711A1 - 一种目标关联视频追踪处理方法和装置 - Google Patents

一种目标关联视频追踪处理方法和装置 Download PDF

Info

Publication number
WO2022247711A1
WO2022247711A1 PCT/CN2022/093647 CN2022093647W WO2022247711A1 WO 2022247711 A1 WO2022247711 A1 WO 2022247711A1 CN 2022093647 W CN2022093647 W CN 2022093647W WO 2022247711 A1 WO2022247711 A1 WO 2022247711A1
Authority
WO
WIPO (PCT)
Prior art keywords
target
video
probability
targets
time
Prior art date
Application number
PCT/CN2022/093647
Other languages
English (en)
French (fr)
Inventor
秦军瑞
吴劲
李启文
段志奎
邝伟锋
许剑锋
邓锐
李洋
Original Assignee
广州智慧城市发展研究院
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 广州智慧城市发展研究院 filed Critical 广州智慧城市发展研究院
Publication of WO2022247711A1 publication Critical patent/WO2022247711A1/zh

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N7/00—Television systems
    • H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
    • H04N7/181—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a plurality of remote sources

Definitions

  • the present disclosure relates to the technical field of object tracking, in particular to a method and device for object-associated video tracking processing.
  • Computer vision algorithm is currently a widely and effective target recognition technology. It has been widely used in target recognition and target tracking in public places, but computer vision algorithms alone are not enough for target-associated video tracking in public places. In the existing target-associated video tracking methods, most methods only focus on the recognition of the target in the video, largely ignoring the feature calculation of the target's location and time, and it is difficult to calculate the action trend of the target's movement.
  • the present disclosure provides a target-associated video tracking processing method and device, which collects multiple groups of videos with different monitoring cameras set in multiple regions, and calculates the image, time and location characteristics of the target in the video image frame sequence, thereby realizing real-time monitoring and action-related A function of the probability value of sexually cooperative action.
  • a method and device for tracking and processing target-associated video are provided, and the method includes the following steps:
  • Step 1 collect multiple videos through monitoring cameras in different locations set in multiple areas, and use the target detection algorithm to detect multiple targets obtained from each video as a set T;
  • Step 2 each section of video is processed into an image frame sequence S marked with the time and place of acquisition;
  • Step 3 by calculating the time and place characteristics of each target in different image frame sequences S, the probability deviation D marked as the same target is obtained;
  • Step 4 according to the calculated eigenvalues of the same target from the current collection location l i to the next collection location location l i+1 , to obtain its spatial connectivity C;
  • Step 5 Using the moving time and moving route of the same target, compare the moving time and moving route of each target in the set T according to D and C, and calculate the probability value of the action correlation between the targets in the set T.
  • each segment of video is processed into an image frame sequence S marked with the time and location of collection: use the camera to record the time and location of collection, mark the collection time and collection time of each image frame recorded in the video frame set P ⁇ location, so that each video frame set is processed into an image frame sequence S marked with collection time and location, and each data item s in the sequence S consists of an image frame, its corresponding collection time, and its corresponding collection location.
  • step 2 the following steps are also included: the image frame sequence S is transmitted to the server back-end database through the wireless network connection of each camera for long-term data storage, or the real-time data set of S is directly stored on the server.
  • step 3 by calculating the time and location characteristics of each target in different image frame sequences S, the probability deviation difference D marked as the same target is calculated, specifically:
  • Step 3.1 take the image frame in each data item s in the sequence S, convert the image frame into a 512 ⁇ 512 image frame array f, let f(m,n) be the mth row and nth column of the array f Value, both m and n are positive integers less than or equal to 512;
  • Step 3.3 set the function K(f,d,l) to extract the eigenvalue k i of the data item s,
  • Step 3.4 set different targets a and b in the set T, process any two videos corresponding to a and b into image frame sequences S a and S b marked with acquisition time and place, and calculate two different sequences S a and S
  • the probability difference of b is to judge the probability difference D ab of the target in the two videos.
  • step 4 according to the calculated eigenvalues of the same target from the current collection location l i to the next collection location location l i+1 , the method to obtain its spatial connectivity C is: use the sequence obtained in step 3
  • the longitude and dimension of the corresponding collection location in the i-th data item in S are a 2-dimensional array l i , which is connected to form a route according to the target’s movement trajectory ⁇ l 1 ,l 2 ,...,l i-1 ,l i ⁇ L i , take the location of the target's next collection location l i+1 , and calculate the connectivity between the positioning spaces of l i and l i+1 as To measure the connection probability of the two positioning spaces.
  • step 5 the moving time and moving route of the same target are used to compare the moving time and moving route of each target in the set T according to D and C, and the probability of action correlation between the targets in the set T is calculated value
  • the specific method is: pass any different sequence S a and S b of two targets a and b in the set T through the probability deviation gap D ab of the two sequences, and at the same time locate l i and locate each collection location in each route L i
  • the connectivity C i,i +1 of l i+ 1 is calculated to get ⁇ C 1,2 ,C 2,3 ,...,C i-1,i ,C i,i+1 ⁇ , assuming that targets a and b have actions
  • the probability value of correlation is ⁇ , then Indicates that when targets a and b have a displacement action from location l i to location l i+1 in route L i , the probability value that the two targets are considered to have action correlation is calculated as From this, the probability value of the action correlation between
  • the probability threshold value is [ 0.8,1] or the probability threshold is set as the arithmetic mean of the probability values of all targets in the set T to carry out action-relevant cooperative actions.
  • step 5 it also includes: setting the target person as a, through the judgment of relevance to a, filtering out and storing all videos containing the corresponding video of the target related to a and storing it in the database, and then Delete the videos of other targets that are not related to the target person a, without storing the videos of all the targets that are not related to a, so as to achieve the effect of targeted and large-scale video compression.
  • An object-associated video tracking processing device includes: a processor, a memory, and a computer program stored in the memory and run on the processor, and the processor implements the object association when executing the computer program
  • the object-associated video tracking processing device runs on a desktop computer, a notebook computer, a palmtop computer or a computing device in a cloud data center.
  • the object-associated video tracking processing device may also be referred to as an object-associated video tracking processing system.
  • the present disclosure provides a method and device for target-associated video tracking processing, which collects multiple groups of videos with different surveillance cameras set in multiple regions, and calculates the image and time and location characteristics of the target in the video image frame sequence , so as to realize the function of real-time monitoring of the probability value of action-related cooperative actions.
  • it has the following advantages: (1) make full use of the time and place characteristics of the video monitoring target to track the target; (2) effectively monitor the action correlation probability between targets, and realize the probability threshold monitoring; ( 3) To achieve the effect of targeted large-scale video compression.
  • Fig. 1 shows a flow chart of a target-related video tracking processing method and device
  • Figure 2 shows the calculation flow chart of the probability deviation gap D
  • Fig. 3 is a flow chart of calculating the probability value of action relevance.
  • FIG. 1 is a flow chart of an object-associated video tracking processing method and device according to the present disclosure. The following describes an object-associated video tracking processing method and device according to an embodiment of the present disclosure with reference to FIG. 1 .
  • the present disclosure proposes a target-associated video tracking processing method and device, which specifically includes the following steps:
  • Step 1 collect multiple videos through monitoring cameras in different positions set in multiple areas, and use the target detection algorithm to perform target detection on each video to obtain multiple targets as a set T;
  • Step 2 each section of video is processed into an image frame sequence S marked with the time and place of acquisition;
  • Step 3 by calculating the time and place characteristics of each target in different image frame sequences S, the probability deviation D marked as the same target is obtained;
  • Step 4 according to the calculated eigenvalues of the same target from the current collection location l i to the next collection location location l i+1 , to obtain its spatial connectivity C;
  • Step 5 Using the moving time and moving route of the same target, compare the moving time and moving route of each target in the set T according to D and C, and calculate the probability value of the action correlation between the targets in the set T.
  • each segment of video is processed into an image frame sequence S marked with the time and location of collection: use the camera to record the time and location of collection, mark the collection time and collection time of each image frame recorded in the video frame set P ⁇ location, so that each video frame set is processed into an image frame sequence S marked with collection time and location, and each data item s in the sequence S consists of an image frame, its corresponding collection time, and its corresponding collection location.
  • step 2 the following steps are also included: the image frame sequence S is transmitted to the server back-end database through the wireless network connection of each camera for long-term data storage, or the real-time data set of S is directly stored on the server.
  • step 3 by calculating the time and location characteristics of each target in different image frame sequences S, the probability deviation difference D marked as the same target is calculated, specifically:
  • Step 3.1 take the image frame in each data item s in the sequence S, convert the image frame into a 512 ⁇ 512 image frame array f, let f(m,n) be the mth row and nth column of the array f Value (m, n are both positive integers less than or equal to 512);
  • Step 3.3 set the function K(f,d,l) to extract the eigenvalue k i of the data item s,
  • Step 3.4 set different targets a and b in the set T, process any two videos corresponding to a and b into image frame sequences S a and S b marked with acquisition time and place, and calculate two different sequences S a and S
  • the probability difference of b is to judge the probability difference D ab of the target in the two videos.
  • step 4 according to the calculated eigenvalues of the same target from the current collection location l i to the next collection location location l i+1 , the method to obtain its spatial connectivity C is: use the sequence obtained in step 3
  • the longitude and dimension of the corresponding collection location in the i-th data item in S are a 2-dimensional array l i , which is connected to form a route according to the target’s movement trajectory ⁇ l 1 ,l 2 ,...,l i-1 ,l i ⁇ L i , take the location of the target's next collection location l i+1 , and calculate the connectivity between the positioning spaces of l i and l i+1 as To measure the connection probability of the two positioning spaces.
  • step 5 the moving time and moving route of the same target are used to compare the moving time and moving route of each target in the set T according to D and C, and the probability of action correlation between the targets in the set T is calculated value
  • the specific method is: pass any different sequence S a and S b of two targets a and b in the set T through the probability deviation gap D ab of the two sequences, and at the same time locate l i and locate each collection location in each route L i
  • the connectivity C i,i +1 of l i+ 1 is calculated to get ⁇ C 1,2 ,C 2,3 ,...,C i-1,i ,C i,i+1 ⁇ , assuming that targets a and b have actions
  • the probability value of correlation is ⁇ , then Indicates that when targets a and b have a displacement action from location l i to location l i+1 in route L i , the probability value that the two targets are considered to have action correlation is calculated as From this, the probability value of the action correlation between
  • the probability threshold value is [ 0.8,1] or the probability threshold is set as the arithmetic mean of the probability values of all targets in the set T to carry out action-relevant cooperative actions.
  • the object-associated video tracking processing device includes: a processor, a memory, and a computer program stored in the memory and running on the processor, and the processor implements the one when executing the computer program.
  • the object-related video tracking processing device runs on a desktop computer, a notebook computer, a palmtop computer or a computing device in a cloud data center.
  • the embodiment of the present disclosure provides a target-associated video tracking processing method and device, which uses different surveillance cameras set in multiple areas to collect multiple groups of videos, and calculates the image, time, and location characteristics of the target in the video image frame sequence, thereby realizing real-time tracking.
  • the method described in this disclosure makes full use of the time and place characteristics of the video monitoring target to track the target, and can effectively monitor the probability of action correlation between targets, realize the probability threshold monitoring, and achieve effective The effect of targeted large-scale video compression.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Closed-Circuit Television Systems (AREA)
  • Image Analysis (AREA)

Abstract

本公开提供了一种目标关联视频追踪处理方法和装置，以多区域设置的不同监控摄像头采集多组视频，通过计算视频图像帧序列中目标的图像和时间、地点特征，从而实现实时监测有行动关联性合作行动的概率值的功能。计算标注为同一目标的概率偏差距D，并获取其空间连接性C，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值。相比现有的目标追踪技术，有如下优点：(1)充分利用了视频监测目标的时间地点特征，进行目标追踪；(2)有效监测目标之间行动关联性概率，实现概率阈值监控；(3)达到有针对性大幅度视频压缩的效果。

Description

一种目标关联视频追踪处理方法和装置 技术领域
本公开涉及目标追踪技术领域，具体涉及一种目标关联视频追踪处理方法和装置。
背景技术
计算机视觉算法是目前广泛有效的目标识别技术，在公共场合的目标识别和目标追踪有广泛的应用，但单纯使用计算机视觉算法不足以进行公共场所的目标关联视频追踪。在已有的目标关联视频追踪方法技术中，绝大多数方法仅仅着眼于视频中目标的识别，大程度忽略了对目标的地点和时间的特征计算，难以计算目标移动的行动趋势。
发明内容
本公开提供一种目标关联视频追踪处理方法和装置，以多区域设置的不同监控摄像头采集多组视频，通过计算视频图像帧序列中目标的图像和时间、地点特征，从而实现实时监测有行动关联性合作行动的概率值的功能。
为了实现上述目的，根据本公开的一方面，提供一种目标关联视频追踪处理方法和装置，所述方法包括以下步骤：
步骤1，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法对各个视频进行目标检测得到的多个目标作为集合T；
步骤2，将每一段视频处理为标记有采集时间地点的图像帧序列S；
步骤3，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D；
步骤4，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C；
步骤5，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值。
进一步地，在步骤1中，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法对各个视频进行目标检测得到的多个目标作为集合T的方法为：在多个区域 的公共场所或人行道放置多个不同位置的摄像头，全天候采集行人视频信息，提取视频段V的视频帧P＝{P t,…,P t-n}(t为视频段V的总帧数，n为(0,t)的正整数)，利用Spatial-Temporal Graph Transformer即简称为STGT算法(参考文献为：Chu P,Wang J,You Q,et al.Spatial-Temporal Graph Transformer for Multiple Object Tracking[J].2021.)或利用SiamFC++算法(参考文献为：Xu Y,Wang Z,Li Z,et al.SiamFC++:Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines[J].2019.)，对视频帧P进行筛选预处理，输出多个目标目标作为集合T以及含有检测目标的视频帧集P`。
进一步地，在步骤2中，将每一段视频处理为标记有采集时间地点的图像帧序列S：用记录摄像头采集时间和采集地点，标注视频帧集P`中各图像帧记录的采集时间和采集地点，由此将每一视频帧集处理为标记有采集时间地点的图像帧序列S，序列S中每个数据项s由图像帧、其对应的采集时间、其对应的采集地点组成。
进一步地，在步骤2中，还包括以下步骤：将图像帧序列S，通过各摄像头的无线网络连接输送到服务器后端数据库进行数据长期存储，或者直接在服务器上储存S的实时数据集。
进一步地，在步骤3中，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D，具体为：
步骤3.1，取序列S中每个数据项s中的图像帧，将其图像帧转化为512×512的图像帧数组f,令f(m,n)为数组f的第m行第n列取值，m和n皆为小于等于512的正整数；
步骤3.2，令n序列S的长度，设数据项s的序号i取值范围属于[1，n]，则S中第i个数据项s i中的图像帧的图像帧矩阵为f i，该第i个数据项中对应的读取时间取年、月、日、时、分、秒为一个6维数组表示为d i,该第i个数据项中对应的读取位置定位取经度o i、纬度a i为一个2维数组表示为l i＝[o i,a i]，则有S中第i个数据项s i数学表示为s i＝[f i，d i，l i]；
步骤3.3，设函数K(f,d,l)以提取数据项s的特征值k i，
Figure PCTCN2022093647-appb-000001
Figure PCTCN2022093647-appb-000002
其中R(d,l)为d i,l i的处理过程，令d(m,n),l(m,n)为数组d i,l i的第m行第n列取值，具体计算为R(d,l)＝[exp(d(1,1)*l(1,1))+exp(d(1,1)*l(1,2)]+[exp(d(1,2)*l(1,1))+exp(d(1,2)*l(1,2)]+[exp(d(1,3)*l(1,1))+exp(d(1,3)*l(1,2)]+[exp(d(1,4)*l(1,1))+exp(d(1,4)*l(1,2)]+[exp(d(1,5)*l(1,1))+exp(d(1,5)*l(1,2)]+[exp(d(1,6)* l(1,1))+exp(d(1,6)*l(1,2)]，即s i＝[f i，d i，l i]通过函数K(s i)＝K(f i，d i，l i)可以得到k i＝K(s i)，则有序列S可表示为K(S)＝{k 1,k 2,…,k i-1,k i}；
步骤3.4，设集合T中不同目标a和b,将对应含有a和b的任意两个视频处理为标记有采集时间地点的图像帧序列S a和S b，计算两段不同序列S a和S b的概率偏差距，判断两段视频中的目标的概率偏差距D a-b，具体计算为：K(S a)＝{k a,1,k a,2,…,k a,i-1,k a,i},K(S b)＝{k b,1,k b,2,…,k b,i-1,k b,i},
Figure PCTCN2022093647-appb-000003
Figure PCTCN2022093647-appb-000004
k a,i和k b,i分别为S a和S b中数据项的通过步骤3.3得到的特征值。
进一步地，在步骤4中，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C的方法为：用步骤3所得的序列S中第i个数据项中对应的采集地点定位取经度、维度为一个2维数组l i，根据目标的移动轨迹{l 1,l 2,…,l i-1,l i}相连形成路线L i,取该目标的下一采集地点定位l i+1，计算l i和l i+1两者定位空间的连接性为
Figure PCTCN2022093647-appb-000005
以衡量两者定位空间的连接概率。
进一步地，在步骤5中，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值，具体方法为：将集合T中两目标a和b的任意不同序列S a和S b通过两序列的概率偏差距D a-b，同时在各条路线L i中各采集地点定位l i和定位l i+1的连接性C i,i+1计算得到{C 1,2,C 2,3,…,C i-1,i,C i,i+1}，设目标a和b有行动关联性的概率值为β，则
Figure PCTCN2022093647-appb-000006
Figure PCTCN2022093647-appb-000007
表示，在目标a和b于路线L i中均有从定位l i移动到定位l i+1的位移行动时，两目标被视为有行动关联性的概率值算作
Figure PCTCN2022093647-appb-000008
由此检测到目标之间有行动关联性的概率值，实时进行监测，输出概率值，当概率值大于概率阈值时则判断目标a和b有关联性否则无关联性，概率阈值取值为[0.8,1]或概率阈值设为集合T中所有目标两两之间进行有行动关联性合作行动的概率值的算术平均值。
进一步地，在步骤5中，还包括：设目标人物为a，通过与a有关联性的判断，筛选出存储所有视频中包含与a有关联性目标所在的对应视频并存储到数据库中，然后删除掉与目标人物a无关联性的其他目标的视频，无需存储与a无关联性的所有目标的所在视频，达到 有针对性大幅度视频压缩的效果。
一种目标关联视频追踪处理装置包括：处理器、存储器及存储在所述存储器中并在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现所述一种目标关联视频追踪处理方法中的步骤，所述一种目标关联视频追踪处理装置运行于桌上型计算机、笔记本电脑、掌上电脑或云端数据中心的计算设备中。所述一种目标关联视频追踪处理装置也可被称为一种目标关联视频追踪处理系统。
本公开的有益效果为：本公开提供了一种目标关联视频追踪处理方法和装置，以多区域设置的不同监控摄像头采集多组视频，通过计算视频图像帧序列中目标的图像和时间、地点特征，从而实现实时监测有行动关联性合作行动的概率值的功能。相比现有的目标追踪技术，有如下优点：(1)充分利用了视频监测目标的时间地点特征，进行目标追踪；(2)有效监测目标之间行动关联性概率，实现概率阈值监控；(3)达到有针对性大幅度视频压缩的效果。
附图说明
通过对结合附图所示出的实施方式进行详细说明，本公开的上述以及其他特征将更加明显，本公开附图中相同的参考标号表示相同或相似的元素，显而易见地，下面描述中的附图仅仅是本公开的一些实施例，对于本领域普通技术目标来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图，在附图中：
图1所示为一种目标关联视频追踪处理方法和装置的流程图；
图2所示为概率偏差距D的计算流程图；
图3所示为行动关联性的概率值的计算流程图。
具体实施方式
以下将结合实施例和附图对本公开的构思、具体结构及产生的技术效果进行清楚、完整的描述，以充分地理解本公开的目的、方案和效果。需要说明的是，在不冲突的情况下，本申请中的实施例及实施例中的特征可以相互组合。
如图1所示为根据本公开的一种目标关联视频追踪处理方法和装置的流程图，下面结合图1来阐述根据本公开的实施方式的一种目标关联视频追踪处理方法和装置。
本公开提出一种目标关联视频追踪处理方法和装置，具体包括以下步骤：
步骤1，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法 对各个视频进行目标检测得到的多个目标作为集合T；
步骤2，将每一段视频处理为标记有采集时间地点的图像帧序列S；
步骤3，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D；
步骤4，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C；
步骤5，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值。
进一步地，在步骤1中，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法对各个视频进行目标检测得到的多个目标作为集合T的方法为：在多个区域的公共场所或人行道放置多个不同位置的摄像头，全天候采集行人视频信息，提取视频段V的视频帧P＝{P t,…,P t-n}(t为视频段V的总帧数，n为(0,t)的正整数，例如V的总帧数为100帧)，利用Spatial-Temporal Graph Transformer即简称为STGT算法(参考文献为：Chu P,Wang J,You Q,et al.Spatial-Temporal Graph Transformer for Multiple Object Tracking[J].2021.)或利用SiamFC++算法(参考文献为：Xu Y,Wang Z,Li Z,et al.SiamFC++:Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines[J].2019.)，对视频帧P进行筛选预处理，输出多个目标目标作为集合T以及含有检测目标的视频帧集P`。
进一步地，在步骤2中，将每一段视频处理为标记有采集时间地点的图像帧序列S：用记录摄像头采集时间和采集地点，标注视频帧集P`中各图像帧记录的采集时间和采集地点，由此将每一视频帧集处理为标记有采集时间地点的图像帧序列S，序列S中每个数据项s由图像帧、其对应的采集时间、其对应的采集地点组成。
进一步地，在步骤2中，还包括以下步骤：将图像帧序列S，通过各摄像头的无线网络连接输送到服务器后端数据库进行数据长期存储，或者直接在服务器上储存S的实时数据集。
进一步地，在步骤3中，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D，具体为：
步骤3.1，取序列S中每个数据项s中的图像帧，将其图像帧转化为512×512的图像帧数组f,令f(m,n)为数组f的第m行第n列取值(m，n皆为小于等于512的正整数)；
步骤3.2，令n序列S的长度，设数据项s的序号i取值范围属于[1，n]，则S中第i个数据项s i中的图像帧的图像帧矩阵为f i，该第i个数据项中对应的读取时间取年、月、日、时、分、秒为一个6维数组表示为d i,该第i个数据项中对应的读取位置定位取经度o i、纬度a i为一个2维数组表示为l i＝[o i,a i]，则有S中第i个数据项s i数学表示为s i＝[f i，d i，l i]；
步骤3.3，设函数K(f,d,l)以提取数据项s的特征值k i，
Figure PCTCN2022093647-appb-000009
Figure PCTCN2022093647-appb-000010
其中R(d,l)为d i,l i的处理过程，令d(m,n),l(m,n)为数组d i,l i的第m行第n列取值，具体计算为R(d,l)＝[exp(d(1,1)*l(1,1))+exp(d(1,1)*l(1,2)]+[exp(d(1,2)*l(1,1))+exp(d(1,2)*l(1,2)]+[exp(d(1,3)*l(1,1))+exp(d(1,3)*l(1,2)]+[exp(d(1,4)*l(1,1))+exp(d(1,4)*l(1,2)]+[exp(d(1,5)*l(1,1))+exp(d(1,5)*l(1,2)]+[exp(d(1,6)*l(1,1))+exp(d(1,6)*l(1,2)]，即s i＝[f i，d i，l i]通过函数K(s i)＝K(f i，d i，l i)可以得到k i＝K(s i)，则有序列S可表示为K(S)＝{k 1,k 2,…,k i-1,k i}；
步骤3.4，设集合T中不同目标a和b,将对应含有a和b的任意两个视频处理为标记有采集时间地点的图像帧序列S a和S b，计算两段不同序列S a和S b的概率偏差距，判断两段视频中的目标的概率偏差距D a-b，具体计算为：K(S a)＝{k a,1,k a,2,…,k a,i-1,k a,i},K(S b)＝{k b,1,k b,2,…,k b,i-1,k b,i},
Figure PCTCN2022093647-appb-000011
Figure PCTCN2022093647-appb-000012
k a,i和k b,i分别为S a和S b中数据项的通过步骤3.3得到的特征值。
进一步地，在步骤4中，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C的方法为：用步骤3所得的序列S中第i个数据项中对应的采集地点定位取经度、维度为一个2维数组l i，根据目标的移动轨迹{l 1,l 2,…,l i-1,l i}相连形成路线L i,取该目标的下一采集地点定位l i+1，计算l i和l i+1两者定位空间的连接性为
Figure PCTCN2022093647-appb-000013
以衡量两者定位空间的连接概率。
进一步地，在步骤5中，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值，具体方 法为：将集合T中两目标a和b的任意不同序列S a和S b通过两序列的概率偏差距D a-b，同时在各条路线L i中各采集地点定位l i和定位l i+1的连接性C i,i+1计算得到{C 1,2,C 2,3,…,C i-1,i,C i,i+1}，设目标a和b有行动关联性的概率值为β，则
Figure PCTCN2022093647-appb-000014
Figure PCTCN2022093647-appb-000015
表示，在目标a和b于路线L i中均有从定位l i移动到定位l i+1的位移行动时，两目标被视为有行动关联性的概率值算作
Figure PCTCN2022093647-appb-000016
由此检测到目标之间有行动关联性的概率值，实时进行监测，输出概率值，当概率值大于概率阈值时则判断目标a和b有关联性否则无关联性，概率阈值取值为[0.8,1]或概率阈值设为集合T中所有目标两两之间进行有行动关联性合作行动的概率值的算术平均值。
所述一种目标关联视频追踪处理装置包括：处理器、存储器及存储在所述存储器中并在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现所述一种目标关联视频追踪处理方法中的步骤，所述一种目标关联视频追踪处理装置运行于桌上型计算机、笔记本电脑、掌上电脑或云端数据中心的计算设备中。
本公开的实施例提供的一种目标关联视频追踪处理方法和装置，以多区域设置的不同监控摄像头采集多组视频，通过计算视频图像帧序列中目标的图像和时间、地点特征，从而实现实时监测有行动关联性合作行动的概率值的功能。相比现有的目标追踪技术，本公开所述方法充分利用了视频监测目标的时间地点特征，进行目标追踪，并能有效监测目标之间行动关联性概率，实现了概率阈值监控，并且达到有针对性大幅度视频压缩的效果。
尽管本公开的描述已经相当详尽且特别对几个所述实施例进行了描述，但其并非旨在局限于任何这些细节或实施例或任何特殊实施例，从而有效地涵盖本公开的预定范围。此外，上文以发明人可预见的实施例对本公开进行描述，其目的是为了提供有用的描述，而那些目前尚未预见的对本公开的非实质性改动仍可代表本公开的等效改动。

Claims (9)

  1. 一种视频目标跟踪路线标注方法，其特征在于，所述方法包括以下步骤：
    步骤1，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法对各个视频进行目标检测得到的多个目标作为集合T；
    步骤2，将每一段视频处理为标记有采集时间地点的图像帧序列S；
    步骤3，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D；
    步骤4，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C；
    步骤5，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值。
  2. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤1中，通过多个区域设置的不同位置的监控摄像头采集多个视频，通过目标检测算法对各个视频进行目标检测得到的多个目标作为集合T的方法为：在多个区域的公共场所或人行道放置多个不同位置的摄像头，全天候采集行人视频信息，提取视频段V的视频帧P＝{P t,…,P t-n}，t为视频段V的总帧数，n为(0,t)的正整数，利用Spatial-Temporal Graph Transformer即简称为STGT算法或利用SiamFC++算法对视频帧P进行筛选预处理，输出多个目标目标作为集合T以及含有检测目标的视频帧集P`。
  3. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤2中，将每一段视频处理为标记有采集时间地点的图像帧序列S：用记录摄像头采集时间和采集地点，标注视频帧集P`中各图像帧记录的采集时间和采集地点，由此将每一视频帧集处理为标记有采集时间地点的图像帧序列S，序列S中每个数据项s由图像帧、其对应的采集时间、其对应的采集地点组成。
  4. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤2中，还包括以下步骤：将图像帧序列S，通过各摄像头的无线网络连接输送到服务器后端数据库进行数据长期存储，或者直接在服务器上储存S的实时数据集。
  5. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤3中，通过计算不同图像帧序列S的各个目标的时间地点特征，求出标注为同一目标的概率偏差距D，具体为：
    步骤3.1，取序列S中每个数据项s中的图像帧，将其图像帧转化为512×512的图像帧 数组f,令f(m,n)为数组f的第m行第n列取值；
    步骤3.2，令n序列S的长度，设数据项s的序号i取值范围属于[1，n]，则S中第i个数据项s i中的图像帧的图像帧矩阵为f i，该第i个数据项中对应的读取时间取年、月、日、时、分、秒为一个6维数组表示为d i,该第i个数据项中对应的读取位置定位取经度o i、纬度a i为一个2维数组表示为l i＝[o i,a i]，则有S中第i个数据项s i数学表示为s i＝[f i，d i，l i]；
    步骤3.3，设函数K(f,d,l)以提取数据项s的特征值k i，
    Figure PCTCN2022093647-appb-100001
    Figure PCTCN2022093647-appb-100002
    其中R(d,l)为d i,l i的处理过程，令d(m,n),l(m,n)为数组d i,l i的第m行第n列取值，具体计算为R(d,l)＝[exp(d(1,1)*l(1,1))+exp(d(1,1)*l(1,2)]+[exp(d(1,2)*l(1,1))+exp(d(1,2)*l(1,2)]+[exp(d(1,3)*l(1,1))+exp(d(1,3)*l(1,2)]+[exp(d(1,4)*l(1,1))+exp(d(1,4)*l(1,2)]+[exp(d(1,5)*l(1,1))+exp(d(1,5)*l(1,2)]+[exp(d(1,6)*l(1,1))+exp(d(1,6)*l(1,2)]，即s i＝[f i，d i，l i]通过函数K(s i)＝K(f i，d i，l i)可以得到k i＝K(s i)，则有序列S可表示为K(S)＝{k 1,k 2,…,k i-1,k i}；
    步骤3.4，设集合T中不同目标a和b,将对应含有a和b的任意两个视频处理为标记有采集时间地点的图像帧序列S a和S b，计算两段不同序列S a和S b的概率偏差距，判断两段视频中的目标的概率偏差距D a-b，具体计算为：K(S a)＝{k a,1,k a,2,…,k a,i-1,k a,i},K(S b)＝{k b,1,k b,2,…,k b,i-1,k b,i},
    Figure PCTCN2022093647-appb-100003
    Figure PCTCN2022093647-appb-100004
    k a,i和k b,i分别为S a和S b中数据项的通过步骤3.3得到的特征值。
  6. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤4中，根据计算所得同一目标在当前采集地点l i到下一采集地点定位l i+1的特征值，来获取其空间连接性C的方法为：用步骤3所得的序列S中第i个数据项中对应的采集地点定位取经度、维度为一个2维数组l i，根据目标的移动轨迹{l 1,l 2,…,l i-1,l i}相连形成路线L i,取该目标的下一采集地点定位l i+1，计算l i和l i+1两者定位空间的连接性为
    Figure PCTCN2022093647-appb-100005
    Figure PCTCN2022093647-appb-100006
    以衡量两者定位空间的连接概率。
  7. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤5中，以同一目标的移动时间和移动路线，根据D和C对比集合T中各目标的移动时间和移动路线，计算出集合T中各目标之间有行动关联性的概率值，具体方法为：将集合T中两目标a和b的任意不同序列S a和S b通过两序列的概率偏差距D a-b，同时在各条路线L i中各采集地点定位 l i和定位l i+1的连接性C i,i+1计算得到{C 1,2,C 2,3,…,C i-1,i,C i,i+1}，设目标a和b有行动关联性的概率值为β，则
    Figure PCTCN2022093647-appb-100007
    表示，在目标a和b于路线L i中均有从定位l i移动到定位l i+1的位移行动时，两目标被视为有行动关联性的概率值算作
    Figure PCTCN2022093647-appb-100008
    由此检测到目标之间有行动关联性的概率值，实时进行监测，输出概率值，当概率值大于概率阈值时则判断目标a和b有关联性否则无关联性，概率阈值取值为[0.8,1]或概率阈值设为集合T中所有目标两两之间有行动关联性的概率值的算术平均值。
  8. 根据权利要求1所述的一种目标关联视频追踪处理方法，其特征在于，在步骤5中，还包括：设目标人物为a，通过权利要求7所述方法，筛选出所有视频中包含与a有关联性目标所在的对应视频并存储到数据库中。
  9. 一种目标关联视频追踪处理装置，其特征在于，所述一种目标关联视频追踪处理装置包括：处理器、存储器及存储在所述存储器中并在所述处理器上运行的计算机程序，所述处理器执行所述计算机程序时实现权利要求1至8中任一项所述的一种目标关联视频追踪处理方法中的步骤，所述一种目标关联视频追踪处理装置运行于桌上型计算机、笔记本电脑、掌上电脑或云端数据中心的计算设备中。
PCT/CN2022/093647 2021-05-24 2022-05-18 一种目标关联视频追踪处理方法和装置 WO2022247711A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202110567222.1A CN113343795B (zh) 2021-05-24 2021-05-24 一种目标关联视频追踪处理方法
CN202110567222.1 2021-05-24

Publications (1)

Publication Number Publication Date
WO2022247711A1 true WO2022247711A1 (zh) 2022-12-01

Family

ID=77471140

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2022/093647 WO2022247711A1 (zh) 2021-05-24 2022-05-18 一种目标关联视频追踪处理方法和装置

Country Status (2)

Country Link
CN (1) CN113343795B (zh)
WO (1) WO2022247711A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116109966A (zh) * 2022-12-19 2023-05-12 中国科学院空天信息创新研究院 一种面向遥感场景的视频大模型构建方法

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113343795B (zh) * 2021-05-24 2024-04-26 广州智慧城市发展研究院 一种目标关联视频追踪处理方法
CN113850837B (zh) * 2021-11-25 2022-02-08 腾讯科技（深圳）有限公司 视频处理方法、装置、电子设备、存储介质及计算机产品

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9443320B1 (en) * 2015-05-18 2016-09-13 Xerox Corporation Multi-object tracking with generic object proposals
CN110717414A (zh) * 2019-09-24 2020-01-21 青岛海信网络科技股份有限公司 一种目标检测追踪方法、装置及设备
CN111107319A (zh) * 2019-12-25 2020-05-05 眸芯科技（上海）有限公司 基于区域摄像头的目标追踪方法、装置及系统
CN111882581A (zh) * 2020-07-21 2020-11-03 青岛科技大学 一种深度特征关联的多目标跟踪方法
CN113343795A (zh) * 2021-05-24 2021-09-03 广州智慧城市发展研究院 一种目标关联视频追踪处理方法和装置

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109241173A (zh) * 2017-07-10 2019-01-18 华为技术有限公司 查询追踪目标的方法及查询设备
CN111932582A (zh) * 2020-06-04 2020-11-13 广东技术师范大学 一种视频图像中的目标跟踪方法及装置

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9443320B1 (en) * 2015-05-18 2016-09-13 Xerox Corporation Multi-object tracking with generic object proposals
CN110717414A (zh) * 2019-09-24 2020-01-21 青岛海信网络科技股份有限公司 一种目标检测追踪方法、装置及设备
CN111107319A (zh) * 2019-12-25 2020-05-05 眸芯科技（上海）有限公司 基于区域摄像头的目标追踪方法、装置及系统
CN111882581A (zh) * 2020-07-21 2020-11-03 青岛科技大学 一种深度特征关联的多目标跟踪方法
CN113343795A (zh) * 2021-05-24 2021-09-03 广州智慧城市发展研究院 一种目标关联视频追踪处理方法和装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116109966A (zh) * 2022-12-19 2023-05-12 中国科学院空天信息创新研究院 一种面向遥感场景的视频大模型构建方法
CN116109966B (zh) * 2022-12-19 2023-06-27 中国科学院空天信息创新研究院 一种面向遥感场景的视频大模型构建方法

Also Published As

Publication number Publication date
CN113343795A (zh) 2021-09-03
CN113343795B (zh) 2024-04-26

Similar Documents

Publication Publication Date Title
WO2022247711A1 (zh) 一种目标关联视频追踪处理方法和装置
Elharrouss et al. Gait recognition for person re-identification
Derpanis et al. Action spotting and recognition based on a spatiotemporal orientation analysis
US20040017930A1 (en) System and method for detecting and tracking a plurality of faces in real time by integrating visual ques
US9245247B2 (en) Queue analysis
CN116340801B (zh) 一种畜牧养殖环境数据异常智能监控方法
CN105389562A (zh) 一种时空约束的监控视频行人重识别结果的二次优化方法
CN112541403B (zh) 一种利用红外摄像头的室内人员跌倒检测方法
Li et al. Multi-view crowd congestion monitoring system based on an ensemble of convolutional neural network classifiers
CN111967443A (zh) 基于图像处理与bim的档案馆内感兴趣区域分析方法
Juyal et al. Locating people in real-world for assisting crowd behaviour analysis using SSD and deep SORT algorithm
Huang et al. Tracking multiple deformable objects in egocentric videos
Seidenari et al. Dense spatio-temporal features for non-parametric anomaly detection and localization
Lin et al. Moving camera analytics: Emerging scenarios, challenges, and applications
WO2022247712A1 (zh) 一种小区监控视频目标跟踪安防预警方法和装置
CN115830076B (zh) 一种人员轨迹视频智能分析系统
Napolean et al. Running event visualization using videos from multiple cameras
JP7195892B2 (ja) 座標変換行列推定方法及びコンピュータプログラム
CN112131950A (zh) 一种基于Android手机的步态识别方法
CN113627497B (zh) 一种基于时空约束的跨摄像头行人轨迹匹配方法
CN110688895A (zh) 基于多模板学习的井下跨视域目标检测跟踪方法
Rachna et al. Real-time violence detection using deep neural networks and DTW
Elaoud et al. Person Re-Identification from different views based on dynamic linear combination of distances
CN115272967A (zh) 一种跨摄像机行人实时跟踪识别方法、装置及介质
CN115641527A (zh) 一种监控视频智能协同跟踪与溯源方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22810438

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22810438

Country of ref document: EP

Kind code of ref document: A1