WO2020150897A1 - 视频目标的多目标跟踪方法、装置及存储介质 - Google Patents

视频目标的多目标跟踪方法、装置及存储介质 Download PDF

Info

Publication number
WO2020150897A1
WO2020150897A1 PCT/CN2019/072683 CN2019072683W WO2020150897A1 WO 2020150897 A1 WO2020150897 A1 WO 2020150897A1 CN 2019072683 W CN2019072683 W CN 2019072683W WO 2020150897 A1 WO2020150897 A1 WO 2020150897A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
target
target object
similarity
observation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2019/072683
Other languages
English (en)
French (fr)
Inventor
李良群
湛西羊
谢维信
刘宗香
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Priority to PCT/CN2019/072683 priority Critical patent/WO2020150897A1/zh
Publication of WO2020150897A1 publication Critical patent/WO2020150897A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Definitions

  • This application relates to the field of electronic technology, and in particular to a method, device and storage medium for multi-target tracking of video targets.
  • Video multi-target tracking is one of the important research contents of machine vision, which mainly obtains basic motion information such as the position, posture, and trajectory of the moving target in the video.
  • video multi-target tracking has opened many research fields and application fields, involving intelligent video surveillance, virtual reality, human-computer interaction, autonomous driving, traffic control, oceanography, intelligent robots, remote sensing , Biomedicine and other fields have attracted more and more scholar and researchers to actively participate and have achieved a lot of research results.
  • the key problem of the detection-based multi-target tracking method is the data association between the online detection result received by the detector and the existing target.
  • scholars and researchers at home and abroad have proposed many types of data association methods in the past 30 years.
  • most tracking algorithms can be divided into the following two categories: generative methods and discriminative methods.
  • the tracking algorithm based on the generative method uses the appearance and movement of the target to construct a target model, and considers the similarity between the target and the observation during the tracking process for tracking association.
  • Ross et al. gradually learn the low-dimensional subspace feature representation of the target, which can effectively adapt to the changes in the appearance of the target.
  • Babu et al. proposed tracking based on the reliable feature points of the target.
  • Azab et al. used fuzzy integration to fuse multiple features in the particle filter framework.
  • Zhang et al. used a particle filter-based sparse representation of the target model to select the candidate template association with the smallest reconstruction error of the target model.
  • Zhang and Liu et al. used two-layer convolutional neural networks to train robust features for association. But based on generative association methods, they sometimes cannot distinguish the target from the background well, because the background pixels and other target pixels in the bounding box of the target are inevitably regarded as part of the target, thus making the target appearance model The presence of inaccurate information prevents it from correctly distinguishing the object from the background, which may lead to tracking failures.
  • the discriminant method is designed to better separate the target from the background. For example, Jang et al.
  • fuzzy set theory in multi-target tracking has become more and more widespread, it is pointed out that using fuzzy mathematics to model uncertain information in the tracking system can help improve tracking performance.
  • association algorithms based on intuitionistic fuzzy set theory are also More and more.
  • fuzzy orthogonal particle filter Such as fuzzy orthogonal particle filter, intuitionistic fuzzy joint probability data association filter, fuzzy Kalman filter and fuzzy coding histogram.
  • the literature proposes an object tracking algorithm based on active contours for modeling neural fuzzy networks.
  • the contour-based model is used to extract the feature vector of the object.
  • the self-built neuro-fuzzy inference network is used, and the human contour histogram in the horizontal and vertical projections is subjected to the Discrete Fourier Transform (DFT). ) Transformation.
  • DFT Discrete Fourier Transform
  • Li et al. proposed an online video multi-target tracking algorithm based on intuitionistic fuzzy sets, which uses the obtained intuitionistic fuzzy membership to replace the correlation between the target and the observation to achieve the goal and For the correlation between observations, Zhao et al. proposed a reasoning algorithm based on intuition logic, and Revathi et al. proposed a histogram fuzzy logic block matching algorithm to track multiple targets in video.
  • N is an integer greater than 1;
  • the target object is related to the observation object, and if so, the target model is updated; if not, the target trajectory management is performed on the target object.
  • the solution of this application obtains N sets of feature similarity results by calculating the similarity of the image features between the target object and the observation object; then the N sets of feature similarity results are screened, and the screened results are merged to obtain the feature fusion result ; Based on the maximum entropy intuitionistic fuzzy clustering, the correlation cost matrix calculation is performed according to the feature fusion result, and the correlation cost matrix calculation result is obtained; finally, according to the correlation cost matrix calculation result, it is determined whether the target object is related to the observation object
  • the above scheme has the following advantages: First, the feature attributes of the video target are diverse, and the importance of the target feature is not the same for different times and locations.
  • the proposed scheme of this application uses the neighborhood rough set to select features and eliminate Features with low discrimination can improve the accuracy of feature similarity; second, introduce the local information of the target, and use (OSPA, Optimal Sub-patten Assignment) distance to define the local information distance measure to further improve the target The accuracy of distance measurement; third, using the principle of maximum entropy, a new maximum entropy intuitionistic fuzzy clustering method is proposed, which uses intuitionistic fuzzy membership to replace the correlation probability, avoids the constraints of traditional probability and statistics methods, and realizes the rapid correlation probability Accurate calculation.
  • OSPA Optimal Sub-patten Assignment
  • Fig. 1-a is a schematic diagram of an implementation process of a method for multi-target tracking of video targets provided by an embodiment of the application;
  • Fig. 1-b is a schematic diagram of occlusion between a target object and a target object provided by an embodiment of this application;
  • Fig. 1-c is a schematic diagram of the occlusion between the target object A and the target object B according to an embodiment of the application;
  • FIG. 2 is a schematic diagram of the structure of an electronic device provided by an embodiment of the application.
  • FIG. 3 is a schematic diagram of the hardware structure of an electronic device provided by another embodiment of the application.
  • the embodiments of the present application provide a method, device and storage medium for multi-target tracking of video targets, which are used for data association of video multi-target tracking.
  • the multi-target tracking method of the video target mainly includes the following steps:
  • the target object is the target object tracked in the video for target tracking.
  • the first image feature is an image feature corresponding to the target object.
  • the observation object is an environmental object other than the target object in the video for target tracking, and is used for comparison and analysis with the target object.
  • the second image feature is the image feature corresponding to the observation object.
  • observation objects there may be multiple observation objects. For example, it is defined that there are j observation objects, where j is an integer greater than 1.
  • N feature categories are used to comprehensively define the feature similarity between the target object and the observation object.
  • the feature category can be features of different dimensions such as distance, color, and shape.
  • N can be specifically equal to 7, and the seven feature categories are: distance, color, edge, texture, shape, movement direction, and overlap area.
  • the N groups of feature similarity results are screened based on the feature similarity of the rough set, and the screened results are fused to obtain a feature fusion result.
  • neighborhood rough set is a model proposed to solve the problem that the classic rough set cannot handle continuous data.
  • the neighborhood of the model is defined according to the maximum distance from the center of the neighborhood to the boundary in a certain metric, and the attributes are reduced based on the dependence of decision attributes on conditional attributes.
  • the interference between adjacent targets is one of the factors that affect the correct association of the targets, because these adjacent targets are very close in position, they will also have relatively similar appearance characteristics due to factors such as occlusion, and errors occur during the association process. The possibility of matching is greater. Therefore, how to select features that can better distinguish between targets and targets, targets and false observations is the key to video multi-target tracking.
  • the neighborhood rough set will be used to select and merge features.
  • N feature similarities are selected to measure the similarity relationship between the target and the observation. If these features are only linearly accumulated during fusion, the reliability of the fusion result will be affected. Because not all features are well distinguished, the target can be mapped to the observation one by one. In fact, some features will also affect the accuracy of the association. For example, when the target is occluded by each other, the visual characteristics of the target are affected, but the characteristics such as the movement direction and overlap area are more robust. The linear fusion of features may result in wrong association results. In this regard, we will use the attribute reduction ability of the neighborhood rough set to select the target feature, reduce the number of features according to the dependence of the decision attribute on the universe, and realize the selection of useful features.
  • the correlation cost matrix calculation is performed according to the feature fusion result, and the correlation cost matrix calculation result is obtained.
  • the local information is the local information of the target object.
  • the embodiment of the present application uses the OSPA distance to define the local information distance measure, which further improves the accuracy of the target distance measure.
  • OSPA is a consistency measurement method for the overall performance evaluation of the target tracking system.
  • a measurement distance is defined in the system state space, which can be used to measure the error between the true track and the estimated track.
  • fuzzy clustering used in the embodiments of the present application confirms the degree of uncertainty of the sample to be classified into each category, and expresses the fuzzy concept that the sample category is one and the other.
  • the set obtained by fuzzy clustering is the intuitionistic fuzzy set, and the intuitionistic fuzzy entropy is used to express the degree of uncertainty of the intuitionistic fuzzy set.
  • the principle of maximum entropy used in the embodiments of this application is a criterion for selecting the statistical characteristics of random variables that best meets the objective situation.
  • the probability distribution of random variables is difficult to determine. Generally, only various means (such as mathematical expectations, Variance, etc.) or known values under certain limited conditions (such as peak value, number of values, etc.), the distributions that conform to the measured values can be multiple or even infinite, usually, one of the distributions has the largest entropy . Choosing the distribution with the maximum entropy as the distribution of the random variable is an effective treatment method and criterion.
  • a new maximum entropy intuitionistic fuzzy clustering method is proposed by using the principle of maximum entropy. Intuitionistic fuzzy membership is used instead of the associated probability.
  • the intuitionistic fuzzy membership in the method proposed in the embodiments of this application can be effective Instead of the degree of relevance, it can associate the target with the observation and improve the accuracy of the association.
  • the target object is related to the observation object, and if so, the target model is updated; if not, the target trajectory management is performed on the target object.
  • the target model is a tracking model corresponding to the target object, which records the tracking status of the target object in real time.
  • the target model can be updated to maintain information such as the position, color, and texture characteristics of the target object.
  • the observation object if the observation object is not associated with the target object, initialize a temporary target trajectory, and when the number of occurrences of the temporary target trajectory is greater than or equal to ⁇ init times, then The temporary target trajectory becomes a stable target trajectory; when the number of occurrences of the temporary target trajectory is less than ⁇ init times, the temporary target trajectory is deleted; the ⁇ init is the target starting threshold;
  • the target object For an unassociated target object, if the target object is not associated with the observation object, determine whether the centroid coordinates of the target object's trajectory are at the boundary, when the distance between the centroid coordinates and the video boundary is less than 3 When the pixel coordinate distance is determined, it is determined that the target object has left in the video. Otherwise, the Kalman filter is used to predict the state of the target object. When the target object is not associated with the observed object for ⁇ term consecutive times, the It is deleted from the current target trajectory, and the ⁇ term is the target termination threshold.
  • the solution of this application obtains N sets of feature similarity results by calculating the similarity of image features between the target object and the observed object; then the N sets of feature similarity results are screened, and the screened results are merged, Obtain the feature fusion result; then based on the maximum entropy intuitionistic fuzzy clustering, perform the correlation cost matrix calculation according to the feature fusion result to obtain the correlation cost matrix calculation result; finally, judge the target object and the said target object according to the correlation cost matrix calculation result Whether the observation object is related or not; the above scheme has the following advantages: First, the feature attributes of the video target are diverse, and the importance of the target feature is not the same for different locations at different times.
  • the solution of this application proposes to use the neighborhood rough set to pair the features Make selections to eliminate features with low discrimination and improve the accuracy of feature similarity; second, introduce the local information of the target, and OSPA distance defines the local information distance measure to further improve the accuracy of the target distance measure; third, use the maximum entropy Principle, a new maximum entropy intuitionistic fuzzy clustering method is proposed, which replaces the associated probability with intuitionistic fuzzy membership, avoids the constraints of traditional probability statistical methods, and realizes the fast and accurate calculation of the associated probability.
  • the embodiment of this application introduces a method for multi-target tracking of video targets through specific formulas and algorithm flows, which specifically include:
  • the neighborhood rough set is a model proposed to solve the problem that the classic rough set cannot handle continuous data.
  • the neighborhood of the model is defined according to the maximum distance from the center of the neighborhood to the boundary in a certain metric, and the attributes are reduced based on the dependence of decision attributes on conditional attributes.
  • ⁇ (x, x i ) is the distance function.
  • the distance function in this paper is selected as the infinite norm distance, which is defined as:
  • the decision attribute D divides the universe into D U , then the upper and lower approximations of D U are defined as:
  • the similarity between the target object o i and the observation object z k is comprehensively calculated through 7 features such as distance, color, edge, texture, shape, movement direction, and overlap area, and the 7 feature similarity
  • 7 features such as distance, color, edge, texture, shape, movement direction, and overlap area
  • f 1 (o i , z k ) is the spatial distance feature similarity measurement function
  • f 2 (o i , z k ) is the geometric dimension feature similarity measurement function
  • f 3 (o i , z k ) is the direction of motion Feature similarity measurement function
  • f 4 (o i ,z k ) is the color feature similarity measurement function
  • f 5 (o i ,z k ) is the gradient direction feature similarity measurement function
  • f 6 (o i ,z k ) Is the similarity measurement function of cultural and scientific features
  • f 7 (o i , z k ) is the similarity measurement function of the overlapping area
  • (x o , y o ) is the center coordinates of the target object o i
  • Figure 1-b is a schematic diagram of the occlusion between the prediction results of different target objects in the present invention.
  • the shapes of the tracking icons of prediction result A and prediction result B are rectangles, and there is overlap between the two
  • the parameter expression of prediction result A is: [x,y ,w,d]
  • x,y represent the coordinates of the rectangular frame
  • w represents the width of the rectangular frame
  • d represents the height of the rectangular frame
  • the parameters of the prediction result B are expressed as: [x',y',w',h ′], where x′, y′ represent the coordinates of the rectangular frame
  • w′ represents the width of the rectangular frame
  • h′ represents the height of the rectangular frame
  • the shaded part between the prediction result A and the prediction result B is expressed as: [x o ,y o ,w o ,h o ]
  • its overlapping part is expressed as:
  • the area of the overlap between the prediction result A and the prediction result B is expressed as w o *h o . If the above w o and ho do not satisfy w o >0 or h o >0, no overlapping rectangle is formed between the two tracking rectangles, that is, the area of the overlapping rectangle is 0.
  • the occlusion degree between the two is defined as:
  • s( ⁇ ) represents the area of the area, and the occlusion degree satisfies 0 ⁇ w(A,B) ⁇ 1.
  • w(A,B) is greater than 0, it means that there is an occlusion between the prediction result A and the prediction result B.
  • y A and y B at the bottom of the two tracking rectangles representing the prediction result A and the prediction result B respectively that if y A > y B , it means that the prediction result B is blocked by the prediction result A, otherwise, It means that the prediction result A is blocked by the prediction result B.
  • the embodiment of the present invention uses the attribute reduction ability of the neighborhood rough set to select the image feature of the target object, reduces the number of features according to the dependence of the decision attribute on the universe, and realizes the selection of useful features.
  • the feature is an effective feature and can be fused later.
  • the threshold ⁇ cf and ⁇ are set to 0.7 and 0.1 respectively through experiments.
  • the objective function of the proposed maximum entropy intuitionistic fuzzy clustering method with local information is as follows:
  • ⁇ ik represents the observation object And cluster centers
  • the fuzzy membership between Represents the observation object And cluster centers the distance between.
  • ⁇ ik is the weight of the local information.
  • the objective optimization function is:
  • ⁇ k ⁇ [0.40.6] is usually called the difference factor.
  • the fuzzy membership degree ⁇ ik can be obtained by optimization as:
  • the membership function is extended from the traditional fuzzy set to the intuitionistic fuzzy set, and the membership function is modified by introducing the intuition index, which is defined as a new intuitionistic fuzzy membership:
  • ⁇ is a normal number.
  • the value range is [0.40.8].
  • our main problem is to solve the distance And local information distance measure Calculation.
  • the distance between the target object and the observed object is measured by the similarity of feature fusion.
  • Observed object at time t And cluster centers the distance between Can be defined as:
  • the reference topology set takes the neighboring target objects that are occluded with the target object o i as a query, and retrieves the neighboring target objects that meet a certain degree of occlusion to form a reference topology set of the target object.
  • the reference topology set of the observation object z k can be defined:
  • the OSPA distance can be used to calculate the distance between the target object and the reference topology set of the observed object
  • the OSPA distance considers both the number of sets and the distance between sets.
  • the OSPA distance for the reference topology set of the target object o i and the observation object z k is as follows:
  • c is the penalty for unmatched
  • q h represents the number of successful pairings between the reference set of the target object and the reference topology set, which is defined as:
  • h kl is defined as:
  • the reference topology set of target object 1 is target object 2.
  • target object 1 is severely occluded by target object 2, causing it to lose most of its appearance information, but its reference topology
  • the information of the set is relatively complete, so the distance between the target object and the reference topology set of the observation object can be used Make an association.
  • ⁇ ik Considering the influence of neighboring targets on the local information weight ⁇ ik of the target object o i , ⁇ ik is defined as:
  • N i indicates the number of objects close to the target audience
  • the audience its meaning is i o of occlusion greater than a prescribed threshold value w (i, j)> ⁇ 1 near the target object o of the number j
  • N i (j) Indicates the influence factor of the neighboring target object of target object i, defined as:
  • w(i,j) represents the degree of occlusion between the target object and other target objects.
  • the corresponding tracking method needs to automatically initialize the target track.
  • the corresponding track needs to be automatically terminated. Therefore, in order to better manage the target trajectory, the following two target trajectory management rules are formulated:
  • the observations that are not associated with the target are initialized as a temporary target trajectory.
  • the temporary target trajectory continuously appears ⁇ init the temporary target becomes a stable target trajectory, otherwise the temporary trajectory is deleted.
  • the target After the target is associated, there is no target trajectory associated with the observation. Determine whether the centroid coordinate of the target trajectory is at the boundary. When the distance from the video boundary is less than 3 pixel coordinate distance, it is determined that the target has left in the video, otherwise The Kalman filter is used to predict the state of the target. When the target is not associated with continuous ⁇ term frames, it is deleted from the current target trajectory.
  • Step 1 Use the hybrid Gaussian detection method to detect the t-th frame image
  • Step 2 Obtain the image features of the target object and the observed object in the t-th frame of image, and perform screening based on the rough set of the neighborhood;
  • the feature f n is an effective feature and can be fused later.
  • Step 3 Perform multi-feature fusion distance calculation on the selected features
  • Step 4 Calculate the reference topological set distance between the target object and the observation object
  • Step 5 Calculate the intuitionistic fuzzy membership of the target object and the observed object
  • Step 6 According to the intuitionistic fuzzy membership of the target object and the observed object, construct an association cost matrix to associate the target object.
  • the correlation cost matrix is constructed.
  • Step 7 The status of the target object is updated.
  • the associated observation ⁇ (o i ,z k ) ⁇ is used to update the target model, and the Kalman filtering method is used to predict and update the state of the target at the next moment.
  • the observation object For unrelated observation objects, if the observation object is not associated with the target object, initialize a temporary target trajectory. When the number of occurrences of the temporary target trajectory is greater than or equal to ⁇ init times, the temporary target trajectory Become a stable target trajectory; when the number of occurrences of the temporary target trajectory is less than ⁇ init times, delete the temporary target trajectory; the ⁇ init is the target starting threshold;
  • the target object For an unassociated target object, if the target object is not associated with the observation object, determine whether the centroid coordinates of the target object's trajectory are at the boundary, when the distance between the centroid coordinates and the video boundary is less than 3 When the pixel coordinate distance is determined, it is determined that the target object has left in the video. Otherwise, the Kalman filter is used to predict the state of the target object. When the target object is not associated with the observed object for ⁇ term consecutive times, the It is deleted from the current target trajectory, and the ⁇ term is the target termination threshold.
  • FIG. 2 provides an electronic device according to an embodiment of the application.
  • the electronic device can be used to implement the multi-target tracking method for video targets provided in the embodiment shown in FIG. 1-a.
  • the electronic device mainly includes:
  • the acquiring unit 210 is configured to acquire the first image feature of the target object and the second image feature of the observation object respectively;
  • the similarity calculation unit 220 is configured to perform feature similarity calculations of N feature categories on the first image feature and the second image feature to obtain N sets of feature similarity results, where N is an integer greater than 1;
  • the feature fusion unit 230 is configured to screen the N groups of feature similarity results based on the feature similarity of the rough set, and fuse the screened results to obtain a feature fusion result;
  • the correlation calculation unit 240 is configured to perform correlation cost matrix calculation based on the maximum entropy intuitionistic fuzzy clustering according to the feature fusion result to obtain the correlation cost matrix calculation result; judge the target object and the correlation cost matrix calculation result according to the correlation cost matrix calculation result. Whether the observation object is related, if so, update the target model; if not, perform target trajectory management on the target object.
  • each functional module is only an example. In actual applications, the configuration requirements of the corresponding hardware or the convenience of the implementation of the software can be considered according to needs.
  • the above-mentioned function allocation is completed by different functional modules, that is, the internal structure of the electronic device is divided into different functional modules to complete all or part of the functions described above.
  • the corresponding functional modules in this embodiment may be implemented by corresponding hardware, or may be implemented by corresponding hardware executing corresponding software.
  • the various embodiments provided in this specification can all apply the above description principles, which will not be repeated hereafter.
  • An embodiment of the present application provides an electronic device. Please refer to FIG. 3.
  • the electronic device includes:
  • the processor 302 executes the computer program, it realizes multiple video targets described in the embodiment shown in FIG. 1-a. Target tracking method.
  • the electronic device further includes:
  • At least one input device 303 and at least one output device 304 At least one input device 303 and at least one output device 304.
  • the aforementioned memory 301, processor 302, input device 303, and output device 304 are connected via a bus 305.
  • the input device 303 may specifically be a camera, a touch panel, a physical button or a mouse, etc.
  • the output device 304 may specifically be a display screen.
  • the memory 301 may be a high-speed random access memory (RAM, Random Access Memory) memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory.
  • RAM Random Access Memory
  • non-volatile memory non-volatile memory
  • the memory 301 is used to store a set of executable program codes, and the processor 302 is coupled with the memory 301.
  • the embodiments of the present application also provide a computer-readable storage medium.
  • the computer-readable storage medium may be provided in the electronic device in each of the foregoing embodiments, and the computer-readable storage medium may be the one shown in FIG. 3 above. Shows the memory in the embodiment.
  • a computer program is stored on the computer-readable storage medium, and when the program is executed by a processor, the method for multi-target tracking of video targets described in the embodiment shown in FIG. 1-a is realized.
  • the computer storage medium may also be a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a RAM, a magnetic disk, or an optical disk, etc., which can store program codes.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

一种视频目标的多目标跟踪方法、装置及存储介质。其中，该视频目标的多目标跟踪方法包括：获取分别目标对象的第一图像特征和观测对象的第二图像特征；对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果；基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。

Description

视频目标的多目标跟踪方法、装置及存储介质 技术领域
本申请涉及电子技术领域，尤其涉及一种视频目标的多目标跟踪方法、装置及存储介质。
背景技术
视频多目标跟踪是机器视觉的重要研究内容之一，主要获取视频中运动目标的位置、姿态、轨迹等基本运动信息。随着数字计算技术的发展，视频多目标跟踪已经打开多个研究领域及应用领域，涉及到智能视频监控、虚拟现实、人机交互、自动驾驶、交通管制、海洋学、智能机器人、遥控传感、生物医学等领域，吸引了越来越多的学者和研究人员的积极参与，取得了大量的研究成果。但在复杂背景和目标密集遮挡的环境下，多目标跟踪过程中存在很多不确定不完备的信息，如行人的形变、光照变化、虚假观测、目标之间的相互遮挡等，导致难以用传统的概率统计方法准确描述目标与观测的关系。
目前，由于检测技术的发展，基于目标检测的视频目标跟踪算法得到了快速的发展和广泛的应用。这类方法根据检测的目标的位置、大小，标签、特征等估计目标的当前状态，不需要人工标记，并且在跟踪过程中可根据检测结果实时更新目标的状态。近年来，随着深度学习方法在视频目标检测上的应用，目标检测技术不断进步，视频多目标检测技术已经非常成熟，即使在杂波环境下，仍可获得较准确的检测结果，数据关联的准确性成为影响跟踪发正确性的主要因素，所以基于检测的多目标数据关联方法得到了广泛的研究。
基于检测的多目标跟踪方法的关键问题是检测器接收的在线检测结果和已有目标之间的数据关联。为解决数据关联问题，近三十年来，国内外的学者和研究人员提出了很多类型的数据关联方法。根据数据关联方式不同，大多数跟踪算法可以分为以下两类：生成式方法和判别式方法。基于生成式方法下的跟踪算法利用目标的外观、运动等多个特征构建目标模型，在跟踪过程中考虑目标与观测之间的相似性进行跟踪关联。Ross等人通过逐步学习目标的低维子空间特征表示，可以有效适应目标外观的变化，Babu等人提出根据目标可靠的特征点进行跟踪，Azab等人利用模糊积分融合多个特征在粒子滤波框架中进行跟踪，Zhang等人利用基于粒子滤波器的稀疏表示目标模型，选择与目标模型重建误差最小的候选模板关联，Zhang和Liu等人利用双层卷积神经网络训练鲁棒的特征进行关联。但基于生成式的关联方法，它们有时不能很好地将目标和背景区分开，因为在目标的边界框内的背景像素和其他目标像素不可避免地被认为是目标的一部分，从而使目标外观模型存在不准确信息，使其无法正确区分对象和背景，导致跟踪可能失败。判别式方法旨在更好地将目标从背景中分离出来，如Jang等人提出融合粒子滤波观测模型的误差最小的投影网络模型，Mei等人提出利用最小二乘法和贝叶斯状态估计对目标和观测关联。但判别方法不能适应目标的外观变化，故Babenko等人提出使用多实例学习训练目标模型，Kalal等人提出基于光流和动态外观模型的监 督编码规则，Tuzel等人解决多个目标的遮挡跟踪问题，并将其作为二元模式分类问题，Wu等人提出了一个区域深度学习跟踪器，通过多个子区域观察目标，并通过深度学习模型观察每个区域，与大多数仅利用二维彩色或灰色图像来在线学习被跟踪对象的外观模型的现有跟踪器相比，取得了更好的效果。另外，基于相关滤波器的方法和基于深度学习的方法已经取得了优异的表现并引起了更多的关注。
近年，随着模糊集理论在多目标跟踪的应用越来越广泛，指出利用模糊数学对跟踪系统中的不确定信息进行建模有助于提高跟踪性能，基于直觉模糊集理论的关联算法成果也越来愈多。如模糊正交粒子滤波，直觉模糊联合概率数据关联滤波，模糊卡尔曼滤波和模糊编码直方图。文献提出建模神经模糊网络基于主动轮廓的对象跟踪算法。基于轮廓的模型被用来提取对象的特征向量，为了训练和识别运动物体，利用自建的神经模糊推理网络，同时将水平和垂直投影中的人体轮廓直方图进行了离散傅里叶变换(DFT)变换。直觉模糊集理论处理不确定信息的优势愈加明显，Li等人提出了一种基于直觉模糊集的在线视频多目标跟踪算法，利用得到的直觉模糊隶属度代替目标与观测的关联度，实现目标与观测间的关联，Zhao等人提出基于直觉逻辑的推理算法，Revathi等人提出基于直方图模糊逻辑块匹配算法对视频多目标进行跟踪。
技术问题
如何有效地对视频中的多个目标对象进行跟踪。
技术解决方案
获取分别目标对象的第一图像特征和观测对象的第二图像特征；
对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果，所述N为大于1的整数；
基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；
基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；
根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。
有益效果
本申请方案通过目标对象与观测对象之间图像特征的相似度计算，得到N组特征相似度结果；然后对N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；再基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；最后根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联；上述方案有以下优点：第一， 视频目标的特征属性多样，对于不同时刻不同地点，目标特征的重要性并不一样，对此，本申请方案提出利用邻域粗糙集对特征进行选择，剔除区分度低的特征，提高特征相似度的准确性；第二，引入目标的局部信息，并利用(OSPA，Optimal Sub-patten Assignment，最优次模式分配)距离定义局部信息距离测度，进一步提高目标距离测度的准确性；第三，利用最大熵原理，提出了一种新的最大熵直觉模糊聚类方法，用直觉模糊隶属度代替关联概率，避免传统概率统计方法的约束，实现关联概率的快速精确计算。
附图说明
图1-a为本申请实施例提供的视频目标的多目标跟踪方法的实现流程示意图；
图1-b为本申请实施例提供的目标对象与目标对象之间遮挡示意图；
图1-c为本申请实施例提供的目标对象A与目标对象B之间遮挡示意图；
图2为本申请一实施例提供的电子装置结构示意图；
图3为本申请另一实施例提供的电子装置硬件结构示意图。
本发明的最佳实施方式
实施例一
本申请实施例提供一种视频目标的多目标跟踪方法、装置及存储介质，用于视频多目标跟踪的数据关联。
请参阅图1-a，该视频目标的多目标跟踪方法主要包括以下步骤：
101、获取分别目标对象的第一图像特征和观测对象的第二图像特征；
该目标对象为进行目标跟踪的视频中跟踪的标的对象。第一图像特征为目标对象对应的图像特征。
该观测对象为进行目标跟踪的视频中除目标对象外的环境对象，用于与目标对象进行比对分析。第二图像特征为观测对象对应的图像特征。
具体的，在实际应用中，观测对象可以有多个，如，定义观测对象有j个，所述j为大于1的整数。
102、对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算；
对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果，所述N为大于1的整数。
在本申请实施例中通过N种特征类别来综合定义目标对象和观测对象之间的特征相似度。其中，特征类别可以为距离、颜色、形状等不同维度的特征。
进一步地，在实际应用中，N具体可以等于7，该7种特征类别分别为：距离、颜色、边缘、文理、形状、运动方向和重叠面积。
103、基于粗糙集的特征相似度对特征相似度结果进行筛选及融合；
基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果。
在本申请实施例中，引入了邻域粗糙集的概念，邻域粗糙集是为了解决经典粗糙集无法处理连续型数据问题提出的模型。该模型的邻域根据邻域中心点到边界在某一度量上的最大距离进行定义的，以决策属性对条件属性的依赖度对属性进行约简。
由于相邻目标之间的干扰性是影响目标正确关联的因素之一，因为这些相邻的目标在位置上十分接近，也会因遮挡等因素具有较为相似的外观特征，在关联过程中发生误匹配的可能性较大。因此，如何选取更能区分目标与目标、目标与虚假观测的特征，是视频多目标跟踪的关键。在本申请实施例中，将利用邻域粗糙集对特征进行选取和融合。
在本申请实施例中，选取了N个特征相似度衡量目标和观测之间的相似关系，如果在融合时仅对这些特征线性累加，会影响融合结果的可靠性。因为不是所有的特征都有良好的区分性，可以将目标与观测一一对应。事实上，有些特征还会影响关联的准确度，比如在目标相互遮挡情况时，目标的视觉特征受到影响，但运动方向和重叠面积等特征具有更好地鲁棒性，如果这时候直接将这些特征进行线性融合，可能得到错误的关联结果。对此，我们将利用邻域粗糙集的属性约简能力对目标特征进行选择，根据决策属性对论域的依赖度减少特征个数，实现对有用特征的选取。
104、基于最大熵直觉模糊聚类并根据所述特征融合结果进行关联代价矩阵计算；
基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果。
在本申请实施例中，引入了局部信息的最大熵直觉模糊聚类。
局部信息为目标对象的局部信息，本申请实施例利用OSPA距离定义局部信息距离测度，进一步提高目标距离测度的准确性。其中，OSPA是一种目标跟踪系统整体性能评价的一致性度量方法，在系统状态空间上定义一种测度距离，可以用来衡量真实航迹和估计航迹之间的误差大小。
传统的聚类分析是一种硬划分，它把每个待辨别的对象严格地划分到某个类中，这种分类的类别界限是分明的。而本申请实施例所使用的模糊聚类，是确认待分类样本属于各个类别的不确定性程度，表达了样本类属亦此亦彼的模糊概念。模糊聚类所得到的集合为直觉模糊集，直觉模糊熵用于表示直觉模糊集的不确定程度。
本申请实施例中使用的最大熵原理是一种选择随机变量统计特性最符合客观情况的准则，随机量的概率分布是很难测定的，一般只能测得其各种均值(如数学期望、方差等)或已知某些限定条件下的值(如峰值、取值个数等)，符合测得这些值的分布可有多种、以至无穷多种，通常，其中有一种分布的熵最大。选用这种具有最大熵的 分布作为该随机变量的分布，是一种有效的处理方法和准则。
在本申请实施例中，利用最大熵原理，提出了一种新的最大熵直觉模糊聚类方法，用直觉模糊隶属度代替关联概率，本申请实施例所提出方法中的直觉模糊隶属度可以有效地代替关联度对目标和观测进行关联，并且提高关联的准确性。
105、根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联。
根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。
该目标模型为目标对象所对应的跟踪模型，实时记录目标对象的跟踪状态。所述目标模型中可以更新维护目标对象的位置、颜色、纹理特征等信息。
示例性的，对于未关联上的观测对象，若所述观测对象没有与所述目标对象关联，则初始化一个临时目标轨迹，当所述临时目标轨迹出现的次数大于或等于τ init次时，所述临时目标轨迹变成稳定目标轨迹；当所述临时目标轨迹出现的次数小于τ init次时，删除所述临时目标轨迹；所述τ init为目标起始阈值；
对于未关联上的目标对象，若所述目标对象没有与所述观测对象关联，则判断所述目标对象的轨迹的质心坐标是否在边界处，当所示质心坐标与视频边界的距离小于3个像素坐标距离时，判定所述目标对象已经在视频中离开，否则利用卡尔曼滤波器对所述目标对象状态进行预测，当所述目标对象连续τ term次没有与所述观测对象关联时，将其从当前目标轨迹中删除，所述τ term为目标终止阈值。
由上可见，本申请方案通过目标对象与观测对象之间图像特征的相似度计算，得到N组特征相似度结果；然后对N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；再基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；最后根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联；上述方案有以下优点：第一，视频目标的特征属性多样，对于不同时刻不同地点，目标特征的重要性并不一样，对此，本申请方案提出利用邻域粗糙集对特征进行选择，剔除区分度低的特征，提高特征相似度的准确性；第二，引入目标的局部信息，OSPA距离定义局部信息距离测度，进一步提高目标距离测度的准确性；第三，利用最大熵原理，提出了一种新的最大熵直觉模糊聚类方法，用直觉模糊隶属度代替关联概率，避免传统概率统计方法的约束，实现关联概率的快速精确计算。
本发明的实施方式
实施例二
本申请实施例通过具体的公式以及算法流程介绍视频目标的多目标跟踪方法，具体包括：
对本申请实施例所使用到的一些参数概念进行定义，包括：
领域粗糙集
邻域粗糙集是为了解决经典粗糙集无法处理连续型数据问题提出的模型。该模型的邻域根据邻域中心点到边界在某一度量上的最大距离进行定义的，以决策属性对条件属性的依赖度对属性进行约简。
假设在实数空间上定义的有限非空集合U＝{x 1,x 2,…,x n}，D为决策属性，对任意x i∈U，它的δ邻域定义为：
δ(x i)＝{x|x∈U,Δ(x,x i)≤δ}        (1)
其中，Δ(x,x i)为距离函数，本文的距离函数选用为无穷范数距离，定义为：
Δ(x,x i)＝|x-x i|               (2)
决策属性D将论域划分的集合为D U，则D U的上近似、下近似定义为：
Figure PCTCN2019072683-appb-000001
Figure PCTCN2019072683-appb-000002
给定一个信息系统IS＝(U,D,V)，其中U是论域，V是属性值的集合，D是决策属性，
Figure PCTCN2019072683-appb-000003
决策属性D将论域划分的集合为D U，定义决策属性D对论域U的依赖度为：
Figure PCTCN2019072683-appb-000004
其中|·|表示个数。当决策属性D对论域的依赖度越大，表明该决策属性的边界域越小，其决策属性越能将论域区分，反之依赖度低的条件属性在分类时粗糙性和不确定更大，不利于决策。
特征相似度
在本申请实施例中，通过距离、颜色、边缘、文理、形状、运动方向和重叠面积等7个特征来综合计算目标对象o i与观测对象z k之间的相似度，7个特征相似度函数定义如下：
Figure PCTCN2019072683-appb-000005
其中，f 1(o i,z k)为空间距离特征相似性度量函数，f 2(o i,z k)为几何尺寸特征相似性度量函数，f 3(o i,z k)为运动方向特征相似性度量函数，f 4(o i,z k)为颜色特征相似性度量函数，f 5(o i,z k)为梯度方向特征相似性度量函数，f 6(o i,z k)为文理特征相似性度量函数，f 7(o i,z k)为重叠面积相似性度量函数；(x o,y o)为目标对象o i的中心坐标，(x z,y z)为观测对象z k的中心坐标，h o为目标对象o i的图像高度，
Figure PCTCN2019072683-appb-000006
为空间距离方差常量，h z为观测对象z k的图像高度，
Figure PCTCN2019072683-appb-000007
为几何尺寸方差常量，(x' o,y' o)为上一时刻目标对象o i的中心坐标，
Figure PCTCN2019072683-appb-000008
为上一时刻目标对象o i的速度在图像坐标轴上的投影，
Figure PCTCN2019072683-appb-000009
为运动方向方差常量，ρ(·)表示求巴氏系数，H r(·)表示颜色直方图，
Figure PCTCN2019072683-appb-000010
为目标模型方差常量，H g(·)表示分块梯度方向直方图特征，
Figure PCTCN2019072683-appb-000011
为梯度方向方差常量，H l(·)表示文理特征直方图，
Figure PCTCN2019072683-appb-000012
为纹理特征方差常量，w(o i,z k)表示目标对象o i与观测对象z k之间的遮挡度，
Figure PCTCN2019072683-appb-000013
为重叠面积方差常量。
对于w(o i,z k)的定义如下：图1-b为本发明不同目标对象的预测结果之间遮挡示意图。如图1-b所示，在当前视频帧中，预测结果A和预测结果B的跟踪图标形状均为矩形，且二者之间有重叠，且预测结果A的参数表述为：[x,y,w,d]，其中，x,y表示矩形框的坐标，w表示矩形框宽度，d表示矩形框的高度，且预测结果B的参数表述为：[x′,y′,w′,h′]，其中，x′,y′表示矩形框的坐标，w′表示矩形框宽度，h′表示矩形框的高度，预测结果A和预测结果B之间阴影部分表示为：[x o,y o,w o,h o]，且其重叠部分表示为：
Figure PCTCN2019072683-appb-000014
由此可知，预测结果A和预测结果B之间重叠部分的面积表示为w o*h o。若上述的w o、h o不满足w o>0或者h o>0，则两跟踪矩形框之间不构成重叠矩形，也即重叠矩形面积为0。
假设预测结果A与预测结果B发生如图2所示的遮挡，且两跟踪矩形框之间重叠的阴影部分表示遮挡区域，则定义二者之间的遮挡度为：
Figure PCTCN2019072683-appb-000015
其中，s(·)表示区域面积，且遮挡度满足0≤w(A,B)≤1。当w(A,B)大于0，则说明预测结果A与预测结果B之间发生遮挡。且进一步由分别代表预测结果A与预测结果B的两跟踪矩形框底部的纵向图像坐标值y A与y B可知，若y A>y B，则说明预测结果B被预测结果A遮挡，反之，则说明预测结果A被预测结果B遮挡。
特征筛选
本发明实施例利用邻域粗糙集的属性约简能力对目标对象的图像特征进行选择，根据决策属性对论域的依赖度减少特征个数，实现对有用特征的选取。对于特征f n(n＝1,2,...7)的选择方法具体如下：
1)计算目标对象o i和所有观测对象z对于特征f n的相似度，将其作为论域U n。
2)决策属性D n的下近似定义为
Figure PCTCN2019072683-appb-000016
为所有相似度f n大于τ cf的观测对象的集合
Figure PCTCN2019072683-appb-000017
然后利用式(5)计算特征f n的决策属性对论域U n的依赖度γ n。
3)当特征的依赖度γ n≥τ cf时，特征为有效特征，可以进行后面的融合。
决策属性对论域的依赖度越大，表明该条件属性更有利于决策属性进行决策，决策D的下近似越大，则边界域越小，该特征能够更好地将目标对象对应的观测对象和其他观测对象中区分开来。本文中阈 值τ cf和δ经过试验分别设置为0.7和0.1。
通过邻域粗糙集剔除掉一些不具有区分性的特征后，再对剩下的特征进行融合。本文采用相似度融合的方法进行多特征融合。假设目标对象o i与观测对象z k由n个特征描述，第i种特征的相似度为f i(o i,z k)，则该特征的权值和融合方式如下：
Figure PCTCN2019072683-appb-000018
Figure PCTCN2019072683-appb-000019
引入局部信息的最大熵直觉模糊聚类
假设t-1时刻的目标对象的集合为
Figure PCTCN2019072683-appb-000020
根据Kalman滤波器可以预测其在t时刻目标对象的集合为
Figure PCTCN2019072683-appb-000021
t时刻的观测对象为
Figure PCTCN2019072683-appb-000022
以目标对象的预测位置作为聚类中心，提出的加入局部信息的最大熵直觉模糊聚类方法的目标函数如下：
Figure PCTCN2019072683-appb-000023
其中，μ ik表示观测对象
Figure PCTCN2019072683-appb-000024
和聚类中心
Figure PCTCN2019072683-appb-000025
之间的模糊隶属度，
Figure PCTCN2019072683-appb-000026
表示观测对象
Figure PCTCN2019072683-appb-000027
和聚类中心
Figure PCTCN2019072683-appb-000028
之间的距离。
Figure PCTCN2019072683-appb-000029
表示观测对象与聚类中心之间的局部信息距离测度，ω ik为局部信息的权值。其中，
目标优化函数为：
Figure PCTCN2019072683-appb-000030
其中α k∈[0.40.6]通常被称为差异因子。
利用拉格朗日乘子法，优化可得模糊隶属度μ ik为：
Figure PCTCN2019072683-appb-000031
为了结合直觉模糊特性，将隶属度函数由传统的模糊集扩展为直觉模糊集，通过引入直觉指数来修正隶属度函数，定义成新的直觉模糊隶属度：
Figure PCTCN2019072683-appb-000032
其中，
Figure PCTCN2019072683-appb-000033
为新的直觉模糊隶属度，
Figure PCTCN2019072683-appb-000034
表示直觉指数π ik上隐含的隶属度比重，π ik由式(14)给出。相应的直觉指数表示为：
Figure PCTCN2019072683-appb-000035
其中，φ为一个正常数。取值范围为[0.40.8]。
对直觉模糊隶属度进行归一化，得到最终的直觉模糊隶属度：
Figure PCTCN2019072683-appb-000036
局部信息距离测度的近似计算
根据提出的模糊聚类方法，现在我们的主要问题就是解决距离
Figure PCTCN2019072683-appb-000037
和局部信息距离测度
Figure PCTCN2019072683-appb-000038
的计算。根据前面式(6)定义的特征相似度，目标对象和观测对象之间的距离用特征融合的相似度衡量。t时刻的观测对象
Figure PCTCN2019072683-appb-000039
和聚类中心
Figure PCTCN2019072683-appb-000040
之间的距离
Figure PCTCN2019072683-appb-000041
可以定义为：
Figure PCTCN2019072683-appb-000042
为了引入目标对象的局部信息，定义如下的目标对象o i的参考拓扑集：
Figure PCTCN2019072683-appb-000043
其中w(o i,o j)为目标对象o i与目标对象o j之间的重叠比例。参考拓扑集以与目标对象o i发生遮挡的邻近目标对象作为查询，并检索与其满足一定遮挡度的邻近目标对象形成该目标对象的参考拓扑集。同理，可以定义观测对象z k的参考拓扑集：
Figure PCTCN2019072683-appb-000044
其中，
Figure PCTCN2019072683-appb-000045
和
Figure PCTCN2019072683-appb-000046
分别表示目标对象和观测对象的参考拓扑集合的
元素个数。
于是，可以利用OSPA距离计算目标对象与观测对象的参考拓扑集的距离
Figure PCTCN2019072683-appb-000047
OSPA距离同时考虑了集合的数目和集合之间的距离两个因素，对于目标对象o i和观测对象z k的参考拓扑集的OSPA距离如下：
Figure PCTCN2019072683-appb-000048
其中 c是对未配对上的惩罚项，q h数目表示目标对象参考集与参考拓扑集成功配对数目，定义为：
Figure PCTCN2019072683-appb-000049
其中h kl定义为：
Figure PCTCN2019072683-appb-000050
当两个参考拓扑集中的
Figure PCTCN2019072683-appb-000051
和
Figure PCTCN2019072683-appb-000052
质心距离最小，且符合||(x o,y o)-(x z,y z)|| 2≤T时，两个拓扑集中的元素配对成功。
故
Figure PCTCN2019072683-appb-000053
表示配对的数目和不配对数目之和，
Figure PCTCN2019072683-appb-000054
表示配对不成功的数目，
Figure PCTCN2019072683-appb-000055
为参考拓扑集上
Figure PCTCN2019072683-appb-000056
元素质心之间的距离，定义为：
Figure PCTCN2019072683-appb-000057
其中
Figure PCTCN2019072683-appb-000058
和
Figure PCTCN2019072683-appb-000059
分别表示目标对象o i的参考拓扑集
Figure PCTCN2019072683-appb-000060
上第l个元素的质心坐标和观测对象z k的参考拓扑集
Figure PCTCN2019072683-appb-000061
上第r个元素的质心坐标，
Figure PCTCN2019072683-appb-000062
为空间距离方差常量，
Figure PCTCN2019072683-appb-000063
为重叠面积方差常量。
如图1-c所示，目标对象1的参考拓扑集为目标对象2，在第33帧时，目标对象1因被目标对象2严重遮挡，导致它失去大部分外观信息，但它的参考拓扑集的信息保存较为完整，所以可以利用目标对象与观测对象的参考拓扑集之间的距离
Figure PCTCN2019072683-appb-000064
进行关联。
考虑相邻目标对目标对象o i局部信息权值ω ik的影响，ω ik定义为：
Figure PCTCN2019072683-appb-000065
其中N i表示目标对象附近的目标对象个数，其含义为与目标对象o i的遮挡度大于指定阈值w(i,j)>τ 1的邻近目标对象o j个数，N i(j)表示目标对象i的邻近目标对象对其的影响因子，定义为：
Figure PCTCN2019072683-appb-000066
其中w(i,j)表示目标对象与其他目标对象的遮挡度，当目标对象 与其他邻近目标对象超过遮挡度特定阈值τ 2(0<τ 1<τ 2)时，表示该目标对象受到邻近目标对象较严重的遮挡的影响，在本文中将τ 1、τ 2设置为τ 1＝0.1，τ 2＝0.6。
轨迹管理
当目标进入视频场景时，相应的跟踪方法需要自动初始化目标轨迹。当目标离开视频场景时，与之对应的轨迹需要自动终止。因此，为了更好地对目标轨迹进行管理，制定如下两条目标轨迹管理规则：
在目标关联后，没有与目标关联的观测初始化为一个临时目标轨迹，当临时目标轨迹连续出现τ init时，该临时目标变成稳定目标轨迹，否则删除该条临时轨迹。
在目标关联后，没有与观测关联的目标轨迹，判断该目标轨迹的质心坐标是否在边界处，当其与视频边界的距离小于3个像素坐标距离时，判定该目标已经在视频中离开，否则利用卡尔曼滤波器对目标状态进行预测，当目标连续τ term帧没有观测与其关联时，将其从当前目标轨迹中删除。
这里τ init和τ term为目标起始和终止的判断阈值，根据经验设置将其分别设为τ init＝3，τ term＝8。
以下通过步骤流程对本申请实施例中的视频目标的多目标跟踪方法进行说明，包括：
步骤一、利用混合高斯检测方法对第t帧图像进行检测；
利用混合高斯检测方法对第t帧图像进行检测，得到观测对象集
Figure PCTCN2019072683-appb-000067
并假设当前有目标对象集
Figure PCTCN2019072683-appb-000068
本申请实施例需要对目标对象的状态预测，根据目标对象的前一帧状态
Figure PCTCN2019072683-appb-000069
i＝1,2,…,c t-1，利用卡尔曼滤波方法预测下一帧目标对象的位置，得到目标对象的预测状态
Figure PCTCN2019072683-appb-000070
i＝1,2,…,c t-1。
步骤二、获取第t帧图像中目标对象和观测对象的图像特征，并进行根据邻域粗糙集进行筛选；
根据邻域粗糙集进行筛选筛选具体为：
a)计算目标对象
Figure PCTCN2019072683-appb-000071
和所有观测对象
Figure PCTCN2019072683-appb-000072
对于特征f n的相似度，将其作为论域U n；
b)决策属性D n的下近似定义为
Figure PCTCN2019072683-appb-000073
为所有相似度f n大于τ cf的观测对象的集合
Figure PCTCN2019072683-appb-000074
然后利用公式(5)计算特征f n的决策属性对论域U n的依赖度γ n；
c)当特征f n的依赖度γ n≥τ cf时，特征f n为有效特征，可以进行后面的融合。
步骤三、对筛选后的特征进行多特征融合距离的计算；
根据上一步计算的特征相似度f n，利用式公(10)计算目标对象的集合
Figure PCTCN2019072683-appb-000075
和观测对象的集合
Figure PCTCN2019072683-appb-000076
的多特征融合的相似度
Figure PCTCN2019072683-appb-000077
利用式(17)求目标对象与观测对象的多特征融合距离
Figure PCTCN2019072683-appb-000078
i＝1,2,…,c t-1,k＝1,2,…,n t。
步骤四、计算目标对象和观测对象之间的参考拓扑集距离；
根据上述得到的多特征融合距离
Figure PCTCN2019072683-appb-000079
分别计算目标对象集合
Figure PCTCN2019072683-appb-000080
和观测对象集合
Figure PCTCN2019072683-appb-000081
的参考拓扑集，利用公式(20)求目标对象的参考拓扑集和观测对象的参考拓扑集之间的距离
Figure PCTCN2019072683-appb-000082
步骤五、计算目标对象与观测对象的直觉模糊隶属度；
根据目标对象和观测对象之间的参考拓扑集距离，以及公式(16)求得计算归一化后目标对象与观测对象的直觉模糊隶属度
Figure PCTCN2019072683-appb-000083
步骤六、根据目标对象与观测对象的直觉模糊隶属度，构建关联代价矩阵对目标对象进行关联。
根据目标对象与观测对象的直觉模糊隶属度，构建关联代价矩阵。
对于t时刻的目标对象O t和观测对象Z t，计算每一对
Figure PCTCN2019072683-appb-000084
的直觉模糊隶属度
Figure PCTCN2019072683-appb-000085
来构建关联代价矩阵S：
Figure PCTCN2019072683-appb-000086
i∈{1,2,…,c t-1},k∈{1,2,…,n t}
本文中τ a设置为τ a＝0.6。利用得到的关联代价矩阵S，依照最大关联度对视频帧的目标对象和观测对象进行关联，其具体步骤如下：
a)从关联代价矩阵S找到最大的隶属度值
Figure PCTCN2019072683-appb-000087
当s pq≥τ a时，当将其所在的行和列对应的目标对象p和观测对象q关联，同时将该行该列的其他元素置0。
b)重复前一步骤，直到s pq<τ a时，找到所有满足条件的关联对后，停止关联。
步骤七、目标对象的状态更新。
若目标对象关联成功，利用关联观测{(o i,z k)}对目标模型进行更新，同时利用Kalman滤波方法对目标下一时刻的状态进行预测更新。
对于未关联上的观测对象，若所述观测对象没有与所述目标对象关联，则初始化一个临时目标轨迹，当所述临时目标轨迹出现的次数大于或等于τ init次时，所述临时目标轨迹变成稳定目标轨迹；当所述临时目标轨迹出现的次数小于τ init次时，删除所述临时目标轨迹；所述τ init为目标起始阈值；
对于未关联上的目标对象，若所述目标对象没有与所述观测对象关联，则判断所述目标对象的轨迹的质心坐标是否在边界处，当所示质心坐标与视频边界的距离小于3个像素坐标距离时，判定所述目标对象已经在视频中离开，否则利用卡尔曼滤波器对所述目标对象状态进行预测，当所述目标对象连续τ term次没有与所述观测对象关联时，将其从当前目标轨迹中删除，所述τ term为目标终止阈值。
实施例三
请参阅图2，为本申请实施例提供一种电子装置。该电子装置可用于实现上述图1-a所示实施例提供的视频目标的多目标跟踪方法。如图2所示，该电子装置主要包括：
获取单元210，用于获取分别目标对象的第一图像特征和观测对象的第二图像特征；
相似度计算单元220，用于对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果，所述N为大于1的整数；
特征融合单元230，用于基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；
关联计算单元240，用于基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。
需要说明的是，以上图2示例的电子装置的实施方式中，各功能模块的划分仅是举例说明，实际应用中可以根据需要，例如相应硬件的配置要求或者软件的实现的便利考虑，而将上述功能分配由不同的 功能模块完成，即将电子装置的内部结构划分成不同的功能模块，以完成以上描述的全部或者部分功能。而且，在实际应用中，本实施例中的相应的功能模块可以是由相应的硬件实现，也可以由相应的硬件执行相应的软件完成。本说明书提供的各个实施例都可应用上述描述原则，以下不再赘述。
本实施例提供的电子装置中各功能模块实现各自功能的具体过程，请参见上述图1-a所示实施例中描述的具体内容，此处不再赘述。
实施例四
本申请实施例提供一种电子装置，请参阅图3，该电子装置包括：
存储器301、处理器302及存储在存储器301上并可在处理器302上运行的计算机程序，处理器302执行该计算机程序时，实现前述图1-a所示实施例中描述的视频目标的多目标跟踪方法。
进一步的，该电子装置还包括：
至少一个输入设备303以及至少一个输出设备304。
上述存储器301、处理器302、输入设备303以及输出设备304，通过总线305连接。
其中，输入设备303具体可为摄像头、触控面板、物理按键或者鼠标等等。输出设备304具体可为显示屏。
存储器301可以是高速随机存取记忆体(RAM，Random Access Memory)存储器，也可为非不稳定的存储器(non-volatile memory)，例如磁盘存储器。存储器301用于存储一组可执行程序代码，处理器302与存储器301耦合。
进一步的，本申请实施例还提供了一种计算机可读存储介质，该计算机可读存储介质可以是设置于上述各实施例中的电子装置中，该计算机可读存储介质可以是前述图3所示实施例中的存储器。该计算机可读存储介质上存储有计算机程序，该程序被处理器执行时实现前述图1-a所示实施例中描述的视频目标的多目标跟踪方法。进一步的，该计算机可存储介质还可以是U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、RAM、磁碟或者光盘等各种可以存储程序代码的介质。
以上为对本申请所提供的视频目标的多目标跟踪方法、装置及存储介质的描述，对于本领域的技术人员，依据本申请实施例的思想，在具体实施方式及应用范围上均会有改变之处，综上，本说明书内容不应理解为对本申请的限制。

Claims (11)

  1. 一种视频目标的多目标跟踪方法，其特征在于，包括：
    获取分别目标对象的第一图像特征和观测对象的第二图像特征；
    对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果，所述N为大于1的整数；
    基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；
    基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；
    根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。
  2. 根据权利要求1所述的方法，其特征在于，所述观测对象有j个，所述j为大于1的整数；
    所述对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，包括：计算所述第一图像特征分别与j个所述第二图像特征的特征相似度。
  3. 根据权利要求1所述的方法，其特征在于，
    所述N等于7，7种特征类别分别为：距离、颜色、边缘、文理、形状、运动方向和重叠面积。
  4. 根据权利要求3所述的方法，其特征在于，
    所述7种特征类别对应的相似性度量函数，包括：
    Figure PCTCN2019072683-appb-100001
    Figure PCTCN2019072683-appb-100002
    Figure PCTCN2019072683-appb-100003
    Figure PCTCN2019072683-appb-100004
    Figure PCTCN2019072683-appb-100005
    Figure PCTCN2019072683-appb-100006
    Figure PCTCN2019072683-appb-100007
    其中，f 1(o i,z k)为空间距离特征相似性度量函数，f 2(o i,z k)为几何 尺寸特征相似性度量函数，f 3(o i,z k)为运动方向特征相似性度量函数，f 4(o i,z k)为颜色特征相似性度量函数，f 5(o i,z k)为梯度方向特征相似性度量函数，f 6(o i,z k)为文理特征相似性度量函数，f 7(o i,z k)为重叠面积相似性度量函数；(x o,y o)为目标对象o i的中心坐标，(x z,y z)为观测对象z k的中心坐标，h o为目标对象o i的图像高度，
    Figure PCTCN2019072683-appb-100008
    为空间距离方差常量，h z为观测对象z k的图像高度，
    Figure PCTCN2019072683-appb-100009
    为几何尺寸方差常量，(x' o,y' o)为上一时刻目标对象o i的中心坐标，
    Figure PCTCN2019072683-appb-100010
    为上一时刻目标对象o i的速度在图像坐标轴上的投影，
    Figure PCTCN2019072683-appb-100011
    为运动方向方差常量，ρ(·)表示求巴氏系数，H r(·)表示颜色直方图，
    Figure PCTCN2019072683-appb-100012
    为颜色直方图方差常量，H g(·)表示分块梯度方向直方图特征，
    Figure PCTCN2019072683-appb-100013
    为梯度方向方差常量，H l(·)表示文理特征直方图，
    Figure PCTCN2019072683-appb-100014
    为纹理特征方差常量，w(o i,z k)表示目标对象o i与观测对象z k之间的遮挡度，
    Figure PCTCN2019072683-appb-100015
    为重叠面积方差常量。
  5. 根据权利要求4所述的方法，其特征在于，所述基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，包括：
    计算目标对象o i和观测对象z k对于特征f n的相似度，计算结果作为论域U n；所述特征f n为所述N种特征类别的任意一种特征；
    利用公式
    Figure PCTCN2019072683-appb-100016
    计算特征f n的决策属性对论域U n的依赖度γ n；所述
    Figure PCTCN2019072683-appb-100017
    为所有特征f n的相似度大于τ cf的观测集合，其中，
    Figure PCTCN2019072683-appb-100018
    当特征f n对应的依赖度γ n大于或等于相似度阈值τ cf时，则所述特征f n为有效特征，可参与后续的融合。
  6. 根据权利要求5所述的方法，其特征在于，
    所述将筛选后的结果进行融合，包括：
    根据公式
    Figure PCTCN2019072683-appb-100019
    设置相应特征类别的权值；
    根据公式
    Figure PCTCN2019072683-appb-100020
    对设置权值后的所述特征相似度结果进行融合；
    其中，α i为第i种特征的权值，f i(o i,z k)为第i种特征的相似度，s(o i,z k)为特征融合结果。
  7. 根据权利要求6所述的方法，其特征在于，
    所述基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果，包括：
    根据所述特征融合结果，计算目标对象和观测对象之间的参考拓 扑集距离；
    根据所述目标对象和观测对象之间的参考拓扑集距离，计算目标对象与观测对象的直觉模糊隶属度；
    根据所述目标对象与观测对象的直觉模糊隶属度，构建关联代价矩阵对目标对象进行关联，得到关联代价矩阵计算结果。
  8. 根据权利要求6所述的方法，其特征在于，
    所述对所述目标对象进行目标轨迹管理，包括：
    对于未关联上的观测对象，若所述观测对象没有与所述目标对象关联，则初始化一个临时目标轨迹，当所述临时目标轨迹出现的次数大于或等于τ init次时，所述临时目标轨迹变成稳定目标轨迹；当所述临时目标轨迹出现的次数小于τ init次时，删除所述临时目标轨迹；所述τ init为目标起始阈值；
    对于未关联上的目标对象，若所述目标对象没有与所述观测对象关联，则判断所述目标对象的轨迹的质心坐标是否在边界处，当所示质心坐标与视频边界的距离小于3个像素坐标距离时，判定所述目标对象已经在视频中离开，否则利用卡尔曼滤波器对所述目标对象状态进行预测，当所述目标对象连续τ term次没有与所述观测对象关联时，将其从当前目标轨迹中删除，所述τ term为目标终止阈值。
  9. 一种电子装置，其特征在于，包括：
    获取单元，用于获取分别目标对象的第一图像特征和观测对象的第二图像特征；
    相似度计算单元，用于对所述第一图像特征和所述第二图像特征进行N种特征类别的特征相似度计算，得到N组特征相似度结果，所述N为大于1的整数；
    特征融合单元，用于基于粗糙集的特征相似度对所述N组特征相似度结果进行筛选，并将筛选后的结果进行融合，得到特征融合结果；
    关联计算单元，用于基于最大熵直觉模糊聚类，根据所述特征融合结果进行关联代价矩阵计算，得到关联代价矩阵计算结果；根据所述关联代价矩阵计算结果判断所述目标对象与所述观测对象是否关联，若是，则更新目标模型；若否，则对所述目标对象进行目标轨迹管理。
  10. 一种电子装置，其特征在于，包括：存储器、处理器及存储在所述存储器上并可在所述处理器上运行的计算机程序，其特征在于，所述处理器执行所述计算机程序时，实现权利要求1至8中的任意一项所述方法。
  11. 一种计算机可读存储介质，其上存储有计算机程序，其特征在于，所述计算机程序被处理器执行时，实现权利要求1至8中的任意一项所述方法。
PCT/CN2019/072683 2019-01-22 2019-01-22 视频目标的多目标跟踪方法、装置及存储介质 Ceased WO2020150897A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/072683 WO2020150897A1 (zh) 2019-01-22 2019-01-22 视频目标的多目标跟踪方法、装置及存储介质

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2019/072683 WO2020150897A1 (zh) 2019-01-22 2019-01-22 视频目标的多目标跟踪方法、装置及存储介质

Publications (1)

Publication Number Publication Date
WO2020150897A1 true WO2020150897A1 (zh) 2020-07-30

Family

ID=71736475

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/072683 Ceased WO2020150897A1 (zh) 2019-01-22 2019-01-22 视频目标的多目标跟踪方法、装置及存储介质

Country Status (1)

Country Link
WO (1) WO2020150897A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120876914A (zh) * 2025-07-28 2025-10-31 杭州万兆科技有限公司 基于深度学习的视频监控智能预警与通讯方法及系统
CN121117621A (zh) * 2025-11-12 2025-12-12 苏州昆仑重型装备制造有限公司 一种基于数据处理的模锻齿轮力学性能检测方法

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6658399B1 (en) * 1999-09-10 2003-12-02 Intel Corporation Fuzzy based thresholding technique for image segmentation
CN107516321A (zh) * 2017-07-04 2017-12-26 深圳大学 一种视频多目标跟踪方法及装置
CN107545582A (zh) * 2017-07-04 2018-01-05 深圳大学 基于模糊逻辑的视频多目标跟踪方法及装置
CN108564111A (zh) * 2018-03-26 2018-09-21 太原理工大学 一种基于邻域粗糙集特征选择的图像分类方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6658399B1 (en) * 1999-09-10 2003-12-02 Intel Corporation Fuzzy based thresholding technique for image segmentation
CN107516321A (zh) * 2017-07-04 2017-12-26 深圳大学 一种视频多目标跟踪方法及装置
CN107545582A (zh) * 2017-07-04 2018-01-05 深圳大学 基于模糊逻辑的视频多目标跟踪方法及装置
CN108564111A (zh) * 2018-03-26 2018-09-21 太原理工大学 一种基于邻域粗糙集特征选择的图像分类方法

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN120876914A (zh) * 2025-07-28 2025-10-31 杭州万兆科技有限公司 基于深度学习的视频监控智能预警与通讯方法及系统
CN121117621A (zh) * 2025-11-12 2025-12-12 苏州昆仑重型装备制造有限公司 一种基于数据处理的模锻齿轮力学性能检测方法

Similar Documents

Publication Publication Date Title
CN109859245B (zh) 视频目标的多目标跟踪方法、装置及存储介质
WO2020150896A1 (zh) 视频目标的数据关联方法、装置及存储介质
CN114972418B (zh) 基于核自适应滤波与yolox检测结合的机动多目标跟踪方法
CN119648536B (zh) 基于多尺度点云特征的三维点云数据分割方法及系统
CN108304798B (zh) 基于深度学习及运动一致性的街面秩序事件视频检测方法
CN117949942B (zh) 基于雷达数据和视频数据融合的目标跟踪方法及系统
Hu et al. Single and multiple object tracking using log-Euclidean Riemannian subspace and block-division appearance model
CN101800890B (zh) 一种高速公路监控场景下多车辆视频跟踪方法
CN112052802B (zh) 一种基于机器视觉的前方车辆行为识别方法
CN107423686B (zh) 视频多目标模糊数据关联方法及装置
CN109829405A (zh) 视频目标的数据关联方法、装置及存储介质
CN110569782A (zh) 一种基于深度学习目标检测方法
WO2018227491A1 (zh) 视频多目标模糊数据关联方法及装置
CN109146925B (zh) 一种动态场景下显著性目标检测方法
CN119580985A (zh) 基于多模态图像分析的机器人辅助伤口治疗方法及系统
Ming et al. Monocular depth-ordering reasoning with occlusion edge detection and couple layers inference
CN107833239B (zh) 一种基于加权模型约束的寻优匹配目标跟踪方法
Huang et al. Automatic building change image quality assessment in high resolution remote sensing based on deep learning
CN106709938B (zh) 基于改进tld的多目标追踪方法
Chan et al. A 3-D-point-cloud system for human-pose estimation
CN119516161A (zh) 一种基于目标识别检测的人员状态识别方法及系统
CN120195652A (zh) 基于毫米波雷达的轻量化ai人员感知方法及系统
WO2025060970A1 (zh) 一种遥感影像变化检测方法
CN110349188A (zh) 基于tsk模糊模型的多目标跟踪方法、装置及存储介质
CN118795878A (zh) 用于双臂机器人的自主导航方法、装置及电子设备

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19911131

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19911131

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 09.11.2021)

122 Ep: pct application non-entry in european phase

Ref document number: 19911131

Country of ref document: EP

Kind code of ref document: A1