WO2021007984A1 - 基于tsk模糊分类器的目标跟踪方法、装置及存储介质 - Google Patents

基于tsk模糊分类器的目标跟踪方法、装置及存储介质 Download PDF

Info

Publication number
WO2021007984A1
WO2021007984A1 PCT/CN2019/112693 CN2019112693W WO2021007984A1 WO 2021007984 A1 WO2021007984 A1 WO 2021007984A1 CN 2019112693 W CN2019112693 W CN 2019112693W WO 2021007984 A1 WO2021007984 A1 WO 2021007984A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature
fuzzy
target
observation
tsk
Prior art date
Application number
PCT/CN2019/112693
Other languages
English (en)
French (fr)
Inventor
李良群
严明月
李小香
刘宗香
Original Assignee
深圳大学
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大学 filed Critical 深圳大学
Publication of WO2021007984A1 publication Critical patent/WO2021007984A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30241—Trajectory

Definitions

  • the present invention relates to the technical field of target tracking, in particular to a target tracking method, device and storage medium based on a TSK fuzzy classifier.
  • Multi-target tracking is to use the measurement obtained by the sensor to automatically detect the target of interest, and to continuously and accurately identify and track multiple targets.
  • Video multi-target tracking has achieved a lot of results and has been widely used in practical engineering. However, how to quickly, accurately and stably achieve multi-target tracking in a complex environment is still a challenging subject and the main research difficulty Uncertainty from the tracking process: First, during the tracking process, the target may change due to various factors, including the target's own scale change, posture change, and its own deformation.
  • the light The change of the target, the interference of the clutter, and the sudden change of the background will all affect the target, causing the target information to be uncertain and making tracking difficult;
  • the target may be affected by other objects in the video frame Occlusion, the extracted target features will be mixed into clutter interference, resulting in the loss of part or all of the target information;
  • the appearance of new targets, the disappearance of old targets, and the missed detection of targets caused by occlusion make The number of targets in each frame is unpredictable.
  • the data association method usually used is more traditional, such as nearest neighbor, joint probability data association method, network flow method, etc. These methods are hard decision methods, and the reliability decreases when the association is fuzzy.
  • the main purpose of the embodiments of the present invention is to provide a target tracking method, device and storage medium based on a TSK fuzzy classifier, which can at least solve the accuracy of associating the target and the observation when the hard decision method is used for target tracking in related technologies The problem is not high.
  • the first aspect of the embodiments of the present invention provides a target tracking method based on a TSK fuzzy classifier, the method including:
  • each feature in the feature set includes motion features and directional gradient HOG features
  • the subsequent parameters of the TSK fuzzy classifier based on the motion feature and the j-th stable track of the HOG feature are trained, and are based on the trained The obtained subsequent parameters construct the corresponding TSK fuzzy classifier;
  • a target tracking device based on a TSK fuzzy classifier the device including:
  • the extraction module is used to extract all feature sets of m stable tracks, and construct a multi-output regression data set for the feature set; wherein, each feature in the feature set includes a motion feature and a directional gradient HOG feature;
  • the calculation module is used to divide different targets into different fuzzy sets, and calculate the fuzzy membership degree of each feature in the feature set relative to the k'th fuzzy rule;
  • the classification module is used to detect the moving target in the image to obtain an observation set, and input the observation set to the TSK fuzzy classifier to obtain a label vector matrix;
  • the association module is used to perform data association on the label vector matrix and determine the association pairs between all observation objects and target objects;
  • the management module is used for track management based on the data association result.
  • a third aspect of the embodiments of the present invention provides an electronic device, which includes: a processor, a memory, and a communication bus;
  • the communication bus is used to implement connection and communication between the processor and the memory
  • the processor is configured to execute one or more programs stored in the memory to implement the steps of any of the above-mentioned target tracking methods based on the TSK fuzzy classifier.
  • a fourth aspect of the embodiments of the present invention provides a computer-readable storage medium, the computer-readable storage medium stores one or more programs, and the one or more programs can be processed by one or more The device executes to implement the steps of any of the above-mentioned target tracking methods based on the TSK fuzzy classifier.
  • a multi-output regression data set is constructed for the feature set of the stable track, and the fuzzy membership of each feature in the feature set relative to the fuzzy rule is calculated Degree; then based on the multi-output regression data set and fuzzy membership to train the subsequent parameters of the TSK fuzzy classifier based on motion features and HOG features, and construct the corresponding TSK fuzzy classifier; then input the observation set to the TSK fuzzy classifier Obtain the label vector matrix, and perform data association on the label vector matrix to obtain the correct correlation between the target and the observation; finally, filter and track the target to obtain the final trajectory of the target.
  • a TSK fuzzy classifier is trained using multi-frame information, and a multi-feature learning mechanism is added in the training process, which increases the learning ability of the classifier, and can effectively deal with the uncertainty in the data association process and improve Accuracy of target tracking.
  • FIG. 1 is a schematic flowchart of a target tracking method provided by the first embodiment of the present invention
  • FIG. 2 is a schematic diagram of observations output in a real scene provided by the first embodiment of the present invention.
  • FIG. 3 is a schematic diagram of the occlusion between the target and the observation provided by the first embodiment of the present invention
  • FIG. 4 is a schematic flowchart of a track management method provided by the first embodiment of the present invention.
  • FIG. 5 is a schematic structural diagram of a target tracking device provided by a second embodiment of the present invention.
  • FIG. 6 is a schematic structural diagram of an electronic device provided by a third embodiment of the present invention.
  • this embodiment proposes a target tracking method based on the TSK fuzzy classifier, as shown in Figure 1. Shown as a schematic diagram of the basic flow of the target tracking method provided in this embodiment, the target tracking method proposed in this embodiment includes the following steps:
  • Step 101 Extract all feature sets of m stable tracks, and construct a multi-output regression data set for the feature sets; wherein, each feature in the feature set includes a motion feature and a HOG feature.
  • dual features including motion features and Histogram of Oriented Gradient (HOG) features are used to describe the target in the TSK fuzzy classifier, so as to obtain a classifier model with better performance.
  • HOG Histogram of Oriented Gradient
  • Step 102 Divide different targets into different fuzzy sets, and calculate the fuzzy membership degree of each feature in the feature set with respect to the k'th fuzzy rule.
  • the FCM clustering algorithm is used to identify the antecedent parameters
  • the number of rules of the TSK fuzzy classifier is set to K'
  • the number of input samples is l'
  • the class number is K', and the fuzzy partition matrices S′ 1 and S′ 2 can be obtained.
  • (x', z') is the motion feature
  • ho is the HOG feature
  • Motion feature center vector And the HOG feature center vector Both are the center vector of the k'th rule obtained by the FCM algorithm on the training sample. The calculation process is as follows:
  • h' is a scalar, which can be set manually or determined by some learning strategy.
  • Step 103 Based on the multi-output regression data set and the fuzzy membership, train the subsequent parameters of the TSK fuzzy classifier based on the motion feature and the HOG feature of the j-th stable track, and respectively based on the trained subsequent parameters Construct the corresponding TSK fuzzy classifier.
  • a TSK fuzzy classifier model is trained using multiple features, and a multi-feature learning mechanism is incorporated in the training process, so that each feature is The classification results are as consistent as possible.
  • This method can not only use the independent information of each feature, but also comprehensively consider the associated information between each feature.
  • the TSK fuzzy classifier obtained by this algorithm can better achieve the goal and observation. Data association.
  • the ridge regression model is used to train the TSK fuzzy classifier, let:
  • the final optimization result of the TSK classifier based only on the j-th stable track based on the motion feature is:
  • constructing corresponding TSK fuzzy classifiers based on the trained subsequent parameters respectively includes: performing multi-feature learning on the trained subsequent parameters; based on multi-feature learning The subsequent parameters construct the corresponding TSK fuzzy classifiers respectively.
  • the multi-feature learning mechanism of this embodiment is as follows:
  • f() is the output value of the consequent parameter trained according to a single feature, It is the output value of the subsequent parameter trained after adding multi-feature learning.
  • TSK fuzzy classifier based on motion features
  • the IF part is the antecedent of the rule
  • the THEN part is the subsequent part of the rule
  • K' is the number of fuzzy rules.
  • the output of the j-th TSK fuzzy classifier based on motion features is:
  • TSK fuzzy classifier based on HOG features
  • the IF part is the antecedent of the rule
  • the THEN part is the subsequent part of the rule
  • K' is the number of fuzzy rules. Is the fuzzy subset corresponding to the input variable ho of the k-th rule, and is the fuzzy connection operator, and f k′ (u) is the output result of each fuzzy rule.
  • the output of the j-th TSK fuzzy classifier based on HOG features is:
  • Step 104 Detect the moving target in the image to obtain an observation set, and input the observation set to the TSK fuzzy classifier to obtain a label vector matrix.
  • each target with a stable track has a TSK fuzzy classifier model based on two features.
  • Each model can be identified and trained.
  • For a test observation sample its motion features and HOG features are extracted and input into
  • the output matrix can be expressed as:
  • a mixed Gaussian background model may be used to detect moving targets.
  • the Gaussian background model treats all the gray values of a pixel in the video as a random process, and uses Gaussian distribution to describe the probability density function of the pixel value.
  • the definition I (x, y, t) represents the pixel value of the pixel (x, y) at time t, then:
  • ⁇ is the Gaussian probability density function
  • ⁇ t and ⁇ t are the mean value and standard deviation of the pixel (x, y) at time t , respectively.
  • N represents the number of image frames of the video
  • ⁇ 0 (x, y) is the average gray value of the pixel with coordinates (x, y)
  • ⁇ 0 (x, y) is the gray value of the pixel (x, y) Worth the variance.
  • the gray value I(x, y, t) of the pixel (x, y) is determined according to the following formula, and o represents the output image:
  • T p is the probability threshold.
  • an equivalent threshold is usually used instead of the probability threshold.
  • I (x, y, t) is determined as the background pixel point
  • I (x, y, t) is determined as Foreground pixels.
  • ⁇ t (x,y) (1- ⁇ ) ⁇ t (x,y)+ ⁇ I(x,y,t)
  • ⁇ is called the learning factor, which reflects the speed of the change of the background information in the video. If the value of ⁇ is too small, the change of the background model will be slower than the change of the actual real scene, which will result in many holes in the detected target, and vice versa. Make the slow moving foreground become part of the background.
  • I (x, y, t) represents the pixel value of the pixel (x, y) at time t
  • ⁇ represents the Gaussian probability density function
  • ⁇ t and ⁇ t respectively represent the pixel point (x, y) at time t
  • T P represents the probability threshold
  • all pixels in the image can be divided into foreground pixels and background pixels, and then a binary image containing the foreground and background is obtained, and the moving pixels in the image are detected, supplemented by median filtering And simple morphological processing, finally get the moving target in the image, and then compose an observation set based on the detected moving target.
  • Step 105 Perform data association on the label vector matrix, and determine the association pairs between all observation objects and target objects.
  • input N observation sets and pass the classifier to get an m ⁇ 2N output matrix
  • the greedy algorithm can be used to analyze and process the matrix to obtain the correct correlation pair between the target and the observation.
  • Step 106 Perform track management based on the data association result.
  • Fig. 2 is a schematic diagram of the observation output in the real scene provided by this embodiment, in which the white rectangular frame represents the target state at the current moment, and the black rectangular frame represents the false observation. It can be seen from Figure 2 that there is a clear occlusion between these false observations and the target. After the fuzzy data is associated, these false observations will become uncorrelated observations, and the observations corresponding to the new target will have a lower degree of fuzzy membership to the currently recorded target, and they will also become uncorrelated observations.
  • this embodiment proposes to use space-time clues to analyze the occlusion situation between unrelated observations and the current target, so as to determine the observation corresponding to the new target and start a new target trajectory for it.
  • Figure 3 is a schematic diagram of the occlusion between the target and the observation provided in this embodiment.
  • the occlusion degree ⁇ is defined in this article. Assume that the target object A and the unrelated observation object B are occluded as shown in Figure 4, where the overlapped shadow between the rectangular frame A and the rectangular frame B represents the occlusion area, and defines the occlusion degree ⁇ between A and B (A,B) is:
  • r( ⁇ ) represents the area of the region
  • ⁇ (A,B) represents the degree of occlusion between A and B
  • a and B will occur The occlusion.
  • the value of the longitudinal coordinate value of the image at the bottom of the rectangular frame B y A longitudinal rectangular base frame image coordinates Y A B further that, if y A> y B, then the B is A blocked.
  • the new target discriminant function ⁇ is expressed as follows:
  • ⁇ is Constant parameter
  • FIG. 4 is a schematic flowchart of the trajectory management method provided in this embodiment, which specifically includes the following steps:
  • Step 401 Determine the observation object corresponding to the new target object from the observation objects that have never been associated;
  • Step 402 Establish a new temporary trajectory for the observation object corresponding to each new target object, and determine whether the temporary trajectory is continuously associated with a preset number of frames;
  • Step 403 Convert the temporary trajectory into a valid target trajectory when the continuous preset frames of the temporary trajectory are all associated.
  • Step 404 Use a Kalman filter to filter and predict each temporary trajectory and the effective target trajectory.
  • this embodiment combines the new target discriminant function and adopts target trajectory management rules to solve the smoothing and prediction of valid target trajectories, the termination of invalid target trajectories, and the start of new target trajectories.
  • the target trajectory management rules adopted specifically include:
  • the Kalman filter is used to update the target trajectory; for the unassociated observation, the target trajectory management rule (1 ) Create a new target trajectory and update the target trajectory label; for unrelated targets, delete the target trajectory label and status according to the target trajectory management rule (4); finally, predict and update all target trajectories according to the trajectory management rule (3) .
  • a multi-output regression data set is constructed for the feature set of the stable track, and the fuzzy membership degree of each feature in the feature set relative to the fuzzy rule is calculated;
  • the output regression data set and fuzzy membership training are based on the subsequent parameters of the TSK fuzzy classifier of the motion feature and the HOG feature respectively, and the corresponding TSK fuzzy classifier is constructed; then the observation set is input to the TSK fuzzy classifier to obtain the label vector matrix, And perform data association on the label vector matrix to get the correct association between the target and the observation; finally, filter and track the target to get the final trajectory of the target.
  • a TSK fuzzy classifier is trained using multi-frame information, and a multi-feature learning mechanism is added in the training process, which increases the learning ability of the classifier, and can effectively deal with the uncertainty in the data association process and improve Accuracy of target tracking.
  • this embodiment proposes a target tracking device based on the TSK fuzzy classifier. Please refer to the figure for details.
  • the target tracking device shown in 5, the target tracking device of this embodiment includes:
  • the extraction module 501 is used to extract all feature sets of m stable tracks, and construct a multi-output regression data set for the feature sets; wherein, each feature in the feature set includes motion features and HOG features;
  • the calculation module 502 is used to divide different targets into different fuzzy sets, and calculate the fuzzy membership degree of each feature in the feature set relative to the k'th fuzzy rule;
  • the construction module 503 is used to train the subsequent parameters of the TSK fuzzy classifier based on the motion feature and the HOG feature of the j-th stable track based on the multi-output regression data set and the fuzzy membership, and respectively based on the training obtained
  • the subsequent parameters construct the corresponding TSK fuzzy classifier
  • the classification module 504 is configured to detect the moving target in the image to obtain an observation set, and input the observation set to the TSK fuzzy classifier to obtain a label vector matrix;
  • the association module 505 is used to perform data association on the label vector matrix and determine the association pairs between all observation objects and target objects;
  • the management module 506 is configured to perform track management based on the data association result.
  • the calculation module 502 is specifically configured to divide different targets into different fuzzy sets, and calculate the fuzzy membership degree of each feature in the feature set with respect to the k'th fuzzy rule through a preset Gaussian membership function ;
  • Gaussian membership functions are expressed as follows:
  • (x', z') is the motion feature
  • ho is the HOG feature
  • the construction module 503 when the construction module 503 constructs the corresponding TSK fuzzy classifier based on the trained consequent parameters, it is specifically used to perform multi-feature learning on the trained subsequent parameters; The learned subsequent parameters construct the corresponding TSK fuzzy classifiers respectively.
  • the construction module 503 when the construction module 503 respectively constructs corresponding TSK fuzzy classifiers based on the subsequent parameters after multi-feature learning, it is specifically used to:
  • the IF part is the antecedent of the rule
  • the THEN part is the subsequent part of the rule
  • K' is the number of fuzzy rules.
  • the IF part is the antecedent of the rule
  • the THEN part is the subsequent part of the rule
  • K' is the number of fuzzy rules. Is the fuzzy subset corresponding to the input variable ho of the k-th rule, and is the fuzzy connection operator, and f k′ (u) is the output result of each fuzzy rule.
  • the classification module 504 when the classification module 504 detects the moving target in the image to obtain the observation set, it is specifically configured to divide all pixels in the image into foreground pixels and background pixels through the Gaussian mixture background model. Obtain a binary image containing the foreground and background; detect the moving pixels in the binary image, and perform median filtering and morphological processing to determine the moving target; form an observation set based on the detected moving target.
  • the mixed Gaussian background model is expressed as follows:
  • I (x, y, t) represents the pixel value of the pixel point (x, y) at time t
  • ⁇ represents the Gaussian probability density function
  • ⁇ t and ⁇ t represent the pixel point (x, y) at time t.
  • Mean and standard deviation k is the number of Gaussian distribution components
  • w i is the weight of the i-th Gaussian distribution ⁇ i (I, ⁇ t , ⁇ t )
  • o represents the output image
  • T P represents the probability threshold.
  • I (x, y, t) When the probability is greater than When it is equal to the probability threshold, I (x, y, t) is determined as the background pixel, and when the probability is less than the probability threshold, I (x, y, t) is determined as the foreground pixel.
  • the management module 606 is specifically configured to determine the observation object corresponding to the new target object from the observation objects that have not been associated; establish a new temporary trajectory for the observation object corresponding to each new target object , And determine whether the temporary trajectory has been associated with the consecutive preset frames; when the temporary trajectory has been associated with the consecutive preset frames, the temporary trajectory is converted into a valid target trajectory; the Kalman filter is used for each temporary track and Effective target trajectory is filtered and predicted.
  • the management module 606 when the management module 606 determines the observation object corresponding to the new target object from the observation objects that have not been associated, it is specifically configured to use a preset occlusion degree calculation formula to calculate The occlusion degree between the unrelated observation object and the target object; the calculated occlusion degree is substituted into the preset new target discrimination function to determine the observation object corresponding to the new target object.
  • the occlusion calculation formula is as follows:
  • A represents the target object
  • B represents the observation object
  • r( ⁇ ) represents the area of the region
  • ⁇ (A,B) represents the degree of occlusion between A and B, and 0 ⁇ 1, when ⁇ (A,B )>0, A and B are blocked;
  • the new target discriminant function is expressed as follows:
  • ⁇ is a constant parameter
  • the target tracking method in the foregoing embodiment can be implemented based on the target tracking device provided in this embodiment, and those of ordinary skill in the art can clearly understand that for the convenience and conciseness of the description, the description in this embodiment For the specific working process of the target tracking device described, reference may be made to the corresponding process in the foregoing method embodiment, which will not be repeated here.
  • a multi-output regression data set is constructed for the feature set of the stable track, and the fuzzy membership degree of each feature in the feature set relative to the fuzzy rule is calculated; then based on the multi-output Regression data set and fuzzy membership training are based on the subsequent parameters of the TSK fuzzy classifier of motion features and HOG features respectively, and construct the corresponding TSK fuzzy classifier; then input the observation set to the TSK fuzzy classifier to obtain the label vector matrix, and Perform data association on the label vector matrix to get the correct association between the target and the observation; finally, filter and track the target to get the final trajectory of the target.
  • a TSK fuzzy classifier is trained using multi-frame information, and a multi-feature learning mechanism is added in the training process, which increases the learning ability of the classifier, and can effectively deal with the uncertainty in the data association process and improve Accuracy of target tracking.
  • This embodiment provides an electronic device. As shown in FIG. 6, it includes a processor 601, a memory 602, and a communication bus 603.
  • the communication bus 603 is used to implement connection and communication between the processor 601 and the memory 602; processing
  • the device 601 is configured to execute one or more computer programs stored in the memory 602 to implement at least one step of the method in the first embodiment.
  • This embodiment also provides a computer-readable storage medium, which is included in any method or technology for storing information (such as computer-readable instructions, data structures, computer program modules, or other data). Volatile or non-volatile, removable or non-removable media.
  • Computer-readable storage media include but are not limited to RAM (Random Access Memory), ROM (Read-Only Memory, read-only memory), EEPROM (Electrically Erasable Programmable read only memory, charged Erasable Programmable Read-Only Memory) ), flash memory or other storage technology, CD-ROM (Compact Disc Read-Only Memory), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, Or any other medium that can be used to store desired information and can be accessed by a computer.
  • the computer-readable storage medium in this embodiment may be used to store one or more computer programs, and the stored one or more computer programs may be executed by a processor to implement at least one step of the method in the first embodiment.
  • This embodiment also provides a computer program, which can be distributed on a computer-readable medium and executed by a computer-readable device to implement at least one step of the method in the first embodiment; and in some cases At least one of the steps shown or described can be performed in a different order from the order described in the foregoing embodiment.
  • This embodiment also provides a computer program product, including a computer readable device, and the computer readable device stores the computer program as shown above.
  • the computer-readable device in this embodiment may include the computer-readable storage medium as shown above.
  • communication media usually contain computer-readable instructions, data structures, computer program modules, or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium. Therefore, the present invention is not limited to any specific combination of hardware and software.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Multimedia (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Analysis (AREA)

Abstract

根据本发明实施例公开的一种基于TSK模糊分类器的目标跟踪方法、装置及存储介质，首先对稳定航迹的特征集合构建多输出回归数据集，并计算各特征相对模糊规则的模糊隶属度；然后基于多输出回归数据集及模糊隶属度训练分别基于运动特征和HOG特征的TSK模糊分类器的后件参数，并构建对应的分类器；再将观测集输入至分类器得到标签向量矩阵，并对标签向量矩阵进行数据关联得到目标和观测的正确关联；最后对目标进行滤波和轨迹管理得到目标的最终轨迹。通过本发明的实施，利用多帧信息训练出TSK模糊分类器，并在训练的过程中加入多特征学习机制，增加了分类器的学习能力，可有效处理数据关联过程中的不确定性，提高目标跟踪的准确性。

Description

基于TSK模糊分类器的目标跟踪方法、装置及存储介质 技术领域
本发明涉及目标跟踪技术领域，尤其涉及一种基于TSK模糊分类器的目标跟踪方法、装置及存储介质。
背景技术
多目标跟踪是利用传感器所获得的量测，自动地检测出感兴趣的目标，并且对多个目标进行持续和准确的识别、跟踪。
视频多目标跟踪已经取得了很多成果，在实际工程中也得到了广泛的应用，然而如何在复杂的环境下快速准确稳定的实现多目标跟踪，仍是一个具有挑战性的课题，主要的研究难点来自于跟踪过程中的不确定性：其一，在跟踪过程中，目标可能会因为各种因素发生变化，包括目标本身的尺度变化、姿势变化、自身的形变等，同时在复杂环境下，光照的变化、杂波的干扰、背景的突变都会对目标产生影响，造成目标信息具有不确定性，给跟踪带来困难；其二，在目标跟踪过程中，目标可能会被视频帧中的其他物体所遮挡，提取到的目标特征会混入杂波干扰，导致目标部分或者全部信息丢失；另外，在真实的视频帧中，新目标的出现、旧目标的消失、以及遮挡导致的目标漏检，使得每一帧的目标数目都是无法预测得到的。这些不确定性因素是导致多目标数据关联模糊的基本原因。
而在实际应用中，通常所采用的数据关联法较为传统，如最近邻、联合概率数据关联法、网络流法等，这类方法均为硬判决方法，在关联出现模糊时可靠性下降。
技术问题
本发明实施例的主要目的在于提供一种基于TSK模糊分类器的目标跟踪方法、装置及存储介质，至少能够解决相关技术中采用硬判决方法进行目标跟踪时，对目标与观测进行关联的准确性不高的问题。
技术解决方案
为实现上述目的，本发明实施例第一方面提供了一种基于TSK模 糊分类器的目标跟踪方法，该方法包括：
提取m条稳定航迹的所有特征集合，并对所述特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及方向梯度HOG特征；
将不同目标划分到不同模糊集，计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度；
基于所述多输出回归数据集以及所述模糊隶属度，训练出分别基于所述运动特征以及所述HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器；
对图像中的运动目标进行检测得到观测集，并将所述观测集输入至所述TSK模糊分类器，得到标签向量矩阵；
对所述标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对；
基于数据关联结果进行轨迹管理。
为实现上述目的，本发明实施例第二方面提供了一种基于TSK模糊分类器的目标跟踪装置，该装置包括：
提取模块，用于提取m条稳定航迹的所有特征集合，并对所述特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及方向梯度HOG特征；
计算模块，用于将不同目标划分到不同模糊集，计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度；
构建模块，用于基于所述多输出回归数据集以及所述模糊隶属度，训练出分别基于所述运动特征以及所述HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器；
分类模块，用于对图像中的运动目标进行检测得到观测集，并将所述观测集输入至所述TSK模糊分类器，得到标签向量矩阵；
关联模块，用于对所述标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对；
管理模块，用于基于数据关联结果进行轨迹管理。
为实现上述目的，本发明实施例第三方面提供了一种电子装置，该电子装置包括：处理器、存储器和通信总线；
所述通信总线用于实现所述处理器和存储器之间的连接通信；
所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现上述任意一种基于TSK模糊分类器的目标跟踪方法的步骤。
为实现上述目的，本发明实施例第四方面提供了一种计算机可读存储介质，该计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现上述任意一种基于TSK模糊分类器的目标跟踪方法的步骤。
有益效果
根据本发明实施例提供的基于TSK模糊分类器的目标跟踪方法、装置及存储介质，首先对稳定航迹的特征集合构建多输出回归数据集，并计算特征集合中各特征相对模糊规则的模糊隶属度；然后基于多输出回归数据集及模糊隶属度训练分别基于运动特征和HOG特征的TSK模糊分类器的后件参数，并构建对应的TSK模糊分类器；再将观测集输入至TSK模糊分类器得到标签向量矩阵，并对标签向量矩阵进行数据关联得到目标和观测的正确关联；最后对目标进行滤波和轨迹管理得到目标的最终轨迹。通过本发明的实施，利用多帧信息训练出TSK模糊分类器，并且在训练的过程中加入多特征学习机制，增加了分类器的学习能力，可有效处理数据关联过程中的不确定性，提高目标跟踪的准确性。
本发明其他特征和相应的效果在说明书的后面部分进行阐述说明，且应当理解，至少部分效果从本发明说明书中的记载变的显而易见。
附图说明
为了更清楚地说明本发明实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本发明的一些实施例，对于本领域 技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他的附图。
图1为本发明第一实施例提供的目标跟踪方法的流程示意图；
图2为本发明第一实施例提供的真实场景中所输出的观测示意图；
图3为本发明第一实施例提供的目标与观测之间的遮挡示意图；
图4为本发明第一实施例提供的轨迹管理方法的流程示意图；
图5为本发明第二实施例提供的目标跟踪装置的结构示意图；
图6为本发明第三实施例提供的电子装置的结构示意图。
本发明的实施方式
为使得本发明的发明目的、特征、优点能够更加的明显和易懂，下面将结合本发明实施例中的附图，对本发明实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本发明一部分实施例，而非全部实施例。基于本发明中的实施例，本领域技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本发明保护的范围。
第一实施例：
为了解决相关技术中采用硬判决方法进行目标跟踪时，对目标与观测进行关联的准确性不高的技术问题，本实施例提出了一种基于TSK模糊分类器的目标跟踪方法，如图1所示为本实施例提供的目标跟踪方法的基本流程示意图，本实施例提出的目标跟踪方法包括以下的步骤：
步骤101、提取m条稳定航迹的所有特征集合，并对特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及HOG特征。
具体的，本实施例中使用包括运动特征及方向梯度(HOG，Histogram of Oriented Gradient)特征的双特征在TSK模糊分类器中对目标进行描述，以得到性能更好的分类器模型。
在本实施例中，若当前帧中稳定航迹个数m≥1，即出现了稳定航迹，m条稳定航迹的所有特征集合U′＝{u′ 1,u′ 2,…,u′ m}，其中，u′ j为前T-1个时刻，第j条稳定航迹的运动特征以及HOG特征集合：u′ j＝{(x′ j,t, ′z′ j,t),(ho j,t)}，t＝1,2,…,T-1，(x′ t,z′ t)为t时刻目标矩形框的中心坐标，ho t为t时刻该目标的HOG特征；对于包含m个类的数据{u′ j,y el}，y el∈{1,2,…,m}，本实施例构建一个多输出回归数据集
Figure PCTCN2019112693-appb-000001
若{u′ j,y el}原始类标签y el＝r(1≤r≤m)在构造的多输出回归数据集
Figure PCTCN2019112693-appb-000002
y el∈{1,2,…,m}中，包含m个输出的相应输出向量定义为：
Figure PCTCN2019112693-appb-000003
在此输出向量中，只有
Figure PCTCN2019112693-appb-000004
的第r个元素是1，而其余元素被设置为-1，表明该目标属于第r条稳定航迹。
步骤102、将不同目标划分到不同模糊集，计算特征集合中各特征相对第k’个模糊规则的模糊隶属度。
在本实施例中，采用FCM聚类算法进行前件参数辨识，TSK模糊分类器的规则数设定为K’，输入为U′＝{u′ 1,u′ 2,…,u′ m}，其中，u′ j＝{(x′ j,t,z′ j,t),(ho j,t)}，t＝1,2,…,T-1，输入样本数为l’，聚类数为K’，可以得到模糊划分矩阵S′ 1、S′ 2，矩阵S′ 1的元素S′ 1w′k′∈[0,1]表示基于运动特征的第，w′(w′＝1,2,…,l’)个输入样本到第k′(k′＝1,2,…,K’)个规则的隶属度，模糊集
Figure PCTCN2019112693-appb-000005
可以用以下常见的高斯隶属函数表示：
Figure PCTCN2019112693-appb-000006
Figure PCTCN2019112693-appb-000007
Figure PCTCN2019112693-appb-000008
Figure PCTCN2019112693-appb-000009
Figure PCTCN2019112693-appb-000010
其中，(x’，z’)为运动特征，ho为HOG特征。 运动特征中心向量
Figure PCTCN2019112693-appb-000011
以及HOG特征中心向量
Figure PCTCN2019112693-appb-000012
都是通过FCM算法对训练样本获得的第k’个规则的中心向量，计算过程如下所示：
Figure PCTCN2019112693-appb-000013
Figure PCTCN2019112693-appb-000014
Figure PCTCN2019112693-appb-000015
Figure PCTCN2019112693-appb-000016
Figure PCTCN2019112693-appb-000017
Figure PCTCN2019112693-appb-000018
其中，h’是一个标量，可以通过手动设置或由某些学习策略确定。
步骤103、基于多输出回归数据集以及模糊隶属度，训练出分别基于运动特征以及HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器。
具体的，在本实施例中，为了更好地利用不确定性信息中的有用信息，利用多个特征训练出TSK模糊分类器模型，并且在训练过程中融入多特征学习机制，使得各个特征的分类结果尽量一致，这种方法不仅能够利用每个特征的独立信息，而且综合考虑到了各个特征之间存在的关联信息，通过该算法得到的TSK模糊分类器能够更好得实现目标与观测之间的数据关联。
在本实施例中，利用岭回归模型对TSK模糊分类器进行训练，令：
Figure PCTCN2019112693-appb-000019
Figure PCTCN2019112693-appb-000020
Figure PCTCN2019112693-appb-000021
Figure PCTCN2019112693-appb-000022
Figure PCTCN2019112693-appb-000023
仅基于运动特征的目标函数如下：
Figure PCTCN2019112693-appb-000024
仅基于HOG特征的目标函数如下：
Figure PCTCN2019112693-appb-000025
其中，
Figure PCTCN2019112693-appb-000026
是仅基于运动特征的第j个稳定航迹的TSK分类器的后件参数，
Figure PCTCN2019112693-appb-000027
是仅基于HOG特征的第j个稳定航迹的TSK分类器的后件参数，
Figure PCTCN2019112693-appb-000028
是输入变量的m维标签向量，m为稳定航迹个数。如果
Figure PCTCN2019112693-appb-000029
的第r维是1而其他维是-1，则意味着输入变量属于第r个稳定航迹。根据优化理论，可以得到仅基于运动特征的第j个稳定航迹的TSK分类器最后优化结果为：
Figure PCTCN2019112693-appb-000030
仅基于HOG特征的第j个稳定航迹的TSK分类器最后优化结果为：
Figure PCTCN2019112693-appb-000031
应当说明的是，通过上式得到的后件参数
Figure PCTCN2019112693-appb-000032
以及
Figure PCTCN2019112693-appb-000033
是仅基于运动特征以及仅基于HOG特征训练得到的后件参数，本实施例需要对其进行多特征学习，得到更具全局性的TSK模糊分类器。基于此，在本实施例一种可选的实施方式中，分别基于所训练得到的后件参数构建对应的TSK模糊分类器包括：对所训练得到后件参数进行多特征学习；基于多特征学习后的后件参数分别构建对应的TSK模糊分类器。本实施例的多特征学习机制如下：
Figure PCTCN2019112693-appb-000034
Figure PCTCN2019112693-appb-000035
式中，f()为根据单一特征训练出的后件参数的输出值，
Figure PCTCN2019112693-appb-000036
为加入多特征学习后训练出的后件参数的输出值。
加入多特征学习机制后每个特征的目标函数为：
Figure PCTCN2019112693-appb-000037
根据优化理论，我们可以得到第a个特征的第j个稳定航迹的TSK模糊分类器模型的后件参数最后的优化结果为：
Figure PCTCN2019112693-appb-000038
Figure PCTCN2019112693-appb-000039
表示经过多特征学习后，TSK模糊分类器模型的后件参数。
构建基于运动特征的TSK模糊分类器为：
Figure PCTCN2019112693-appb-000040
Figure PCTCN2019112693-appb-000041
其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
Figure PCTCN2019112693-appb-000042
分别为第k条规则的输入变量x’、z’对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果。
基于运动特征的第j个TSK模糊分类器的输出为：
Figure PCTCN2019112693-appb-000043
构建基于HOG特征的TSK模糊分类器为：
Figure PCTCN2019112693-appb-000044
其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
Figure PCTCN2019112693-appb-000045
为第k条规则的输入变量ho对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果。
基于HOG特征的第j个TSK模糊分类器的输出为：
Figure PCTCN2019112693-appb-000046
步骤104、对图像中的运动目标进行检测得到观测集，并将观测集输入至TSK模糊分类器，得到标签向量矩阵。
具体的，每个拥有稳定航迹的目标都有分别基于两个特征的TSK模糊分类器模型，每个模型都得以辨识以及训练，对于一个测试观测样本，提取出其运动特征以及HOG特征输入到上述训练好的TSK模糊分类器中，输出矩阵可以表示为：
Figure PCTCN2019112693-appb-000047
应当说明的是，在本实施例中，可以采用混合高斯背景模型对运动目标进行检测。高斯背景模型，是将一个像素点在视频中所有的灰度值看成一个随机过程，利用高斯分布描述像素点像素值的概率密度函数。
其中，定义I(x,y,t)表示像素点(x,y)在t时刻的像素值，则有：
Figure PCTCN2019112693-appb-000048
式中，η为高斯概率密度函数，μ t和σ t分别是像素点(x,y)在t时刻的均值和标准差。假设有图像序列I(x,y,0)，I(x,y,1)，…，I(x,y,N-1)，那么对于像素点(x,y)，它的初始背景模型的期望值μ 0(x,y)和偏差σ 0(x,y)分别用下述的公式计算：
Figure PCTCN2019112693-appb-000049
Figure PCTCN2019112693-appb-000050
式中，N表示视频的图像帧数，μ 0(x,y)是坐标为(x,y)的像素的平均灰度值，σ 0(x,y)是像素(x,y)灰度值得方差。在t时刻，按照下式对像素(x,y)的灰度值I(x,y,t)进行判定，用o表示输出图像：
Figure PCTCN2019112693-appb-000051
其中T p为概率阈值，在实际应用中，通常用等价的阈值替代概率阈值。在本实施例中，在判定概率大于或等于概率阈值时，将I(x,y,t)确定为背景像素点，在判定概率小于概率阈值时，将I(x,y,t)确定为前景像素点。在检测完毕后，对被判定为背景的像素的背景模型采用下式进行更新：
μ t(x,y)＝(1-α)μ t(x,y)+αI(x,y,t)
Figure PCTCN2019112693-appb-000052
式中，α称为学习因子，反映视频中背景信息的变化快慢，如果α取值太小，背景模型的变化慢于实际真实场景的变化，将导致检测出的目标存在很多空洞，反之，会使得运动较慢的前景变成背景的一部分。
在本实施例中，为增强高斯背景鲁棒性，选用多个高斯分布加权
Figure PCTCN2019112693-appb-000053
式中，I(x,y,t)表示像素点(x,y)在t时刻的像素值，η表示高斯概 率密度函数，μ t和σ t分别表示像素点(x,y)在t时刻的均值和标准差，k为高斯分布分量个数，w i为第i个高斯分布η i(I,μ t,σ t)的权重，o表示输出图像，T P表示概率阈值；若I(x,y,t)对于这k个高斯分布，概率都大于概率阈值T P(或对于任意的η i(I,μ t,σ t)，|I(x,y,t)-μ t|≤2.5σ t都满足)，则I(x,y,t)为图像背景，否则为前景。混合高斯背景模型更新时，只对概率大于概率阈值T P(或满足|I(x,y,t)-μ t|≤2.5σ t)的高斯分量进行更新。
利用本实施例的混合高斯模型，能够对图像中所有像素划分为前景像素点和背景像素点，进而得到一个包含前景和背景的二值图像，检测出图像中运动的像素，辅以中值滤波和简单的形态学处理，最终得到图像中运动的目标，然后基于所检测出的运动目标组成观测集。
步骤105、对标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对。
在本实施例中，输入N个观测集合，通过分类器，将会得到一个m×2N的输出矩阵
Figure PCTCN2019112693-appb-000054
本实施例可以利用贪婪算法对矩阵进行分析处理，得到目标与观测之间的正确关联对。
步骤106、基于数据关联结果进行轨迹管理。
在复杂环境下，由于背景干扰、目标自身形变等多种因素的影响，在保持高检测率的条件下，目标检测器将难以避免的会产生如图2中所示的虚假观测。如图2所示为本实施例提供的真实场景中所输出的观测示意图，其中，白色矩形框表示当前时刻目标状态，黑色矩形框表示虚假观测。从图2可以看出，这些虚假观测与目标之间发生了明显的遮挡。经过模糊数据关联之后，这些虚假观测将成为未被关联上的观测，而新目标所对应的观测对当前已记录目标的模糊隶属度较低，其同样也将成为未被关联上的观测。因此，如果为所有未被关联上的观测均建立新的目标轨迹，则可能导致为虚假观测错误的进行了轨迹起始。基于此，本实施例提出利用空时线索对未被关联上的观测与当前目标间的遮挡情况进行分析，从而判别出对应于新目标的观测，并为其起始新的目标轨迹。
如图3所示为本实施例提供的目标与观测之间的遮挡示意图，为 了对未被关联上的观测与当前目标间的遮挡程度进行度量，本文定义了遮挡度ω。假设目标对象A与未被关联上的观测对象B发生如图4所示的遮挡，其中矩形框A与矩形框B之间重叠的阴影部分表示遮挡区域，定义A与B之间的遮挡度ω(A,B)为：
Figure PCTCN2019112693-appb-000055
式中，r(·)表示区域的面积，ω(A,B)表示A与B之间的遮挡度，且0≤ω≤1，当ω(A,B)>0时，A与B发生了遮挡。并且，根据矩形框A底部的纵向图像坐标值y A与矩形框B底部的纵向图像坐标值y B可进一步得知，如果y A>y B，则说明B被A遮挡。
然后，将所计算的遮挡度代入预设的新目标判别函数，确定新目标对象所对应的观测对象；新目标判别函数φ表示如下：
Figure PCTCN2019112693-appb-000056
其中，O＝{o 1,...,o L}表示目标集，Ω＝{d 1,...,d k}表示经过模糊数据关联之后，仍未被关联上的观测对象，β为常量参数，且0<β<1，在本实施例中可以取β＝0.5。在φ(d i)＝1时，未被关联上的观测对象为新目标对象所对应的观测对象，在φ(d i)＝0时，未被关联上的观测对象为虚假观测对象。
可选的，本实施例提供了一种轨迹管理方法，如图4为本实施例提供的轨迹管理方法的流程示意图，具体包括以下步骤：
步骤401、从未被关联上的观测对象中确定新目标对象所对应的观测对象；
步骤402、为各新目标对象所对应的观测对象建立新的临时轨迹，并判断临时轨迹是否连续预设帧数均被关联上；
步骤403、在临时轨迹连续预设帧数均被关联上时，将临时轨迹转化为有效目标轨迹；
步骤404、采用卡尔曼滤波器对每条临时轨迹以及有效目标轨迹进行滤波及预测。
具体的，本实施例结合新目标判别函数，采用目标轨迹管理规则 解决有效目标轨迹的平滑与预测、无效目标轨迹的终止以及新目标轨迹的起始等问题。所采用的目标轨迹管理规则具体包括：
(1)为每个φ(d)＝1的观测d建立新的临时轨迹；
(2)若临时轨迹连续λ 1帧都被关联上，则将其转化为有效目标轨迹，否则，删除该临时轨迹，其中λ 1为常量参数，并且λ 1>1；
(3)采用卡尔曼滤波器对每条临时轨迹、有效目标轨迹进行滤波以及预测；
(4)对连续预测λ 2帧后仍未被关联上的临时轨迹、有效目标轨迹进行删除，其中λ 2为常量参数，并且λ 2>1。
从而，在本实施例中，对于已关联上的目标，根据规则(2)和(3)，利用卡尔曼滤波器对目标轨迹进行更新；对于未关联上的观测，根据目标轨迹管理规则(1)建立新的目标轨迹，更新目标轨迹标签；对于未关联上的目标，根据目标轨迹管理规则(4)删除目标轨迹标签以及状态；最后再根据轨迹管理规则(3)对所有目标轨迹进行预测更新。
根据本发明实施例提供的基于TSK模糊分类器的目标跟踪方法，首先对稳定航迹的特征集合构建多输出回归数据集，并计算特征集合中各特征相对模糊规则的模糊隶属度；然后基于多输出回归数据集及模糊隶属度训练分别基于运动特征和HOG特征的TSK模糊分类器的后件参数，并构建对应的TSK模糊分类器；再将观测集输入至TSK模糊分类器得到标签向量矩阵，并对标签向量矩阵进行数据关联得到目标和观测的正确关联；最后对目标进行滤波和轨迹管理得到目标的最终轨迹。通过本发明的实施，利用多帧信息训练出TSK模糊分类器，并且在训练的过程中加入多特征学习机制，增加了分类器的学习能力，可有效处理数据关联过程中的不确定性，提高目标跟踪的准确性。
第二实施例：
为了解决相关技术中采用硬判决方法进行目标跟踪时，对目标与观测进行关联的准确性不高的技术问题，本实施例提出了一种基于TSK模糊分类器的目标跟踪装置，具体请参见图5所示的目标跟踪装 置，本实施例的目标跟踪装置包括：
提取模块501，用于提取m条稳定航迹的所有特征集合，并对特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及HOG特征；
计算模块502，用于将不同目标划分到不同模糊集，计算特征集合中各特征相对第k’个模糊规则的模糊隶属度；
构建模块503，用于基于多输出回归数据集以及模糊隶属度，训练出分别基于运动特征以及HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器；
分类模块504，用于对图像中的运动目标进行检测得到观测集，并将观测集输入至TSK模糊分类器，得到标签向量矩阵；
关联模块505，用于对标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对；
管理模块506，用于基于数据关联结果进行轨迹管理。
在本实施例的一些实施方式中，计算模块502具体用于将不同目标划分到不同模糊集，通过预设的高斯隶属函数，计算特征集合中各特征相对第k’个模糊规则的模糊隶属度；高斯隶属函数分别表示如下：
Figure PCTCN2019112693-appb-000057
Figure PCTCN2019112693-appb-000058
其中，
Figure PCTCN2019112693-appb-000059
Figure PCTCN2019112693-appb-000060
Figure PCTCN2019112693-appb-000061
其中，
Figure PCTCN2019112693-appb-000062
为运动特征中心向量，
Figure PCTCN2019112693-appb-000063
为HOG特征中 心向量，(x’，z’)为运动特征，ho为HOG特征。
在本实施例的一些实施方式中，构建模块503在分别基于所训练得到的后件参数构建对应的TSK模糊分类器时，具体用于对所训练得到后件参数进行多特征学习；基于多特征学习后的后件参数分别构建对应的TSK模糊分类器。
进一步地，在本实施例的一些实施方式中，构建模块503在基于多特征学习后的后件参数分别构建对应的TSK模糊分类器时，具体用于：根据多特征学习后的基于运动特征的第j个稳定航迹的TSK模糊分类器的后件参数，构建基于运动特征的第j个稳定航迹的TSK模糊分类器：
Figure PCTCN2019112693-appb-000064
其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
Figure PCTCN2019112693-appb-000065
分别为第k条规则的输入变量x’、z’对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果；
以及，根据多特征学习后的基于HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，构建基于HOG特征的第j个稳定航迹的TSK模糊分类器：
Figure PCTCN2019112693-appb-000066
其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
Figure PCTCN2019112693-appb-000067
为第k条规则的输入变量ho对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果。
在本实施例的一些实施方式中，分类模块504在对图像中的运动目标进行检测得到观测集时，具体用于通过混合高斯背景模型将图像中所有像素划分为前景像素点和背景像素点，得到包含前景和背景的二值图像；检测二值图像中运动的像素，并进行中值滤波及形态学处理，确定运动目标；基于所检测出的运动目标组成观测集。混合高斯 背景模型表示如下：
Figure PCTCN2019112693-appb-000068
Figure PCTCN2019112693-appb-000069
其中，I(x,y,t)表示像素点(x,y)在t时刻的像素值，η表示高斯概率密度函数，μ t和σ t分别表示像素点(x,y)在t时刻的均值和标准差，k为高斯分布分量个数，w i为第i个高斯分布η i(I,μ t,σ t)的权重，o表示输出图像，T P表示概率阈值，在判定概率大于或等于概率阈值时，将I(x,y,t)确定为背景像素点，在判定概率小于概率阈值时，将I(x,y,t)确定为前景像素点。
在本实施例的一些实施方式中，管理模块606具体用于从未被关联上的观测对象中确定新目标对象所对应的观测对象；为各新目标对象所对应的观测对象建立新的临时轨迹，并判断临时轨迹是否连续预设帧数均被关联上；在临时轨迹连续预设帧数均被关联上时，将临时轨迹转化为有效目标轨迹；采用卡尔曼滤波器对每条临时轨迹以及有效目标轨迹进行滤波及预测。
进一步地，在本实施例的一些实施方式中，管理模块606在从未被关联上的观测对象中确定新目标对象所对应的观测对象时，具体用于采用预设的遮挡度计算公式，计算未被关联上的观测对象与目标对象之间的遮挡度；将所计算的遮挡度代入预设的新目标判别函数，确定新目标对象所对应的观测对象。遮挡度计算公式表示如下：
Figure PCTCN2019112693-appb-000070
其中，A表示目标对象，B表示观测对象，r(·)表示区域的面积，ω(A,B)表示A与B之间的遮挡度，且0≤ω≤1，当ω(A,B)>0时，A与B发生了遮挡；
新目标判别函数表示如下：
Figure PCTCN2019112693-appb-000071
其中，O＝{o 1,...,o L}表示目标集，Ω＝{d 1,...,d k}表示未被关联上的观测 对象，β为常量参数，且0<β<1，在φ(d i)＝1时，未被关联上的观测对象为新目标对象所对应的观测对象，在φ(d i)＝0时，未被关联上的观测对象为虚假观测对象。
应当说明的是，前述实施例中的目标跟踪方法均可基于本实施例提供的目标跟踪装置实现，所属领域的普通技术人员可以清楚的了解到，为描述的方便和简洁，本实施例中所描述的目标跟踪装置的具体工作过程，可以参考前述方法实施例中的对应过程，在此不再赘述。
采用本实施例提供的基于TSK模糊分类器的目标跟踪装置，首先对稳定航迹的特征集合构建多输出回归数据集，并计算特征集合中各特征相对模糊规则的模糊隶属度；然后基于多输出回归数据集及模糊隶属度训练分别基于运动特征和HOG特征的TSK模糊分类器的后件参数，并构建对应的TSK模糊分类器；再将观测集输入至TSK模糊分类器得到标签向量矩阵，并对标签向量矩阵进行数据关联得到目标和观测的正确关联；最后对目标进行滤波和轨迹管理得到目标的最终轨迹。通过本发明的实施，利用多帧信息训练出TSK模糊分类器，并且在训练的过程中加入多特征学习机制，增加了分类器的学习能力，可有效处理数据关联过程中的不确定性，提高目标跟踪的准确性。
第三实施例：
本实施例提供了一种电子装置，参见图6所示，其包括处理器601、存储器602及通信总线603，其中：通信总线603用于实现处理器601和存储器602之间的连接通信；处理器601用于执行存储器602中存储的一个或者多个计算机程序，以实现上述实施例一中的方法的至少一个步骤。
本实施例还提供了一种计算机可读存储介质，该计算机可读存储介质包括在用于存储信息(诸如计算机可读指令、数据结构、计算机程序模块或其他数据)的任何方法或技术中实施的易失性或非易失性、可移除或不可移除的介质。计算机可读存储介质包括但不限于RAM(Random Access Memory，随机存取存储器),ROM(Read-Only  Memory，只读存储器),EEPROM(Electrically Erasable Programmable read only memory，带电可擦可编程只读存储器)、闪存或其他存储器技术、CD-ROM(Compact Disc Read-Only Memory，光盘只读存储器)，数字多功能盘(DVD)或其他光盘存储、磁盒、磁带、磁盘存储或其他磁存储装置、或者可以用于存储期望的信息并且可以被计算机访问的任何其他的介质。
本实施例中的计算机可读存储介质可用于存储一个或者多个计算机程序，其存储的一个或者多个计算机程序可被处理器执行，以实现上述实施例一中的方法的至少一个步骤。
本实施例还提供了一种计算机程序，该计算机程序可以分布在计算机可读介质上，由可计算装置来执行，以实现上述实施例一中的方法的至少一个步骤；并且在某些情况下，可以采用不同于上述实施例所描述的顺序执行所示出或描述的至少一个步骤。
本实施例还提供了一种计算机程序产品，包括计算机可读装置，该计算机可读装置上存储有如上所示的计算机程序。本实施例中该计算机可读装置可包括如上所示的计算机可读存储介质。
可见，本领域的技术人员应该明白，上文中所公开方法中的全部或某些步骤、系统、装置中的功能模块/单元可以被实施为软件(可以用计算装置可执行的计算机程序代码来实现)、固件、硬件及其适当的组合。在硬件实施方式中，在以上描述中提及的功能模块/单元之间的划分不一定对应于物理组件的划分；例如，一个物理组件可以具有多个功能，或者一个功能或步骤可以由若干物理组件合作执行。某些物理组件或所有物理组件可以被实施为由处理器，如中央处理器、数字信号处理器或微处理器执行的软件，或者被实施为硬件，或者被实施为集成电路，如专用集成电路。
此外，本领域普通技术人员公知的是，通信介质通常包含计算机可读指令、数据结构、计算机程序模块或者诸如载波或其他传输机制 之类的调制数据信号中的其他数据，并且可包括任何信息递送介质。所以，本发明不限制于任何特定的硬件和软件结合。
以上内容是结合具体的实施方式对本发明实施例所作的进一步详细说明，不能认定本发明的具体实施只局限于这些说明。对于本发明所属技术领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干简单推演或替换，都应当视为属于本发明的保护范围。

Claims (10)

  1. 一种基于TSK模糊分类器的目标跟踪方法，其特征在于，包括：
    提取m条稳定航迹的所有特征集合，并对所述特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及方向梯度HOG特征；
    将不同目标划分到不同模糊集，计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度；
    基于所述多输出回归数据集以及所述模糊隶属度，训练出分别基于所述运动特征以及所述HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器；
    对图像中的运动目标进行检测得到观测集，并将所述观测集输入至所述TSK模糊分类器，得到标签向量矩阵；
    对所述标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对；
    基于数据关联结果进行轨迹管理。
  2. 如权利要求1所述的目标跟踪方法，其特征在于，所述计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度包括：
    通过预设的高斯隶属函数，计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度；所述高斯隶属函数分别表示如下：
    Figure PCTCN2019112693-appb-100001
    Figure PCTCN2019112693-appb-100002
    其中，
    Figure PCTCN2019112693-appb-100003
    Figure PCTCN2019112693-appb-100004
    Figure PCTCN2019112693-appb-100005
    其中，
    Figure PCTCN2019112693-appb-100006
    为运动特征中心向量，
    Figure PCTCN2019112693-appb-100007
    为HOG特征中心向量，(x’，z’)为运动特征，ho为HOG特征。
  3. 如权利要求1所述的目标跟踪方法，其特征在于，所述分别基于所训练得到的后件参数构建对应的TSK模糊分类器包括：
    对所训练得到后件参数进行多特征学习；
    基于多特征学习后的后件参数分别构建对应的TSK模糊分类器。
  4. 如权利要求3所述的目标跟踪方法，其特征在于，所述基于多特征学习后的后件参数分别构建对应的TSK模糊分类器包括：
    根据多特征学习后的基于所述运动特征的第j个稳定航迹的TSK模糊分类器的后件参数，构建基于运动特征的第j个稳定航迹的TSK模糊分类器：
    Figure PCTCN2019112693-appb-100008
    Figure PCTCN2019112693-appb-100009
    其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
    Figure PCTCN2019112693-appb-100010
    分别为第k条规则的输入变量x’、z’对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果；
    根据多特征学习后的基于所述HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，构建基于HOG特征的第j个稳定航迹的TSK模糊分类器：
    Figure PCTCN2019112693-appb-100011
    Figure PCTCN2019112693-appb-100012
    其中，IF部分为规则前件，THEN部分为规则后件,K’是模糊规则的数量，
    Figure PCTCN2019112693-appb-100013
    为第k条规则的输入变量ho对应的模糊子集，and是模糊连接算子，f k′(u)为每条模糊规则的输出结果。
  5. 如权利要求1所述的目标跟踪方法，其特征在于，所述对图像中的运动目标进行检测得到观测集包括：
    通过混合高斯背景模型将图像中所有像素划分为前景像素点和 背景像素点，得到包含前景和背景的二值图像；所述混合高斯背景模型表示如下：
    Figure PCTCN2019112693-appb-100014
    Figure PCTCN2019112693-appb-100015
    其中，I(x,y,t)表示像素点(x,y)在t时刻的像素值，η表示高斯概率密度函数，μ t和σ t分别表示像素点(x,y)在t时刻的均值和标准差，k为高斯分布分量个数，w i为第i个高斯分布η i(I,μ t,σ t)的权重，o表示输出图像，T P表示概率阈值，在判定概率大于或等于概率阈值时，将I(x,y,t)确定为背景像素点，在判定概率小于概率阈值时，将I(x,y,t)确定为前景像素点；
    检测所述二值图像中运动的像素，并进行中值滤波及形态学处理，确定运动目标；
    基于所检测出的运动目标组成观测集。
  6. 如权利要求1至5中任意一项所述的目标跟踪方法，其特征在于，所述基于数据关联结果进行轨迹管理包括：
    从未被关联上的观测对象中确定新目标对象所对应的观测对象；
    为各所述新目标对象所对应的观测对象建立新的临时轨迹，并判断所述临时轨迹是否连续预设帧数均被关联上；
    在所述临时轨迹连续预设帧数均被关联上时，将所述临时轨迹转化为有效目标轨迹；
    采用卡尔曼滤波器对每条临时轨迹以及有效目标轨迹进行滤波及预测。
  7. 如权利要求6所述的目标跟踪方法，其特征在于，所述从未被关联上的观测对象中确定新目标对象所对应的观测对象包括：
    采用预设的遮挡度计算公式，计算未被关联上的观测对象与目标对象之间的遮挡度；所述遮挡度计算公式表示如下：
    Figure PCTCN2019112693-appb-100016
    其中，A表示目标对象，B表示观测对象，r(·)表示区域的面积，ω(A,B)表示A与B之间的遮挡度，且0≤ω≤1，当ω(A,B)>0时，A与B发生了遮挡；
    将所计算的所述遮挡度代入预设的新目标判别函数，确定新目标对象所对应的观测对象；所述新目标判别函数表示如下：
    Figure PCTCN2019112693-appb-100017
    其中，O＝{o 1,...,o L}表示所述目标集，Ω＝{d 1,...,d k}表示所述未被关联上的观测对象，β为常量参数，且0<β<1，在φ(d i)＝1时，所述未被关联上的观测对象为新目标对象所对应的观测对象，在φ(d i)＝0时，所述未被关联上的观测对象为虚假观测对象。
  8. 一种基于TSK模糊分类器的目标跟踪装置，其特征在于，包括：
    提取模块，用于提取m条稳定航迹的所有特征集合，并对所述特征集合构建多输出回归数据集；其中，特征集合中的每个特征包括运动特征以及方向梯度HOG特征；
    计算模块，用于将不同目标划分到不同模糊集，计算所述特征集合中各特征相对第k’个模糊规则的模糊隶属度；
    构建模块，用于基于所述多输出回归数据集以及所述模糊隶属度，训练出分别基于所述运动特征以及所述HOG特征的第j个稳定航迹的TSK模糊分类器的后件参数，并分别基于所训练得到的后件参数构建对应的TSK模糊分类器；
    分类模块，用于对图像中的运动目标进行检测得到观测集，并将所述观测集输入至所述TSK模糊分类器，得到标签向量矩阵；
    关联模块，用于对所述标签向量矩阵进行数据关联，确定所有观测对象与目标对象的关联对；
    管理模块，用于基于数据关联结果进行轨迹管理。
  9. 一种电子装置，其特征在于，包括：处理器、存储器和通信总线；
    所述通信总线用于实现所述处理器和存储器之间的连接通信；
    所述处理器用于执行所述存储器中存储的一个或者多个程序，以实现如权利要求1至7中任意一项所述的基于TSK模糊分类器的目标跟踪方法的步骤。
  10. 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质存储有一个或者多个程序，所述一个或者多个程序可被一个或者多个处理器执行，以实现如权利要求1至7中任意一项所述的基于TSK模糊分类器的目标跟踪方法的步骤。
PCT/CN2019/112693 2019-07-18 2019-10-23 基于tsk模糊分类器的目标跟踪方法、装置及存储介质 WO2021007984A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201910650057.9A CN110349187B (zh) 2019-07-18 2019-07-18 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
CN201910650057.9 2019-07-18

Publications (1)

Publication Number Publication Date
WO2021007984A1 true WO2021007984A1 (zh) 2021-01-21

Family

ID=68178698

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2019/112693 WO2021007984A1 (zh) 2019-07-18 2019-10-23 基于tsk模糊分类器的目标跟踪方法、装置及存储介质

Country Status (2)

Country Link
CN (1) CN110349187B (zh)
WO (1) WO2021007984A1 (zh)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113126082A (zh) * 2021-03-31 2021-07-16 中山大学 一种群目标航迹起始方法、系统、装置及存储介质
CN113158813A (zh) * 2021-03-26 2021-07-23 精英数智科技股份有限公司 流动目标实时统计方法及装置
CN113247720A (zh) * 2021-06-02 2021-08-13 浙江新再灵科技股份有限公司 基于视频的智能梯控方法及系统
CN114119970A (zh) * 2022-01-29 2022-03-01 中科视语(北京)科技有限公司 目标跟踪方法及装置
CN116630751A (zh) * 2023-07-24 2023-08-22 中国电子科技集团公司第二十八研究所 一种融合信息瓶颈和不确定性感知的可信目标检测方法

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110349187B (zh) * 2019-07-18 2023-04-14 深圳大学 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
CN111444937B (zh) * 2020-01-15 2023-05-12 湖州师范学院 一种基于集成tsk模糊分类器的众包质量提升的方法
CN112305915B (zh) * 2020-10-28 2023-02-07 深圳大学 Tsk迭代回归模型的标签多伯努利多目标跟踪方法及系统
CN112632854B (zh) * 2020-12-17 2022-04-12 衡阳师范学院 基于类人学习能力的tsk模糊模型的故障预测方法及系统
CN113534127B (zh) * 2021-07-13 2023-10-27 深圳大学 一种多目标数据关联方法、装置及计算机可读存储介质
CN114998999B (zh) * 2022-07-21 2022-12-06 之江实验室 一种基于多帧输入与轨迹平滑的多目标跟踪方法和装置

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160377698A1 (en) * 2015-06-25 2016-12-29 Appropolis Inc. System and a method for tracking mobile objects using cameras and tag devices
CN107545582A (zh) * 2017-07-04 2018-01-05 深圳大学 基于模糊逻辑的视频多目标跟踪方法及装置
CN109543615A (zh) * 2018-11-23 2019-03-29 长沙理工大学 一种基于多级特征的双学习模型目标跟踪方法
CN110349187A (zh) * 2019-07-18 2019-10-18 深圳大学 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
CN110363165A (zh) * 2019-07-18 2019-10-22 深圳大学 基于tsk模糊系统的多目标跟踪方法、装置及存储介质

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9342759B1 (en) * 2013-03-14 2016-05-17 Hrl Laboratories, Llc Object recognition consistency improvement using a pseudo-tracklet approach
CN107161207B (zh) * 2017-05-08 2023-08-22 江苏大学 一种基于主动安全的智能汽车轨迹跟踪控制系统及控制方法
CN107274297A (zh) * 2017-06-14 2017-10-20 贵州中北斗科技有限公司 一种土地作物种植适宜性评估方法

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20160377698A1 (en) * 2015-06-25 2016-12-29 Appropolis Inc. System and a method for tracking mobile objects using cameras and tag devices
CN107545582A (zh) * 2017-07-04 2018-01-05 深圳大学 基于模糊逻辑的视频多目标跟踪方法及装置
CN109543615A (zh) * 2018-11-23 2019-03-29 长沙理工大学 一种基于多级特征的双学习模型目标跟踪方法
CN110349187A (zh) * 2019-07-18 2019-10-18 深圳大学 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
CN110363165A (zh) * 2019-07-18 2019-10-22 深圳大学 基于tsk模糊系统的多目标跟踪方法、装置及存储介质

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113158813A (zh) * 2021-03-26 2021-07-23 精英数智科技股份有限公司 流动目标实时统计方法及装置
CN113126082A (zh) * 2021-03-31 2021-07-16 中山大学 一种群目标航迹起始方法、系统、装置及存储介质
CN113126082B (zh) * 2021-03-31 2023-09-26 中山大学 一种群目标航迹起始方法、系统、装置及存储介质
CN113247720A (zh) * 2021-06-02 2021-08-13 浙江新再灵科技股份有限公司 基于视频的智能梯控方法及系统
CN114119970A (zh) * 2022-01-29 2022-03-01 中科视语(北京)科技有限公司 目标跟踪方法及装置
CN114119970B (zh) * 2022-01-29 2022-05-03 中科视语(北京)科技有限公司 目标跟踪方法及装置
CN116630751A (zh) * 2023-07-24 2023-08-22 中国电子科技集团公司第二十八研究所 一种融合信息瓶颈和不确定性感知的可信目标检测方法
CN116630751B (zh) * 2023-07-24 2023-10-31 中国电子科技集团公司第二十八研究所 一种融合信息瓶颈和不确定性感知的可信目标检测方法

Also Published As

Publication number Publication date
CN110349187B (zh) 2023-04-14
CN110349187A (zh) 2019-10-18

Similar Documents

Publication Publication Date Title
WO2021007984A1 (zh) 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
He et al. Bounding box regression with uncertainty for accurate object detection
CN108470354B (zh) 视频目标跟踪方法、装置和实现装置
CN106846355B (zh) 基于提升直觉模糊树的目标跟踪方法及装置
Roshtkhari et al. An on-line, real-time learning method for detecting anomalies in videos using spatio-temporal compositions
Meshgi et al. An occlusion-aware particle filter tracker to handle complex and persistent occlusions
CN110782483B (zh) 基于分布式相机网络的多视图多目标跟踪方法及系统
CN110363165B (zh) 基于tsk模糊系统的多目标跟踪方法、装置及存储介质
CN110349188B (zh) 基于tsk模糊模型的多目标跟踪方法、装置及存储介质
WO2018107488A1 (zh) 基于提升直觉模糊树的目标跟踪方法及装置
Riegler et al. Hough Networks for Head Pose Estimation and Facial Feature Localization.
CN111582349B (zh) 一种基于YOLOv3和核相关滤波改进的目标跟踪算法
Liang-qun et al. Fuzzy logic approach to visual multi-object tracking
CN115063454B (zh) 多目标跟踪匹配方法、装置、终端及存储介质
Bae et al. A novel real time video tracking framework using adaptive discrete swarm optimization
Li et al. Robust object tracking via multi-feature adaptive fusion based on stability: contrast analysis
Chen et al. Learning to count with back-propagated information
Amisse et al. Fine-tuning deep learning models for pedestrian detection
Fang et al. SCENT: A new precipitation nowcasting method based on sparse correspondence and deep neural network
Xu et al. An accurate multi-cell parameter estimate algorithm with heuristically restrictive ant system
Huang et al. Cost-sensitive sparse linear regression for crowd counting with imbalanced training data
CN117372928A (zh) 一种视频目标检测方法、装置及相关设备
Ait Abdelali et al. Visual vehicle tracking via deep learning and particle filter
CN111291785A (zh) 目标检测的方法、装置、设备及存储介质
Lu et al. Particle filter vehicle tracking based on surf feature matching

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19937410

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 19.05.2022)

122 Ep: pct application non-entry in european phase

Ref document number: 19937410

Country of ref document: EP

Kind code of ref document: A1