WO2018227491A1 - 视频多目标模糊数据关联方法及装置 - Google Patents

视频多目标模糊数据关联方法及装置 Download PDF

Info

Publication number
WO2018227491A1
WO2018227491A1 PCT/CN2017/088495 CN2017088495W WO2018227491A1 WO 2018227491 A1 WO2018227491 A1 WO 2018227491A1 CN 2017088495 W CN2017088495 W CN 2017088495W WO 2018227491 A1 WO2018227491 A1 WO 2018227491A1
Authority
WO
WIPO (PCT)
Prior art keywords
result
observation
prediction
prediction result
occlusion
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/CN2017/088495
Other languages
English (en)
French (fr)
Inventor
李良群
湛西羊
罗升
刘宗香
谢维信
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shenzhen University
Original Assignee
Shenzhen University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shenzhen University filed Critical Shenzhen University
Priority to PCT/CN2017/088495 priority Critical patent/WO2018227491A1/zh
Publication of WO2018227491A1 publication Critical patent/WO2018227491A1/zh
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments

Definitions

  • the present invention relates to the field of target tracking, and in particular to a video multi-target fuzzy data association method and apparatus.
  • Video multi-target tracking has always been a hot research topic in the field of computer vision. It combines image processing, pattern recognition, artificial intelligence, automatic control and computer in one, in intelligent video surveillance, human-computer interaction, robot vision navigation, virtual reality, The fields of medical diagnosis, traffic control and monitoring have very important practical value.
  • the invention provides a video multi-objective fuzzy data association method and device, which can improve the accuracy of multi-target tracking in a complex environment.
  • a technical solution adopted by the present invention is to provide a video multi-target fuzzy data association method, comprising: performing online target motion detection on a current video frame, and detecting a possible moving object as an observation result; Determining an occlusion between prediction results of different targets in a current video frame, wherein the prediction result is obtained by using at least a trajectory of a target of a previous video frame; and determining each prediction according to the occlusion degree Whether occlusion occurs between the result and the other predicted results; if no occlusion occurs between the predicted result and any other of the predicted results, performing a first data association on the predicted result and the observed result; And performing occlusion between the prediction result and the other prediction result, and performing a second data association on the prediction result and the observation result, wherein the first data association and the first data association are different.
  • a technical solution adopted by the present invention is to provide a video multi-target fuzzy data association device, including: a processor, configured to perform online target motion on a current video frame acquired from the camera Detecting, detecting the obtained possible moving object as an observation result; calculating an occlusion degree between prediction results of different targets in the current video frame, wherein the prediction result is obtained by predicting at least the trajectory of the target of the previous video frame Determining whether occlusion occurs between each of the prediction results and the other prediction results according to the occlusion degree; if no occlusion occurs between the prediction result and any other of the prediction results, The prediction result and the observation result are associated with the first data; if an occlusion occurs between the prediction result and the other prediction result, And performing a second data association on the prediction result and the observation result, wherein the first data association and the first data association are different.
  • the invention has the beneficial effects of providing a method and a device for video multi-objective fuzzy data association. Firstly, by determining whether occlusion occurs between prediction results of all targets in the current video frame, occlusion occurs between the prediction results of the target respectively. With and without occlusion, the data association between the prediction result and the observation result can accurately track multiple targets under the conditions of apparent similarity, frequent interaction, occlusion, and background interference, and has strong robustness and accuracy. .
  • FIG. 1 is a flow chart of a first embodiment of a video multi-target fuzzy data association method according to the present invention
  • FIG. 2 is a schematic diagram of occlusion between prediction results of different targets of the present invention.
  • FIG. 3 is a flowchart of a second embodiment of a video multi-target fuzzy data association method according to the present invention.
  • step S14 in FIG. 2 is a schematic flow chart of an embodiment of step S14 in FIG. 2;
  • FIG. 5 is a schematic flowchart of an embodiment of step S143 in FIG. 4;
  • FIG. 6 is a third embodiment of a video multi-objective fuzzy data association method according to the present invention.
  • step S24 in FIG. 6 is a schematic flow chart of an embodiment of step S24 in FIG. 6;
  • step S242 in FIG. 7 is a schematic flow chart of an embodiment of step S242 in FIG. 7;
  • Figure 9 is a fuzzy input variable f k (i, j) of the present invention, Schematic diagram of the membership function
  • FIG. 10 is a schematic diagram of a membership function of the output fuzzy variable ⁇ k of the present invention.
  • FIG. 11 is a schematic structural diagram of multi-feature clue fusion according to the present invention.
  • step S244 in FIG. 7 is a schematic flow chart of an embodiment of step S244 in FIG. 7;
  • FIG. 13 is a flowchart of a fourth embodiment of a video multi-target fuzzy data association method according to the present invention.
  • step S5 in FIG. 13 is a schematic flow chart of an embodiment of step S5 in FIG. 13;
  • step S52 in FIG. 14 is a schematic flow chart of an embodiment of step S52 in FIG. 14;
  • 16 is a schematic structural diagram of a first embodiment of a video multi-target fuzzy data association apparatus according to the present invention.
  • FIG. 17 is a schematic structural diagram of a second embodiment of a video multi-target fuzzy data association apparatus according to the present invention.
  • the first embodiment of the video multi-target fuzzy data association method of the present invention includes:
  • S1 Perform online target motion detection on the current video frame, and detect the possible moving object as an observation result.
  • the motion detection algorithm such as frame difference method, optical flow method and background subtraction method is used to detect the motion of the current video frame, so as to find out the pixels belonging to the foreground of the motion, supplemented by median filtering and simple morphological processing, and finally obtain the current video.
  • Possible moving objects in the frame are used as observation objects.
  • An observation object is an image block in the current video frame. Generally, the shape of the observation object is a rectangle.
  • the prediction result of the target in this step is obtained by predicting at least the trajectory of the target of the previous video frame.
  • an occlusion calculation is performed between the prediction results of all the targets in the current frame video to determine whether occlusion occurs between the prediction results of all the targets in the current frame video.
  • FIG. 2 is a schematic diagram of occlusion between prediction results of different targets of the present invention.
  • the tracking icon shapes of the prediction result A and the prediction result B are all rectangular, and there is overlap between the two, and the parameter of the prediction result A is expressed as: [x, y, w , d], where x, y represent the coordinates of the rectangular frame, w represents the width of the rectangular frame, d represents the height of the rectangular frame, and the parameter of the prediction result B is expressed as: [x', y', w', h'] Where x', y' represents the coordinates of the rectangular frame, w' represents the width of the rectangular frame, h' represents the height of the rectangular frame, and the shaded portion between the predicted result A and the predicted result B is expressed as: [x o , y o , w o ,h o ], and the overlapping parts are expressed as:
  • the area of the overlap between the prediction result A and the prediction result B is expressed as w o *h o . If the above w o , h o does not satisfy w o >0 or h o >0, the two tracking rectangles do not form an overlapping rectangle, that is, the overlapping rectangle area is 0.
  • the occlusion degree between the two is defined as:
  • s( ⁇ ) represents the area of the area
  • the occlusion degree satisfies 0 ⁇ ⁇ (A, B) ⁇ 1.
  • ⁇ (A, B) is greater than 0, it means that occlusion occurs between the prediction result A and the prediction result B.
  • the occlusion degree is determined for the prediction result of all the targets in the current video frame scene, and the overlap ratio ⁇ ij of the tracking rectangle between the different target prediction results of the current video frame is calculated according to the formula (1) (different targets).
  • the occlusion degree between the prediction results is determined, and it is judged whether the occlusion degree of each prediction result and other prediction results is smaller than the first occlusion determination threshold ⁇ over .
  • the first occlusion determination threshold ⁇ over satisfies ⁇ over ⁇ [0, 1]. If ⁇ ij is smaller than the first occlusion determination threshold ⁇ over , occlusion is considered to occur between the prediction results. If ⁇ over is equal to 0, it indicates that no occlusion occurs between the prediction results.
  • the first data is associated with the observation result in the current video frame. . Further, an occlusion occurs between the prediction result and the other prediction result, and the second data is associated with the observation result in the current video frame.
  • the first data association is different from the first data association, and the second data association is more complex than the first data association.
  • occlusion and non-occlusion are respectively performed between the prediction results of the target, and data association between the prediction result and the observation result is performed. It can accurately track multiple targets under the conditions of apparent similarity, frequent interaction, occlusion and background interference, and has strong robustness and accuracy.
  • FIG. 3 is a second embodiment of a video multi-target fuzzy data association method according to the first embodiment of the video multi-objective fuzzy data association method of the present invention, which is related to the video multi-target fuzzy data association of the present invention.
  • the method is further extended by the first embodiment, and therefore the same steps as the first embodiment of the video multi-objective fuzzy data association method of the present invention are not described herein.
  • This embodiment includes:
  • S11 Perform online target motion detection on the current video frame, and detect the possible moving object as an observation result.
  • step S14 further includes the following sub-steps:
  • the first similarity measure is used to measure the predicted result and the distance between the observed result.
  • the first similarity measure includes: a spatial distance feature similarity measure and an appearance feature similarity measure.
  • the spatial distance feature is one of the features that can more effectively match the observation and prediction results of the target.
  • the spatial distance feature similarity measure f D ( ⁇ ) between the observation d and the prediction result o is defined as:
  • the appearance feature similarity measure f S ( ⁇ ) between the observation result d and the prediction result o is defined as:
  • the multiplicative fusion is used to fuse the spatial distance feature similarity measure and the appearance feature similarity measure to obtain the correlation between the observation result and the prediction result, and is defined as:
  • the correlation cost matrix between the observation result and the prediction result is obtained according to the degree of association, and is defined as:
  • step S143 further includes the following sub-steps:
  • S1432 Determine whether the maximum value is the maximum value in the row and column, and meet the greater than the first threshold.
  • the spatial distance feature similarity measure and the appearance feature similarity measure between the observation result and the prediction result are fused to obtain an associated cost matrix of the two.
  • the optimization solution can find the correct correlation observations and prediction results.
  • FIG. 6 is a third embodiment of a video multi-target fuzzy data association method according to the first embodiment of the video multi-objective fuzzy data association method of the present invention, which is related to the video multi-target fuzzy data association of the present invention.
  • the method is further extended by the first embodiment, and therefore the same steps as the first embodiment of the video multi-objective fuzzy data association method of the present invention are not described herein.
  • This embodiment includes:
  • S21 Perform online target motion detection on the current video frame, and detect the possible moving object as an observation result.
  • the fusion of the two features cannot be completed due to the simple multiplicative fusion strategy, and the correlation between the prediction result and the observation result cannot be completed.
  • a logical multi-feature clue weighted fusion strategy At the same time, after the target's prediction result is in the occlusion state, since it loses most of the shape features, its model is no longer updated. At this time, the most reliable is the motion information of the prediction result, because the target's predicted result motion will have inertia.
  • the running speed generally does not change greatly in a short time, and the position before the occlusion can be generated based on the prediction result, and the motion trajectory of the target prediction result is predicted by Kalman filtering.
  • step S24 further includes the following sub-steps:
  • the color feature has better ability to resist the deformation of the target, but it lacks the description of the spatial structure of the prediction result, and is sensitive to illumination, and the edge feature can well describe the edge of the human body.
  • the variation and the small amount of offset are insensitive, so the color and edge features have complementary characteristics, so the present invention uses these two kinds of information fusion to establish the appearance characteristics of the prediction result.
  • the distance between the observation result and the prediction result is measured by the second similarity measure, and the second similarity measure includes an appearance feature similarity measure, a geometric feature similarity measure, and a motion feature similarity measure. And spatial distance feature similarity measures.
  • the appearance feature similarity measure f A ( ⁇ ) between the observation result d and the prediction result o is defined as:
  • H c ( ⁇ ) is a color histogram feature weighted by the background of the current video frame image
  • H g ( ⁇ ) is a block gradient direction histogram feature
  • Is a variance constant Is the variance constant.
  • the motion feature similarity measure f M ( ⁇ ) between the observation d and the prediction result o is defined as:
  • the spatial distance feature similarity measure f D ( ⁇ ) between the observation d and the prediction result o is defined as:
  • the appearance feature similarity measure f S ( ⁇ ) between the observation d and the prediction result o is defined as:
  • the target model and the candidate model corresponding to the appearance feature similarity measure and the geometric feature similarity measure are respectively defined as:
  • Bhattacharyya coefficient which is defined as:
  • the motion model of the predicted result of the target is described by the coordinates and velocity of its centroid.
  • the motion of the video target motion is not very large.
  • the tracking rectangle (x, y, w, h) of the prediction result of each target establishes a motion state parameter model based on position, size, and velocity. Then define the state variable X k of the Kalman filter as:
  • x and y respectively represent the horizontal and vertical coordinates of the centroid of the tracking rectangle of the observation result of the kth frame, and respectively represent the velocity of the centroid of the tracking rectangle of the observation result of the kth frame in the x-axis and the y-axis direction.
  • S242 Calculating a weight value of each feature similarity measure in the second similarity measure by using a fuzzy inference system model.
  • the U i of the domain is a fuzzy set whose membership function is defined as
  • the i-th fuzzy inference rule of the fuzzy inference system is defined as:
  • the weighted average clearing method is used to obtain the total output of the fuzzy inference system model, which is defined as:
  • Membership function The point at which the maximum value is taken.
  • step S242 further includes the following sub-steps:
  • the input of the fuzzy inference system is a similarity measure of each feature of the current video frame, and the output is the weight of each corresponding feature ⁇ k ⁇ k ⁇ A, M, S, D ⁇ , wherein the feature k can be included but not It is limited to appearance feature similarity measure, geometric feature similarity measure and motion feature similarity measure.
  • the control rules are complex and the calculation amount is huge. To reduce the calculation amount and reduce the control rules, the controller is simplified. Since the weight value of each feature depends on the similarity measure of the feature and the relative size of the remaining feature similarity measure, the similarity measure of each feature and the mean of all feature similarity measures can be used as the fuzzy input respectively. Determine the weight value of the feature. Therefore, the fuzzy controller can be simplified into four two-dimensional fuzzy controllers.
  • the motion feature similarity measure is the first fuzzy input variable
  • the similarity measure of the remaining three features is the second fuzzy input variable
  • the calculation of the similarity mean of the remaining three features can be defined. for:
  • FIG. 9 is a fuzzy input variable f k (i, j) according to the present invention. Schematic diagram of the membership function. Wherein, in the two-dimensional fuzzy controller, the input variable f k (i, j) and Fuzzy to ⁇ Zo, PS, PM, PL, PE ⁇ , each fuzzy language means: Zo (zero), PS (positive small), PM (median), PL (positive), PE (maximum), input variables f k (i,j) and See Figure 9 for the membership function.
  • FIG. 10 is a schematic diagram of the membership function of the output fuzzy variable ⁇ k according to the present invention.
  • the meaning of each fuzzy language is: VL (very small), L (small), M (medium), H (high), The membership function of VH (very high) and EH (very high) output variable ⁇ k can be seen in Figure 4.
  • Table 1 is a fuzzy rule table of the input and output variables of the fuzzy inference system. It can be seen that the similarity measure f k (i, j) of the feature k increases and the other characteristic similarities Mean mean The decrease, the greater the fusion weight value of the output, and vice versa. Thereby, the weight value of the reliable feature information is given to the tracking period, and the weight value of the unreliable feature information is given less.
  • FIG. 11 is a schematic structural diagram of multi-feature clue fusion according to the present invention.
  • the motion feature similarity measure f M (i, j) is the first fuzzy input variable, and the appearance feature similarity measure, the spatial distance feature similarity measure, and the geometric feature similarity measure are used.
  • Mean for the second fuzzy input variable the feature weight value ⁇ M of the motion feature similarity measure is determined.
  • the appearance feature similarity measure f S (i, j) is the first fuzzy input variable, and the mean of the motion feature similarity measure, the spatial distance feature similarity measure and the geometric feature similarity measure For the second fuzzy input variable, combined with the fuzzy inference rule, the feature weight value ⁇ S of the appearance feature similarity measure can be obtained.
  • the feature weight value of the spatial distance feature similarity measure and the calculation method of the feature weight value ⁇ D and ⁇ A of the geometric feature similarity measure are similar to the method for calculating the feature weight value of the motion feature similarity measure described above.
  • the feature weight values ⁇ k ⁇ k ⁇ A, M, S, D ⁇ of the feature similarity measure are sequentially calculated according to the above method, and are described in detail above, and are not described herein again.
  • the feature weight value outputs of all the feature similarity measures are normalized to obtain the fusion coefficients of the features at the current time, which are defined as:
  • Multi-feature cues fusion is used to weight and fuse the appearance feature similarity measure, geometric feature similarity measure, motion direction feature similarity measure and spatial distance feature similarity measure, and the correlation cost matrix between the predicted result and the observed result can be obtained. ,defined as:
  • ⁇ k ⁇ k ⁇ A, M, S, D ⁇ is the fusion coefficient of each feature similarity measure and satisfies f k (i,j) k ⁇ A,M,S,D ⁇ are observations And forecast results A measure of similarity between each feature.
  • step S244 further includes the following sub-steps:
  • S2442 Determine whether the maximum value is the maximum value in the rank and the greater than the second threshold.
  • FIG. 13 is a flowchart of a fourth embodiment of a video multi-target fuzzy data association method according to the first to third embodiments of the video multi-target fuzzy data association method of the present invention, which is a video of the present invention.
  • the multi-target fuzzy data association method is further extended by the first to third embodiments, and therefore the same steps as those in the first to third embodiments of the video multi-target fuzzy data association method of the present invention are not described herein.
  • the embodiment further includes:
  • the occlusion degree is first determined, and if the occlusion is not occluded between the prediction result and the other prediction results, the first data is correlated with the observation result to obtain a correctly correlated prediction. Results and observations are recorded and recorded in a set of correlated predictions and observations. If occlusion occurs between the prediction result and other prediction results, the second data is correlated with the observation result to obtain the correct correlation prediction result and observation result, and recorded in the set of the associated prediction result and the observation result.
  • the present invention further adopts the third data association.
  • the step S5 further includes the following sub-steps:
  • the occlusion degree calculation is performed according to the equation (2) in the first embodiment after all the unrelated prediction targets and observations after the first data association and the second data association are performed, and the occlusion matrix of the two is further obtained.
  • Step S52 further includes the following sub-steps:
  • S521 Determine whether an occlusion degree between the uncorrelated observation result and the unassociated prediction result is greater than a second occlusion determination threshold.
  • the second occlusion determination threshold ⁇ 2 satisfies ⁇ 2 ( ⁇ 2 ⁇ [0.4, 0.6]).
  • the threshold value is greater than the second occlusion determination threshold, the correlation between the uncorrelated observation result and the unrelated prediction result is associated.
  • the two are correctly associated and recorded in the set of associated predictions and observations. Further, the uncorrelated prediction result smaller than the second occlusion determination threshold ⁇ 2 is recorded in the uncorrelated prediction result set.
  • occlusion is greater than a third occlusion determination threshold ⁇ 3 , wherein the third occlusion determination threshold ⁇ 3 satisfies ⁇ 3 ( ⁇ 3 ⁇ [0.4, 0.6]). If it is greater than, the observation is a false observation and is eliminated. Conversely, if the occlusion is less than the third occlusion determination threshold ⁇ 3 , the uncorrelated observations are recorded in the uncorrelated observation result set.
  • S6 Perform trajectory management on the observation results and the prediction results that are still not associated after the first data association, the second data association, and the third data association.
  • Trajectory management of unrelated observations and unrelated predictions to establish temporary target trajectories and delete invalid targets includes: for unrelated observations, using unrelated observations and predictions The occlusion degree, the discriminant function of the unrelated observation result is obtained, and the discriminant function is used to determine whether the temporary target trajectory is established for the unrelated observation result, and the uncorrelated prediction result is continuous if the uncorrelated prediction result corresponds to the target If the K frame is not associated, it is judged that the target is invalid and the invalid target is deleted, and K is an integer greater than 1.
  • the occlusion degree between the observation result and the prediction result, and the discriminant function for acquiring the uncorrelated observation result includes: obtaining the occlusion between the unrelated observation result z ⁇ ⁇ and the target prediction result o ⁇ O according to the equation (2) Degree, and obtain the discriminant function of each unrelated observation z ⁇ :
  • ⁇ is a constant parameter and 0 ⁇ 1;
  • Determining whether to establish a temporary target trajectory according to the discriminant function for the unrelated observation result includes: for each unrelated observation result, if the discriminant function is 1, the temporary target trajectory is established for the unrelated observation result, if If the discriminant function is 0, it is not established.
  • the effective target trajectory and the temporary target trajectory are filtered to obtain a set of target states of the current video frame, and the prediction is performed by using the set of target states of the current video frame, including: using the Kalman filter to the effective target trajectory and the temporary target The trajectory is filtered and predicted.
  • the prediction result and the observation result are respectively calculated for the first and second data associations for the different occlusion situations, and the first and second data are performed.
  • the third data association is performed on the prediction results and the observation results that are still not associated, which can effectively improve the correct association between the multi-objective and the observation, and in the case of apparent similarity, frequent interaction, occlusion, and background interference. Multi-target for accurate tracking, with strong robustness and accuracy.
  • This embodiment is a combination of the first to fourth embodiments of the present invention, and two publicly available PETS2009 are used in this embodiment. The data set and the TownCentre data set are verified.
  • TC_ODAL video multi-target online tracking algorithm
  • the trajectory confidence based on trajectory detectability and continuity is proposed.
  • the high confidence trajectory is locally correlated with the online detection response, while the low confidence trajectory segment is globally correlated with other original trajectories and detection responses.
  • an incremental linear discriminant analysis algorithm is used to project the apparent model of the trajectory into the discriminative projection space.
  • Association tracking algorithm based on self-supervised apparent model learning (ALExTRAC).
  • ALExTRAC self-supervised apparent model learning
  • a self-supervising method is proposed to learn the related objects and observations in the video sequence. It learns an appearance similarity model to estimate the cost value of the data association, and collects samples with significant changes in the sliding window. The set is used to reinforce the similarity model.
  • the algorithm in the present invention performs Matlab simulation on a PC with a CPU of Intel Core i5, 3.10 GHz and a memory of 4 GB.
  • the average processing speed of the fuzzy multi-target video tracking algorithm and the comparison algorithm proposed by the present invention is shown in Table 3.
  • the running speed of the algorithm proposed by the present invention has a great relationship with the number of targets in the scene and the number of frames of the video segment.
  • the average algorithm speed is 1.8f/s
  • the average algorithm speed is 0.8f/s
  • the video sequence has an average algorithm speed of 0.7f/s.
  • the main loss of the algorithm processing speed proposed by the invention lies in the construction of the appearance model of the target prediction result, and the calculation of the multi-attribute feature similarity.
  • the correlation cost of the observation result of the target and the observation result three fuzzy system inference processes are needed.
  • Table 3 is the average processing speed of the video multi-target tracking algorithm and the comparison algorithm based on fuzzy logic.
  • Table 3 gives an average performance comparison of the algorithm of the present invention with the two multi-target tracking algorithms described above on the video sequence PETS2009.S2L1.
  • the present invention uses the same target detection result and real position data of the TC_ODAL algorithm and the ALEXTRAC algorithm, and uses the same evaluation code to calculate the performance index.
  • Table 4 is the test result of the video sequence PETS2009.S2L1 of the present invention, as follows:
  • the MOT_FL algorithm proposed by the present invention is higher than the TC_ODAL algorithm and the ALExTRAC algorithm by 5% and 20%, respectively, in the multi-target tracking accuracy (MOTA) performance index, which indicates the proposed fuzzy data association.
  • the method can effectively deal with the noise present in the detection results.
  • the performance of the multi-target tracking accuracy (MOTP) and the number of trajectory interruptions (FG) is slightly worse, which indicates that the tracking result of the proposed method has a certain deviation from the real target, mainly because of the video sequence.
  • Most of the targets do not satisfy the simple uniform motion model, so the accuracy of the target state after Kalman filtering is relatively low.
  • the method of the present invention significantly reduces the missed detection (F.Neg), and at the same time, in the vast majority of the correctly tracked number of tracks (MT) and the vast majority of the number of tracked tracks (ML).
  • the index is slightly better than the comparison algorithm, which indicates that the algorithm proposed by the invention can accurately track multiple targets.
  • the video multi-target fuzzy data association algorithm of the present invention is applied to the relatively congested video sequence PETS2009.S2L2, and compared with the average performance of the above two multi-target tracking algorithms, see the table. 5.
  • Table 5 is the test result of the video sequence PETS2009.S2L2 of the present invention, as follows:
  • the fuzzy multi-target video tracking algorithm proposed by the present application greatly reduces the missed detection (F.Neg) and the target identity exchange times (IDS). , a significant increase in the multi-target tracking accuracy (MOTA) score for overall performance.
  • MOTA multi-target tracking accuracy
  • the algorithm of the present application is superior to the TC_ODAL algorithm and the ALEXTRAC algorithm in the vast majority of the correctly tracked number of tracks (MT) and the majority of the number of tracked tracks (ML).
  • the method further demonstrates that the video multi-objective fuzzy data association algorithm proposed by the present application can provide a more complete target trajectory.
  • the video multi-target fuzzy data association algorithm proposed in the present application is higher than the TC_ODAL algorithm and the ALExTRAC algorithm in the multi-target tracking accuracy (MOTA) performance index, respectively, which indicates that the proposed method can be effective.
  • the noise present in the test results is processed.
  • the majority of the correctly tracked number of tracks (MT) and the majority of the number of tracked tracks (ML) are slightly better than the comparison algorithm, indicating that the algorithm proposed in this application can achieve multiple targets. Accurate tracking, and can provide a more complete target trajectory output. This is due to the validity of data associations based on fuzzy logic. It is better than the ALExTRAC algorithm but worse than the TC_ODAL algorithm in the number of target identity exchanges (IDS).
  • the MOT_FL algorithm assumes that each target motion is uniform motion, the MOT_FL algorithm cannot accurately estimate and predict this situation due to the sudden change of the nonlinear motion state of the target.
  • the number of trajectories is interrupted (FG).
  • the indicator is worse than the TC_ODAL algorithm and the ALExTRAC algorithm.
  • FIG. 16 is a schematic structural diagram of a first embodiment of a video multi-target fuzzy data association apparatus according to the present invention, including:
  • the detecting module 11 is configured to perform online target motion detection on the current video frame, and detect the obtained possible moving object as an observation result.
  • the calculating module 12 is configured to calculate an occlusion degree between prediction results of different targets in the current video frame, wherein the prediction result is obtained by predicting at least the trajectory of the target of the previous video frame.
  • the determining module 13 is configured to determine whether occlusion occurs between each prediction result and other prediction results according to the occlusion degree.
  • the data association module 14 is configured to perform a first data association between the prediction result and the observation result if no occlusion occurs between the prediction result and any other prediction result; and if the occlusion occurs between the prediction result and other prediction results, the prediction is performed
  • the result and the observation result are associated with the second data, wherein the first data association It is different from the second data association.
  • FIG. 17 is a schematic structural diagram of a second embodiment of a video multi-target fuzzy data association apparatus according to the present invention, including: a processor 110 and a camera 120.
  • the camera 120 can be a local camera, the processor 110 is connected to the camera 120 through a bus; the camera 120 can also be a remote camera, and the processor 110 is connected to the camera 120 via a local area network or the Internet.
  • the processor 110 controls the operation of the video multi-target fuzzy data association device, which may also be referred to as a CPU (Central Processing Unit).
  • Processor 110 may be an integrated circuit chip with signal processing capabilities.
  • the processor 110 can also be a general purpose processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), an off-the-shelf programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, and discrete hardware components.
  • the general purpose processor may be a microprocessor or the processor or any conventional processor or the like.
  • the video multi-target fuzzy data association device may further include a memory (not shown) for storing instructions and data necessary for the processor 110 to operate, and may also store video data captured by the transmitter 120.
  • the processor 110 is configured to perform online target motion detection on the current video frame acquired from the camera 120, detect the obtained possible moving object as an observation result, and calculate an occlusion degree between prediction results of different targets in the current video frame, wherein the prediction result is Obtaining at least the trajectory of the target of the previous video frame; determining whether occlusion occurs between each prediction result and other prediction results according to the occlusion degree; if no occlusion occurs between the prediction result and any other prediction result, And performing a first data association on the prediction result and the observation result; if an occlusion occurs between the prediction result and the other prediction result, performing a second data association on the prediction result and the observation result, where the first data association is different from the first data association .
  • each part of the video multi-target fuzzy data association device of the present invention can be referred to the description in the corresponding embodiments of the online target tracking method of the present invention, and details are not described herein again.
  • the present invention provides a video multi-target fuzzy data association method and apparatus for determining whether occlusion occurs between prediction results of targets in a current video frame, and for different occlusion situations, respectively.
  • Calculating the prediction result and the observation result to perform the first and second data association, and after performing the first and second data association, performing the third data association on the prediction result and the observation result that are still not associated, can effectively improve the multi-target and
  • the correct correlation between observations, accurate tracking of multiple targets under the conditions of apparent similarity, frequent interaction, occlusion and background interference, has strong robustness and accuracy.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Analysis (AREA)

Abstract

本申请公开了一种视频多目标模糊数据关联的方法及装置，方法包括：对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；计算当前视频帧中不同目标的预测结果之间的遮挡度；根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡；若预测结果与任何其他预测结果之间均未发生遮挡，则对预测结果和观测结果进行第一数据关联；若预测结果与其他预测结果之间发生遮挡，则对预测结果和观测结果进行第二数据关联，其中，第一数据关联和第二数据关联不同。通过上述方式，能够提高在复杂环境下多目标跟踪的准确性。

Description

视频多目标模糊数据关联方法及装置 【技术领域】
本发明涉及目标跟踪领域，特别是涉及一种视频多目标模糊数据关联方法及装置。
【背景技术】
视频多目标跟踪一直是计算机视觉领域中的热点研究问题，其融合了图像处理、模式识别、人工智能、自动控制以及计算机于一体，在智能视频监控、人机交互、机器人视觉导航、虚拟现实、医学诊断、交通管制及监控等领域有着非常重要的实用价值。
但针对复杂背景环境下的视频目标，开发出一种鲁棒高效的多目标跟踪算法仍存在诸多困难，如目标的相互遮挡、光照和姿态的突然变化、尺度变化、背景拥簇快速运动等。这些情况在实际的行人目标跟踪中具有很强的随意性和不确定性，用传统概率方法并不能很好的进行建模。
【发明内容】
本发明提供一种视频多目标模糊数据关联方法及装置，能够提高在复杂环境下多目标跟踪的准确性。
为了解决上述技术问题，本发明采用的一个技术方案是：提供一种视频多目标模糊数据关联方法，包括：对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；计算所述当前视频帧中不同目标的预测结果之间的遮挡度，其中所述预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的；根据所述遮挡度分别判断每一所述预测结果与其他所述预测结果之间是否发生遮挡；若所述预测结果与任何其他所述预测结果之间均未发生遮挡，则对所述预测结果和所述观测结果进行第一数据关联；若所述预测结果与其他所述预测结果之间发生遮挡，则对所述预测结果和所述观测结果进行第二数据关联，其中，所述第一数据关联和所述第一数据关联不同。
为了解决上述技术问题，本发明采用的一个技术方案是：提供一种视频多目标模糊数据关联装置，包括：处理器，所述处理器用于对从所述摄像机获取的当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；计算所述当前视频帧中不同目标的预测结果之间的遮挡度，其中所述预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的；根据所述遮挡度分别判断每一所述预测结果与其他所述预测结果之间是否发生遮挡；若所述预测结果与任何其他所述预测结果之间均未发生遮挡，则对所述预测结果和所述观测结果进行第一数据关联；若所述预测结果与其他所述预测结果之间发生遮挡， 则对所述预测结果和所述观测结果进行第二数据关联，其中，所述第一数据关联和所述第一数据关联不同。
本发明的有益效果是：提供一种视频多目标模糊数据关联的方法及装置，首先通过判定当前视频帧中所有目标的预测结果之间是否发生遮挡，在分别对目标的预测结果之间发生遮挡和不遮挡的情况，进行预测结果和观测结果间的数据关联，能够对表观相似、频繁交互、遮挡以及背景干扰等情况下的多目标进行准确跟踪，具有较强的鲁棒性和准确性。
【附图说明】
图1是本发明视频多目标模糊数据关联方法第一实施例的流程图；
图2是本发明不同目标的预测结果之间遮挡示意图；
图3是本发明视频多目标模糊数据关联方法第二实施例的流程图；
图4是图2中步骤S14一实施方式的流程示意图；
图5是图4中步骤S143一实施方式的流程示意图；
图6是本发明视频多目标模糊数据关联方法第三实施例；
图7是图6中步骤S24一实施方式的流程示意图；
图8是图7中步骤S242一实施方式的流程示意图；
图9是本发明模糊输入变量fk(i，j)、
Figure PCTCN2017088495-appb-000001
的隶属度函数示意图；
图10是本发明输出模糊变量αk的隶属度函数示意图；
图11是本发明多特征线索融合的结构示意图；
图12是图7中步骤S244一实施方式的流程示意图；
图13是本发明视频多目标模糊数据关联方法第四实施例的流程图；
图14是图13中步骤S5一实施方式的流程示意图；
图15是图14中步骤S52一实施方式的流程示意图；
图16是本发明视频多目标模糊数据关联装置第一实施例的结构示意图；
图17是本发明视频多目标模糊数据关联装置第二实施例的结构示意图。
【具体实施方式】
如图1所示，本发明视频多目标模糊数据关联方法第一实施例包括：
S1：对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果。
使用帧差法、光流法、背景减除法等运动检测算法对当前视频帧进行运动检测，以从中找出属于运动前景的像素，辅以中值滤波和简单的形态学处理，最终得到当前视频帧中的可能运动对象作为观测对象。一个观测对象是当前视频帧中的一个图像块，一般而言，观测对象的形状为矩形。
S2：计算当前视频帧中不同目标的预测结果之间的遮挡度。
本步骤中的目标的预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的。首先对于当前帧视频中的所有目标的预测结果之间进行遮挡度计算，以判定当前帧视频中的所有目标的预测结果之间是否发生遮挡。
请进一步参阅图2，图2为本发明不同目标的预测结果之间遮挡示意图。如图2所示，在当前视频帧中，预测结果A和预测结果B的跟踪图标形状均为矩形，且二者之间有重叠，且预测结果A的参数表述为：[x，y，w，d]，其中，x，y表示矩形框的坐标，w表示矩形框宽度，d表示矩形框的高度，且预测结果B的参数表述为：[x′，y′，w′，h′]，其中，x′，y′表示矩形框的坐标，w′表示矩形框宽度，h′表示矩形框的高度，预测结果A和预测结果B之间阴影部分表示为：[xo，yo，wo，ho]，且其重叠部分表示为：
Figure PCTCN2017088495-appb-000002
由此可知，预测结果A和预测结果B之间重叠部分的面积表示为wo*ho。若上述的wo、ho不满足wo＞0或者ho＞0，则两跟踪矩形框之间不构成重叠矩形，也即重叠矩形面积为0。
假设预测结果A与预测结果B发生如图2所示的遮挡，且两跟踪矩形框之间重叠的阴影部分表示遮挡区域，则定义二者之间的遮挡度为：
Figure PCTCN2017088495-appb-000003
其中，s(·)表示区域面积，且遮挡度满足0≤ω(A，B)≤1。当ω(A，B)大于0，则说明预测结果A与预测结果B之间发生遮挡。且进一步由分别代表预测结果A与预测结果B的两跟踪矩形框底部的纵向图像坐标值yA与yB可知，若yA＞yB，则说明预测结果B被预测结果A遮挡，反之，则说明预测结果A被预测结果B遮挡。
S3：根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡。
本步骤中，对于当前视频帧场景中的全部目标的预测结果进行遮挡度判断，并按照式(1)计算当前视频帧不同目标预测结果之间的跟踪矩形框的重叠率ωij(不同目标即预测结果之间的遮挡度)，并判断每一预测结果与其他预测结果的遮挡度是否小于第一遮挡判定阈值τover。其中，第一遮挡判定阈值τover满足τover∈[0，1]。若ωij小于第一遮挡判定阈值τover则认为预测结果之间发生遮挡，若τover等于0，则表明预测结果之间未发生遮挡。
S4：若预测结果与任何其他预测结果之间均未发生遮挡，则对预测结果和观测结果进行第一数据关联；若预测结果与其他预测结果之间发生遮挡，则对预测结果和观测结果进行第二数据关联。
由步骤S3对当前视频帧中所有目标的预测结果进行遮挡度判定后，对预测结果与任何其他预测结果之间均未发生遮挡的，将其与当前视频帧中的观测结果进行第一数据关联。进一步，对预测结果与其他预测结果之间发生遮挡，将其与当前视频帧中的观测结果进行第二数据关联。其中，第一数据关联和第一数据关联不同，且第二数据关联比第一数据关联复杂。
上述实施方式中，首先通过判定当前视频帧中所有目标的预测结果之间是否发生遮挡，在分别对目标的预测结果之间发生遮挡和不遮挡的情况，进行预测结果和观测结果间的数据关联，能够对表观相似、频繁交互、遮挡以及背景干扰等情况下的多目标进行准确跟踪，具有较强的鲁棒性和准确性。
请参阅图3，图3为本发明视频多目标模糊数据关联方法第二实施例，是在本发明视频多目标模糊数据关联方法第一实施例的基础上，是对本发明视频多目标模糊数据关联方法第一实施例的进一步扩展，因此与本发明视频多目标模糊数据关联方法第一实施例相同的步骤在此不再赘述。本实施例包括：
S11：对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果。
S12：计算当前视频帧中不同目标的预测结果之间的遮挡度。
S13：根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡。
S14：若预测结果与任何其他预测结果之间均未发生遮挡，则对预测结果和观测结果进行第一数据关联。
参阅图4，步骤S14进一步包括如下子步骤：
S141，计算观测结果和预测结果之间的第一相似性度量。
采用第一相似性度量对预测结果和与观测结果间的距离进行度量。其中，第一相似性度量包括：空间距离特征相似性度量以及外观特征相似性度量。通常，目标在相邻帧图像之间的位置不会发生较大变化，因此，空间距离特征是能够较为有效地匹配目标的观测结果与预测结果的特征之一。在具体实施例中，观测结果d与预测结果o之间的空间距离特征相似性度量fD(·)定义为：
Figure PCTCN2017088495-appb-000004
其中，||·||2为二范数，(xo，yo)为预测结果o在当前视频帧中的中心坐标，(xd，yd)为观测结果d在当前视频帧中的中心坐标，ho为预测结果o在当前视频帧中的高度，
Figure PCTCN2017088495-appb-000005
为方差常量。
进一步，观测结果d与预测结果o之间的外观特征相似性度量fS(·)定义为：
Figure PCTCN2017088495-appb-000006
其中，hd为观测结果d在当前视频帧中的高度，
Figure PCTCN2017088495-appb-000007
为方差常量。
S142，利用第一相似性度量计算观测结果和预测结果之间的关联代价矩阵。
采用乘性融合对空间距离特征相似性度量以及外观特征相似性度量进行融合，以得到观测结果和预测结果之间的关联度，且定义为：
sij＝fD(o，d)×fs(o，d)  (4)
根据关联度得到观测结果和预测结果之间的关联代价矩阵，且定义为：
S＝[sij]n×l  (5)
其中，i＝1，2，…n，j＝1，2，…，l。
S143，采用贪婪算法对关联代价矩阵进行优化求解，找出关联的观测结果和预测结果。
采用贪婪算法实现预测结果与观测结果之间的正确的关联，从而得到预测结果与观测结果之间的关联对。请参阅图5，该步骤S143进一步包括如下子步骤：
S1431，找出关联代价矩阵S中未被标记的所有元素中的最大值。
找出关联代价矩阵S中未被标记的所有元素中的最大值Spq＝max([Sij]n*l)，其中，p＝1，2，3......n，q＝1，2，3......l，并标记该关联代价矩阵S中该最大值spq所在的第p行及第q列的所有元素。
S1432，判断最大值是否为所在行列中的最大值，且满足大于第一阈值。
判断该最大值spq大是否为所在行和所在列中的最大值，即是否满足：spq≥{spj}j＝1，2，…l、spq≥{siq}i＝1，2，…，n。进一步判断该最大值spq是否大于第一阈值λ1，即预测结果p与观测结果q的关联概率是否大于第一阈值λ1，且该第一阈值的满足λ1∈[0.6，0.9]。
S1433，若大于，则观测结果和预测结果正确关联。
该最大值spq满足上述判定条件，则认为预测结果p与观测结果q和之间正确关联，则将该关联对记录进已关联预测结果和观测结果的集合中。循环执行上述步骤直至关联代价矩阵S中的所有行或所有列均被标记。
上述实施方式，通过判定当前视频帧中目标的预测结果之间未发生遮挡，对观测结果和预测结果之间空间距离特征相似性度量以及外观特征相似性度量进行融合以得到二者的关联代价矩阵，优化求解能够找出正确关联的观测结果和预测结果。
请参阅图6，图6为本发明视频多目标模糊数据关联方法第三实施例，是在本发明视频多目标模糊数据关联方法第一实施例的基础上，是对本发明视频多目标模糊数据关联方法第一实施例的进一步扩展，因此与本发明视频多目标模糊数据关联方法第一实施例相同的步骤在此不再赘述。本实施例包括：
S21：对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果。
S22：计算当前视频帧中不同目标的预测结果之间的遮挡度。
S23：根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡。
S24：若预测结果与其他预测结果之间发生遮挡，则对预测结果和所测结果进行第二数据关联。
在视频帧中不同目标的预测结果之间有遮挡的情况下，由于采用简单的乘性融合策略对两种特征进行融合不能完成预测结果与观测结果之间的关联，这种情况下采用基于模糊逻辑多特征线索加权的融合策略。同时，目标的预测结果处于遮挡状态后，由于其丢失了大部分外形特征，所以不再更新其模型，而此时最可靠的是预测结果的运动信息，因为目标的预测结果运动会存在惯性，其运行速度在短时间内一般不会发生很大变化，可以根据的预测结果发生遮挡前的位置，利用卡尔曼(Kalman)滤波对目标预测结果的运动轨迹进行预测。
请参阅图7，步骤S24进一步包括如下子步骤：
S241：计算观测结果和预测结果之间的第二相似性度量。
在当前视频帧中，由于颜色特征具有较好的抵抗目标形变的能力，但其缺少对预测结果空间结构的描述，且对光照敏感，而边缘特征能够很好地描述人体的边缘，它对光照变化和小量的偏移不敏感，因此颜色与边缘特征具有互补特性，故本发明采用这两种信息融合建立预测结果的外观特征。在本发明中，利用第二相似性度量对观测结果和预测结果之间的距离进行度量，且该第二相似性度量包括外观特征相似性度量、几何形状特征相似性度量、运动特征相似性度量以及空间距离特征相似性度量。
其中，观测结果d与预测结果o之间的外观特征相似性度量fA(·)定义为：
Figure PCTCN2017088495-appb-000008
其中，ρ(·)为巴氏(Bhattacharyya)系数，Hc(·)为所述当前视频帧图像背景加权的颜色直方图特征，Hg(·)为分块梯度方向直方图特征，
Figure PCTCN2017088495-appb-000009
为方差常量，为方差常量。
观测结果d与预测结果o之间的运动特征相似性度量fM(·)定义为：
Figure PCTCN2017088495-appb-000011
其中，(x′o，y′o)为上一时刻所述预测结果o的中心坐标，(xo，yo)为所述预测结果o的中心坐标，
Figure PCTCN2017088495-appb-000012
为上一时刻所述预测结果o的速度在坐标轴上的投影，
Figure PCTCN2017088495-appb-000013
为方差常量；
观测结果d与预测结果o之间的空间距离特征相似性度量fD(·)定义为：
Figure PCTCN2017088495-appb-000014
其中，||·||2为二范数，(xo，yo)为预测结果o的中心坐标，(xd，yd)为观测结果d 的中心坐标，ho为预测结果o的高度，
Figure PCTCN2017088495-appb-000015
为方差常量。
观测结果d与预测结果o之间的外观特征相似性度量fS(·)定义为：
Figure PCTCN2017088495-appb-000016
其中，hd为观测结果d的高度，
Figure PCTCN2017088495-appb-000017
为方差常量。
其中，外观特征相似性度量、几何形状特征相似性度量对应的目标模型与候选模型分别定义为：
目标模型：
Figure PCTCN2017088495-appb-000018
候选模型：
Figure PCTCN2017088495-appb-000019
为了衡量目标模型和候选模型之间的相似度，本发明采用巴氏(Bhattacharyya)系数来描述，且该巴氏系数定义为：
Figure PCTCN2017088495-appb-000020
目标的预测结果的运动模型由其质心的坐标和速度来描述。在视频多目标跟踪中，由于相邻两帧视频序列图像间隔时间很短，视频目标运动的机动性不是很大，在大多数情况下，可以假设每一个目标的运动为匀速运动，因此可以为每个目标的预测结果的跟踪矩形框(x，y，w，h)建立基于位置、大小以及速度的运动状态参数模型。则定义卡尔曼滤波器的状态变量Xk为：
Xk＝[x，vx，y，vy]T  (15)
其中，x、y分别表示第k帧观测结果的跟踪矩形框质心的横纵坐标，分别表示第k帧观测结果的跟踪矩形框的质心在x轴和y轴方向上的速度。
S242：采用模糊推理系统模型计算第二相似性度量中每一特征相似性度量的权重值。
其中，一个模糊推理系统包括n个输入变量x＝[x1，x2，…，xn]T，且每一输入变量x的每一分量xi的模糊语言变量集合定义为：
Figure PCTCN2017088495-appb-000021
其中，
Figure PCTCN2017088495-appb-000022
是分量xi的第j个语言变量值，且
Figure PCTCN2017088495-appb-000023
为定义在论域的Ui一个模糊集合，其隶属度函数定义为
Figure PCTCN2017088495-appb-000024
其中，i＝1，2，…，n；j＝1，2，…，mi。
进一步，一个模糊推理系统还包括m个输出模糊变量y，每一输出模糊变量y的模糊语言变量集合定义为：
Figure PCTCN2017088495-appb-000025
其中，Bj(j＝1，2，…，my)是所述输出模糊变量y的第j个语言变量值，且Bj(j＝1，2，…，my)为定义在论域的Ui一个模糊集合，其隶属度函数定义为
Figure PCTCN2017088495-appb-000026
进一步，该模糊推理系统的第i条模糊推理规则定义为：
Figure PCTCN2017088495-appb-000027
其中，i＝1，2，…，m，m为规则总数，且每条模糊推理规则的适用度定义为：
Figure PCTCN2017088495-appb-000028
通过模糊推理系统可得到对于每条模糊规则的输出模糊集合的隶属度函数定义为：
Figure PCTCN2017088495-appb-000029
从而得到总的输出模糊集合，定义为：
Figure PCTCN2017088495-appb-000030
采用加权平均清晰化方法，得到该模糊推理系统模型总输出，定义为：
Figure PCTCN2017088495-appb-000031
其中，
Figure PCTCN2017088495-appb-000032
为隶属度函数
Figure PCTCN2017088495-appb-000033
取最大值时的点。
请参阅图8，进一步该步骤S242还包括如下子步骤：
S2421，计算模糊推理系统的输入变量。
模糊推理系统的输入为当前视频帧各特征的相似性度量，其输出为对应的各特征的权重{αk}k∈{A，M，S，D}，其中，特征k可以为包括但不限于外观特征相似性度量、几何形状特征相似性度量及运动特征相似性度量。作为一个多输入多输出的模糊控制器，控制规则复杂，计算量庞大，为降低计算量，减少控制规则，对此控制器进行简化。由于每个特征的权重值取决于该特征的相似性度量和其余特征相似性度量的相对大小，因此可以分别将每个特征的相似性度量与其佘所有特征相似性度量的均值作为模糊输入，来确定该特征的权重值。因此，该模糊控制器可以简化为四个二维模糊控制器。
在本发明的一应用场景中，以运动特征相似性度量为第一模糊输入变量，其余3个特征的相似度量均值为第二模糊输入变量，且其余3个特征的相似度量均值的计算可定义为：
Figure PCTCN2017088495-appb-000034
将
Figure PCTCN2017088495-appb-000035
分别作为模糊逻辑系统的第一及第二模糊输入变量。其中，ei为特征i的相似性度量，
Figure PCTCN2017088495-appb-000036
为t-1时刻的特征k的融合系数，fM(i，j)运动特征相似性度量，
Figure PCTCN2017088495-appb-000037
为佘3个特征相似性度量的加权均值。
S2422，采用所模糊推理系统的推理规则得到第二相似性度量中每一特征相似性度量的权重值。
请参阅图9，图9为本发明模糊输入变量fk(i，j)、
Figure PCTCN2017088495-appb-000038
的隶属度函数示意图。其中，在二维模糊控制器中，将输入变量fk(i，j)和
Figure PCTCN2017088495-appb-000039
模糊化为{Zo、PS、PM、PL、PE}，各模糊语言含义为：Zo(零)、PS(正小)、PM(正中)、PL(正大)、PE(极 大)，输入变量fk(i，j)和
Figure PCTCN2017088495-appb-000040
的隶属度函数可以参见图9。
请参阅图10，图10为本发明输出模糊变量αk的隶属度函数示意图。对于输出模糊变量αk模糊化为{VL、L、M、H、VH、EH}，各模糊语言含义为：VL(非常小)、L(小)、M(中)、H(高)、VH(非常高)以及EH(极高)输出变量αk的隶属度函数可以参见图4。
进一步，如表1所示，表1为该模糊推理系统的输入和输出变量的模糊规则表，可以看出随着特征k的相似性度量fk(i，j)的增大和其余特性相似性度量均值
Figure PCTCN2017088495-appb-000041
的减小，输出的融合权重值越大，反之越小。从而对跟踪期间赋予可靠的特征信息较大的权重值，相反，赋予不可靠的特征信息较小的权重值。
Figure PCTCN2017088495-appb-000042
表1
请一并参阅图11，图11为本发明多特征线索融合的结构示意图。参见图11，本实施例中，以运动特征相似性度量fM(i，j)为第一模糊输入变量，以外观特征相似性度量、空间距离特征相似性度量以及几何形状特征相似性度量的均值
Figure PCTCN2017088495-appb-000043
为第二模糊输入变量，来确定该运动特征相似性度量的特征权重值αM。同理，以外观特征相似性度量fS(i，j)为第一模糊输入变量，以运动特征相似性度量、空间距离特征相似性度量以及几何形状特征相似性度量的均值
Figure PCTCN2017088495-appb-000044
为第二模糊输入变量，结合模糊推理规则，能够得到该外观特征相似性度量的特征权重值αS。进一步，空间距离特征相似性度量的特征权重值以及几何形状特征相似性度量的特征权重值αD及αA的计算方法，与上述的运动特征相似性度量的特征权重值的计算方法相似，本实施例中，按照上述方法依次算出个特征相似性度量的各特征权重值{αk}k∈{A，M，S，D}，且详见上文描述，此处不再赘述。
本实施例中通过判断各个特征的可信程度，自适应给不同特征分配不同的权重值，很好地解决了在复杂背景、互相遮挡情况下的跟踪问题。
S2423，归一化每一特征相似性度量的权重值，以得到每一特征相似性度量的融合系数。
在得到上述所有特征相似度量的特征权重值后，将所有特征相似度量的特征权重值输出归一化，以得到当前时刻的各特征的融合系数，其定义为：
Figure PCTCN2017088495-appb-000045
S243，对权重值和第二相似性度量进行多特征线索融合，以得到观测结果和预测结果之间的关联代价矩阵。
采用多特征线索融合对外观特征相似性度量、几何形状特征相似性度量、运动方向特征相似性度量以及空间距离特征相似性度量进行加权和融合，可以得到预测结果和观测结果之间的关联代价矩阵，定义为：
Figure PCTCN2017088495-appb-000046
其中，{αk}k∈{A，M，S，D}为每一特征相似性度量的融合系数，且满足
Figure PCTCN2017088495-appb-000047
fk(i，j)k∈{A，M，S，D}为观测结果
Figure PCTCN2017088495-appb-000048
和预测结果
Figure PCTCN2017088495-appb-000049
之间的每一特征相似性度量。
S244，采用贪婪算法对关联代价矩阵进行优化求解，找出关联的观测结果和预测结果。
采用贪婪算法实现预测结果与观测结果之间的正确的关联，从而得到预测结果与观测结果之间的关联对。参阅图12，步骤S244进一步包括如下子步骤：
S2441，找出关联代价矩阵sij中未被标记的所有元素中的最大值。
找出关联代价矩阵sij中未被标记的所有元素中的最大值Spq＝max([Sij]n*l)，其中，p＝1，2，3......n，q＝1，2，3......l，并标记该关联代价矩阵S中该最大值spq所在的第p行及第q列的所有元素。
S2442，判断最大值是否为所在行列中的最大值，且满足大于第二阈值。
判断该最大值spq大是否为所在行和所在列中的最大值，即是否满足：spq≥{spj}j＝1，2，…l、spq≥{Siq}i＝1，2，…，r。进一步判断该最大值spq是否大于第二阈值λ2，即预测结果p与观测结果q的关联概率是否大于第二阈值λ2，且该第二阈值的满足λ2∈[0.6，0.9]。
S2443，若大于，则观测结果和预测结果正确关联。
该最大值spq满足上述判定条件，则认为预测结果p与观测结果q和之间正确关联，则将该关联对记录进已关联预测结果和观测结果的集合中。进一步，若该关联代价矩阵Sij中还存在未被标记的行和列，则继续上述步骤S2441。
上述实施方式，通过判定当前视频帧中目标的预测结果之间发生遮挡，计算预测结果和观测结果之间的第二特征相似性度量，引入模糊推理系统，利用基于模糊逻辑的方法，根据当前跟踪环境自适应给不同特征信息分配不同的权重值，得到多属性特征的加权和融合，构成该帧目标的预测结果与观测结果的关联代价矩阵，再使用贪婪算法优化求解分配，能够有效提高多目标与观测之 间的正确关联，对表观相似、频繁交互、遮挡以及背景干扰等情况下的多目标进行准确跟踪，具有较强的鲁棒性和准确性。
请参阅图13，图13为本发明视频多目标模糊数据关联方法第四实施例的流程图，是在本发明视频多目标模糊数据关联方法第一至三实施例的基础上，是对本发明视频多目标模糊数据关联方法第一至三实施例的进一步扩展，因此与本发明视频多目标模糊数据关联方法第一至三实施例中相同的步骤在此不再赘述。在上述实施例的基础上，本实施例进一步包括：
S5，对未被关联上的观测结果和未被关联上的预测结果进行第三数据关联。
在对当前视频帧场景中的全部目标的预测结果首先进行遮挡度判断，对预测结果与其它预测结果间均未发生遮挡的，将其与观测结果进行第一数据关联，以得到正确关联的预测结果和观测结果，并记录进已关联预测结果和观测结果的集合中。对预测结果与其他预测结果之间发生遮挡的，将其与观测结果进行第二数据关联，以得到正确的关联的预测结果和观测结果，并记录进已关联预测结果和观测结果的集合中。
对于上述经过第一数据关联和第二数据关联后仍未关联上的预测结果和观测结果，则本发明进一步对其采用第三数据关联，参阅图14，该步骤S5进一步包括如下子步骤：
S51，计算经过第一数据关联和第二数据关联之后的未被关联上的观测结果和所未被关联上的预测结果之间的遮挡度，得到两者之间的遮挡度矩阵。
对经过第一数据关联和第二数据关联之后的所有未被关联上的预测目标及观测结果按照第一实施例中的式(2)进行遮挡度计算，并进一步得到二者的遮挡度矩阵。
S52，使用贪婪算法求解遮挡度矩阵，找出关联的观测结果和预测结果。
步骤S52进一步包括如下子步骤：
S521，判断未被关联上的观测结果和未被关联上的预测结果之间的遮挡度是否大于第二遮挡判定阈值。
其中，第二遮挡判定阈值τ2满足τ2(τ2∈[0.4，0.6])。
S522，若大于遮第二遮挡判定阈值，则未被关联上的观测结果和未被关联上的预测结果之间关联。
若被关联上的观测结果和未被关联上的预测结果之间的遮挡度大于第二遮挡判定阈值τ2，则二者正确关联并记录进已关联的预测结果和观测结果的集合中。此外，对于小于第二遮挡判定阈值τ2的未被关联上的预测结果记录进未关联预测结果集合中。
S523，计算小于第二遮挡判定阈值的未被关联上的观测结果和当前视频帧中所有预测结果之间的遮挡度。
本步骤中，对于小于第二遮挡判定阈值的未被关联上的观测结果，将其与 当前视频帧中所有的预测结果按照第一实施例中的式(2)进行遮挡度的计算。
S524，若遮挡度大于第三遮挡判定阈值，则未被关联上的观测结果为虚假观测。
判定该遮挡度是否大于第三遮挡判定阈值τ3，其中，该第三遮挡判定阈值τ3满足τ3(τ3∈[0.4，0.6])。若大于，则该观测结果为虚假观测，将其剔除。相反，若该遮挡度小于第三遮挡判定阈值τ3，则将该未被关联上的观测结果记录进未关联观测结果集合中。
S6，对经过第一数据关联、第二数据关联以及第三数据关联后仍未关联上的观测结果和预测结果进行轨迹管理。
对未被关联的观测结果和未被关联的预测结果进行轨迹管理，以建立临时目标轨迹、删除无效目标包括：对未被关联的观测结果，利用未被关联的观测结果与预测结果之间的遮挡度，获取未被关联的观测结果的判别函数，根据判别函数判断是否为未被关联的观测结果建立临时目标轨迹，对未被关联的预测结果，若未被关联的预测结果对应的目标连续K帧未被关联，则判断目标无效并删除无效目标，K为大于1的整数。
其中，未被关联的观测结果的集合为Ω＝{z1，...，zm}，目标的预测结果的集合为O＝{o1，...，ol}；利用未被关联的观测结果与预测结果之间的遮挡度，获取未被关联的观测结果的判别函数包括：按照式(2)获取未被关联的观测结果z∈Ω与目标预测结果o∈O之间的遮挡度，并获取每个未被关联的观测结果z∈Ω的判别函数：
Figure PCTCN2017088495-appb-000050
其中γ为常量参数，且0＜γ＜1；
根据判别函数判断是否为未被关联的观测结果建立临时目标轨迹包括：对每个未被关联的观测结果，若其判别函数为1，则为未被关联的观测结果建立临时目标轨迹，若其判别函数为0则不建立。
其中，对所有有效目标轨迹和临时目标轨迹进行滤波以得到当前视频帧的目标状态的集合，并利用当前视频帧的目标状态的集合进行预测包括：使用卡尔曼滤波器对有效目标轨迹和临时目标轨迹进行滤波和预测。
上述实施方式，通过判定当前视频帧中目标的预测结果之间是否发生遮挡，针对不同的遮挡情况，分别计算预测结果和观测结果进行第一及第二数据关联，在进行第一及第二数据关联后，对仍未被关联上的预测结果和观测结果进行第三数据关联，能够有效提高多目标与观测之间的正确关联，对表观相似、频繁交互、遮挡以及背景干扰等情况下的多目标进行准确跟踪，具有较强的鲁棒性和准确性。
下面为使用本发明视频多目标模糊数据关联方法一实施例进行实验验证比较的结果，本实施例是本发明第一至第四实施例的结合，且本实施例中分别使用2个公开的PETS2009数据集和TownCentre数据集进行验证。
2)实验对比算法
本实施例中，选用目前性能最优的两种算法与本实施例中的算法进行比较：
1.基于判别性表观模型学习和轨迹置信度的视频多目标在线跟踪算法(TC_ODAL)。
在该算法提出了基于轨迹可检测性和连续性的轨迹置信度，对于高置信度的轨迹与在线检测响应进行局部关联，而低置信度的轨迹片段与其他原有轨迹和检测响应进行全局关联，为了能使轨迹的表观特性更具区别性，其采用了一种基于增量线性判别分析算法，将轨迹的表观模型投影到判别性投影空间中。
2.基于自我监督表观模型学习的关联跟踪算法(ALExTRAC)。在该算法中提出了一种自我监督的方法去学习在视频序列中关联目标与观测，其通过学习一个外观相似性模型来估计数据关联的代价值，并在滑动窗口内收集外观显著变化的样本集用于强化相似性模型。
3)实验运行性能
本发明中的算法在CPU为Intel Core i5，3.10GHz和内存为4GB的PC机上进行Matlab仿真实现。本发明提出的基于模糊逻辑的视频多目标跟踪算法与对比算法的平均处理速度如表3所示。本发明提出的算法的运行速度与场景中的目标数量及视频片段的帧数有较大的关系。对于行人密度较低的PETS2009-S2L1视频序列，算法平均速度为1.8f/s，而对于行人密度很大的PETS2009-S2L2视频序列，算法平均速度为0.8f/s，而对于高频率遮挡的TownCentre视频序列，算法平均速度为0.7f/s。本发明提出的算法处理速度主要的损耗在于目标的预测结果外观模型的构建，多属性特征相似性的计算，对于目标的观测结果与观测结果的关联代价需要进行三次模糊系统推理过程。详见表3为本发明基于模糊逻辑的视频多目标跟踪算法与对比算法的平均处理速度。
Figure PCTCN2017088495-appb-000051
表3
4)跟踪性能比较
表3给出了本发明算法与上述的两个多目标跟踪算法在视频序列PETS2009.S2L1上的平均性能比较。为了保证比较的公平性，本发明采用TC_ODAL算法和ALExTRAC算法都相同的目标检测结果和真实位置数据，并使用相同的评估代码计算性能指标。进一步请参阅表4，表4为本发明视频序列PETS2009.S2L1上的测试结果，如下：
Figure PCTCN2017088495-appb-000052
表4
从表4可以看出，本发明所提出的MOT_FL算法在多目标跟踪准确度(MOTA)这一性能指标上分别高于TC_ODAL算法和ALExTRAC算法接近5％、20％，这说明提出的模糊数据关联方法能够有效地处理检测结果中存在的噪声。然而在多目标跟踪精确度(MOTP)和轨迹中断次数(FG)这两项性能指标上表现的略差，这表明本申请提出方法的跟踪结果与真实目标存在一定的偏差，主要是因为视频序列中的大多数目标不满足简单的匀速运动模型，因此由Kalman滤波后的目标状态的精度比较低。与TC_ODAL算法和ALExTRAC算法相比，本发明方法明显减少了漏检(F.Neg)，同时在绝大多数正确跟踪的轨迹数目(MT)和绝大多数错误跟踪的轨迹数目(ML)这两项指标上略优于对比算法，则表明本发明提出的算法能实现对多个目标的准确跟踪。
在本发明一具体的应用场景中，本发明视频多目标模糊数据关联算法应用在比较拥堵的视频序列PETS2009.S2L2上，并与上述两种多目标跟踪算法的平均性能进行了对比，请参见表5，表5为本发明视频序列PETS2009.S2L2上的测试结果，如下：
Figure PCTCN2017088495-appb-000053
表5
由表5所得的数据可知，与TC_ODAL算法和ALExTRAC算法相比，本申请所提出的基于模糊逻辑的视频多目标跟踪算法大幅度减少了漏检(F.Neg)以及目标身份交换次数(IDS)，表示整体性能的多目标跟踪准确度(MOTA)分数的显著性提升。这表明模糊数据关联的框架的有效地实现了目标与检测响应的数据关联。本申请算法在绝大多数正确跟踪的轨迹数目(MT)和绝大多数错误跟踪的轨迹数目(ML)这两项指标上优于TC_ODAL算法和ALExTRAC算 法，且进一步表明本申请提出的视频多目标模糊数据关联算法能提供更加完整的目标轨迹。
为了进一步验证本申请视频多目标模糊数据关联算法的有效性，本申请在行人间容易出现频繁动态遮挡情况的视频序列TownCentre上进行了试验，与其他两种多目标跟踪算法的对比实验结果如表6所示，表6为视频序列TownCentre上的测试结果，具体如下：
Figure PCTCN2017088495-appb-000054
表6
从表6可以看出，本申请中所提出的视频多目标模糊数据关联算法在多目标跟踪准确度(MOTA)这一性能指标上分别高于TC_ODAL算法和ALExTRAC算法，这说明提出的方法能够有效地处理检测结果中存在的噪声。同时，在绝大多数正确跟踪的轨迹数目(MT)和绝大多数错误跟踪的轨迹数目(ML)这两项指标上略优于对比算法，则表明本申请提出的算法能实现对多个目标的准确跟踪，而且能提供更加完整的目标轨迹输出。这归功于基于模糊逻辑的数据关联的有效性。而在目标身份交换次数(IDS)上要优于ALExTRAC算法但差于TC_ODAL算法。由于本文MOT_FL算法假设为每一个目标运动为匀速运动，但由于目标的非线性运动状态的突变，MOT_FL算法无法准确的对这种情况进行准确估计与预测，其在轨迹中断次数(FG)这一指标上要差于TC_ODAL算法和ALExTRAC算法。
如图16所示，图16为本发明视频多目标模糊数据关联装置第一实施例的结构示意图，包括：
检测模块11，用于对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果。
计算模块12，用于计算当前视频帧中不同目标的预测结果之间的遮挡度，其中预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的。
判断模块13，用于根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡。
数据关联模块14，用于若预测结果与任何其他预测结果之间均未发生遮挡，则对预测结果和观测结果进行第一数据关联；若预测结果与其他预测结果之间发生遮挡，则对预测结果和观测结果进行第二数据关联，其中，第一数据关联 和第二数据关联不同。
如图17所示，图17为本发明视频多目标模糊数据关联装置第二实施例的结构示意图，包括：处理器110和摄像机120。
其中，摄像机120可以为本地摄像机，处理器110通过总线连接摄像机120；摄像机120也可以为远程摄像机，处理器110通过局域网或互联网连接摄像机120。
处理器110控制视频多目标模糊数据关联装置的操作，处理器110还可以称为CPU(Central Processing Unit，中央处理单元)。处理器110可能是一种集成电路芯片，具有信号的处理能力。处理器110还可以是通用处理器、数字信号处理器(DSP)、专用集成电路(ASIC)、现成可编程门阵列(FPGA)或者其他可编程逻辑器件、分立门或者晶体管逻辑器件、分立硬件组件。通用处理器可以是微处理器或者该处理器也可以是任何常规的处理器等。
视频多目标模糊数据关联装置可以进一步包括存储器(图中未画出)，存储器用于存储处理器110工作所必需的指令及数据，也可以存储传输器120拍摄的视频数据。
处理器110用于对从摄像机120获取的当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；计算当前视频帧中不同目标的预测结果之间的遮挡度，其中预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的；根据遮挡度分别判断每一预测结果与其他预测结果之间是否发生遮挡；若预测结果与任何其他预测结果之间均未发生遮挡，则对预测结果和观测结果进行第一数据关联；若预测结果与其他预测结果之间发生遮挡，则对预测结果和观测结果进行第二数据关联，其中，第一数据关联和第一数据关联不同。
本发明视频多目标模糊数据关联装置包括的各部分的功能可参考本发明在线目标跟踪方法各对应实施例中的描述，在此不再赘述。
综上所述，本领域技术人员容易理解，本发明提供一种视频多目标模糊数据关联方法及装置，通过判定当前视频帧中目标的预测结果之间是否发生遮挡，针对不同的遮挡情况，分别计算预测结果和观测结果进行第一及第二数据关联，在进行第一及第二数据关联后，对仍未被关联上的预测结果和观测结果进行第三数据关联，能够有效提高多目标与观测之间的正确关联，对表观相似、频繁交互、遮挡以及背景干扰等情况下的多目标进行准确跟踪，具有较强的鲁棒性和准确性。
以上所述仅为本发明的实施方式，并非因此限制本发明的专利范围，凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换，或直接或间接运用在其他相关的技术领域，均同理包括在本发明的专利保护范围内。

Claims (12)

  1. 一种视频多目标模糊数据关联方法，其特征在于，所述方法包括：
    对当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；
    计算所述当前视频帧中不同目标的预测结果之间的遮挡度，其中所述预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的；
    根据所述遮挡度分别判断每一所述预测结果与其他所述预测结果之间是否发生遮挡；
    若所述预测结果与任何其他所述预测结果之间均未发生遮挡，则对所述预测结果和所述观测结果进行第一数据关联；若所述预测结果与其他所述预测结果之间发生遮挡，则对所述预测结果和所述观测结果进行第二数据关联，其中，所述第一数据关联和所述第二数据关联不同。
  2. 根据权利要求1所述的方法，其特征在于，不同预测结果A与预测结果B之间的遮挡度定义为：
    Figure PCTCN2017088495-appb-100001
    其中，s(·)表示区域面积。
  3. 根据权利要求1所述的方法，其特征在于，若所述预测结果与任何其他所述预测结果之间均未发生遮挡，则对所述预测结果和所述观测结果进行第一数据关联包括：
    计算所述观测结果和所述预测结果之间的第一相似性度量，所述第一相似性度量包括空间距离特征相似性度量以及外观特征相似性度量；
    利用所述第一相似性度量计算所述观测结果和所述预测结果之间的关联代价矩阵；
    采用贪婪算法对所述关联代价矩阵进行优化求解，找出关联的所述观测结果和所述预测结果。
  4. 根据权利要求3所述的方法，其特征在于，观测结果d与预测结果o之间的所述空间距离特征相似性度量fD(·)定义为：
    Figure PCTCN2017088495-appb-100002
    其中，||·||2为二范数，(xo，yo)为所述预测结果o的中心坐标，(xd，yd)为所述观测结果d的中心坐标，ho为所述预测结果o的高度，
    Figure PCTCN2017088495-appb-100003
    为方差常量；
    所述观测结果d与所述预测结果o之间的所述外观特征相似性度量fS(·)定义为：
    Figure PCTCN2017088495-appb-100004
    其中，hd为所述观测结果d的高度，
    Figure PCTCN2017088495-appb-100005
    为方差常量；
    利用所述第一相似性度量计算所述观测结果和所述预测结果之间的关联代价矩阵包括：
    采用乘性融合对所述空间距离特征相似性度量以及外观特征相似性度量进行融合，以得到所述观测结果和所述预测结果之间的关联度，定义为：
    sij＝fD(o，d)×fs(o，d)(4)
    根据所述关联度得到所述观测结果和所述预测结果之间的关联代价矩阵，定义为：
    S＝[sij]n×l(5)
    其中，i＝1，2，…n，j＝1，2，…，l。
  5. 根据权利要求3所述的方法，其特征在于，所述采用贪婪算法对所述关联代价矩阵进行优化求解，找出关联的观测结果和预测结果包括：
    找出所述关联代价矩阵S中未被标记的所有元素中的最大值；
    判断所述最大值是否为所在行列中的最大值，且大于第一阈值；
    若大于，则所述观测结果和所述预测结果正确关联。
  6. 根据权利要求1所述的方法，其特征在于，若所述预测结果与其他所述预测结果之间发生遮挡，则对所述预测结果和所述观测结果进行第二数据关联包括：
    计算所述观测结果和所述预测结果之间的第二相似性度量，所述第二相似性度量包括外观特征相似性度量、几何形状特征相似性度量、运动特征相似性度量以及空间距离特征相似性度量；
    采用模糊推理系统模型计算所述第二相似性度量中每一特征相似性度量的权重值；
    对所述权重值和所述第二相似性度量进行多特征线索融合，以得到所述观测结果和所述预测结果之间的关联代价矩阵；
    采用贪婪算法对所述关联代价矩阵进行优化求解，找出关联的所述观测结果和所述预测结果。
  7. 根据权利要求6所述的方法，其特征在于，观测结果d与预测结果o之间的外观特征相似性度量fA(·)定义为：
    Figure PCTCN2017088495-appb-100006
    其中，ρ(·)为巴氏系数，Hc(·)为所述当前视频帧图像背景加权的颜色直方图特征，Hg(·)为分块梯度方向直方图特征，
    Figure PCTCN2017088495-appb-100007
    为方差常量，
    Figure PCTCN2017088495-appb-100008
    为方差常量；
    所述观测结果d与所述预测结果o之间的运动特征相似性度量fM(·)定义为：
    Figure PCTCN2017088495-appb-100009
    其中，(x′o，y′o)为上一时刻所述预测结果o的中心坐标，(xo，yo)为所述预测结果o的中心坐标，
    Figure PCTCN2017088495-appb-100010
    为上一时刻所述预测结果o的速度在坐标轴上的投影，
    Figure PCTCN2017088495-appb-100011
    为方差常量；
    所述观测结果d与所述预测结果o之间的所述空间距离特征相似性度量fD(·)定义为：
    Figure PCTCN2017088495-appb-100012
    其中，||·||2为二范数，(xo，yo)为所述预测结果o的中心坐标，(xd，yd)为所述观测结果d的中心坐标，ho为所述预测结果o的高度，
    Figure PCTCN2017088495-appb-100013
    为方差常量；
    所述观测结果d与所述预测结果o之间的所述几何形状特征相似性度量fS(·)定义为：
    Figure PCTCN2017088495-appb-100014
    其中，hd为所述观测结果d的高度，
    Figure PCTCN2017088495-appb-100015
    为方差常量；
    所述采用模糊推理系统计算所述第二相似性度量中每一特征相似性度量的权重值包括：
    计算所述模糊推理系统的输入变量；
    采用所述模糊推理系统的推理规则得到所述第二相似性度量中每一特征相似性度量的权重值；
    归一化所述每一特征相似性度量的权重值，以得到每一所述特征相似性度量的融合系数。
  8. 根据权利要求6所述的方法，其特征在于，所述模糊推理系统包括n个输入变量x，每一所述输入变量x的每一分量xi的模糊语言变量集合定义为：
    Figure PCTCN2017088495-appb-100016
    其中，
    Figure PCTCN2017088495-appb-100017
    是所述分量xi的第j个语言变量值，所述
    Figure PCTCN2017088495-appb-100018
    为定义在论域的Ui一个模糊集合，其隶属度函数定义为
    Figure PCTCN2017088495-appb-100019
    其中，i＝1，2，…，n；j＝1，2，…，mi；
    所述模糊推理系统包括m个输出模糊变量y变，每一所述输出模糊变量y的模糊语言变量集合定义为：
    Figure PCTCN2017088495-appb-100020
    其中，Bj(j＝1，2，…，my)是所述输出模糊变量y的第j个语言变量值，所述Bj(j＝1，2，…，my)为定义在论域的Ui一个模糊集合，其隶属度函数定义为
    Figure PCTCN2017088495-appb-100021
    所述模糊推理系统的第i条模糊推理规则定义为：
    Figure PCTCN2017088495-appb-100022
    其中，i＝1，2，…，m，m为规则总数，且每条所述模糊推理规则的适用度定义为：
    Figure PCTCN2017088495-appb-100023
    所述每条模糊推理规则的输出模糊集合的隶属度函数为：
    Figure PCTCN2017088495-appb-100024
    总的所述输出模糊集合，定义为：
    Figure PCTCN2017088495-appb-100025
    采用加权平均清晰化方法，得到所述模糊推理系统模型总输出，定义为：
    Figure PCTCN2017088495-appb-100026
    其中，
    Figure PCTCN2017088495-appb-100027
    为所述隶属度函数
    Figure PCTCN2017088495-appb-100028
    取最大值时的点。
  9. 根据权利要求6所述的方法，其特征在于，所述观测结果和所述预测结果之间的关联代价矩阵定义为：
    Figure PCTCN2017088495-appb-100029
    其中，{αk}k∈{A，M，S，D}为每一所述特征相似性度量的融合系数，且满足
    Figure PCTCN2017088495-appb-100030
    为所述观测结果
    Figure PCTCN2017088495-appb-100031
    和所述预测结果
    Figure PCTCN2017088495-appb-100032
    之间的每一所述特征相似性度量；
    所述采用贪婪算法对所述关联代价矩阵进行优化求解，找出关联的观测结果和预测结果包括：
    找出所述关联代价矩阵sij中未被标记的所有元素中的最大值；
    判断所述最大值是否为所在行列中的最大值，且满足大于第二阈值；
    若大于，则所述观测结果和所述预测结果正确关联。
  10. 根据权利要求1-9中任一所述的方法，其特征在于，进一步包括：
    对未被关联上的观测结果和未被关联上的预测结果进行第三数据关联，包括：
    计算经过所述第一数据关联和所述第二数据关联之后的所述未被关联上的观测结果和所述未被关联上的预测结果之间的遮挡度，得到两者之间的遮挡度矩阵；
    使用贪婪算法求解遮挡度矩阵，找出关联的所述观测结果和所述预测结果，包括：
    判断所述未被关联上的观测结果和所述未被关联上的预测结果之间的遮挡度是否大于第二遮挡判定阈值；
    若大于所述遮第二遮挡判定阈值，则所述未被关联上的观测结果和所述未被关联上的预测结果之间关联；
    计算小于所述第二遮挡判定阈值的所述未被关联上的观测结果和所述当前视频帧中所有所述预测结果之间的遮挡度；
    若所述遮挡度大于第三遮挡判定阈值，则所述未被关联上的观测结果为虚假观测。
  11. 根据权利要求1-10所述的方法，其特征在于，进一步包括：
    对经过所述第一数据关联、所述第二数据关联以及所述第三数据关联后仍未关联上的所述观测结果和所述预测结果进行轨迹管理。
  12. 一种视频多目标模糊数据关联的装置，其特征在于，包括：处理器和摄像机，所述处理器连接所述摄像机；
    所述处理器用于对对从所述摄像机获取的当前视频帧进行在线目标运动检测，检测得到的可能运动对象作为观测结果；计算所述当前视频帧中不同目标的预测结果之间的遮挡度，其中所述预测结果是至少利用前一视频帧的目标的轨迹进行预测而得到的；根据所述遮挡度分别判断每一所述预测结果与其他所述预测结果之间是否发生遮挡；若所述预测结果与任何其他所述预测结果之间均未发生遮挡，则对所述预测结果和所述观测结果进行第一数据关联；若所述预测结果与其他所述预测结果之间发生遮挡，则对所述预测结果和所述观测结果进行第二数据关联，其中，所述第一数据关联和所述第一数据关联不同。
PCT/CN2017/088495 2017-06-15 2017-06-15 视频多目标模糊数据关联方法及装置 Ceased WO2018227491A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/088495 WO2018227491A1 (zh) 2017-06-15 2017-06-15 视频多目标模糊数据关联方法及装置

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/CN2017/088495 WO2018227491A1 (zh) 2017-06-15 2017-06-15 视频多目标模糊数据关联方法及装置

Publications (1)

Publication Number Publication Date
WO2018227491A1 true WO2018227491A1 (zh) 2018-12-20

Family

ID=64658991

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2017/088495 Ceased WO2018227491A1 (zh) 2017-06-15 2017-06-15 视频多目标模糊数据关联方法及装置

Country Status (1)

Country Link
WO (1) WO2018227491A1 (zh)

Cited By (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111292352A (zh) * 2020-01-20 2020-06-16 杭州电子科技大学 多目标跟踪方法、装置、设备及存储介质
CN112148256A (zh) * 2020-09-11 2020-12-29 中国运载火箭技术研究院 标志类变量容错方法、装置及计算机存储介质、电子设备
CN112991280A (zh) * 2021-03-03 2021-06-18 望知科技(深圳)有限公司 视觉检测方法、系统及电子设备
CN113808158A (zh) * 2020-06-15 2021-12-17 中移(苏州)软件技术有限公司 视频中群体对象运动的分析方法、装置、设备及存储介质
CN114241373A (zh) * 2021-12-14 2022-03-25 北京理工大学重庆创新中心 一种端到端的车辆行为检测方法、系统、设备及存储介质
CN116109965A (zh) * 2022-12-13 2023-05-12 济南铁路信息技术有限公司 一种语义关联的多源监控视频融合分析方法及系统
CN116228814A (zh) * 2023-01-13 2023-06-06 湖北星纪时代科技有限公司 目标追踪的数据关联方法、电子设备及存储介质
CN116543417A (zh) * 2023-05-04 2023-08-04 重庆特斯联启智科技有限公司 人体姿态估计方法、装置、设备以及存储介质
CN116718197A (zh) * 2023-08-09 2023-09-08 腾讯科技（深圳）有限公司 轨迹处理方法、装置、电子设备及存储介质
CN116954113A (zh) * 2023-06-05 2023-10-27 深圳市机器时代科技有限公司 智能机器人驱动传感智能控制系统及其方法
CN121415352A (zh) * 2025-12-29 2026-01-27 上海克劳安智能科技有限公司 基于人脸识别的安全防范管理方法

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103281477A (zh) * 2013-05-17 2013-09-04 天津大学 基于多级别特征数据关联的多目标视觉跟踪方法
CN104091348A (zh) * 2014-05-19 2014-10-08 南京工程学院 融合显著特征和分块模板的多目标跟踪方法
WO2016077026A1 (en) * 2014-11-12 2016-05-19 Nec Laboratories America, Inc. Near-online multi-target tracking with aggregated local flow descriptor (alfd)
CN105894542A (zh) * 2016-04-26 2016-08-24 深圳大学 一种在线目标跟踪方法及装置
CN106846361A (zh) * 2016-12-16 2017-06-13 深圳大学 基于直觉模糊随机森林的目标跟踪方法及装置
CN106846355A (zh) * 2016-12-16 2017-06-13 深圳大学 基于提升直觉模糊树的目标跟踪方法及装置
CN106846373A (zh) * 2016-11-16 2017-06-13 浙江工业大学 一种融合目标外观模型和博弈论的视频目标互遮挡处理方法

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103281477A (zh) * 2013-05-17 2013-09-04 天津大学 基于多级别特征数据关联的多目标视觉跟踪方法
CN104091348A (zh) * 2014-05-19 2014-10-08 南京工程学院 融合显著特征和分块模板的多目标跟踪方法
WO2016077026A1 (en) * 2014-11-12 2016-05-19 Nec Laboratories America, Inc. Near-online multi-target tracking with aggregated local flow descriptor (alfd)
CN105894542A (zh) * 2016-04-26 2016-08-24 深圳大学 一种在线目标跟踪方法及装置
CN106846373A (zh) * 2016-11-16 2017-06-13 浙江工业大学 一种融合目标外观模型和博弈论的视频目标互遮挡处理方法
CN106846361A (zh) * 2016-12-16 2017-06-13 深圳大学 基于直觉模糊随机森林的目标跟踪方法及装置
CN106846355A (zh) * 2016-12-16 2017-06-13 深圳大学 基于提升直觉模糊树的目标跟踪方法及装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
LI, L.Q. ET AL.: "A new algorithm for multiple manoeuvring target tracking", IEEE 10TH INTERNATIONAL CONFERENCE ON SIGNAL PROCESSING (PROCEEDINGS), 24 October 2010 (2010-10-24) - 28 October 2010 (2010-10-28), XP031816728, DOI: 10.1109/ICOSP.2010.5655942 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN111292352B (zh) * 2020-01-20 2023-08-25 杭州电子科技大学 多目标跟踪方法、装置、设备及存储介质
CN111292352A (zh) * 2020-01-20 2020-06-16 杭州电子科技大学 多目标跟踪方法、装置、设备及存储介质
CN113808158A (zh) * 2020-06-15 2021-12-17 中移(苏州)软件技术有限公司 视频中群体对象运动的分析方法、装置、设备及存储介质
CN112148256A (zh) * 2020-09-11 2020-12-29 中国运载火箭技术研究院 标志类变量容错方法、装置及计算机存储介质、电子设备
CN112991280A (zh) * 2021-03-03 2021-06-18 望知科技(深圳)有限公司 视觉检测方法、系统及电子设备
CN112991280B (zh) * 2021-03-03 2024-05-28 望知科技(深圳)有限公司 视觉检测方法、系统及电子设备
CN114241373A (zh) * 2021-12-14 2022-03-25 北京理工大学重庆创新中心 一种端到端的车辆行为检测方法、系统、设备及存储介质
CN116109965A (zh) * 2022-12-13 2023-05-12 济南铁路信息技术有限公司 一种语义关联的多源监控视频融合分析方法及系统
CN116228814A (zh) * 2023-01-13 2023-06-06 湖北星纪时代科技有限公司 目标追踪的数据关联方法、电子设备及存储介质
CN116543417A (zh) * 2023-05-04 2023-08-04 重庆特斯联启智科技有限公司 人体姿态估计方法、装置、设备以及存储介质
CN116954113A (zh) * 2023-06-05 2023-10-27 深圳市机器时代科技有限公司 智能机器人驱动传感智能控制系统及其方法
CN116954113B (zh) * 2023-06-05 2024-02-09 深圳市机器时代科技有限公司 智能机器人驱动传感智能控制系统及其方法
CN116718197A (zh) * 2023-08-09 2023-09-08 腾讯科技（深圳）有限公司 轨迹处理方法、装置、电子设备及存储介质
CN116718197B (zh) * 2023-08-09 2023-10-24 腾讯科技（深圳）有限公司 轨迹处理方法、装置、电子设备及存储介质
CN121415352A (zh) * 2025-12-29 2026-01-27 上海克劳安智能科技有限公司 基于人脸识别的安全防范管理方法

Similar Documents

Publication Publication Date Title
CN107423686B (zh) 视频多目标模糊数据关联方法及装置
WO2018227491A1 (zh) 视频多目标模糊数据关联方法及装置
CN107545582B (zh) 基于模糊逻辑的视频多目标跟踪方法及装置
CN109859245B (zh) 视频目标的多目标跟踪方法、装置及存储介质
Liu et al. Human inertial thinking strategy: A novel fuzzy reasoning mechanism for IoT-assisted visual monitoring
CN105405151B (zh) 基于粒子滤波和加权Surf的抗遮挡目标跟踪方法
CN107818571B (zh) 基于深度学习网络和均值漂移的船只自动跟踪方法及系统
CN111488795A (zh) 应用于无人驾驶车辆的实时行人跟踪方法
WO2020150896A1 (zh) 视频目标的数据关联方法、装置及存储介质
WO2017185688A1 (zh) 一种在线目标跟踪方法及装置
WO2021007984A1 (zh) 基于tsk模糊分类器的目标跟踪方法、装置及存储介质
WO2012009947A1 (zh) 特征迹驱动的多目标跟踪装置及方法
CN111626194A (zh) 一种使用深度关联度量的行人多目标跟踪方法
WO2019006632A1 (zh) 一种视频多目标跟踪方法及装置
CN114222986A (zh) 使用社交图网络进行的随机轨迹预测
CN101561928A (zh) 基于属性关系图外观模型的多人体跟踪方法
CN110288627A (zh) 一种基于深度学习和数据关联的在线多目标跟踪方法
CN101840579A (zh) 一种利用视频分割与粒子滤波实现多目标跟踪方法
CN106447698A (zh) 一种基于距离传感器的多行人跟踪方法和系统
CN109829405A (zh) 视频目标的数据关联方法、装置及存储介质
CN115035158B (zh) 目标跟踪的方法及装置、电子设备和存储介质
Liu et al. Vehicle detection and tracking based on improved YOLOv8
Zhang et al. New mixed adaptive detection algorithm for moving target with big data
Yousefi et al. Tracking of moving human in different overlapping cameras using Kalman filter optimized
CN118470061A (zh) 一种基于改进Pointpillars网络的多目标跟踪方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 17913467

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 17913467

Country of ref document: EP

Kind code of ref document: A1

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 09/04/2020)

122 Ep: pct application non-entry in european phase

Ref document number: 17913467

Country of ref document: EP

Kind code of ref document: A1