JP2009281865A - Multi-target tracking device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a multi-target tracking device can switch algorithm according to the degree of difficulty of tracking. <P>SOLUTION: This device includes: a MFA tracking processing part 4 generating the correlation solution with MFA processing while taking multi-dimensional correlation using the observation values over a plurality of flames from a sensor 1 when satisfying the first condition for carrying out the MHT tracking processing at the previous observation time and the second condition for determining that the tracking at the previous observation time is difficult; a MHT tracking processing part 2 generating new track while taking two-dimensional correlation between the latest observation value from the sensor 1 and the existing tracking track and generating a plurality of assumptions when either of the first and the second conditions is not satisfied; a MHT assumption monitoring part 3 determining whether the tracking is difficult or not from the situation of the generated assumption during carrying out the MHT tracking processing; and a MHT assumption setting part 5 replacing the correlation solution generated by the MFA tracking processing part 4 as an assumption of the MHT tracking processing part 2. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a tracking technique for generating a wake using an observation value obtained from a sensor for observing a target position in a sensor system such as a radar, and more particularly, in a technique for solving a correlation problem in a multidimensional manner with respect to this tracking. The present invention relates to a multi-target tracking apparatus for deriving a correlation solution for reducing calculation time and realizing good tracking performance.

  Currently, many papers, patents, etc. have already mentioned the technology for tracking a target using observation values obtained by observing the target with a sensor, and various proposals have been made for the device and method. Has been made.

  When tracking a multi-proximity target or tracking a target in an unnecessary signal environment such as clutter, the correlation determination method is an important part that affects the tracking performance. The correlation indicates “what observation value from observation time t1 to observation time t3 is regarded as the same target” when a plurality of observation values are obtained at each observation time as shown in FIG. .

  As a method of determining this correlation, as shown in FIG. 13, a method of generating a wake at each observation time and associating the wake one observation time before the observation value at the latest observation time has been used. There are various methods for determining the two-dimensional correlation between the track and the observed value, but one example is to construct a hypothesis considering all possible correlations for each existing track, There is a multiple hypothesis tracking (MHT) method that calculates the reliability of a hypothesis based on the likelihood of correlation (for example, see Non-Patent Document 1).

  The above-mentioned technique is a method for determining the allocation of the existing wake and the latest observation value. To further improve the correlation performance, as shown in Fig. 14, the correlation between observation values over multiple observation times is calculated and calculated together. There is a multiple frame assignment (MFA) method as an algorithm for generating a wake (see, for example, Non-Patent Document 2). Here, the frame is a set of observation values obtained at one observation time.

  In this multiple frame allocation method, the correlation determination of observation values over N (natural number) observation times is converted into an N-dimensional allocation problem and solved. The formulated assignment problem is shown below.

Here, cost matrix c _{i1, ..., iN} is observations i _1, ..., a i _N a cost to generate the wake typically use the inverse of track likelihood ratio.

The above assignment problem is a problem of selecting one element from each dimension of the cost matrix and minimizing the selected element. As a simple example, FIG. 15 shows an example of the cost matrix corresponding to the observed value when N = 2. An example of the solution obtained from this cost matrix and the configuration of the corresponding wake are shown in FIG. The cost of this solution is c ₁₁ + c ₂₄ + c ₃₂ + c ₄₃ by taking the sum of the elements of the cost matrix corresponding to the part where “1” is set. The constraint condition indicates that the column of elements set to “1” must not overlap. For example, as shown in FIG. 17, in the correlation determination, “1” is set to overlap the second element at the second observation time, and the constraint condition is not satisfied. This corresponds to the fact that one observation value is used redundantly in tracking.

  The number of solutions that can be realized by the N-dimensional assignment described above (satisfying the constraint condition) increases exponentially according to the number of observation values. Therefore, in order to obtain an optimal solution that minimizes the cost, there is a problem that the processing time increases exponentially according to the number of observed values and the number N of dimensions to be processed. In order to solve this problem, the method shown in Non-Patent Document 2 described above uses a Lagrange relaxation method to obtain a sub-optimal solution while relaxing the N-dimensional assignment problem in two dimensions.

  A conventional multi-target tracking device based on the multi-frame allocation method shown in Non-Patent Document 2 will be described with reference to FIGS. FIG. 18 is a diagram illustrating a configuration of a conventional multi-target tracking apparatus using a multiple frame allocation method. FIG. 19 is a flowchart showing the operation of the conventional multi-target tracking apparatus based on the multiple frame allocation method.

  In FIG. 17, the conventional multi-target tracking apparatus based on the multiple frame allocation method is provided with a sensor 1 for observing a target position, an MFA tracking processing unit 4, and a track display unit 6. In addition, the MFA tracking processing unit 4 can be implemented as a cost matrix calculation unit 41, a Lagrange multiplier setting unit 42, a Lagrange relaxation solution calculation unit 43, a feasible solution calculation unit 44, a relaxation solution and cost storage unit 45. A solution and a cost storage unit 46 are provided.

  First, in step 901, the cost matrix calculation unit 41 inputs observation values for N observation times from the past to the latest obtained from the sensor 1.

  Next, in step 902, the cost matrix calculation unit 41 selects observation values one by one from each observation time, calculates the likelihood ratio of the wake formed by the combination of the observation values, and uses the reciprocal thereof. Calculate the cost matrix.

  Next, in step 903, the Lagrange multiplier setting unit 42 sets a Lagrange multiplier for relaxing the constraint condition. This Lagrange multiplier is set for all observed values from the third observation time to the Nth observation time.

_Let u _{jr be the} Lagrange multiplier for the observed value j _r . By setting this Lagrangian multiplier, it is possible to relax the restrictions on the first to N-th observed values of the N-dimensional cost matrix in two dimensions. By this relaxation, the cost function is transformed into a two-dimensional cost matrix, and its elements can be expressed by the following equations.

  As an example of the processing in step 903, all initial Lagrangian multipliers are set to zero.

  Next, in Step 904, the Lagrange relaxation solution calculation unit 43 calculates a solution of the two-dimensional assignment problem that has been relaxed by the multiplier. This two-dimensional solution calculation method includes a high-speed solution derivation method such as the Hungarian method, and one of them is used. The derived relaxation solution cost is set to q (u) and stored in the relaxation solution and cost storage unit 45.

  Next, in step 905, the feasible solution calculation unit 44 corrects the above relaxation solution and derives a feasible solution that satisfies all the constraint conditions. The derived feasible solution cost is set as ν (overline z) and stored in the feasible solution and cost storage unit 46. Note that the overline z indicates that-is present on z.

  Next, in Step 906, the feasible solution calculation unit 44 calculates a value obtained by dividing the difference between the cost q (u) of the Lagrangian relaxation solution and the cost ν (overline z) of the feasible solution by q (u). If it is below the preset threshold parameter, the process proceeds to step 908. If the threshold condition is not satisfied, the process proceeds to step 907.

  If the threshold condition is not satisfied in step 907, the Lagrange multiplier setting unit 42 resets the Lagrange multiplier. In this resetting, a multiplier is set according to the deviation of the constraint condition in the latest relaxation solution.

  When this Lagrange multiplier resetting step 907 is executed, the Lagrangee relaxation solution calculation step 904 and the feasible solution calculation step 905 are repeated until the threshold condition is satisfied.

  In step 908, the wake display unit 6 outputs the feasible solution as a suboptimal solution of this N-dimensional assignment problem.

  Although the above-described multiple frame allocation method MFA has high correlation performance, it has a problem of heavy computation load. In order to solve this problem, a technique has been proposed in which a tracking method based on the multiple frame allocation method and a tracking method based on two-dimensional correlation are used in combination (for example, see Patent Document 1).

  FIG. 20 is a diagram showing the configuration of this combined technique. In this combined technique, when the value obtained by dividing the observation time by the observation interval (number of processing times) is a multiple of the value n set in advance as a parameter, multiple frame allocation processing is performed, and otherwise, two-dimensional correlation is performed. Perform the calculation process.

JP 2007-212244 A Donald B. Reid “An Algorithm for Tracking Multiple Targets” (IEEE Transactions on Automatic Control, Vol.AC-24, No6, December, 1979) A. B. Poore and A. J. Robertson III “A New Multi-dimensional Data Association Algorithm for Multisensor-Multitarget Tracking” (Proc. Of SPIE vol.2561,1995)

  Correlation processing in tracking is easy to find the correct answer when the detection status of the sensor is good and there are no other targets or unnecessary signals in the surroundings, but target detection is lost or unnecessary signals are generated in the surroundings It becomes difficult to do. If easy, the conventional tracking method using two-dimensional correlation is sufficient, and if difficult, the probability of obtaining a correct answer is generally higher when the MFA tracking method is applied. Therefore, a switching method in which two-dimensional correlation is applied when tracking is easy and MFA tracking processing is applied when difficult is ideal. However, the prior art disclosed in Patent Document 1 has a problem that the switching time of the two tracking algorithms is fixed, and the algorithm cannot be switched depending on the difficulty of tracking.

  The present invention has been made to solve the above-described problems, and its purpose is to switch algorithms according to the difficulty of tracking, to reduce the calculation load and to derive an accurate correlation solution. A multi-target tracking device capable of satisfying both of the above is obtained.

  The multi-target tracking device according to the present invention satisfies the first condition for executing the tracking process by the multiple hypothesis tracking method at the previous observation time and the second condition for determining that tracking is difficult at the previous observation time. Includes an MFA tracking processing unit that generates a correlation solution by a process based on a multi-frame allocation method while obtaining a multidimensional correlation using observation values across a plurality of frames from the sensor, and the first and second conditions. If any of the conditions is not satisfied, the MHT tracking processing unit that generates a new track while generating a plurality of hypotheses by taking a two-dimensional correlation between the latest observed value from the sensor and the existing tracking track, and the MHT An MHT hypothesis monitoring unit that determines whether tracking is difficult from the status of the generated hypothesis during execution of the tracking processing by the tracking processing unit, and the correlation solution generated by the MFA tracking processing unit is the MHT tracking processing unit. of It is provided with a and MHT hypothesis setting unit to replace a theory.

  The multi-target tracking device according to the present invention can perform algorithm switching according to the tracking difficulty level, and has the effect of being able to achieve both reduction in calculation load and accurate derivation of a correlation solution.

Embodiment 1 FIG.
A multi-target tracking device according to Embodiment 1 of the present invention will be described with reference to FIGS. FIG. 1 is a diagram showing the configuration of the multi-target tracking device according to Embodiment 1 of the present invention. In addition, in each figure, the same code | symbol shows the same or equivalent part.

  In FIG. 1, a multi-target tracking device according to Embodiment 1 of the present invention takes a two-dimensional correlation between a sensor 1 for observing a target position, the latest observed value from the sensor 1 and an existing tracking track, and a plurality of hypotheses. MHT tracking processing unit 2 that generates a new track while generating MHT, and an MHT hypothesis monitoring unit that determines whether tracking is difficult from the status of the generated hypothesis during execution of tracking processing by the MHT tracking processing unit 2 3 and an MFA tracking processing unit 4 that generates a correlation solution by a process using a multiple frame allocation method while obtaining a multidimensional correlation using observation values over a plurality of frames from the sensor 1. An MHT hypothesis setting unit 5 that replaces the generated correlation solution as a hypothesis of the MHT tracking processing unit 2, a wake display unit 6 that displays the wake to the user, a wake storage unit 21 that stores the wake, and a temporary storage that stores the hypothesis. A storage unit 22, the observed value of the N frames, and the N-frame data storing unit 23 for storing the track is provided.

  Next, the operation of the multi-target tracking device according to the first embodiment will be described with reference to the drawings. FIG. 2 is a flowchart showing the operation of the multitarget tracking apparatus according to Embodiment 1 of the present invention.

  First, in step 101, the observation value observed from the sensor 1 is input to the apparatus and stored in an observation value storage unit (not shown).

Next, in step 102, when both of the following conditions (a) and (b) are satisfied, the tracking process by the multiple frame allocation method (MFA) is selected, and the process proceeds to step 106.
(A) When tracking processing by multiple hypothesis tracking method (MHT) is executed at the previous observation time.
(B) When it is determined that tracking is difficult at the previous observation time.

  If any of the above conditions is not satisfied, the tracking process by the multiple hypothesis tracking method is selected, and the process of step 103 is performed. At the first observation time, tracking processing by the multiple hypothesis tracking method is selected unconditionally.

  When processing by the multiple hypothesis tracking method is selected, first, in step 103, the MHT tracking processing unit 2 executes the same processing as the processing by the conventional multiple hypothesis tracking method. The details of the processing contents are shown in Non-Patent Document 1, as described above.

  Thereafter, in step 104, the MHT tracking processing unit 2 generates all descendants of the wake included in the best hypothesis at a certain observation time during the N observation times from the observation time, and the wake storage unit 21. To remember. The hypothesis is also stored in the hypothesis storage unit 22.

  Here, the wake and hypothesis parent-child relationship will be described. Based on the wake T generated at a certain observation time, all the wakes generated by updating the motion specification estimates by tracking filter calculation using any of the observation values obtained thereafter are descendants of T. is there. An example of the wake T at a certain observation time is shown in FIG. Thereafter, an example of a wake obtained after two observation times have elapsed is shown in FIG. 3B. Of the four wakes, T11 and T12 are T descendants, and T21 and T22 are not T descendants. That is, the hypothesis generated by combining the wakes is selected according to the reliability, but all the wakes are maintained, which is different from the tracking process by the conventional multiple hypothesis tracking method.

  When tracking processing by the multiple hypothesis tracking method is performed, in step 105, the MHT hypothesis monitoring unit 3 determines whether or not tracking of the multiple hypothesis tracking method is difficult at the next observation time. This determination is performed by any one of the following (1) to (3).

  (1) Among the hypotheses generated by the MHT tracking processing unit 2, when the difference in reliability between the first hypothesis H1 and the second hypothesis H2 is greater than or equal to the threshold th,

  In this case, it is assumed that tracking by the multiple hypothesis tracking method is not difficult. Conversely,

  In this case, it is assumed that tracking by the multiple hypothesis tracking method is difficult. FIG. 4 shows an example of the transition of the reliability of the first hypothesis H1 and the second hypothesis H2. At the observation times (a) and (b), since the difference in reliability between the two hypotheses exceeds the threshold th, it is determined that tracking of the multiple hypothesis tracking method is not difficult. At the observation time (c), since the difference in reliability between the two hypotheses is below the threshold th, it is determined that tracking by the multiple hypothesis tracking method is difficult.

  (2) The MHT tracking processing unit 2 may use the hypothesis number m (natural number) to be finally maintained as a preset parameter. When this parameter is specified, in the conventional multiple hypothesis tracking method, when the reliability difference between the m-th hypothesis Hm and the m + 1-th hypothesis Hm + 1 is greater than or equal to the threshold th, that is,

  In this case, it is assumed that tracking by the multiple hypothesis tracking method is not difficult. Conversely,

  In this case, it is assumed that tracking by the multiple hypothesis tracking method is difficult. FIG. 5 shows an example of transition of reliability of the m-th hypothesis Hm and the m + 1-th hypothesis Hm + 1. At the observation times (a) and (b), since the difference in reliability between the two hypotheses exceeds the threshold th, it is determined that tracking of the multiple hypothesis tracking method is not difficult. At the observation time (c), since the difference in reliability between the two hypotheses is below the threshold th, it is determined that tracking by the multiple hypothesis tracking method is difficult.

  (3) Of the hypotheses generated by the MHT tracking processing unit 2, it is assumed that tracking by the multiple hypothesis tracking method is difficult if the ranks of descendants of the first and second hypotheses are switched from the previous observation time. In addition, although the parent-child relationship of a wake was demonstrated in step 104, a parent-child relationship can be defined also about a hypothesis according to the parent-child relationship of the wake referred to. For example, if there is a hypothesis H1 that adopts a wake T1 at a certain observation time t1, and there is a hypothesis H11 that adopts T11 that is a descendant of the wake T1 at a subsequent observation time t2, the hypothesis H11 is a descendant of the hypothesis H1. FIG. 6 shows an example of transition of reliability of each descendant assuming that there is a first hypothesis H1 and a second hypothesis H2 at a certain observation time. At the observation times (a) and (b), the reliability rankings are not interchanged between the two descendants, so that it is determined that tracking of the multiple hypothesis tracking method is not difficult. On the other hand, at the observation time (c), since the reliability of the descendants of the hypothesis H2 exceeds the reliability of the descendants of the hypothesis H1, it is determined that tracking of the multiple hypothesis tracking method is difficult.

  Next, a case where both of the two conditions (a) and (b) described in step 102 above are satisfied and the tracking process by the multiple frame allocation method is selected will be described.

  First, in step 106, the MFA tracking processing unit 4 executes tracking processing by a conventional multiple frame allocation method. Various techniques can be applied to the processing content, but an example is described in Non-Patent Document 2 described above. In addition, when tracking processing by the multiple frame allocation method is performed, the first frame of the sliding window, which is an observation value for the latest N frames subjected to correlation processing by the multiple frame allocation method, needs to be a smooth value of a wake instead of an observation value. There is. As the smooth value, the smooth value of the first time of the sliding window of the wake included in the hypothesis having the highest reliability generated by the processing by the multiple hypothesis tracking method at the previous observation time is applied. Since all descendants of the wake that realize the smooth value of the first frame are generated by the MHT tracking processing unit 2 side, the MFA tracking processing unit 4 side uses these wakes to calculate the correlation cost for N frames. Generate a matrix.

  Next, in Step 107, the MHT hypothesis setting unit 5 sets the correlation solution generated by the MFA tracking processing unit 4 as the first hypothesis managed by the MHT tracking processing unit 2. In addition, the hypothesis of the 1st place to the (m-1) th place generated by the MHT tracking processing unit 2 at the previous observation time is extended by the simplest correlation process. However, since the correlation solution generated by the MFA tracking processing unit 4 is first, the extended hypothesis is lowered from the previous observation time one by one to be the second to mth hypotheses. An example of this is shown in FIG.

  A hypothesis extension method based on this simple correlation process will be described. A track that is descendant of a track included in each hypothesis and that has the highest likelihood is selected from the tracks generated by the MFA tracking processing unit 4 and included in the extended hypothesis. As a result, when there is an overlap in observation values to be used among a plurality of wakes included in one extension hypothesis, a wake having a high likelihood is left. For a track with a low likelihood, a track of another descendant is selected. Further, if there is no child track of the track included in the hypothesis of the MHT tracking processing unit 2 among all the tracks generated by the MFA tracking processing unit 4, the memory track track (not detected at the current observation time) is included in the extended hypothesis. Wake).

  Since all hypotheses of the second and lower ranks are configured using wakes already calculated by the MFA tracking processing unit 4, as long as the MFA tracking processing unit 4 derives the most likely correlation solution, the reliability ranking May not be replaced, but may have the same content as the second hypothesis. In addition, if the calculation time is not sufficient, the content may be the same as that of a lower hypothesis. Therefore, as post-processing, first, hypotheses of the MFA tracking processing unit 4 and the hypothesis of the MHT tracking processing unit 2 are compared. If they match, the solution of the MFA tracking processing unit 4 is deleted. In addition, when the hierarchical relationship of reliability does not match the hypothesis rank, the hypothesis rank is switched. Note that the processing based on the multiple hypothesis tracking method is always selected at the observation time next to the observation time at which the tracking processing by the multiple frame allocation method is selected.

  In the final step 108, the wake display unit 6 displays the wake included in the first hypothesis in the hypothesis group managed by the MHT tracking processing unit 2 to the user.

  As described above, according to the first embodiment, the processing based on the multiple hypothesis tracking method (MHT) and the processing based on the multiple frame allocation method (MFA) are selectively used, and the tracking is performed according to the hypothesis status in the processing based on the multiple hypothesis tracking method. When difficulty is determined and tracking is difficult, processing by the multiple frame allocation method is performed, so that the processing load can be reduced and an accurate correlation solution can be derived.

Embodiment 2. FIG.
A multi-target tracking device according to Embodiment 2 of the present invention will be described with reference to FIGS. FIG. 8 is a diagram showing the configuration of the multi-target tracking device according to Embodiment 2 of the present invention.

  In FIG. 8, the multi-target tracking device according to Embodiment 2 of the present invention is generated by the sensor 1, the MHT tracking processing unit 2, the MFA tracking processing unit 4, the wake display unit 6, and the MHT tracking processing unit 2. As a comparison result by the solution comparison unit 7 that compares the first hypothesis and the likelihood of the correlation solution generated by the MFA tracking processing unit 4 and the comparison result by the solution comparison unit 7, the likelihood of the correlation solution of the MFA tracking processing unit 4 When the degree is high, the MHT hypothesis correcting unit 8 that replaces the first hypothesis of the MHT tracking processing unit 2 with the correlation solution of the MFA tracking processing unit 4 and the comparison result by the solution comparing unit 7 are used as the comparison result of the MHT tracking processing unit 2. When the likelihood of the first hypothesis is large, the MFA correlation solution correcting unit 9 that replaces the correlation solution of the MFA tracking processing unit 4 with the first hypothesis of the MHT tracking processing unit 2, the track storage unit 21, , Hypothesis storage unit 22 and N frame data And 憶部 23 is provided.

  Next, the operation of the multi-target tracking device according to the second embodiment will be described with reference to the drawings. FIG. 9 is a flowchart showing the operation of the multitarget tracking apparatus according to Embodiment 2 of the present invention.

  First, in step 201, an observation value observed from the sensor 1 is input to the apparatus and is stored in an observation value storage unit (not shown).

  Next, in steps 202 and 203, the tracking process by the multiple hypothesis tracking method and the tracking process by the multiple frame allocation method are executed in parallel.

  In step 202, the MHT tracking processing unit 2 executes the same processing as the processing by the conventional multiple hypothesis tracking method. The details of the processing contents are shown in Non-Patent Document 1, as described above.

  In step 203, the MFA tracking processing unit 4 performs tracking processing based on multi-dimensional correlation for N frames. Various techniques can be applied to the processing content, but as described above, Non-Patent Document 2 shows an example.

  Next, in step 204, the solution comparison unit 7 compares the likelihood of the first hypothesis generated by the process based on the multiple hypothesis tracking method and the likelihood of the correlation solution generated by the process based on the multiple frame allocation method. To do.

As a result of the comparison, when the likelihood of the correlation solution of the multiple frame allocation method exceeds the likelihood of the first hypothesis of the multiple hypothesis tracking method and the difference exceeds the threshold th ₁ , the process proceeds to step 205. .

  In step 205, the MHT hypothesis correcting unit 8 replaces the first hypothesis of the multiple hypothesis tracking method so as to be the same as the correlation solution of the multiple frame allocation method. First, the correlation solution generated by the MFA tracking processing unit 4 is set as the first hypothesis managed by the MHT tracking processing unit 2. In addition, the first to m-1th hypotheses generated at the previous observation time by the MHT tracking processing unit 2 are lowered one by one to be the second to mth hypotheses or all are discarded. To do.

As a comparison result of the solution comparison unit 7, when the likelihood of the first hypothesis of the multiple hypothesis tracking method exceeds the likelihood of the correlation solution of the multiple frame allocation method and the difference exceeds the threshold −th ₂ , The process proceeds to step 206.

  In step 206, the MFA correlation solution correction unit 9 replaces the correlation solution of the MFA tracking processing unit 4 so as to be the same as the first hypothesis of the multiple hypothesis tracking method.

  In the final step 207, the wake display unit 6 displays the wake included in the correlation solution of the MHT tracking processing unit 2 to the user.

  As described above, according to the second embodiment, the processing based on the multiple hypothesis tracking method (MHT) and the processing based on the multiple frame allocation method (MFA) are executed in parallel, and the hypothesis and the correlation solution generated by both are executed. Since they are replaced with each other in accordance with the magnitude of the likelihood, an accurate correlation solution can always be derived even when the tracking performance deteriorates in any process.

Embodiment 3 FIG.
A multi-target tracking device according to Embodiment 3 of the present invention will be described with reference to FIGS. FIG. 10 is a diagram showing a configuration of a multi-target tracking device according to Embodiment 3 of the present invention.

10, the multi-target tracking device according to Embodiment 3 of the present invention includes a sensor 1, an MHT tracking processing unit 2, a plurality of MFA tracking processing units 4, a wake display unit 6, and an MHT tracking processing unit 2. MFA initial solution generation unit 10 that uses the smooth value of the wake included in each hypothesis as the first frame of the respective input data of the plurality of MFA tracking processing units 4 and the plurality of MFA tracking A solution adjustment unit 11 that selects one of the plurality of correlation solutions generated by the processing unit 4 according to the likelihood of the plurality of correlation solutions, and an N frame data storage unit 23 are provided. The MFA tracking processing unit 4 is provided with m MFA tracking processes 4 ₁ , 4 ₂ ,..., 4 _m .

  Next, the operation of the multi-target tracking device according to the third embodiment will be described with reference to the drawings. FIG. 11 is a flowchart showing the operation of the multitarget tracking apparatus according to Embodiment 3 of the present invention.

  First, in step 301, the observation value observed from the sensor 1 is input to the apparatus, and is stored in an observation value storage unit (not shown).

  Next, in step 302, it is determined whether all observation values from the first observation value time of the sensor 1 to the latest observation time are N + 1 frames or more, and the process always proceeds to step 303 until N frames have elapsed from the observation start time. If it has passed, the process proceeds to step 305.

  In step 303, the MHT tracking processing unit 2 executes the same processing as the processing based on the conventional multiple hypothesis tracking method. The details of the processing contents are shown in Non-Patent Document 1.

  Next, when the tracking process by the multiple hypothesis tracking method is executed, in step 304, the MFA initial solution generation unit 10 includes the tracks included in each hypothesis for the top m hypotheses of the hypothesis group generated by the MHT tracking processing unit 2. Is the first frame of the input data of the tracking process by the multiple frame allocation method.

  In step 305, the MFA tracking processing unit 4 performs tracking processing by multidimensional correlation for N frames. Various techniques can be applied to this processing content, but an example is shown in Non-Patent Document 2 described above.

Here, m MFA tracking processes 4 ₁ , 4 ₂ ,..., 4 _m operate in parallel. For the data of N frames that are input to each MFA tracking processing unit and are subject to correlation processing, the contents of the first frame are set as smooth values of wakes included in the m hypotheses generated by the MHT tracking processing unit 2. That is, each MFA tracking processing unit processes data for N frames in which the smooth value of the first frame is different and the observation values of the remaining frames are common. Therefore, a total of m correlation solutions are generated.

  Next, in step 306, the solution adjustment unit 11 selects the most likely correlation solution among the m correlation solutions generated by the MFA tracking processing unit 4.

  In the final step 307, the wake display unit 6 displays the wake included in the correlation solution selected by the solution adjustment unit 11 to the user.

  As described above, according to the third embodiment, processing by a plurality of multiple frame allocation methods (MFA) is performed in parallel based on the initial m hypotheses generated by processing by the multiple hypothesis tracking method (MHT). In addition, a plurality of correlation solutions generated by processing by the multiple frame allocation method (MFA) are selected according to the likelihood and displayed to the user. ) Makes it possible to derive an accurate correlation solution even when the tracking performance deteriorates.

It is a figure which shows the structure of the multi-target tracking apparatus which concerns on Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the multi target tracking apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the example of the wake T of a certain observation time, and the wake example obtained after two observation time passes after that. It is a figure which shows the example of transition of the reliability of 1st hypothesis H1 and 2nd hypothesis H2. It is a figure which shows the example of the transition of the reliability of the hypothesis Hm of the m-th rank and the hypothesis Hm + 1 of the m + 1-th rank. It is a figure which shows the example of transition of the reliability of the descendant of the 1st hypothesis H1, and the descendant of the 2nd hypothesis H2. It is a figure which shows the example of hypothesis replacement | exchange of the multi-target tracking apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the multi-target tracking apparatus which concerns on Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the multi-target tracking apparatus which concerns on Embodiment 2 of this invention. It is a figure which shows the structure of the multi-target tracking apparatus which concerns on Embodiment 3 of this invention. It is a flowchart which shows operation | movement of the multi-target tracking apparatus which concerns on Embodiment 3 of this invention. It is a figure for demonstrating the correlation in multitarget tracking. It is a figure which shows the example of the correlation determination in a two-dimensional correlation system. It is a figure which shows the example of the correlation determination in a MFA correlation system. It is a figure which shows the example of the cost matrix corresponding to the observation value of two dimensions (2 observation time). It is a figure which shows the structure of the wake corresponding to the example of the solution obtained from a cost matrix. It is a figure which shows the example of the correlation determination which does not satisfy | fill a constraint condition. It is a figure which shows the structure of the conventional multi target tracking apparatus by the multiple frame allocation method. It is a flowchart which shows operation | movement of the conventional multi-target tracking apparatus by the multiple frame allocation method. It is a figure which shows the structure of the conventional multi-target tracking apparatus which uses together the tracking system by the multiple frame allocation method and the tracking system by two-dimensional correlation.

Explanation of symbols

  1 sensor, 2 MHT tracking processing unit, 3 MHT hypothesis monitoring unit, 4 MFA tracking processing unit, 5 MHT hypothesis setting unit, 6 track display unit, 7 solution comparison unit, 8 MHT hypothesis correction unit, 9 MFA correlation solution correction unit, 10 MFA initial solution generation unit, 11 solution adjustment unit, 21 track storage unit, 22 hypothesis storage unit, 23 N frame data storage unit.

Claims (8)

When the first condition for executing the tracking process by the multiple hypothesis tracking method at the previous observation time and the second condition for determining that the tracking is difficult at the previous observation time are satisfied, a plurality of frames from the sensor are straddled. An MFA tracking processing unit that generates a correlation solution by processing using a multi-frame allocation method while obtaining multi-dimensional correlation using observation values;
If either of the first and second conditions is not met, a two-dimensional correlation between the latest observation value from the sensor and the existing tracking track is taken, and a new track is generated while generating a plurality of hypotheses. An MHT tracking processing unit,
An MHT hypothesis monitoring unit that determines whether tracking is difficult from the status of the generated hypothesis during execution of the tracking process by the MHT tracking processing unit;
A multi-target tracking device comprising: an MHT hypothesis setting unit that replaces the correlation solution generated by the MFA tracking processing unit as a hypothesis of the MHT tracking processing unit. The MHT hypothesis monitoring unit determines whether tracking is difficult based on a difference in reliability between the first hypothesis and the second hypothesis generated by the MHT tracking processing unit. The multi-target tracking device described. The MHT hypothesis monitoring unit assumes that the MHT tracking processing unit adopts a method in which the hypothesis selection method of the MHT tracking processing unit retains the top m (natural number) hypotheses according to reliability. The multi-target tracking device according to claim 1, wherein it is determined whether or not tracking is difficult based on a difference in reliability between the generated m-th hypothesis and m + 1-th hypothesis. The MHT hypothesis monitoring unit determines whether or not tracking is difficult depending on whether or not the rank of the hypothesis of the first hypothesis and the hypothesis of the second hypothesis at a certain observation time generated by the MHT tracking processing unit is changed. The multi-target tracking device according to claim 1, wherein: A MHT tracking processing unit that takes a two-dimensional correlation between the latest observation value from the sensor and the existing tracking track, and generates a new track while generating a plurality of hypotheses;
In parallel with the MHT tracking processing unit, an MFA tracking processing unit that generates a correlation solution by processing using a multiple frame allocation method while obtaining a multidimensional correlation using observation values across a plurality of frames from a sensor;
A solution comparison unit that compares the first hypothesis generated by the MHT tracking processing unit with the likelihood of the correlation solution generated by the MFA tracking processing unit;
When the likelihood of the correlation solution of the MFA tracking processing unit is large as a comparison result by the solution comparison unit, the MHT hypothesis that replaces the first hypothesis of the MHT tracking processing unit with the correlation solution of the MFA tracking processing unit Correction part,
If the likelihood of the first hypothesis of the MHT tracking processing unit is large as a comparison result by the solution comparison unit, the correlation solution of the MFA tracking processing unit is expressed as the first hypothesis of the MHT tracking processing unit. A multi-target tracking device comprising: a replacement MFA correlation solution correction unit. A MHT tracking processing unit that takes a two-dimensional correlation between the latest observation value from the sensor and the existing tracking track, and generates a new track while generating a plurality of hypotheses;
A plurality of MFA tracking processing units each generating a correlation solution by processing by a multi-frame allocation method while obtaining multi-dimensional correlation using observation values across a plurality of frames from a sensor;
An MFA initial solution generator that uses a smooth value of a wake included in each hypothesis for a plurality of upper hypotheses generated by the MHT tracking processor as a first frame of input data of each of the plurality of MFA tracking processors;
A multi-target tracking apparatus comprising: a solution adjusting unit that selects one of the plurality of correlation solutions generated by the plurality of MFA tracking processing units according to the likelihood of the plurality of correlation solutions. . A MHT tracking processing unit that takes a two-dimensional correlation between the latest observation value from the sensor and the existing tracking track, and generates a new track while generating a plurality of hypotheses;
An MHT hypothesis monitoring unit that determines whether tracking is difficult from the generated hypothesis status during the execution of the tracking process by the MHT tracking processing unit;
When the MHT hypothesis monitoring unit determines that tracking is not difficult, the multi-target tracking device continues processing by the MHT tracking processing unit. When the first condition for executing the tracking process by the multiple hypothesis tracking method at the previous observation time and the second condition for determining that tracking is difficult by the MHT hypothesis monitoring unit at the previous observation time are satisfied, An MFA tracking processing unit that generates a correlation solution by a process using a multi-frame allocation method while obtaining a multi-dimensional correlation using observation values over a plurality of frames from
The multi-target tracking device according to claim 7, further comprising an MHT hypothesis setting unit that replaces the correlation solution generated by the MFA tracking processing unit as a hypothesis of the MHT tracking processing unit.

Images