JP2014074675A - Tracking apparatus, tracking method and tracking program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To track a tracking target with high accuracy even when a target to be tracked exists in a far distance.SOLUTION: A tracking apparatus includes: an extraction part for extracting an observation signal possibly indicating a target to be tracked from an image obtained from a sensor at each sampling time; a generation part which, when a plurality of observation signals is extracted, generates a plurality of hypotheses including a hypothesis indicating a branch track that is a track branched by updating an existing track being a track obtained by past tracking processing of the target by using two or more observation signals selected from the plurality of observation signals; an evaluation prat for evaluating likelihood indicating probability of each generated hypothesis on the basis of a predetermined reference; a tracking part for specifying a past track of the target by selecting at least one hypothesis including a hypothesis having the highest likelihood obtained by the evaluation part from the plurality of hypotheses generated at each sampling time; and a setting part for setting a gate region indicating a range of an image, from which an observation signal corresponding to the target is extracted at a succeeding sampling time, on the basis of an existing track of a branch source of the branch track.

Description

  The present disclosure relates to a tracking device, a tracking method, and a tracking program that track the track of a moving object based on observation data obtained by a radar, an infrared sensor, or the like.

  Various types of observation signals with the possibility of indicating the position of a moving body such as an aircraft are extracted from observation data obtained from radars, infrared sensors, etc., and the track of the moving body is tracked by monitoring the extracted observation signals. Have been proposed (see, for example, Patent Documents 1-3).

  In recent years, tracking devices using multiple hypothesis tracking (MHT) technology have also been proposed in order to track a tracking target such as a moving object, i.e., a target from a stage at a long distance (e.g., patent document). 4).

  The tracking device to which MHT is applied takes into account the possibility of target detection failure and noise generation, and creates multiple hypotheses about the correspondence between the target and the extracted observation points. Tracking can be performed with high accuracy by proceeding the tracking process while evaluating.

JP-A-10-104348 JP 2003-57337 A Japanese Patent Laid-Open No. 2002-341024 JP 2009-264987 A

  In the target tracking technique to which MHT is applied, an observation signal having a possibility of indicating the position of the target is extracted from the observation data obtained at each sampling time, and each of the extracted observation signals is one of the elements included in the target set. Generate hypotheses to assign to. Here, the target set includes, for example, an existing target indicating the target being tracked, a new target indicating another newly appearing target, and an erroneous observation target that is a target that erroneously recognizes noise included in the observation data. Is included as an assignment candidate for each observation signal.

  In the conventional tracking technique using MHT, hypotheses are comprehensively generated by assigning each extracted observation point to any one of the existing target, new target, and erroneous observation target described above, and the likelihood of these hypotheses is estimated. Tracking processing is realized by performing selection based on.

  By the way, in the observation data obtained when the target to be tracked is at a long distance, since the S / N (Signal / noise) ratio is low, one observation signal extracted from the observation data is two or more. It may indicate the target position at the same time. Therefore, when tracking a track with a long-distance moving object as the target to be tracked, the possibility that one track obtained in the past tracking process may branch into two or more tracks during tracking is considered. It is desirable.

  However, in the tracking technique to which the conventional MHT is applied, there is no possibility that the target wake is branched into two or more wakes. Therefore, in the tracking technique using the conventional MHT, when a plurality of observation points are extracted from the vicinity of the estimated position of the target obtained by the past tracking process, another new target is generated in the vicinity of the existing target. Subsequent tracking processing is performed based on a hypothesis that assumes the appearance. For this reason, when tracking the track of a long-distance moving body, tracking accuracy may be lowered.

  It is an object of the present disclosure to provide a tracking device, a tracking method, and a tracking program that can track a tracking target with high accuracy even when the target that is the tracking target is at a long distance.

  A tracking device according to one aspect includes an extraction unit that extracts an observation signal that may be an image of a target that is a tracking target from an image obtained from a sensor at each sampling time, and a plurality of observation signals are obtained by the extraction unit. A track that is branched by updating two or more observation signals selected from the plurality of observation signals, the existing track that is a track obtained in the past tracking process for the target when extracted. A generating unit that generates a plurality of hypotheses including a hypothesis indicating a certain branch track, and an evaluation unit that evaluates likelihoods indicating the certainty of each of the plurality of hypotheses generated by the generating unit based on a predetermined criterion; Selecting at least one hypothesis including a hypothesis with the highest likelihood obtained by the evaluation unit from among a plurality of hypotheses generated by the generation unit at each sampling time. A tracking unit for identifying a past track of the image, and a range of images from which an observation signal corresponding to the target is extracted at the next sampling time when a hypothesis indicating the branching track is included in the hypothesis selected by the tracking unit And a setting unit that sets the gate region based on the existing track of the branch source of the branch track.

  In addition, the tracking method according to another viewpoint extracts an observation signal that may be a target image to be tracked from an image obtained from a sensor at each sampling time, and when a plurality of observation signals are extracted. The bifurcation track which is a bifurcation track which is branched by updating the existing track which is the track obtained by the past tracking process with respect to the target using two or more observation signals selected from the plurality of observation signals. A plurality of hypotheses including hypotheses are generated, and the likelihood indicating the likelihood of each of the generated plurality of hypotheses is evaluated based on a predetermined criterion, and among the plurality of hypotheses generated at each sampling time The at least one hypothesis including the hypothesis with the highest likelihood is selected to track the past track of the target, and the selected hypothesis includes the hypothesis indicating the branch track. Case, the gate region indicating a range of an image to extract the observation signals corresponding to the target in the next sampling time is set based on the branch source of the existing track of the branch track.

  In addition, the tracking program according to another aspect extracts an observation signal that may be a target image to be tracked from an image obtained from a sensor at each sampling time, and multiple observation signals are extracted. A bifurcation track that is a bifurcation track that has been branched by updating an existing track that is a track obtained in the past tracking process for the target using two or more observation signals selected from the plurality of observation signals. A plurality of hypotheses including the hypothesis to be shown, and the likelihood indicating the likelihood of each of the generated hypotheses is evaluated based on a predetermined criterion, and a plurality of hypotheses generated at each sampling time By selecting at least one hypothesis including the hypothesis with the highest likelihood, the past track of the target is tracked, and the branch track is shown in the selected hypothesis. Is included in the computer, the gate region indicating the range of the image from which the observation signal corresponding to the target is extracted at the next sampling time is set based on the existing wake of the branch wake. Let

  According to the tracking device, the tracking method, and the tracking program of the present disclosure, it is possible to track the tracking target with high accuracy even when the target that is the tracking target is at a long distance.

It is a figure which shows one Embodiment of a tracking apparatus. It is a figure which shows the example of extraction of an observation signal. It is a figure which shows the example of the produced | generated hypothesis. It is a figure which shows the example of the matrix expression of a bipartite graph. It is a figure which shows the example of the matrix which shows the log likelihood of each edge. It is a figure which shows the example of a hypothesis tree. It is a figure which shows the example of the gate area | region set corresponding to a branch target. It is a figure which shows another embodiment of a tracking apparatus. It is a figure which shows the example of a 1st hypothesis set and a 2nd hypothesis set. It is a figure which shows the example of extraction of an observation signal. It is a figure which shows another example of a 1st hypothesis set. It is FIG. (1) which shows another example of a 2nd hypothesis set. It is a figure which shows another example of a 2nd hypothesis set (the 2). It is a figure which shows the hardware structural example of a tracking apparatus. It is a figure which shows the example of the flowchart of a tracking process. It is a figure which shows the example of the flowchart of the process which produces | generates a bipartite graph. It is a figure which shows the example of the flowchart of the process which calculates the weight of each bipartite graph. It is a figure which shows the example of the flowchart of the process which sets an edge. It is a figure which shows the example of the flowchart of the process which simplifies a bipartite graph. It is a figure which shows the example of the simplified bipartite graph.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 shows an embodiment of a tracking device. The tracking device 10 shown in FIG. 1 performs processing for tracking the track of at least one target that is a tracking target based on an image obtained from the sensor 1 such as an infrared sensor at each sampling time, and is obtained for each target. The wake is displayed on the display device 2.

  The tracking device 10 illustrated in FIG. 1 includes an extraction unit 11, a generation unit 12, an evaluation unit 13, a tracking unit 14, and a setting unit 15.

  The extraction unit 11 receives an image obtained by the sensor 1 at each sampling time, and extracts a portion that may be an image to be tracked as an observation signal in each image. For example, the extraction unit 11 may detect a portion having a predetermined luminance or higher from the gate region set based on the past tracking process for the target, and extract each detected portion as an observation signal. As will be described later, the gate region indicates a range where an observation signal corresponding to a target is likely to be detected in an image obtained at the next sampling time. The setting of the gate region will be described later.

  FIG. 2 shows an example of observation signal extraction. The horizontal axis x and the vertical axis y shown in FIG. 2 are coordinate axes indicating the position of the observation signal in the image obtained by the sensor 1.

  A symbol t1 illustrated in FIG. 2 indicates an estimated position of the target obtained as a result of the tracking process up to the k-th sampling time. Reference sign G1 indicates a gate region set for an image obtained at the (k + 1) th sampling time corresponding to the estimated position t1.

  The example of FIG. 2 shows a case where two observation signals z1 and z2 are detected from the gate region G1 described above in an image obtained at the (k + 1) th sampling time.

  Here, the gate region G1 corresponding to the estimated position t1 indicates a range in which an observation signal corresponding to the target may be detected in an image obtained at the next sampling time based on the past tracking processing result. ing. The gate region G1 corresponding to the estimated position t1 can be set so as to reflect the variation in the target position expected value and the target speed predicted based on the past tracking processing results.

  In the conventional MHT method, as shown in FIG. 2, when two observation signals z1 and z2 are extracted, a transition can be made from the hypothesis that the target exists at the estimated position t1 at the previous sampling time. A plurality of hypotheses described below are generated as hypotheses. A plurality of hypotheses are generated when each of the observation signals z1 and z2 indicates the position of the target in the latest image, when it indicates the position of the target that has newly appeared, and when noise is mistakenly recognized as the target This is a hypothesis that covers the case of a false observation target.

  By the way, when the target is at a long distance, there is a possibility that a plurality of targets that move while maintaining a close state are captured as one target in the image obtained by the sensor 1 shown in FIG. In this case, there is a possibility that a wake that was visible up to the k-th acquired image appears as a multi-branched wake in the k + 1-th acquired image. That is, there is a possibility that one existing wake obtained by the past tracking process may be branched into a plurality of wakes.

  Correlation between the vector indicating the motion of each target corresponding to each track after branching and the vector indicating the motion of the target corresponding to the existing track before branching when the existing track branches into two or more tracks There is. Therefore, the tracking process for the target corresponding to each track after branching includes information such as the target speed and speed variation obtained by the tracking process for the existing track before branching, that is, the movement of the target. It is desirable to use a state vector indicating

  However, the conventional MHT technique does not assume the wake bifurcation as described above. That is, in the conventional MHT method, as shown in FIG. 2, when a plurality of observation signals are detected in the gate region G1, the existing wake tracked up to the estimated position t1 is changed to one of the observation signals z1 and z2. On the other hand, tracking is started using the other observation signal as a new target.

  Here, when the existing track updated using the position of the observation signal z1 is tracked, the gate region G1_1 set for the image obtained at the next sampling time is the target region obtained in the past tracking process. It is set to reflect the state vector. On the other hand, the gate area set for tracking a new target is often set as a circular area indicating a range in which a target having an assumed maximum speed moves at a sampling interval, for example. The maximum speed of the target that can be assumed is often set larger than the target speed indicated by the state vector estimated based on the past tracking result. Therefore, for example, when tracking is started with the observation signal z2 as a new target, that is, when tracking processing of a new track starting from the position of the observation signal z2 is started, the gate region Gn applied is the gate described above. This is a wider range than the region G1_1.

  For this reason, there is a high possibility that the number of observation signals extracted from the image acquired at the next sampling time using the gate region Gn will increase, and the processing cost for the tracking processing based on the extracted observation signals will increase. Resulting in. Furthermore, since the observation signal is extracted from a wide range, it is highly likely that the tracking result obtained using the observation signal indicating the position of the noise or other target is a reliable tracking result for the target. End up. That is, it is a problem in the conventional MHT method that it is difficult to obtain a correct tracking result when there is a possibility that the existing wake may be branched into two wakes.

  In order to solve the above-described problem, the tracking device 10 disclosed in FIG. 1 includes a generation unit 12 having a function of generating an expanded set of hypotheses including a hypothesis corresponding to a case where an existing track branches. doing. Furthermore, the tracking device 10 of the present disclosure includes a setting unit 15 having a function of using information obtained by tracking processing for an existing track before branching in setting a gate area corresponding to the track after branching. Yes.

  When a plurality of observation signals are extracted by the extraction unit 11, the generation unit 12 updates the existing track obtained by the past tracking process using each of two or more observation signals selected from the plurality of observation signals. A plurality of hypotheses including a hypothesis indicating a branching wake that is branched are generated. The plurality of hypotheses generated by the generation unit 12 is an extended set of hypotheses including a hypothesis indicating a branching track together with a hypothesis generated according to the conventional MHT method.

  The generation unit 12 performs the above-described process of generating a plurality of hypotheses in each target set including each element of the signal set including the observation signal extracted by the extraction unit 11 and an element indicating the target to be tracked. You may process as an assignment problem for determining the correspondence with an element.

  Here, the target set includes, as elements, an existing target that is being tracked, a newly detected new target, and an erroneous observation target that indicates noise that is erroneously detected as the target. In addition to the above three types of elements, the generation unit 12 adds a branch target indicating a target having a track branched from the existing track to the target set, and solves the assignment problem for the target set including the added element. A plurality of hypotheses including a hypothesis indicating a branch of an existing track may be generated.

  The generation unit 12 may represent the association between each element included in the signal set and each element included in the target set using, for example, a bipartite graph as illustrated in FIG.

  FIG. 3 shows an example of a hypothesis generated by the generation unit 12. In addition, the code | symbol T shown in FIG. 3 shows the target set, and the code | symbol Z has shown the signal set.

  As shown in FIG. 2, when the two observation signals z1 and z2 are extracted from the gate region G1, the bipartite graph shown in FIG. 3 shows the elements of the target set T and the elements of the signal set Z. The correspondence relationship which can be allocated is shown.

  In the target set T illustrated in FIG. 3, the symbol t_1 indicates an existing target, and the symbol t_c1 indicates a branch target. Symbols t_f1 and t_f2 indicate erroneous observation targets corresponding to the observation signals z1 and z2, respectively. Similarly, symbols t_n1 and t_n2 indicate new targets corresponding to the observation signals z1 and z2, respectively.

  In the signal set Z, symbols z_1 and z_2 each indicate an observation signal. On the other hand, the symbols z_d1, z_d2, z_d3, and z_d4 are dummy nodes added to make the number of elements of the signal set Z the same as the number of elements of the target set.

  An edge that is a line connecting each element included in the target set T and each element included in the signal set Z indicates that there is a possibility that a pair of elements connected by the edge is associated with each other. . For example, the example of FIG. 3 shows that the existing target t_1 and the branch target t_c1 have a possibility of being associated with all elements of the signal set Z. On the other hand, the erroneous observation target t_f1 indicates a case where the observation signal z1 is noise, and the new target t_n1 is an edge connecting the erroneous observation target t_f1 and the new target t_n1 to the observation signal z2 because the observation signal z1 indicates a new target. Does not exist. Similarly, there is no edge connecting the erroneous observation target t_f2 and the new target t_n2 and the observation signal z1.

  Individual hypotheses that can be considered when updating the existing wake extending to the estimated position t1 shown in FIG. 2 using the observation signals z1 and z2 are the elements of the target set T and the elements of the signal set Z shown in FIG. Can be regarded as one of the allocation methods for one-to-one correspondence. For example, in the example of FIG. 3, the combination of edges indicated by the bold lines indicates that the observation signal signal z1 shown in FIG. 2 indicates the position of the existing target t_1, and the observation signal z2 shown in FIG. A hypothesis representing a certain case is expressed.

  Accordingly, the generation unit 12 illustrated in FIG. 1 covers, for example, the allocation method in which each element of the target set T illustrated in FIG. 3 and each element of the signal set Z are associated one-to-one. A plurality of hypotheses indicating how to update the wake using the positions of the observation signals z1 and z2 can be generated. Here, the generation unit 12 covers the allocation method in which each element of the target set T to which the branch target t_c1 is added is associated with each element of the signal set Z on a one-to-one basis, so that the existing wake becomes the existing target t_1. A set of hypotheses including a hypothesis that branches to the branch target t_c1 can be generated.

  That is, when a plurality of observation signals are extracted by the extraction unit 11 illustrated in FIG. 1, the generation unit 12 generates the above-described hypothesis set using a target set including an element indicating a branch target. A plurality of hypotheses including a hypothesis representing a branch of an existing wake can be generated.

  In addition, when the number of observation signals extracted by the extraction unit 11 is one, it is not necessary to perform the tracking process in consideration of the possibility that the existing track branches. In this case, the generation unit 12 generates a plurality of hypotheses so as to cover the allocation method in which each element of the target set that does not include the branch target and each element of the signal set are associated one-to-one as in the conventional case. May be.

  By the way, the bipartite graph as shown in FIG. 3 can also be expressed by a matrix as shown in FIG.

  FIG. 4 shows an example of a matrix representation of a bipartite graph. 4 that are equivalent to the elements shown in FIG. 3 are denoted by the same reference numerals and description thereof is omitted.

  Each row of the matrix shown in FIG. 4 corresponds to each element of the target set T shown in FIG. 3, and each column corresponds to each element of the signal set Z shown in FIG. Each matrix element shown in FIG. 4 has a value of 1 when there is an edge connecting an element of the target set T indicated by the corresponding row and an element of the signal set Z indicated by the corresponding column, If there is no value, it has a value of 0.

  The generation unit 12 illustrated in FIG. 1 may generate a set of hypotheses for each existing track using a matrix as illustrated in FIG. For example, the generation unit 12 can generate a plurality of hypotheses including a hypothesis representing a branch of an existing track without omission by covering a combination of selecting matrix elements each having a value 1 from each column.

  The evaluation unit 13 illustrated in FIG. 1 evaluates the likelihood indicating the likelihood of each of the plurality of hypotheses generated by the generation unit 12 based on a predetermined criterion described later. For example, the evaluation unit 13 may obtain the likelihood of each hypothesis represented by the combination of edges of the bipartite graph described above as the product of the likelihood of each edge included in the combination.

  Further, the evaluation unit 13 first evaluates the log likelihood indicating the logarithm of the likelihood of each edge represented by the bipartite graph used in the processing of the generation unit 12, and based on the obtained evaluation result Logarithmic likelihood indicating the certainty of each hypothesis may be obtained. When the log likelihood of each edge is evaluated, the evaluation unit 13 obtains the log likelihood of the hypothesis by calculating the sum of the log likelihood of each edge included in the combination of edges representing each hypothesis. Can do.

  Here, the reference | standard used for evaluation of the likelihood of each edge represented by a bipartite graph is demonstrated. In the bipartite graph shown in FIG. 3, the evaluation unit 13 determines the likelihood of the edge connecting the observation signal z1 or the observation signal z2 and the existing target t_1 with respect to the observation signals z1 and z2 in the gate region G1 shown in FIG. Calculate based on position. For example, the evaluation unit 13 sets the likelihood of the edge connecting the observation signal z1 and the existing target t_1 between the expected value of the target predicted position indicated by the state vector used for setting the gate region and the position of the observation signal z1. You may calculate based on distance. Further, the evaluation unit 13 may calculate in advance the likelihood of each edge that associates the observation signal z1 with the new target t_n1 or the erroneous observation target t_f1 by a known method. Similarly, the evaluation unit 13 may also calculate in advance the likelihood of each edge that associates the observation signal z2 with the new target t_n2 or the erroneous observation target t_f2 by a known method. Furthermore, the evaluation unit 13 uses the likelihood of the edge connecting the observation signal z1 or the observation signal z2 and the branch target t_c1, for example, the likelihood obtained for the edge connecting the observation signal z1 or the observation signal z2 and the existing target t_1. Calculate based on For example, the evaluation unit 13 has a high possibility that the wake will branch from the likelihood of the edge connecting the observation signal z1 and the branch target t_c1 to the likelihood obtained for the edge connecting the observation signal z1 and the existing target t_1. You may calculate by multiplying the weight which shows this. In addition, it is desirable to set a weight smaller than the numerical value “0.1” as the weight indicating the magnitude of the possibility that the wake branches.

  FIG. 5 shows an example of a matrix indicating the log likelihood of each edge of the bipartite graph shown in FIG. Note that among the elements shown in FIG. 5, elements equivalent to those shown in FIGS. 3 and 4 are denoted by the same reference numerals and description thereof is omitted.

  Matrix elements L_11 and L_12 illustrated in FIG. 5 indicate log likelihoods of edges connecting the existing target t_1 and the observation signals z1 and z2, respectively. Further, matrix elements L_1d1, L_1d2, L_1d3, and L_1d4 indicate log likelihoods of edges connecting the existing target t_1 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  Further, matrix elements L_c11 and L_c12 indicate log likelihoods of edges connecting the branch target t_c1 and the observation signals z1 and z2, respectively. The matrix elements L_c1d1, L_c1d2, L_c1d3, and L_c1d4 indicate log likelihoods of edges connecting the branch target t_c1 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  The matrix element L_f11 indicates the log likelihood of the edge connecting the erroneous observation target t_f1 and the observation signal z1. The matrix elements L_f1d1, L_f1d2, L_f1d3, and L_f1d4 indicate log likelihoods of edges that connect the erroneous observation target t_f1 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  Similarly, the matrix element L_n11 indicates the log likelihood of the edge connecting the new target t_n1 and the observation signal z1. Further, matrix elements L_n1d1, L_n1d2, L_n1d3, and L_n1d4 indicate log likelihoods of edges connecting the new target t_n1 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  Further, the matrix element L_f22 indicates the log likelihood of the edge connecting the erroneous observation target t_f2 and the observation signal z2. In addition, matrix elements L_f2d1, L_f2d2, L_f2d3, and L_f2d4 indicate log likelihoods of edges connecting the misobservation target t_f2 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  Similarly, the matrix element L_n22 indicates the log likelihood of the edge connecting the new target t_n2 and the observation signal z2. The matrix elements L_n2d1, L_n2d2, L_n2d3, and L_n2d4 indicate log likelihoods of edges connecting the new target t_n2 and the dummy nodes z_d1, z_d2, Z_d3, and z_d4, respectively.

  Of the edges used to express each hypothesis, for example, a constant such as a numerical value “0” is uniformly set as the log likelihood of an edge connecting one of the dummy nodes and one of the elements of the target set. It is desirable to keep it.

  The evaluation unit 13 shown in FIG. 1 creates a matrix as shown in FIG. 5 for each target to be tracked, and based on the values of the matrix elements included in the created matrix, the logarithmic likelihood of each hypothesis. You may ask for the degree.

For example, the log likelihood L of the hypothesis indicated by the combination of edges indicated by bold lines in the bipartite graph shown in FIG. 3 is expressed by the equation (1) using the values of the matrix elements included in the matrix shown in FIG. It can be expressed as
L = L_11 + L_c1d1 + L_f1d2 + L_n1d3 + L_f22 + L_n2d4 (1)
Similarly, the evaluation unit 13 calculates the log likelihood of each hypothesis generated by the generation unit 12 at each sampling time, and passes the calculated log likelihood of each hypothesis to the tracking unit 14 shown in FIG. Thus, it is used for the tracking process by the tracking unit 14.

  The tracking unit 14 repeatedly performs a process of selecting at least one hypothesis including the hypothesis with the highest likelihood obtained by the evaluation unit 13 from the plurality of hypotheses generated by the generation unit 12. Identify past wakes.

  In the MHT method, a hypothesis tree is generated by positioning each hypothesis generated by the generation unit 12 at each sampling time as a child node of a node indicating an existing track, as shown in FIG. A tracking process is executed based on the tree.

  FIG. 6 shows an example of a hypothetical tree. In FIG. 6, symbols k, k + 1, k + 2, and k + 3 indicate frame numbers indicating the acquisition order of each image obtained at each sampling time.

  In the example of FIG. 6, the target position in the image of frame number k is specified by the past tracking process, and a node indicating a hypothesis that the target exists at the specified position is indicated as a root node R.

  In addition, m nodes n1,..., Nm positioned as child nodes of the root node R can transition from hypotheses corresponding to the root node R among hypotheses generated in response to the acquisition of the image of the frame number k + 1. Presents a hypothesis.

  Similarly, nodes n1_1, n1_2,... Positioned as child nodes of the node n1 indicate hypotheses that can be transitioned from the hypothesis corresponding to the node n1 among the hypotheses generated in response to the acquisition of the image of the frame number k + 2. ing. Nodes nm_1, nm_2,... Positioned as child nodes of the node nm indicate hypotheses that can be transitioned from the hypothesis corresponding to the node nm among the hypotheses generated in response to the acquisition of the image of the frame number k + 2. Yes. Although not shown in FIG. 6, the hypothesis tree includes a node indicating a hypothesis that can be transitioned from other child nodes than the nodes n1 and nm.

  Nodes n1_1_1, n1_1_2,... Positioned as child nodes of the node n1_1 indicate hypotheses that can be transitioned from the hypothesis corresponding to the node n1_1 among the hypotheses generated in response to the acquisition of the image of the frame number k + 3. Yes. Similarly, nodes n1_2_1, n1_2_2,... Positioned as child nodes of the node n1_2 indicate hypotheses that can be transitioned from the hypothesis corresponding to the node n1_2 among the hypotheses generated in response to the acquisition of the image of the frame number k + 3. ing. Although not shown in FIG. 6, the hypothesis tree includes a node indicating a hypothesis that can be transitioned from other child nodes than the nodes n1_1 and n1_2.

  Nodes nm_1_1, nm_1_2,... Positioned as child nodes of the node nm_1 indicate hypotheses that can be shifted from the hypothesis corresponding to the node nm_1 among the hypotheses generated in response to the acquisition of the image of the frame number k + 3. Yes. Similarly, nodes nm_2_1, nm_2_2,... Positioned as child nodes of the node nm_2 indicate hypotheses that can be shifted from the hypothesis corresponding to the node nm_2 among the hypotheses generated in response to the acquisition of the image of the frame number k + 3. ing. Although not shown in FIG. 6, the hypothesis tree includes nodes indicating hypotheses that can be transitioned from other child nodes than the nodes nm_1 and nm_2.

  The tracking unit 14 shown in FIG. 1 calculates, for example, the sum of log likelihoods obtained by the evaluation unit 13 for the hypothesis generated in response to the acquisition of the latest image for the hypothesis tree shown in FIG. The hypothesis may be selected by obtaining each branch having a depth of 1 and performing pruning based on the obtained result.

  Here, the pruning of the hypothetical tree will be described with reference to FIG. In the hypothesis tree shown in FIG. 6, the branches having a depth of 1 are branches B1,..., Bm including nodes indicating hypotheses derived from the nodes n1,. Further, each node corresponding to the leaf of these branches B1,..., Bm indicates each hypothesis generated in response to acquisition of an image of frame number k + 3 which is the latest image.

  For example, when the sum of the log likelihoods of hypotheses corresponding to the nodes corresponding to the leaves of the branch B1 is the maximum, the tracking unit 14 performs pruning to delete all other branches while leaving the branch B1. By this pruning process, the tracking unit 14 selects a hypothesis derived from the branch B1 as a hypothesis for subsequent tracking, and associates the target position in the frame number k + 1 with the target by the hypothesis corresponding to the node n1. Based on the observed signal.

  That is, each time a new image is acquired, the tracking unit 14 performs the above-described pruning process and specifies the position of the target in the image indicated by the frame number that is back by a predetermined number. Tracking processing can be realized.

  Note that the example of FIG. 6 shows a case where the target position in the image of frame number k + 1 that is traced back by 2 is specified based on the tracking result from the image of frame number k + 3 to the hypothesis generated. . The number that goes back the frame number may be a number larger than 2 described above. For example, the tracking unit 14 sets the number going back the frame number as n, and tracks the hypotheses included in the branches up to the depth n + 1 in parallel. Processing may be performed. Further, the tracking unit 14 may display the track of the target obtained by the tracking process by passing information indicating the position of the target specified as described above to the display unit 2.

  Based on the tracking processing result obtained as described above, the setting unit 15 shown in FIG. 1 determines that the extraction unit 11 is the tracking target from the image obtained at the next sampling time as described below. A gate region for extracting an observation signal corresponding to a certain target is set.

  First, the setting unit 15 detects a hypothesis in which an observation signal is associated with an existing target or branch target from the hypothesis selected by the tracking unit 14, and includes the position and speed of the existing target or branch target relating to the detected hypothesis. Find the state vector.

  Of the detected hypotheses, the setting unit 15 uses the state vector obtained as a result of the past tracking process and the position of the observation signal for the hypothesis including the association between the existing target and the observation signal, as in the conventional MHT method. Based on the above, a state vector including the position and speed of the existing target is obtained. Here, since the vector indicating the position of the observation signal extracted from the image obtained by the sensor 1 is affected by white noise, the setting unit 15 includes the position and speed in the latest image of the existing target. It is desirable to estimate the vector by a Kalman filter.

  Similarly, for the hypothesis including the correspondence between the branch target and the observation signal, the setting unit 15 also includes the state including the position and the speed based on the state vector obtained as a result of the past tracking process and the position of the observation signal. Find a vector. Here, as the state vector obtained as a result of the past tracking process, the setting unit 15 uses the state vector obtained for the existing wake of the branch source, so that the setting unit 15 reflects the state vector of the existing wake before the branch. A target state vector can be determined.

  In addition, the setting unit 15 may detect an observation signal indicating each of the existing target and the branch target in an image obtained at the next sampling time based on the state vectors obtained for the existing target and the branch target. Each range is obtained and set as a gate region. When the setting unit 15 obtains the gate area to be set corresponding to each of the existing target and the branch target, the speed indicated by the state vector obtained corresponding to each of the existing target and the branch target varies. It is desirable to consider that. For example, the setting unit 15 obtains a range in which the Mahalanobis distance from the expected value of the target position estimated from the speed described above is equal to or less than a predetermined value by using a Kalman filter that reflects the variation in speed indicated by the state vector. The obtained range may be used as the gate region.

  As described above, the state vector obtained corresponding to the branch target reflects the state vector obtained by the past tracking process for the existing track of the branch source. Therefore, the setting unit 15 can set a narrowed range reflecting the state vector of the existing wake before branching as the gate region corresponding to the branch target.

  For example, as the gate region used in the tracking process for the hypothesis in which the observation signal z2 illustrated in FIG. 2 is associated with the branch target, the setting unit 15 illustrated in FIG. 1 replaces the gate region Gn illustrated in FIG. A gate region G1_c as shown in FIG. 7 can be set.

  FIG. 7 shows an example of the gate region set corresponding to the branch target. Of the elements shown in FIG. 7, the same elements as those shown in FIG. 2 are designated by the same reference numerals, and the description thereof is omitted.

  In FIG. 7, the symbol v <b> 1 indicates the target speed indicated by the state vector of the existing target obtained using the position of the observation signal z <b> 1. Moreover, the code | symbol p1 has shown the expected value of the position of the existing target in the image obtained at the (k + 2) th sampling time estimated based on the speed v1 mentioned above.

  Further, in FIG. 7, the symbol v <b> 2 indicates the target speed indicated by the branch target state vector obtained using the position of the observation signal z <b> 2. Moreover, the code | symbol p2 has shown the expected value of the position of the branch target in the image obtained at the k + 2nd sampling time estimated based on the speed v2 mentioned above.

  Comparing the gate region Gn shown in FIG. 2 with the gate region G1_c shown in FIG. 7, it can be seen that the gate region G1_c is narrowed down based on the speed v2 indicated by the branch target state vector. When the hypothesis that the observation signal z2 is assigned to the branch target is correct, the observation signal corresponding to the branch target is highly likely to be detected in the gate region G1_c of the image obtained at the next sampling time. That is, for the hypothesis in which the observation signal z2 is assigned to the branch target, by performing the tracking process using the gate region G1_c set as described above, there is a possibility of obtaining a correct tracking process result when the wake is branched. Can be increased.

  As described above, according to the tracking device 10 of the present disclosure shown in FIG. 1, a hypothesis representing a case where a wake branches is generated, and the tracking process of the branch source corresponding to the hypothesis is performed in the tracking process. It is possible to apply a gate area that reflects existing wake information. As a result, it is possible to increase the possibility of obtaining the correct tracking processing result even for a branch target having a track branched from the existing track, so that even if the target is at a long distance, the target track is tracked with high accuracy. It becomes possible.

  By the way, in MHT, since the tracking process is performed in parallel for all hypotheses that can be transitioned from the target state specified in the past tracking process, the number of hypotheses generated in response to acquisition of a new image is smaller. Is desirable.

  On the other hand, for example, a substantially equivalent hypothesis is included in the set of hypotheses generated by covering the method of selecting matrix elements so that rows do not overlap from each column of the matrix shown in FIG. It is. For example, a hypothesis set in which one of the observation signals is assigned to one of the existing target and the branch target and a dummy node other than the observation signal is assigned to the other is an example of a substantially equivalent hypothesis.

  Therefore, in the following, a method for generating an expanded set of hypotheses including a hypothesis representing a case where a wake branches while suppressing the number of hypotheses generated in response to acquisition of a new image will be described.

  FIG. 8 shows another embodiment of the tracking device 10. 8 that are the same as those shown in FIG. 1 are given the same reference numerals, and descriptions thereof are omitted.

  The generation unit 12 illustrated in FIG. 8 includes a first generation unit 121, a second generation unit 122, and a deletion unit 123. The first generation unit 121 and the second generation unit 122 receive the observation signal extracted by the extraction unit 11, and generate a first hypothesis set and a second hypothesis set described later, respectively. The first generation unit 121 passes the generated first hypothesis set to the evaluation unit 13. On the other hand, the second generation unit 122 passes the generated second hypothesis set to the evaluation unit 13 via the deletion unit 123.

  The first generation unit 121 generates, for each existing track, a first hypothesis set that includes a hypothesis indicating an updated track that can be assumed when the existing track does not branch. That is, the first hypothesis set generated by the first generation unit 121 is equivalent to the hypothesis set generated by the conventional MHT method. For example, like the conventional MHT method, the first generation unit 121 generates a bipartite graph indicating allocation for a target set that does not include a branch target and a signal set that includes the same number of elements as the target set. The first hypothesis set may be generated.

  On the other hand, for each existing track, the second generation unit 122 uses the two or more observation signals selected from the plurality of observation signals obtained by the extraction unit 12 to update the existing track. Is generated as an element. That is, the second hypothesis set generated by the second generation unit 122 is a set including, as elements, hypotheses indicating branch wakes that can be assumed when existing wakes are necessarily branched. The second generation unit 122 generates, for example, a bipartite graph indicating assignments between a target set including an existing target and at least one branch target as elements, and a signal set including an observation signal. A set may be generated.

  As described above, the elements of the first hypothesis set are hypotheses corresponding to the conventional MHT technique that does not assume the wake bifurcation, and the elements of the second hypothesis set are hypotheses that allocate observation signals to the existing target and the branch target, respectively. That is, this is a hypothesis representing a case where the existing wake is necessarily branched. In this way, by generating hypotheses by dividing them into the first hypothesis set and the second hypothesis set, it is possible to avoid the generation of hypotheses that assign existing targets or branch targets to dummy nodes.

  That is, the first generation unit 121 and the second generation unit 122 generate the first hypothesis set and the second hypothesis set, respectively, thereby suppressing the number of hypotheses that increase due to the consideration of the branch of the wake. Can do.

  FIG. 9 shows an example of the first hypothesis set and the second hypothesis set. Note that among the elements shown in FIG. 9, elements equivalent to those shown in FIG. 4 are denoted by the same reference numerals and description thereof is omitted.

  FIG. 9A shows a bipartite graph generated by the first generator 121 when the extraction unit 11 shown in FIG. 8 extracts two observation signals z1 and z2 corresponding to the existing track. A matrix is shown. Each hypothesis included in the first hypothesis set can be obtained as a combination of selecting one by one from a matrix element having a value “1” of each column of the matrix shown in FIG. it can.

  Similarly, in FIG. 9B, the second generation unit 122 generates the two observation signals z1 and z2 corresponding to the existing track by the extraction unit 11 illustrated in FIG. A matrix representing a bipartite graph is shown. Each hypothesis included in the second hypothesis set can be obtained as a combination of selecting one by one from the matrix elements having the value “1” of each column of the matrix shown in FIG. 9B so that the rows do not overlap. it can. In the example of FIG. 9B, the following two hypotheses are included in the second hypothesis set. One of the hypotheses shows a case where the existing target t_1 is assigned to the observation signal z1 and the branch target t_c1 is assigned to the observation signal z2. The other hypothesis conversely indicates a case where the existing target t_1 is assigned to the observation signal z2 and the branch target t_c1 is assigned to the observation signal z1.

  The number of hypotheses generated from the bipartite graph shown in FIG. 4 is 336, whereas the number of hypotheses generated from the bipartite graph shown in FIG. 9A is 48. The number of hypotheses generated from the bipartite graph shown in FIG. 9B is two as described above. In this way, as described above, by generating hypotheses by dividing the first hypothesis set and the second hypothesis set, it is possible to suppress the number of hypotheses that increase due to the consideration of the branch of the wake.

  Here, when a plurality of observation signals are included in a single gate region, the wake updated with the observation signal associated with the existing target t_1 and the wake updated with the observation signal associated with the branch target t_c1 Both can be treated as two branch tracks branched from the existing track. In other words, when multiple observation signals correspond to a single target, there is no difference in treating any branch track obtained by updating the existing track using each of the multiple observation signals as an extension of the existing track. Absent. Thus, for example, two hypotheses represented by the bipartite graph represented by the matrix shown in FIG. 9B are substantially equivalent, and one of such substantially equivalent hypotheses is deleted. Thus, the number of hypotheses can be further suppressed.

  The generation unit 12 illustrated in FIG. 8 uses the hypotheses included in the second hypothesis set in order to avoid the duplication of substantially equivalent hypotheses and the target of the tracking process, as in the two hypotheses described above. A deletion unit 123 that deletes a part is included.

  When the plurality of observation signals correspond to a single target, the deletion unit 123 replaces the two or more selected observation signals selected from the second hypothesis set to correspond to hypotheses that can be expressed. A set is detected, and one hypothesis is left for each detected hypothesis set, and other hypotheses are deleted. For example, in the matrix representing the bipartite graph generated by the second generation unit 122, the deletion unit 123 selects one of the matrix elements of the column corresponding to each observation signal so that the rows do not overlap, and is not selected. The hypothesis may be deleted by setting a value “0” to the matrix element.

  For example, the matrix shown in FIG. 9C shows a result obtained by performing the above-described deletion processing by the deletion unit 123 shown in FIG. 8 on the matrix shown in FIG. 9B.

  In the example of FIG. 9C, the hypothesis of assigning the observation signal z1 to the existing target t_1 and assigning the observation signal z2 to the branch target t_c1 remains, assigning the observation signal z2 to the existing target t_1, and observing signal z1 to the branch target t_c1. The hypothesis of assigning is deleted.

  As described above, according to the generation unit 12 including the deletion unit 123, a plurality of substantially equivalent hypotheses included in the second hypothesis set can be narrowed down to one hypothesis. The number of hypotheses that increase can be further suppressed.

  For example, the number of hypotheses included in the second hypothesis set represented by the matrix shown in FIG. 9C is one. That is, as shown in FIG. 2, when a plurality of extracted observation signals correspond to a single target, the number of observation signals is increased by assuming that the wake is branched using each of these observation signals. The number of hypotheses can be reduced to one.

  By the way, in the actual tracking process, as shown in FIG. 10, a part of the gate region corresponding to a plurality of targets may overlap, and an observation signal may be extracted from the overlapped part. Next, a method in which the first generation unit 121 and the second generation unit 122 described above generate a first hypothesis set and a second hypothesis set in such a complicated case will be described.

  FIG. 10 shows an example of observation signal extraction. In FIG. 10, reference numerals t <b> 1 and t <b> 2 indicate different target estimated positions obtained based on the tracking processing result up to the previous sampling time. Reference sign G1 indicates a gate area set corresponding to the estimated position t1, and reference sign G2 indicates a gate area set corresponding to the estimated position t2. Symbols z1, z2, and z3 indicate observation signals, respectively.

  The two gate regions G1 and G2 shown in FIG. 10 partially overlap, and the positions of the observation signals z2 and z3 are included in the portion where the gate regions G1 and G2 overlap. On the other hand, the position of the observation signal z1 is included in a range that does not overlap with the gate region G2 in the gate region G1.

  As shown in FIG. 10, when observation signals are extracted from overlapping portions of gate regions set corresponding to two targets, a hypothesis that simultaneously shows possible wakes for these targets may be generated. desirable.

  Here, when the observation signals z1, z2, and z3 shown in FIG. 10 are assigned to two existing tracks, in addition to the case where the two existing tracks do not branch together, the existing track indicated by the estimated position t1 branches. And a case where the existing wake indicated by the estimated position t2 branches. For this reason, the second generation unit 122 illustrated in FIG. 8 has second hypothesis sets that can be considered when the existing track indicated by the estimated position t1 branches and when the existing track indicated by the estimated position t2 branches. It is desirable to generate two bipartite graphs to represent That is, the hypothesis set when the observation signals z1, z2, and z3 shown in FIG. 10 are assigned to two existing tracks is the first hypothesis represented by one bipartite graph as shown in FIGS. A set and a second hypothesis set represented by two bipartite graphs.

  FIG. 11 shows another example of the first hypothesis set. In addition, in FIG. 11, the codes | symbols t_1 and t_2 have each shown the existing target. Reference symbols t_f1, t_f2, and t_f3 indicate erroneous observation targets corresponding to the observation signals z1, z2, and z3, respectively. Similarly, symbols t_n1, t_n2, and t_n3 indicate new targets corresponding to the observation signals z1, z2, and z3, respectively. Also, the symbols z_d1, z_d2, z_d3, z_d4, and z_d5 all indicate dummy nodes.

  The matrix shown in FIG. 11 represents a first hypothesis set that includes, as elements, hypotheses when the existing wake indicated by the estimated position t1 and the existing wake indicated by the estimated position t2 shown in FIG. 10 do not branch. A bipartite graph is shown.

  Similarly to the matrixes shown in FIGS. 4 and 9, the values of the matrix elements of the matrix shown in FIG. 11 are the values of the signal set corresponding to the elements and columns of the target set corresponding to the row indicating the matrix element. It indicates whether or not it can be associated with an element. That is, a matrix element having a value of “1” indicates an edge capable of associating a target set element with a signal set element, and a matrix element having a value of “0” indicates a target set element. And edges that can be associated with the elements of the signal set.

  12 and 13 show another example of the second hypothesis set. Of the elements shown in FIGS. 12 and 13, the same elements as those shown in FIG. 11 are designated by the same reference numerals, and the description thereof is omitted.

  In FIG. 12, a symbol t_c1 indicates a branch target corresponding to the existing track indicated by the estimated position t1, and a symbol z_d6 indicates a dummy node. On the other hand, in FIG. 13, a symbol t_c2 indicates a branch target corresponding to the existing wake indicated by the estimated position t2, and a symbol z_d6 similarly indicates a dummy node.

  The matrix shown in FIG. 12 shows a set of hypotheses including, as elements, hypotheses assumed when the existing track indicated by the estimated position t1 branches, and shows a subset of the second hypothesis set.

  In the matrix shown in FIG. 12, among the matrix elements included in the row indicated by the existing target t_1 and the branch target t_c1, the value “0” is set for the matrix elements corresponding to the dummy nodes z_d1 to z_d6. By setting the value “0” to the matrix element described above, the assignment destination of the existing target t_1 and the branch target t_c1 can be limited to any one of the observation signals z_1 to z_3. That is, the matrix shown in FIG. 12 can express a hypothesis that the existing wake indicated by the estimated position t1 is always branched using two observation signals selected from the observation signals z_1 to z_3.

  Similarly, the matrix shown in FIG. 13 shows a set of hypotheses including, as elements, hypotheses assumed when the existing track indicated by the estimated position t2 branches. A set of hypotheses including hypotheses represented by the matrix shown in FIG. 13 is also a subset of the second hypothesis set.

  In the matrix shown in FIG. 13, among the matrix elements included in the row indicated by the existing target t_2 and the branch target t_c2, the value “0” is set for the matrix elements corresponding to the dummy nodes z_d1 to z_d6. By setting the value “0” to the matrix element described above, the assignment destination of the existing target t_2 and the branch target t_c2 can be limited to any one of the observation signals z_1 to z_3. That is, the matrix shown in FIG. 13 can express a hypothesis that the existing track indicated by the estimated position t2 is always branched using two observation signals selected from the observation signals z_1 to z_3.

  Note that the second generation unit 122 illustrated in FIG. 8 includes a second hypothesis set including a hypothesis that branches each existing track to the number of observation signals extracted from the gate region set corresponding to the existing track. May be generated. For example, the second generation unit 122 adds branch targets up to the number obtained by subtracting 1 from the number of observation signals extracted from the gate region, and adds a corresponding dummy node to the signal set. Bipartite graphs may be generated for the target set and signal set. In this manner, the second generation unit 122 may generate a second hypothesis set that includes a set of hypotheses corresponding to various branching methods as a subset. It is desirable to determine what branching method to generate and generate the second hypothesis set in consideration of the high possibility that each branching method will occur. For example, in consideration of the fact that one wake is unlikely to divide into three or more wakes compared to the possibility that one wake divides into two wakes, the number of branches for one wake is two or less. You may restrict to.

  Further, the deletion unit 123 illustrated in FIG. 8 uses the fact that two or more observation signals included in the signal set are always assigned to the existing target and the branch target corresponding to the existing track to be branched. You may perform the process which simplifies the bipartite graph showing a set.

  For example, an observation signal that is surely assigned to an existing target or a branch target is not assigned to a new target or an erroneous observation target. Therefore, by simplifying the bipartite graph representing the second hypothesis set by deleting nodes indicating new targets and false observation targets provided in the target set corresponding to observation signals assigned to existing targets or branch targets Can do. A method for simplifying the bipartite graph will be described later with reference to a flowchart shown in FIG.

  By the way, compared with the possibility that one wake extends to a position indicated by one of the observation signals in the gate region, the possibility that one existing wake branches into two or more wakes is small. Therefore, when comparing the probability of each hypothesis included in the first hypothesis set described above with the probability of each hypothesis included in the second hypothesis set, the possibility that the existing track is extended and the existing track branches. It is desirable to apply a weight that reflects the difference from the possibility of doing so.

  In order to realize the application of such weights, the evaluation unit 13 illustrated in FIG. 8 includes a matrix generation unit 131, a weight setting unit 132, and a likelihood calculation unit 133.

  The matrix generation unit 131 includes a bipartite graph representing the first hypothesis set generated by the first generation unit 121 and the second generation unit 122 generated by the second generation unit 122 and subjected to simplification processing by the deletion unit 123. A bipartite graph representing a hypothesis set is received.

  In response to each received bipartite graph, the matrix generation unit 131 generates a likelihood matrix used for calculating the likelihood of each hypothesis represented by each bipartite graph as follows. For example, the matrix generation unit 131 generates a matrix having the same number of rows and columns as the matrix representation of each bipartite graph as a likelihood matrix, and sets an initial value “NULL” to each matrix element of the generated likelihood matrix. . Next, the matrix generation unit 131 calculates the log likelihood indicating the likelihood of each edge indicated by the matrix element having the value “1” in the matrix representing the bipartite graph received from the generation unit 12 as the likelihood matrix. Set as a matrix element corresponding to the edge.

  When the matrix generation unit 131 generates the above-described likelihood matrix, for example, the matrix element corresponding to the edge that associates each observation signal with the existing target and the branch target is set at a position in the gate region of each observation signal. The logarithm of the likelihood obtained based on this is set. The likelihood based on the position of each observation signal in the gate region is, for example, the distance between the expected value of the estimated position of the existing wake obtained by the past tracking process and the position of each observation signal for the existing wake. It is desirable to calculate such that the smaller the value, the larger the value. On the other hand, when the matrix generation unit 131 generates the above-described likelihood matrix, for example, for a matrix element corresponding to an edge that associates each observation signal with a new target, a predetermined constant that reflects the possibility of occurrence of the new target is used. It is desirable to set Similarly, when generating the above-described likelihood matrix, the matrix generation unit 131 reflects the possibility of occurrence of an erroneous observation target for, for example, a matrix element corresponding to an edge that associates each observation signal with an erroneous observation target. It is desirable to set a predetermined constant. In addition, when generating the above-described likelihood matrix, the matrix generation unit 131 desirably sets the same constant for matrix elements corresponding to edges that associate each element of the target set with a dummy node, for example.

  The weight setting unit 132 sets a first weight, which will be described later, to the likelihood matrix generated by the matrix generation unit 131 corresponding to the bipartite graph representing the first hypothesis set, while the two parts representing the second hypothesis set A second weight described later is set in the likelihood matrix generated corresponding to the graph.

  When only a bipartite graph corresponding to the first hypothesis set is generated as a hypothesis corresponding to one existing wake, the weight setting unit 132 uses the likelihood generated by the matrix generation unit 131 corresponding to the bipartite graph. A weight of 1 is given to the degree matrix.

  On the other hand, when at least one bipartite graph corresponding to the second hypothesis set is generated together with the bipartite graph corresponding to the first hypothesis set as a hypothesis corresponding to one or more existing tracks, the weight setting unit 132 The first weight and the second weight are calculated as follows.

  For example, the weight setting unit 132 may calculate the second weight given to the likelihood matrix corresponding to each bipartite graph included in the second hypothesis set according to the number of branch targets included in the bipartite graph. Good. For example, the weight calculation unit 132 sets in advance a coefficient Pc that reflects the possibility that one existing track branches into two tracks, and the coefficient Pc is set according to the number of branch targets included in each bipartite graph. The second weight given to the corresponding likelihood matrix may be calculated by raising the power. Note that the value of the coefficient Pc is desirably set to a relatively small value such as a numerical value “0.001”.

  In addition, the weight setting unit 132 performs the first hypothesis corresponding to the second hypothesis set based on the second weight calculated for the likelihood matrix corresponding to each bipartite graph included in the second hypothesis set as described above. You may calculate the 1st weight set to the likelihood matrix produced | generated corresponding to the bipartite graph showing a set. For example, the weight setting unit 132 obtains the sum of the second weights obtained for the likelihood matrix corresponding to each bipartite graph representing the second hypothesis set, and obtains a value obtained by subtracting the obtained sum from the numerical value “1” as the first It may be set as a weight.

  The weight setting method by the weight setting unit 132 will be described later with reference to the flowchart shown in FIG.

  In addition, the weight setting unit 132 obtains the logarithm of the first weight and the logarithm of the second weight calculated as described above, and adds the obtained logarithmic weight to each matrix element of the likelihood matrix as described above. May be applied. Further, the weight setting unit 132 passes each likelihood matrix to which the weight is applied in this way to the likelihood calculating unit 133, and is used for calculating the likelihood for each hypothesis.

  The likelihood calculating unit 133 executes a process of calculating the likelihood of each hypothesis based on the likelihood matrix obtained as described above, and the tracking process by the tracking unit 14 is performed on the obtained likelihood of each hypothesis. To serve. The likelihood calculation unit 133 selects, for example, matrix elements set with log likelihood values one by one so that the rows do not overlap among the matrix elements included in each column included in each likelihood matrix, The log likelihood of each hypothesis is calculated by adding the values of the selected matrix elements.

  As described above, the value of each matrix element included in the likelihood matrix generated corresponding to the bipartite graph representing the second hypothesis set reflects the second weight set by the weight setting unit 132. Yes. Therefore, the logarithmic likelihood value calculated corresponding to each hypothesis by the likelihood calculating unit 133 is less likely to branch than each individual track is extended. It is an index for legitimately assessing the certainty.

  As described above, according to the tracking device 10 shown in FIG. 8, the tracking process considering the possibility of the existing track branching while suppressing the number of hypotheses generated for each existing track to be tracked. Can be executed based on a valid evaluation index. That is, according to the tracking device 10 shown in FIG. 8, the tracking processing from the stage where the target is at a long distance can be realized with high accuracy and while suppressing an increase in processing load as much as possible.

  The tracking device 10 of the present disclosure described above can be realized using a computer device.

  FIG. 14 shows an example of the hardware configuration of the tracking device 10. Of the components shown in FIG. 14, components equivalent to those shown in FIG. 1 are denoted by the same reference numerals and description thereof is omitted.

  The computer device 20 shown in FIG. 14 includes a processor 21, a memory 22, a hard disk device 23, a sensor interface 25, a display control unit 25, an input device 26, and an optical drive device 27. The processor 21, the memory 22, the hard disk device 23, the sensor interface 24, the display control unit 25, the input device 26, and the optical drive device 27 are connected to one another via a bus.

  FIG. 14 shows an example in which the tracking device 10 is realized by the processor 21, the memory 22, the hard disk device 23, and the sensor interface 24. By controlling the function of the sensor interface 24, the processor 21 can realize a function of receiving images obtained by the sensor 1 at a predetermined sampling interval. Further, the processor 21 controls the function of the display control unit 25, whereby the image received from the sensor 1 and the tracking processing result can be displayed on the display device 2.

  An optical drive device 27 provided in the computer device 20 can be mounted with a removable disk 28 such as an optical disk, and reads and records information recorded on the mounted removable disk 28. The input device 25 is, for example, a keyboard or a mouse. The operator of the tracking device 10 operates the input device 25 to give an instruction to the tracking device 10 to start receiving an image obtained by the sensor 1, an instruction to start the tracking process, or the like. Can be entered.

  Along with the operating system of the computer device 20, the memory 22 stores an application program for the processor 21 to execute the tracking process described above. The application program for executing the tracking process described above can be recorded and distributed on a removable disk 28 such as an optical disk, for example. Then, an application program for executing the tracking process may be stored in the memory 22 and the hard disk device 23 by mounting the removable disk 28 in the optical drive device 27 and performing a reading process. In addition, an application program for executing tracking processing can be read into the memory 22 and the hard disk device 23 via a communication device (not shown) connected to a network such as the Internet.

  The processor 21 may perform the functions of the extraction unit 11, the generation unit 12, the evaluation unit 13, the tracking unit 14, and the setting unit 15 illustrated in FIG. 1 by executing an application program stored in the memory 22.

  That is, the tracking device 10 disclosed herein can be realized by the cooperation of the processor 21, the memory 22, the hard disk device 23, and the sensor interface 24 included in the computer device 20 described above, for example.

  FIG. 15 shows an example of a flowchart of the tracking process. Each process of step 301 to step 309 illustrated in FIG. 15 is an example of a process included in the application program for the tracking process. The processing of step 301 to step 309 is an implementation example of a tracking method that assumes branching of a wake, and is an embodiment of the tracking method disclosed herein. Further, each of the processes of Step 301 to Step 309 is executed by the processor 21 at, for example, a predetermined sampling interval when the target tracking is continuously performed based on the past tracking process result.

  First, the processor 21 sets a gate area for each existing track as described with reference to FIG. 2 based on the result obtained in the past tracking process (step 301). Next, the processor 21 acquires an image obtained by the sensor 1 via the sensor interface 24 shown in FIG. 14 (step 302). Next, the processor 21 extracts an observation signal from the image acquired from the sensor 1 for each gate region set in step 301 (step 303). In this way, the processor 21 can implement the function of the extraction unit 11 shown in FIG. 1 by executing step 302.

  Next, the processor 21 compares the observation signals extracted from the respective gate regions with each other to detect targets sharing the observation signals used for updating the wake, and groups the detected targets (step 304). For example, as shown in FIG. 10, when an observation signal is extracted from a portion where two gate regions overlap, the processor 21 observes two targets to be tracked corresponding to these gate regions. The signals z2 and z3 are detected as a shared target. Further, the processor 21 forms a group including a plurality of targets detected in this way. Note that when the observation signal extracted from the gate region corresponding to one of the targets to be tracked is not included in the gate region corresponding to the other target, the processor 21 includes the gate including the above-described observation signal. Groups may be formed that include a single goal corresponding to the region.

  Next, for each group formed in step 304, the processor 21 generates a bipartite graph representing a set of hypotheses shown in FIG. 16 in order to collectively generate hypotheses related to the update of the target track belonging to the group. (Step 305).

  FIG. 16 shows an example of a flowchart of processing for generating a bipartite graph. Each process of step 311 to step 319 shown in FIG. 16 is an example of a process for generating a bipartite graph shown in step 305 of FIG. In addition, each processing of step 311 to step 319 is executed by the processor 21.

  First, the processor 21 generates a basic bipartite graph for each group formed in step 304 described above (step 311). As described with reference to FIGS. 2 and 3, the processor 21 includes a target set including targets to be tracked included in each group and observation signals extracted from gate regions corresponding to these targets. For the signal set, a bipartite graph similar to the conventional MHT is generated. In the following description, the bipartite graph generated in step 311 for each group is referred to as a basic bipartite graph corresponding to the group. This basic bipartite graph corresponds to a bipartite graph representing each hypothesis included in the first hypothesis set generated by the first generation unit 121 shown in FIG. That is, the processor 21 executing the process of step 311 is an example of a technique for realizing a part of the function of the first generation unit 121.

  Next, the processor 21 counts the number nt of observation signals extracted from the gate region corresponding to each target in the group (step 312), and determines whether or not the number nt of the obtained observation signals is plural. Is determined (step 313).

  When the number nt of observation signals is larger than the numerical value “1”, the processor 21 performs the processing of step 314 and step 315 as the processing of the affirmative determination route of step 313 and then proceeds to the processing of step 316.

  In step 314, the processor 21 generates at least one copy of the basic bipartite graph described above with the number obtained by subtracting the numerical value “1” from the number nt of the observation signals obtained in step 312 as the upper limit. For example, when three observation signals are extracted from the gate region G1, the processor 21 generates basic hypotheses in consideration of the case where the target track to be tracked branches into two tracks. Generate only one copy of the subgraph. On the other hand, in addition to the case where the wake branches into two wakes, the processor 21 generates two copies of the basic bipartite graph when generating a hypothesis considering the case where the wake branches into three wakes.

  Next, the processor 21 adds, to the at least one bipartite graph generated in the process of step 314, a different number of nodes indicating the branch target, which is not more than the number of duplicated bipartite graphs (step 315). For example, when two copies of the basic bipartite graph are generated in the process of step 314, the processor 21 adds one node indicating a branch target to one of the bipartite graphs generated as the copy, and the other Two nodes are added to the bipartite graph. The at least one bipartite graph to which the node indicating the branch target is added in this way corresponds to the bipartite graph representing the second hypothesis set generated by the second generation unit 122 shown in FIG. That is, the processor 21 executing the processes of steps 314 and 315 is an example of a technique for realizing a part of the function of the second generation unit 122.

  On the other hand, when the count value nt obtained in step 312 is equal to or smaller than the numerical value “1”, the processor 21 determines that it is not necessary to generate a bipartite graph for representing a hypothesis that the target track branches. . In this case, the processor 21 skips the processing of step 314 and step 315 described above according to the negative determination route of step 313, and proceeds to the processing of step 316.

  The affirmative determination route processing and the negative determination route processing in step 313 merge in step 316. In step 316, the processor 21 determines whether or not the bipartite graph generation processing for representing the branch of the wake has been completed for all the targets belonging to the group. When there is a target in the group for which generation processing of the bipartite graph for representing the branch of the wake has not been completed, the processor 21 returns to the processing of step 312 according to the negative determination route of step 316, and the unprocessed target For one, the processing after step 312 is executed.

  When the processing of step 312 to step 316 is executed for all the targets belonging to the group, the processor 21 proceeds to the processing of step 317 according to the affirmative determination route of step 316.

  In step 317, the processor 21 calculates a weight to be set for each bipartite graph shown in FIG. 17 for at least one bipartite graph generated corresponding to the target group to be tracked in the above-described processing. Execute.

  FIG. 17 shows an example of a flowchart of processing for calculating the weight of a bipartite graph. Each process of step 321 to step 329 shown in FIG. 17 is an example of a process for calculating the weight of the bipartite graph shown in step 317 of FIG. In addition, each processing of step 321 to step 329 is executed by the processor 21.

  First, the processor 21 determines whether or not there is a duplicated bipartite graph among the bipartite graphs generated by the processing of the above-described steps 311 to 16 (step 321).

  If the copied bipartite graph includes a duplicated bipartite graph indicating a hypothesis when the wake branches, the processor 21 performs steps 322 to 322 as the affirmative determination route processing of step 321. The process of step 327 is executed.

  In the affirmative determination route of step 321, the processor 21 first initializes the variable S by setting an initial value “0” to the variable S used in the subsequent processing (step 322).

Next, the processor 21 counts the number of branch targets nm included in the target set of each bipartite graph (step 323), and the weight to be applied to the bipartite graph based on the obtained count value nm. A coefficient is calculated (step 324). For example, the processor 21 may calculate the weighting coefficient set in the bipartite graph described above by raising a preset coefficient Pc to the power of the count value nm. That is, the processor 21 may calculate the weighting coefficient P set in the bipartite graph including nm nodes indicating the branch target using the equation (2).
P = (Pc) ^nm (2)
Each time the processor 21 calculates the weighting coefficient in the process of step 324, the processor 21 adds the calculated weighting coefficient to the variable S (step 325), and thereafter, for all duplicated bipartite graphs, the processing from step 323 to step 325 is performed. It is determined whether or not the processing is completed (step 326).

  If there is an unprocessed copied bipartite graph, the processor 21 returns to the process of step 323 according to the negative determination route of step 326, and executes a process of calculating a weighting factor for the newly copied bipartite graph. To do.

  In this way, the processing of Steps 323 to 326 is repeated, and when the processing for all the copied bipartite graphs is completed, the processor 21 proceeds to the processing of Step 327 according to the affirmative determination route of Step 326.

  In step 327, the processor 21 calculates the weight coefficient of the basic bipartite graph by subtracting the value of the variable S obtained in the process so far from the numerical value “1”, and then performs the process of step 329. move on.

  On the other hand, when the copied bipartite graph is not included in the bipartite graph to be processed, that is, when the bipartite graph to be processed is only the basic bipartite graph, the processor 21 performs step 321. As the negative determination route processing of step 328, the processing of step 328 is executed.

  In step 328, the processor 21 sets the weighting coefficient of the basic bipartite graph to a numerical value “1”, and proceeds to the process of step 329.

  The affirmative determination route processing and the negative determination route processing in step 321 merge in step 329. In step 329, the processor 21 calculates the logarithm of the weighting coefficient calculated for each bipartite graph in the above-described processing, and sets the obtained value as a weight for each bipartite graph.

  As described above, the processor 21 executing the processing of step 321 to step 329 in the process of generating the bipartite graph is an example of a technique for realizing the function of the weight setting unit 132 illustrated in FIG. . Note that the weights obtained corresponding to each bipartite graph as described above are stored in the memory 22 or the hard disk device 23 shown in FIG. 14 corresponding to each bipartite graph, for example. Can do. The weights of the individual bipartite graphs held in the memory 22 or the hard disk device 23 can be used when the processor 21 executes a process for calculating the likelihood of each hypothesis described later.

  As described above, after the process of calculating the weight of each bipartite graph generated for the group to be processed is completed, the processor 21 proceeds to the process of step 318 in FIG.

  In step 318, the processor 21 executes processing for setting an edge shown in FIG. 18 for each bipartite graph generated for the group to be processed.

  FIG. 18 shows an example of a flowchart of processing for setting an edge of a bipartite graph. Each process of step 331 to step 340 shown in FIG. 18 is an example of a process for setting an edge of the bipartite graph shown in step 318 of FIG. In addition, each processing of step 331 to step 340 is executed by the processor 21.

  The processor 21 executes the processes of Step 331 to Step 340 described below for each of at least one bipartite graph generated for the group to be processed.

  First, the processor 21 sets an edge connecting each node indicating an existing target and a node indicating each observation signal extracted from the gate region corresponding to the existing target, and calculates the edge for each set edge. The log likelihood is given (step 331). In the matrix representing the bipartite graph to be processed, the processor 21 sets a numerical value “1” in the matrix element indicated by each existing target and the observation signal corresponding thereto, thereby setting the edge corresponding to the matrix element. You may show that. Further, the processor 21 may calculate the logarithm of the likelihood of the edge connecting each existing target and the corresponding observation signal as described in the description of the processing of the matrix generation unit 131 shown in FIG. . Further, the processor 21 provides a likelihood matrix corresponding to each bipartite graph in the memory 22 or the hard disk device 23, and stores the log likelihood obtained for each edge as the value of the matrix element of the corresponding likelihood matrix. Thus, log likelihood may be given to these edges.

  Next, the processor 21 sets an edge connecting each node indicating the branch target and a node indicating each observation signal extracted from the gate region corresponding to the existing target of the branch source, and sets the branch source to each set edge. The log likelihood calculated corresponding to is given (step 332). In the matrix representing the bipartite graph to be processed, the processor 21 sets a numerical value “1” in the matrix element indicated by each branch target and the corresponding observation signal, thereby setting the edge corresponding to the matrix element. You may show that. Further, the processor 21 holds the log likelihood obtained for the edge connecting the existing target of the branch source of each branch target and each observation signal in the processing of step 331, and the stored value is used as the processing of step 332. Can be used. Further, the processor 21 converts the log likelihood obtained corresponding to the existing target of the branch source into a likelihood matrix provided corresponding to the bipartite graph to be processed, for each branch target and each observed signal. A log likelihood may be given to the corresponding edge by holding it as the value of the matrix element shown.

  Next, the processor 21 sets an edge connecting the observation signal and each node indicating a new target and an erroneous observation target, and gives a predetermined log likelihood to each set edge (step 333). In the matrix representing the bipartite graph to be processed, the processor 21 sets a numerical value “1” in the matrix element indicated by each new target and each observation signal corresponding to each false observation target, thereby corresponding to the matrix element. It may indicate that the edge to be set has been set. Further, the processor 21 can obtain a predetermined log likelihood to be given to the edge connecting the new target and the erroneous observation target and each observation signal as described in the description of the matrix generation unit 131 in FIG. In addition, the processor 21 sets the obtained log likelihood to a likelihood matrix provided corresponding to the bipartite graph to be processed, and a matrix corresponding to the edge connecting the new target and the erroneous observation target and each observation signal. A log likelihood may be given to the set edge by setting it as an element value.

  Next, the processor 21 determines whether or not the bipartite graph to be processed includes a node indicating a branch target (step 334).

  In the case of negative determination in step 334, the processor 21 sets an edge connecting each dummy node and all elements of the target set, and gives a predetermined log likelihood to each set edge (step 335). As described above, the processor 21 desirably sets the same constant value (for example, the numerical value “0”) as the log likelihood for the edge connecting the dummy node and each element of the target set. In addition, as a method of giving log likelihood to each of these edges, the predetermined log likelihood described above as the value of the matrix element corresponding to each edge is added to the likelihood matrix provided corresponding to the bipartite graph to be processed. It may be stored.

  On the other hand, when the bipartite graph to be processed includes a node indicating the branch target, the processor 21 executes the processing of step 336 to step 339 as the processing of the affirmative determination route of step 334.

  In step 336, an edge connecting the node indicating the existing target that does not branch, the new target and the erroneous observation target, and the dummy node is set, and a predetermined log likelihood is given to each edge (step 336). As described above, the processor 21 uses the same constant value (for example, the numerical value “0”) as the log likelihood for the edge connecting the dummy node and the existing target that does not branch and the node indicating the new target and the erroneous observation target. It is desirable to set. In addition, as a method of giving log likelihood to each of these edges, the predetermined log likelihood described above as the value of the matrix element corresponding to each edge is added to the likelihood matrix provided corresponding to the bipartite graph to be processed. It may be stored.

  Next, the processor 21 counts the branch target number nc indicating the number of branch targets added to the bipartite graph to be processed and the observation signal number nz indicating the number of observation signals corresponding to the existing target of the branch source. (Step 337). Next, the processor 21 compares the total number of branches obtained by adding 1 to the target branch number nc obtained in step 337 and the observed signal number nz, thereby simplifying the simplification process described later for the bipartite graph to be processed. (Step 338).

  Here, when the total number of branches obtained for the bipartite graph to be processed is equal to the observed signal number nz, all of the observed signals are assigned to either the existing target or the branch target. In this case, as described in the description of the deletion unit 123 illustrated in FIG. 8, a hypothesis that leaves a hypothesis for branching the existing wake obtained in the past tracking process into the number of wakes equal to the number of observation signals nz. , That is, a bipartite graph representing a set of hypotheses can be simplified.

  The processor 21 determines that the process of simplifying the bipartite graph can be executed when the total number of branches obtained for the bipartite graph to be processed is equal to the number of observed signals nz. In this case, the processor 21 proceeds to step 339 according to the affirmative determination route of step 338, and executes processing for simplifying the bipartite graph shown in FIG.

  FIG. 19 shows an example of a flowchart of processing for simplifying a bipartite graph. Each process of step 341 to step 346 shown in FIG. 19 is an example of a process for simplifying the bipartite graph shown in step 339 of FIG. In addition, each processing of step 341 to step 346 is executed by the processor 21.

  First, in the bipartite graph to be processed, the processor 21 sets the edges connecting the existing target and branch target of the branch source and the corresponding observation signals among the edges set in the processing so far as follows. (Step 341). The processor 21 deletes other edges from the above-mentioned edges, for example, leaving only the edges connecting each of the existing target and branch target of the branch source and one of the observation signals. That is, the processor 21 deletes all edges that represent other combinations, leaving one combination among the combinations of edges that represent hypotheses that can be expressed by exchanging observation signals associated with the existing target and the branch target. To do.

  Next, the processor 21 determines whether or not the bipartite graph to be processed includes a node indicating a target that does not branch (step 342).

  For example, as in the example shown in FIG. 10, when the observation signal is detected in an overlapping part of two or more gate regions, the two parts generated as a representation of the second hypothesis set assuming a wake branch The graph includes nodes that indicate targets that do not branch. As described above, when the bipartite graph including the node indicating the target that does not branch is the processing target, the processor 21 executes each process of Step 343 to Step 345 as the process of the affirmative determination route of Step 342.

  In step 343, the processor 21 deletes all edges connecting the node indicating the observation signal corresponding to the target to be branched and the node indicating the target not to be branched (step 343). Next, the processor 21 deletes a node indicating a new target and an erroneous observation target provided corresponding to the observation signal corresponding to the branch target (step 344). Furthermore, the processor 21 deletes the dummy nodes included in the signal set so that the number of nodes in the target set and the number of nodes in the signal set decreased by the above processing become the same (step 345).

  By executing the processes in steps 343 to 345 described above, for example, the bipartite graph shown in FIG. 13 can be simplified as shown in FIG.

  FIG. 20 shows an example of a simplified bipartite graph. Note that among the elements illustrated in FIG. 20, elements equivalent to those illustrated in FIG. 13 are denoted by the same reference numerals, and description thereof is omitted.

  FIG. 20A shows an example of a simplified bipartite graph obtained by applying the simplification process described above to the bipartite graph shown in FIG. The bipartite graph of FIG. 20A fixes the correspondence between the observation signals z_2, z_3, the existing target t_2, and the branch target t2_c1, and the existing target t_1, the new target, and the false observation target where the observation signals z_2, z_3 do not branch A set of hypotheses excluding the possibility of being assigned to is shown.

  20B applies the simplification process described above to the hypothesis set in the case where the existing track corresponding to the estimated position t1 is branched using three observation signals in the example shown in FIG. An example of a bipartite graph obtained by The bipartite graph in FIG. 20B is an example in which the observation signals z_1, z_2, and z_3 are associated with the existing target t_1 and the branch targets t1_c1 and t1_c2 on a one-to-one basis, and the hypothesis that associates the existing target t_2 with the dummy node is narrowed down. It is.

  From the example shown in FIGS. 20A and 20B, it can be seen that the number of hypotheses can be significantly suppressed by performing the above-described simplification.

  On the other hand, when it is determined in step 342 described above that the bipartite graph to be processed does not include a target that does not branch, the processor 21 assumes that the bipartite graph is an observation signal extracted from a single gate region. Is determined to be a bipartite graph. In this case, the processor 21 executes the process of step 346 according to the negative determination route of step 342.

  In step 346, the processor 21 simplifies the bipartite graph by deleting from the bipartite graph all nodes other than the nodes corresponding to the branch target existing target and the branch target and each observation signal, respectively. Thereby, as shown in FIG. 9C, the processor 21 can obtain a simplified bipartite graph representing the association between the existing target and the branch target and any one of the observation signals.

  As described above, in the affirmative determination route in step 338 shown in FIG. 18, the processor 21 executing the processes in steps 341 to 346 described above realizes the function of the deletion unit 123 shown in FIG. 8. It is an example. Note that the processor 21 desirably reflects the result of the above-described simplification process on a likelihood matrix provided corresponding to each bipartite graph.

  The processor 21 proceeds to the processing of step 340 shown in FIG. 18 after the processing of the affirmative determination route or the negative determination route in step 342 described above, and performs processing for all the bipartite graphs belonging to the processing target group. It is determined whether or not is completed.

  When there is an unprocessed bipartite graph, the processor 21 returns to step 331 according to the negative determination route of step 340, and executes the processing of step 331 to step 339 for one of the unprocessed bipartite graphs.

  Thereafter, when the processing for all the bipartite graphs is completed, the processor 21 proceeds to the processing of step 319 of FIG. 16 according to the affirmative determination route of step 340, and all the groups formed in step 304 of FIG. It is determined whether or not the process has been completed.

  If there is a group for which bipartite graph generation has not been completed, the processor 21 returns to step 311 according to the negative determination route of step 319 and executes the processing of step 311 to step 319 for one of the unprocessed groups. .

  As described above, the processor 21 performs the processes of Step 311 to Step 319 for each group, thereby generating the generation unit including the first generation unit 121, the second generation unit 122, and the deletion unit 123 illustrated in FIG. Twelve functions can be realized.

  Thereafter, when the processing for all the groups is completed, the processor 21 proceeds to the processing of step 306 shown in FIG. 15 according to the affirmative determination route of step 319.

  In step 306, the processor 21 determines the log likelihood of each hypothesis based on the likelihood matrix corresponding to each bipartite graph provided in the memory 22 or the hard disk device 23 in the process of generating the above bipartite graph. Is calculated.

  Next, based on the log likelihood of each hypothesis obtained in step 306, the processor 21 performs tracking processing for each target by, for example, selecting hypotheses by the pruning process described with reference to FIG. Is executed (step 307).

  Further, the processor 21 estimates a state vector for each target to be tracked based on the hypothesis narrowed down in step 307, and uses the estimation result for the subsequent tracking process (step 308). The processor 21 performs processing for estimating a state vector for the branch target based on a hypothesis in which one of the observation signals is assigned to the branch target, for example, at the previous sampling time obtained for the existing track of the branch source. Use the estimated position of the target. Thereby, the processor 21 can estimate the state vector reflecting the result of the past tracking process obtained for the track before the branch corresponding to the branch target.

  The processor 21 determines whether or not to continue the subsequent tracking process when the process of step 308 is completed (step 309).

  For example, when the instruction to end the tracking process is not given by the input device 26 shown in FIG. 14 or the like, the processor 21 determines to continue the tracking process. In this case, according to the affirmative determination route of step 309, the process returns to step 301, and the gate region of each existing track is set using the state vector of each target obtained by the process of step 308 described above. Execute tracking processing.

  As described above, in step 308, the processor 21 obtains a state vector for the branch target by estimation processing using the state vector obtained in the past tracking processing for the existing track of the branch source. Based on the state vector thus obtained, the processor 21 executes the process of step 301 to set a range reflecting the result of the past tracking process as the gate area corresponding to the branch track. Is possible. That is, the function of the setting unit 15 shown in FIGS. 1 and 8 is realized by the processor 21 executing the process of step 301 on the image obtained at the next sampling time using the process result of step 308. can do.

  As described above, when the processor 21 executes the processes shown in FIGS. 15 to 19, the computer apparatus 20 shown in FIG. 14 can realize the tracking process in consideration of the branch of the wake. Thereby, it is possible to track the target with high accuracy from the stage where the target is at a long distance where the wake bifurcation occurs.

Regarding the above description, the following items are further disclosed.
(Appendix 1)
An extraction unit that extracts an observation signal having a possibility of being a target image to be tracked from an image obtained from a sensor at each sampling time;
When a plurality of observation signals are extracted by the extraction unit, using two or more observation signals selected from the plurality of observation signals, an existing wake that is a wake obtained by past tracking processing for the target is used. A generator that generates a plurality of hypotheses including a hypothesis indicating a branch wake that is a wake branched by updating;
An evaluation unit that evaluates the likelihood indicating the likelihood of each of the plurality of hypotheses generated by the generation unit based on a predetermined criterion;
By selecting at least one hypothesis including a hypothesis with the highest likelihood obtained by the evaluation unit from among a plurality of hypotheses generated by the generation unit at each sampling time, the past track of the target is selected. A tracking section for identifying
When a hypothesis indicating the branch wake is included in the hypothesis selected by the tracking unit, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at the next sampling time is represented as a branch of the branch wake. And a setting unit configured to set based on the original existing track.
(Appendix 2)
In the tracking device according to attachment 1,
The generator is
A first generation unit that generates a first hypothesis set including a hypothesis indicating an updated wake that can be assumed when the existing wake does not branch;
A second generation unit that generates a second hypothesis set including a hypothesis indicating a branching wake obtained by updating the existing wake using each of two or more observation signals selected from the plurality of observation signals. Have
The evaluation unit is
A weight setting that gives a predetermined first weight to the likelihood of each hypothesis included in the first hypothesis set and gives a second weight smaller than the first weight to the likelihood of each hypothesis included in the second hypothesis set A tracking device characterized by having a section.
(Appendix 3)
In the tracking device according to attachment 2,
The generator is
When the plurality of observation signals correspond to a single target, it is expressed by exchanging the corresponding two or more selected observation signals from the hypotheses included in the second hypothesis set. A tracking device, comprising: a deletion unit that detects a set of possible hypotheses and deletes each of the detected hypotheses while leaving one of the hypotheses included in the set.
(Appendix 4)
From the image obtained from the sensor at each sampling time, extract the observation signal that has the possibility of being the target image to be tracked,
When a plurality of observation signals are extracted, an existing track that is a track obtained in the past tracking process for the target is updated using two or more observation signals selected from the plurality of observation signals. Generate multiple hypotheses, including hypotheses indicating the diverging wakes,
A likelihood indicating the likelihood of each of the plurality of generated hypotheses is evaluated based on a predetermined criterion;
By tracking at least one hypothesis including a hypothesis with the highest likelihood among a plurality of hypotheses generated at each sampling time, the past track of the target is tracked,
When the hypothesis indicating the branch track is included in the selected hypothesis, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at a next sampling time is represented by the branch source of the branch track. A tracking method that is set based on the existing track.
(Appendix 5)
In the tracking method described in Appendix 4,
In generating the plurality of hypotheses,
Generating a first hypothesis set including a hypothesis indicating an updated wake that can be assumed when the existing wake does not branch,
Generating a second hypothesis set including a hypothesis indicating a branching wake obtained by updating the existing wake using each of two or more observation signals selected from the plurality of observation signals;
When evaluating the plurality of hypotheses,
A tracking method characterized in that a predetermined first weight is given to the likelihood of each hypothesis included in the first hypothesis set, and another second weight is given to the likelihood of each hypothesis included in the second hypothesis set. .
(Appendix 6)
From the image obtained from the sensor at each sampling time, extract the observation signal that has the possibility of being the target image to be tracked,
When a plurality of observation signals are extracted, an existing track that is a track obtained in the past tracking process for the target is updated using two or more observation signals selected from the plurality of observation signals. Generate multiple hypotheses, including hypotheses indicating the diverging wakes,
A likelihood indicating the likelihood of each of the plurality of generated hypotheses is evaluated based on a predetermined criterion;
By tracking at least one hypothesis including a hypothesis with the highest likelihood among a plurality of hypotheses generated at each sampling time, the past track of the target is tracked,
When the hypothesis indicating the branch track is included in the selected hypothesis, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at a next sampling time is represented by the branch source of the branch track A tracking program that causes a computer to execute processing that is set based on an existing track.
(Appendix 7)
In the tracking program described in Appendix 6,
In generating the plurality of hypotheses,
Generating a first hypothesis set including a hypothesis indicating an updated wake that can be assumed when the existing wake does not branch,
Generating a second hypothesis set including a hypothesis indicating a branching wake obtained by updating the existing wake using each of two or more observation signals selected from the plurality of observation signals;
When evaluating the plurality of hypotheses,
Giving the computer a predetermined first weight to the likelihood of each hypothesis included in the first hypothesis set, and giving another second weight to the likelihood of each hypothesis included in the second hypothesis set A tracking program characterized by

DESCRIPTION OF SYMBOLS 1 ... Sensor; 2 ... Display apparatus; 11 ... Extraction part; 12 ... Generation | occurrence | production part; 13 ... Evaluation part; 14 ... Tracking part; 15 ... Setting part; 121 ... 1st generation part; Deletion unit 131 131 matrix generation unit 132 weight setting unit 133 likelihood calculation unit 21 processor 22 memory 32 hard disk drive (HDD) 24 sensor interface 25 display control unit 26 ... Input device; 27 ... Optical drive device; 28 ... Removable disk

Claims (5)

An extraction unit that extracts an observation signal having a possibility of being a target image to be tracked from an image obtained from a sensor at each sampling time;
When a plurality of observation signals are extracted by the extraction unit, using two or more observation signals selected from the plurality of observation signals, an existing wake that is a wake obtained by past tracking processing for the target is used. A generator that generates a plurality of hypotheses including a hypothesis indicating a branch wake that is a wake branched by updating;
An evaluation unit that evaluates the likelihood indicating the likelihood of each of the plurality of hypotheses generated by the generation unit based on a predetermined criterion;
By selecting at least one hypothesis including a hypothesis with the highest likelihood obtained by the evaluation unit from among a plurality of hypotheses generated by the generation unit at each sampling time, the past track of the target is selected. A tracking section for identifying
When a hypothesis indicating the branch wake is included in the hypothesis selected by the tracking unit, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at the next sampling time is represented as a branch of the branch wake. And a setting unit configured to set based on the original existing track. The tracking device according to claim 1,
The generator is
A first generation unit that generates a first hypothesis set including a hypothesis indicating an updated wake that can be assumed when the existing wake does not branch;
A second generation unit that generates a second hypothesis set including a hypothesis indicating a branching wake obtained by updating the existing wake using each of two or more observation signals selected from the plurality of observation signals. Have
The evaluation unit is
A weight setting that gives a predetermined first weight to the likelihood of each hypothesis included in the first hypothesis set and gives a second weight smaller than the first weight to the likelihood of each hypothesis included in the second hypothesis set A tracking device characterized by having a section. The tracking device according to claim 2,
The generator is
When the plurality of observation signals correspond to a single target, it is expressed by exchanging the corresponding two or more selected observation signals from the hypotheses included in the second hypothesis set. A tracking device, comprising: a deletion unit that detects a set of possible hypotheses and deletes each of the detected hypotheses while leaving one of the hypotheses included in the set. From the image obtained from the sensor at each sampling time, extract the observation signal that has the possibility of being the target image to be tracked,
When a plurality of observation signals are extracted, an existing track that is a track obtained in the past tracking process for the target is updated using two or more observation signals selected from the plurality of observation signals. Generate multiple hypotheses, including hypotheses indicating the diverging wakes,
A likelihood indicating the likelihood of each of the plurality of generated hypotheses is evaluated based on a predetermined criterion;
By tracking at least one hypothesis including a hypothesis with the highest likelihood among a plurality of hypotheses generated at each sampling time, the past track of the target is tracked,
When the hypothesis indicating the branch track is included in the selected hypothesis, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at a next sampling time is represented by the branch source of the branch track. A tracking method that is set based on the existing track. From the image obtained from the sensor at each sampling time, extract the observation signal that has the possibility of being the target image to be tracked,
When a plurality of observation signals are extracted, an existing track that is a track obtained in the past tracking process for the target is updated using two or more observation signals selected from the plurality of observation signals. Generate multiple hypotheses, including hypotheses indicating the diverging wakes,
A likelihood indicating the likelihood of each of the plurality of generated hypotheses is evaluated based on a predetermined criterion;
By tracking at least one hypothesis including a hypothesis with the highest likelihood among a plurality of hypotheses generated at each sampling time, the past track of the target is tracked,
When the hypothesis indicating the branch track is included in the selected hypothesis, a gate region indicating a range of an image from which an observation signal corresponding to the target is extracted at a next sampling time is represented by the branch source of the branch track. A tracking program that causes a computer to execute processing that is set based on an existing track.