JP4769943B2 - Object tracking device, abnormal state detection device, and object tracking method - Google Patents

Description

  The present invention relates to, for example, an object tracking device that tracks an object in an image, an abnormal state detection device, and an object tracking method.

  Conventionally, many techniques for tracking a notable object in a moving image have been proposed. With this technology, tracking that is robust to some extent without losing sight of the partial occlusion of the tracking target (Patent Document 1), or smoothly tracking from the state where the tracking target is lost due to the short-time occlusion of the tracking target What has been resumed has been proposed (Non-Patent Documents 1 and 2).

There has also been proposed a technique for integrating moving image information from a plurality of different angles and tracking an object to be tracked using the integrated moving image information (Non-patent Document 3). Furthermore, a technique for tracking a plurality of objects using inference based on Bayesian theory by extending a particle filter indicating a standard object tracking method has been proposed (Patent Document 2).
JP 2005-250989 A JP 2005-165688 A K. Nummiaro, et al., “An adaptive color-based particle filter,” Image and Vision Computing, Vol. 21, No. 1, pp. 99-110, 2002. AD Jepson, et al., “Robust online appearance models for visual tracking,” IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 25, No. 10, 2003. Suzuki et al., `` Stabilization of human tracking by introducing environmental model '', IEICE Transactions D-II, Vol.J88-D-II, No.8, pp.1592-1600, 2005.

  However, in the techniques described in Patent Document 1 and Non-Patent Documents 1 and 2 described above, the processing when the tracking target is completely shielded by another object is insufficient, and in particular, the tracking target is shielded for a long time. Or, when the shield moves together with the tracking target after shielding, specifically, when the tracking target is included inside a box-like object, bag, vehicle, etc. as the shielding object It was difficult to continue and it was difficult to continue tracking.

Further, in the technique described in Non-Patent Document 3, the integrated moving image information is used for tracking only under limited circumstances where a plurality of cameras for obtaining moving image information from a plurality of different angles can be used. I could not.
Furthermore, in the technique described in Patent Document 2, the object to be tracked for a long time as described above or the object to be tracked after shielding is tracked only by handling an object that does not overlap at all and a large number of objects that overlap. It was difficult to continue tracking when moving together.

  Therefore, the present invention provides an object tracking device, an abnormal state detection device, and an object tracking method that enable tracking even when the tracking target is shielded for a long time or when the shielding object moves together with the tracking target after shielding. It is intended to provide.

In order to solve the above problems and achieve the object of the present invention, an object tracking device of the present invention includes an image capturing means for capturing an image of an object to be tracked, and a likelihood model of the tracking target from the captured image. The tracking processing means for generating the tracking target likelihood while predicting the motion of the tracking target from the hypothesis based on the object, and comparing the generated tracking target likelihood with a predetermined threshold An abnormal state detection means for detecting an abnormal state of the tracking process of the object, a shielding state determination means for determining that the tracking target is shielded by another tracking target when the abnormal state is continued for a certain period of time, And likelihood model updating means for additionally registering a likelihood model when it is determined to be in a shielded state and updating the threshold model by decreasing the number of times of additional registration .

  Thus, when the abnormal state detection means detects an abnormal state of the object tracking process by comparing the generated likelihood of the tracking target and a predetermined threshold, the likelihood model creation means is different from the tracking target. Create likelihood models for other tracking targets. Therefore, when there is an abnormality in tracking, it is possible to move the tracking target from the original target to the shielding object, recognize the shielding object as a temporary tracking target, and continue tracking.

  Further, the abnormal state detection device of the present invention detects an abnormal state by detecting an abnormal state of the object tracking process by comparing the generated likelihood of the tracking target with a predetermined threshold, and the abnormal state is detected. A likelihood model creation means for creating a likelihood model of another tracking target different from the tracking target, and the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time A shielding state determining means for determining, a likelihood model updating means for additionally registering the likelihood model created when it is determined to be in the shielding state and updating the threshold value for each number of times additionally registered; Is provided.

  As a result, when the abnormal state continues for a certain period of time, the shielding state determination means determines that the tracking target is in a state where it is blocked by another tracking target. When it is determined that the state is occluded, the likelihood model updating means additionally registers the created likelihood model and updates the threshold by decreasing the number of additional registrations. Therefore, by monitoring the duration of the abnormal state and determining that the tracking target is covered with an obstruction when there is no return to the normal state, lowering the threshold for abnormal determination and additionally registering the likelihood model, the tracking target When the occlusion state is canceled, the tracking target is returned from the shielding object with the low likelihood to the original object with the high likelihood, and the tracking can be continued.

  Further, the object tracking method of the present invention generates the tracking target likelihood from the image of the tracking target object based on the hypothesis based on the tracking target likelihood model while predicting the tracking target motion, thereby tracking the object. A tracking processing step for performing processing, an abnormal state detecting step for detecting an abnormal state of the object tracking processing by comparing the likelihood of the generated tracking target with a predetermined threshold, and when an abnormal state is detected A likelihood model creating step of creating a likelihood model of another tracked object different from the tracked object.

  Furthermore, in the object tracking method of the present invention, it is determined that the tracking target is in a shielded state, a shielding state determination step that determines that the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time. A likelihood model added step of adding a likelihood model created at the time and updating the threshold value every time the threshold value is added, and after being determined to be in a shielding state, A shielding state cancellation determination step for determining that the shielding state has been canceled when the likelihood model in which a likelihood greater than the likelihood model is generated exists, and a likelihood model when it is determined that the shielding state has been canceled. And a likelihood model deletion step of updating the threshold value for each number of deleted likelihood models and updating it.

  Therefore, when there is an abnormality in tracking, it is possible to move the tracking target from the original target to the shielding object, recognize the shielding object as a temporary tracking target, and continue tracking. Furthermore, by monitoring the duration of the abnormal state and determining that the tracking target is covered by an obstruction when there is no return to the normal state, lowering the threshold for abnormal determination and additionally registering the likelihood model, the tracking target When the occlusion state is canceled, the tracking target is returned from the shielding object with the low likelihood to the original object with the high likelihood, and the tracking can be continued.

  According to the present invention, tracking is possible even when the tracking target is shielded for a long time, or when the shielding object moves together with the tracking target after shielding. Accordingly, it is possible to more reliably realize tracking of an object in a scene where there are many objects that are likely to be shielding objects such as general streets, buildings, and office spaces. Furthermore, the present invention can provide a great effect when applied to a system that is supposed to be used for tracking objects in a living space, such as a human-symbiotic robot and a surveillance camera.

  In addition, since it is impossible to know in advance which object will be the shielding object, it is practically impossible to prepare a likelihood model for the shielding object in advance. However, according to the present invention, it is possible to determine that a shielding object appears between the tracking target and the camera from the change in the acquired image information, and thus a likelihood model can be dynamically constructed. It is possible to perform reliable object tracking without using prior information. In particular, when the environment in the image changes dynamically, or when the environment itself is static, the camera itself can move to a relative moving environment including one that moves. . It is also possible to construct a dynamic likelihood model by fusing image information obtained from a camera and distance information.

  In addition, it is possible to flexibly switch attention from an obstructer with a low likelihood to an original tracking target with a high likelihood by performing a decrease update of likelihood and a threshold value for abnormality determination when generating a plurality of likelihood models. Processing becomes possible. Such a flexible process is, for example, a tracking process in which the original tracking target appears when the shielding state is canceled while paying attention to the original tracking target at the same time as tracking the shielding object. Especially effective. Thereby, for example, it is possible to realize natural and flexible interaction closer to humans in application to an intelligent robot.

Embodiments of the present invention will be described below with reference to the drawings as appropriate.
FIG. 1 is a diagram illustrating a system configuration of an object tracking apparatus.
In FIG. 1, an object tracking device 1 includes a camera device 2 that captures an image of an object to be tracked, an initial setting unit 3 that performs various initial settings, and a likelihood of a tracking target from images captured by the camera device 2. And an object tracking processing unit 4 for generating the likelihood of the tracking target while predicting the motion of the tracking target from the hypothesis based on the above and performing the object tracking process.

  Here, the camera device 2 can acquire distance information together with an image of an object to be tracked. The initial setting unit 3 performs various processes necessary for starting tracking, such as setting a likelihood model of a tracking target and initializing a hypothesis group used in a particle filter described later of the object tracking processing unit 4. These processes may be set from the external input device 14 as necessary.

  A general-purpose particle filter is used for the object tracking processing unit 4. Although the particle filter having the most basic configuration is assumed here, the configuration can be changed as appropriate. The object tracking processing unit 4 includes three components: a hypothesis generation unit 5, an operation prediction unit 6, and an observation / evaluation unit 7.

  The hypothesis generation unit 5 generates a new hypothesis group by sampling each hypothesis (particle) with a probability proportional to its likelihood. The motion prediction unit 6 predicts the position on the image to be tracked by applying a predefined motion model, and updates each hypothesis. Any behavior model can be used. That is, if there is prior knowledge about the motion to be tracked, it can be included in the motion model, but if not, a general model can be used. In addition to this, it is also possible to introduce an arbitrary mechanism for learning the dynamics of motion into the motion model.

The observation / evaluation unit 7 observes the image information and evaluates the likelihood of the updated hypothesis. Likelihood L (s) for hypothesis s is calculated as in the following equation (1).
[Equation 1]
^N
L (s) = Σ [L _i (s) / α ⁱ ]
_{i = 0}

Here, L _i (s) is the likelihood function of each likelihood model registered in the “system tracking target likelihood model” by the initial setting unit 3 and the likelihood model update unit 11, and α is 1 set in advance. These are real numbers. When such likelihood evaluation is performed, the likelihood function of the model registered in the “system tracking target likelihood model” has a greater influence, so the current target with a low likelihood is tracked. On the other hand, when an object registered with a higher likelihood appears, it is possible to preferentially track the higher likelihood. The initial setting unit 3 registers only the likelihood model to be tracked as L ₀ (s).
Also, a hypothesis that is representative of the entire system (hereinafter referred to as a representative hypothesis) and its likelihood are calculated. The representative hypothesis can be selected according to the application, such as the hypothesis corresponding to the average of the hypothesis group, the hypothesis corresponding to the mode of the probability density function that the hypothesis group approximates, or the hypothesis having the maximum likelihood. .

The reason why the α function is used in the likelihood function of Equation 1 will be described below.
This is to maintain the relationship that the likelihood on the side to be shielded (i-th model) is always 1 / α times the likelihood on the side to be shielded (i-th model). . For example, when α = 10, the likelihood on the side to be shielded is one order of magnitude higher than the likelihood on the side to be shielded, and is one when the object on the side to be shielded appears again from behind the shield. It is possible to return the attention of the tracking target by the appearance of an object having an order of magnitude higher likelihood.

  Further, the object tracking device 1 detects an abnormal state of the object tracking process by comparing the likelihood of the tracking target generated by the object tracking processing unit 4 with a threshold set in advance by the threshold setting unit 12. Unit 8 and a likelihood model creation unit 9 that creates a likelihood model of another tracking target that is different from the tracking target when an abnormal state is detected by the abnormal state detection unit 8.

  Further, the object tracking device 1 includes a shielding state determination unit 10 that determines that the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time by the detection of the abnormal state detection unit 8; The likelihood model created by the likelihood model creation unit 9 when it is determined by the shielding state determination unit 10 to be in the shielding state is additionally registered in the “system tracking target likelihood model” and the threshold update unit 13 is used. And a likelihood model updating unit 11 that updates the threshold set by the threshold setting unit 12 by reducing the threshold every time the likelihood model is additionally registered.

Further, when the shielding state determination unit 10 determines that the state is the shielding state, there is a likelihood model having a likelihood function value larger than the likelihood function value of the last registered likelihood model. The likelihood model update unit 11 deletes all registrations of likelihood models added after the likelihood model having the maximum likelihood, and uses the threshold update unit 13. The threshold set by the threshold setting unit 12 is updated for each number of likelihood models that have been deleted.
An output such as a tracking state of the object tracking device 1 is output to an external output device 15.

The threshold value set in advance in the threshold value setting unit 12 is used to determine whether or not the abnormal state detection unit 8 is in an abnormal state. Here, the likelihood model update unit 11 registers one shield model. Each time the threshold value updating unit 13 is used, the threshold value set by the threshold setting unit 12 is multiplied by 1 / α. This is because the likelihood is 1 / α level before adding the model in the state of “tracking the shield normally”. Conversely, when the shielding state determination unit 10 determines that the shielding state has been eliminated, the threshold set by the threshold setting unit 12 using the threshold updating unit 13 by the number of models to be deleted by the likelihood model updating unit 11. Is multiplied by α.
In the object tracking device 1 described above, the initial setting unit 3 to the threshold update unit 13 other than the camera device 2 are described above using, for example, a CPU as a control unit and a memory as a storage unit for programs and control data. It is comprised so that it may have each function.

Hereinafter, the operation of the object tracking apparatus configured as described above will be described based on the flowchart shown in FIG.
The flowchart of FIG. 2 shows the processing of each unit shown in FIG. 1 continuously as a series of flows.
In FIG. 2, first, an initial setting process is performed (step S1). Specifically, the initial setting unit 3 shown in FIG. 1 sets system default values. For example, the initial setting unit 3 registers the likelihood model of the tracking target generated by the hypothesis generation unit 5 of the object tracking processing unit 4 in the “system tracking target likelihood model” as the “0th model”, and the likelihood The number of models N = 0. The initial setting unit 3 sets the “tracking state” of the system to “normal”.

  The “likelihood model” makes it possible to evaluate, as a numerical value (likelihood), how accurately the “hypothesis” represents the position of the tracking target by the likelihood function. The likelihood function is normalized so that the maximum value is 1 and the minimum value is 0. Any likelihood model can be used, but it is desirable to satisfy the condition that it can be used universally without requiring prior knowledge of the object. Here, it is assumed that “color distribution in an image region represented by a hypothesis” is used as a likelihood model. In this case, the degree of similarity between the color distribution of the model and the color distribution observed in the image area represented by the hypothesis is determined, and the closer to the model, the higher likelihood is given.

  Next, each process of the object tracking process is performed (step S2). Specifically, the hypothesis generation unit 5 of the object tracking processing unit 4 shown in FIG. 1 generates a likelihood model of a hypothesis to be tracked (step S3), and the motion prediction unit 6 applies a predefined motion model. Thus, the position on the image to be tracked is predicted / updated (step S4), and the observation / evaluation unit 7 observes the image information and evaluates the likelihood of the updated hypothesis. Based on this, a representative hypothesis and its likelihood are calculated (step S5).

  Next, it is determined whether the state immediately before the object to be tracked is normal or abnormal (step S6). Specifically, the object tracking processing unit 4 shown in FIG. 1 determines whether the “tracking state” of the system is “normal” or “abnormal” when the likelihood of the hypothesis is evaluated by the observation / evaluation unit 7. Determine whether it is set to. The subsequent processing flow branches according to the determination result of the determination step S6.

When it is determined in the determination step S6 that the immediately preceding state is normal, it is subsequently determined whether or not an abnormal state is detected (step S7). More specifically, the abnormal state detection unit 8 shown in FIG. 1 compares the likelihood of the representative hypothesis to be tracked generated by the object tracking processing unit 4 with the threshold value set in advance by the threshold setting unit 12. Whether the tracking process is in an abnormal state is detected.
For example, if the likelihood of the representative hypothesis generated in step S2 exceeds a predetermined threshold, the tracking process is considered to be executed normally and the tracking process is continued.

  When it is determined in the determination step S7 that the abnormal state detection determination result is normal, it is subsequently determined whether or not the shielding state has been eliminated (step S8). Then, if it is determined in the determination step S8 that it has not been solved, that is, if it is not in the shielding state, it is next determined whether or not it is an end condition (step S9). Specifically, the following can be considered as examples of the termination condition. First, when the preset time (number of loops) is reached, second, the tracking target moves out of the screen and becomes untraceable, and when a certain time has passed, third, the robot or monitoring This is the case when the user gives an instruction to end tracking in application to a camera or the like.

  When it is determined in the determination step S9 that the end condition is not satisfied, the process returns to step S3, and the processes and determinations from step S3 to step S9 are repeated.

  When it is determined in the determination step S6 that the previous state is normal, but in the determination step S7, it is determined that the abnormal state detection determination is abnormal, the likelihood model creating unit 9 shown in FIG. A creation process is performed (step S17). Specifically, the abnormal state detection unit 8 detects that the object tracking process is abnormal by comparing the likelihood of the representative hypothesis to be tracked generated by the object tracking processing unit 4 with the threshold value set in advance by the threshold setting unit 12. It is detected whether it is in a state. For example, when the likelihood generated by the object tracking process in step S2 is less than a predetermined threshold, it is determined that the tracking target has been lost for some reason. In this case, there are two possibilities: the possibility that the object to be tracked is temporarily lost due to a sudden movement or the like, and the possibility that the object to be tracked is blocked by another object.

In the likelihood model creation process in step S17, as described above, the state immediately before the tracking target is normal in step S6, and the abnormal state detection unit 8 detects that the current state is abnormal in step S7. In actuality, the likelihood model creating unit 9 shown in FIG. 1 is actually a shielding target other than the tracking target in preparation for the possibility that the current tracking target object is shielded. Create an object likelihood model.
Then, the state determined to be normal in the determination step S6 is set to “abnormal” (step S18). Specifically, the abnormal state detection unit 8 shown in FIG. 1 sets the “tracking state” of this system from “normal” to “abnormal”.

  The likelihood model creation process in step S17 described above is performed when the abnormal state detection unit 8 detects an abnormality in the tracking process. However, the likelihood model created in step S17 is not registered in the “system tracking target likelihood model” until the abnormality of the tracking process shown in step S14 described later continues for a certain time.

The specific likelihood model creation process is as follows.
First, the likelihood model creation unit 9 illustrated in FIG. 1 extracts an image region to be tracked using distance information acquired from the camera device 2. More specifically, the likelihood model creation unit 9 is closer to the average distance at the previous time of the tracking target in the image region represented by the representative hypothesis generated by the hypothesis generation unit 5 of the object tracking processing unit 4. That is, an image region on the side close to the camera device 2 is extracted. This extracted image area represents an area where there is an object that is likely to be a shield for the tracking target object.
Next, the likelihood model creation unit 9 creates a likelihood model of the shielding object based on the image information in the extracted image region.

  Also, when it is determined in the determination step S6 that the state immediately before the tracking target is abnormal, a determination is made as to whether or not an abnormal state is detected as in the determination step S7 (step S10). . More specifically, the abnormal state detection unit 8 shown in FIG. 1 compares the likelihood of the representative hypothesis to be tracked generated by the object tracking processing unit 4 with the threshold value set in advance by the threshold setting unit 12. Whether the tracking process is in an abnormal state is detected.

  When it is determined in this determination step S10 that the abnormal state detection determination is normal, the state determined as abnormal in the determination step S6 is set to “normal” (step S11). Specifically, the abnormal state detection unit 8 shown in FIG. 1 sets the “tracking state” of the system from “abnormal” to “normal”. Then, likelihood model discarding processing is performed (step S12). Specifically, the likelihood model update unit 11 shown in FIG. 1 discards the likelihood model created in step S17.

  When it is determined in the determination step S10 that the abnormal state detection determination is abnormal, the abnormal state is then monitored to determine whether or not the abnormal state continues for a certain time (step S13). Specifically, if the previous state is abnormal and the abnormal state detection unit 8 shown in FIG. 1 detects that the current state is also abnormal, the state in which the object to be tracked is still lost is continued. In preparation for the possibility of the failure, it is determined whether or not the time during which the abnormal state continues has exceeded a predetermined time.

When it is determined in the determination step S13 that the abnormal state has continued for a certain period of time, the likelihood model update unit 11 shown in FIG. 1 creates a likelihood model of an obstruction that is another tracking target different from the tracking target. Then, it is additionally registered in the “system tracking target likelihood model” (step S14).
Specifically, the shielding state determination unit 10 illustrated in FIG. 1 has a state in which the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time by the detection of the abnormal state detection unit 8. The likelihood model updating unit 11 additionally registers the likelihood model created by the likelihood model creating unit 9. In this way, when the abnormality of the tracking process is continued for a certain period of time, the likelihood model addition process in step S14 is executed. At this time, the likelihood model update unit 11 increments the value of the number N of registered models by one, and additionally registers it in the “system tracking target likelihood model”.

Then, the threshold value is multiplied by 1 / α (step S15). Specifically, the likelihood model update unit 11 illustrated in FIG. 1 additionally registers the likelihood model created by the likelihood model creation unit 9 when the shielding state determination unit 10 determines that the state is in the shielding state. At the same time, the threshold updating unit 13 is used to update the threshold set by the threshold setting unit 12 by decreasing the number of times the likelihood model is additionally registered.
Further, the state determined to be abnormal in the determination step S10 is set to “normal” (step S16). Specifically, the abnormal state detection unit 8 shown in FIG. 1 sets the “tracking state” of the system from “abnormal” to “normal”.

  When it is determined in the determination step S13 that the abnormal state has not continued for a certain period of time, the process proceeds to the object tracking process in step S2 through the processing in the determination steps S8 and S9 while maintaining the abnormal state in the determination step S13. Then, the object tracking process in step S2 and the monitoring process in determination step S13 are continued. This process is continued because the tracked object is not occluded, but if it is temporarily lost, the tracking state of this system is expected to return to a normal state within a certain period of time. Because.

  If the tracking state is set to “abnormal” in step S18, the likelihood model is discarded in step S12, the tracking state is set to “normal” in step S16, and the abnormal state is determined in determination step S13. Is determined not to continue for a certain period of time, the process proceeds to determination step S8 to determine whether or not the shielding state has been eliminated.

If it is determined in this determination step S8 that the shielding state has not been eliminated, the process proceeds to the determination of the end condition in step S9 already described. If it is determined that the end condition is not satisfied, the process returns to the temporary generation process in step S3. If it is determined that the end condition is satisfied, the process ends.

If it is determined in the determination step S8 that the shielding state has been eliminated, the likelihood model is deleted (step S19). Then, a process of multiplying the threshold value set by the threshold setting unit 12 by α by the number of likelihood models from which the threshold value set by the threshold setting unit 12 has been deleted is executed (step S20). Then, the process proceeds to determination of whether or not the termination condition in step S9 is satisfied.

  The above process will be described in more detail. Consider a case in which an abnormality determination is made in determination step S7, but the object to be tracked is only temporarily lost. At this time, a likelihood model is created in step S17, and after the “tracking state” of the system in step S18 is set to “abnormal”, the process returns to the object tracking process in determination step S2 through determination steps S8 and S9. In such a case, since it is determined that the normal state is returned in the next determination step S10, the state is set to “normal” in step S11, and the likelihood model in step S12 is discarded. The tracking process is continued from the next time until the end condition (step S9) is satisfied.

  Further, as described above, when it is determined in the determination step S13 that the abnormal state continues for a certain period of time and the object to be tracked is in the shielding state, and the likelihood model of the shielding object is added in step S14, the step S15 is performed. The threshold value is multiplied by 1 / α. Then, after the “tracking state” of the system in step S16 is set to “normal”, it is determined whether or not the shielding state has been resolved (step S8). Specifically, after the shielding state determination unit 10 illustrated in FIG. 1 determines that the state is the shielding state, the likelihood model in which a likelihood greater than the likelihood model added last is generated. When it exists, it determines with the shielding state having been eliminated.

Explaining this in detail, the shielding state determination unit 10 indicates that the value of the number N of registered models by the likelihood model creation unit 9 is 1 or more and the likelihood of Formula 1 for the representative hypothesis by the object tracking processing unit 4 In the evaluation, when all of the two conditions that the maximum value among the likelihood functions L _i (s) is the likelihood function of a model other than the Nth model are satisfied, the tracking target is the maximum Moving to a model having a likelihood function that takes a value, it is determined that the shielding state has been eliminated.

  When it is determined in the determination step S8 that the shielding state has been eliminated, likelihood model deletion processing is performed (step S19). Specifically, the likelihood model update unit 11 illustrated in FIG. 1 deletes the likelihood model added in step S14. Here, the deletion process of the likelihood model is performed with all the models registered in step S14 after the model having the maximum likelihood in the determination step S8 as a deletion target. To describe this process in more detail, the likelihood model update unit 11 first deletes all registrations of likelihood models to be deleted from the “system tracking target likelihood model” and sets the value of the number N of registered models. This means that the process of reducing the number of deleted models is executed.

  Then, the threshold value is multiplied by the number of likelihood models from which the threshold value has been deleted (step S20). Specifically, the likelihood model update unit 11 illustrated in FIG. 1 uses the threshold update unit 13 to update the threshold set by the threshold setting unit 12 every time the shielding state is eliminated. Here, every time one model of an obstruction is deleted by the likelihood model update unit 11, the threshold value set by the threshold setting unit 12 using the threshold update unit 13 is multiplied by α.

  When the shielding state is not resolved at the determination step S8 and when the shielding state is resolved at the determination step S8, the likelihood model deletion process is performed at the step S9, and the number of likelihood models for which the threshold value is deleted at the step S10 is α After doubling, the end condition is determined in determination step S9, and the process ends when the above-described end condition is satisfied.

According to the present embodiment described above, the determination of shielding with respect to the tracking target can be dealt with no matter how many times it occurs. For example, when a new occlusion occurs while tracking an object that occludes the original tracking target object, the likelihood model of the new occlusion object is registered in the “system tracking target likelihood model”. Accordingly, it is possible to continue tracking, and it is possible to determine that a plurality of occlusions are released and the original tracking target appears.
It goes without saying that the present invention is not limited to the above-described embodiment, and can be appropriately changed without departing from the gist of the present invention.

It is a figure which shows the system configuration | structure of an object tracking apparatus. It is a flowchart which shows operation | movement of an object tracking apparatus.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Object tracking apparatus, 2 ... Camera apparatus, 3 ... Initial setting part, 4 ... Object tracking process part, 5 ... Hypothesis generation part, 6 ... Motion prediction part, 7 ... Observation / evaluation part, 8 ... Abnormal state detection part, DESCRIPTION OF SYMBOLS 9 ... Likelihood model preparation part, 10 ... Shielding state determination part, 11 ... Likelihood model update part, 12 ... Threshold setting part, 13 ... Threshold update part, 14 ... Input device, 15 ... Output device

Claims (5)

Image capturing means for capturing an image of an object to be tracked;
From the captured image, tracking processing means for generating the tracking target likelihood while predicting the operation of the tracking target from a hypothesis based on the likelihood model of the tracking target, and performing the tracking process of the object,
An abnormal state detection means for detecting an abnormal state of the tracking processing of the object by comparing the likelihood of the tracking target generated and a predetermined threshold;
Shielding state determination means for determining that the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time;
Likelihood model updating means for additionally registering the likelihood model when it is determined to be in the shielding state and updating the threshold by decreasing the number of additional registrations;
An object tracking device comprising: The shielding state determination unit determines that the shielding state is present when the likelihood model in which a likelihood greater than the likelihood model added and registered last exists after it is determined to be the shielding state. Is determined to have been resolved,
The likelihood model updating means deletes all the registrations of the additionally registered likelihood models after the likelihood model in which the maximum likelihood is generated, and sets the threshold value of the deleted likelihood model. The object tracking device according to claim 1 , wherein the object tracking device is updated by increasing each number. Abnormality when performing tracking processing of the object by generating the likelihood of the tracking target while predicting the motion of the tracking target from a hypothesis based on the likelihood model of the tracking target from the image of the object to be tracked An abnormal state detection device for detecting a state,
An abnormal state detection means for detecting an abnormal state of the tracking processing of the object by comparing the likelihood of the tracking target generated and a predetermined threshold;
Likelihood model creation means for creating a likelihood model of another tracking target different from the tracking target when the abnormal state is detected;
Shielding state determination means for determining that the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time;
A likelihood model updating means for additionally registering the created likelihood model when it is determined to be in the shielding state and updating the threshold by decreasing the number of additional registrations;
An abnormal state detection device comprising: The shielding state determination unit determines that the shielding state is present when the likelihood model in which a likelihood greater than the likelihood model added and registered last exists after it is determined to be the shielding state. Is determined to have been resolved,
The likelihood model updating means deletes all the registrations of the additionally registered likelihood models after the likelihood model in which the maximum likelihood is generated, and sets the threshold value of the deleted likelihood model. The abnormal state detection apparatus according to claim 3 , wherein the number is updated for each number. A tracking processing step for generating the tracking target likelihood from the image of the tracking target object based on a hypothesis based on the tracking target likelihood model while generating the tracking target likelihood and performing the tracking process of the object When,
An abnormal state detection step of detecting an abnormal state of the tracking process of the object by comparing the generated likelihood of the tracking target and a predetermined threshold;
A likelihood model creation step of creating a likelihood model of another tracking target different from the tracking target when the abnormal state is detected;
A shielding state determination step for determining that the tracking target is shielded by another tracking target when the abnormal state continues for a certain period of time;
A likelihood model adding step of adding the likelihood model created when it is determined to be in the shielding state and updating the threshold value by decreasing the number of additions;
After determining that the shielding state is present, the shielding state for determining that the shielding state has been eliminated when the likelihood model in which a likelihood greater than the added likelihood model is finally present exists. Cancellation determination step;
A likelihood model deletion step of deleting the likelihood model when it is determined that the shielding state has been canceled and updating the threshold value for each number of the deleted likelihood models; and
An object tracking method comprising:

Images