JP2017010224A - Object tracking apparatus, object tracking method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To properly track an object which intersects with another object.SOLUTION: A foreground area is extracted from a detection area of an object, to detect an intersection on the basis of a foreground mask corresponding to a recognition model used in detecting the object. Only when an intersection is detected, a template indicating the feature quantity of the object is updated. When no object is detected, the object is tracked by performing a collation by using a template formed on the basis of an image area in a past frame. In the template, information on another object is unlikely to be included in the feature quantity before intersection, thereby increasing collation accuracy.SELECTED DRAWING: Figure 1

Description

  In particular, the present invention relates to an object tracking device, an object tracking method, and a program suitable for use in tracking an object in a situation where crossing of objects occurs.

  In recent years, intelligentization has become widespread in video equipment. For example, a human body detection function is installed in a surveillance camera, and functions such as counting the number of people, analyzing customer intentions, detecting abnormal operations, and detecting intrusions in dangerous areas have been proposed. In addition, a function that automatically controls the focus and exposure of a camera by specifying and tracking the position of a person in a video imaged by an imaging device such as a digital camera has been attracting attention. In addition to people, a function for automatically controlling a camera by paying attention to general objects such as dogs, cats, and flowers has also become widespread.

  The foundation of intelligent video equipment is object detection technology based on machine learning. In machine learning, a feature quantity that distinguishes an object from a non-object is extracted from a large number of objects and non-object learning samples, and a recognition model is created. When detecting an object from an image, first, a pyramid layer is created by scaling the size of the original image. Then, a raster scan is performed for each pyramid layer, and objects of different sizes are detected by combining the responses of the discriminators for each feature amount described in the recognition model.

  A typical method for creating this recognition model is a method for creating a cascade classifier by adaboost learning. In this method, an object to be recognized is classified into a plurality of categories, and a recognition model is created for each category. For example, the human body is divided into three categories of a front human body, a side human body, and a half human body, and similar human bodies are collected to learn a recognition model, thereby improving the recognition accuracy. Further, the cascade type discriminator described above is configured by connecting a plurality of weak discriminators in series. Each weak discriminator outputs the likelihood of the object likeness of the current scan window based on the learned recognition model and compares it with a threshold value to determine whether the object is an object. The scan window that has passed through all the weak classifiers is determined to be an object.

  On the other hand, when moving images captured by a camera are converted into still images as time-series images and object detection technology is applied to successive frames of the time-series images, it is not sufficient to detect an object using only the cascade classifier. is there. For example, there are cases where the posture of the object changes or the illumination conditions change in the frames before and after the time-series image, and in this case, the appearance of the object changes, so it often happens that it cannot be detected by the discriminator. To do. Therefore, using the tracking technique, in the frame in which the object can be detected in the search area, the trajectory is updated in association with the trajectory in which the detection result of the discriminator is stored. On the other hand, the object is tracked in the frame in which the object cannot be detected in the search area, and the track is updated in association with the track storing the tracking result. Then, a search area for the next frame is set based on the updated trajectory.

  However, in general, when tracking an object, if the object crosses or is shielded, the tracking accuracy decreases. Therefore, as a conventional technique for tracking an object, the method described in Non-Patent Document 1 uses a weight based on the accuracy of prediction and tracking from the position of the object predicted by the motion model and the position of the object observed by tracking. The average position is obtained according to the above, and the trajectory is smoothed and stabilized.

  In the method described in Non-Patent Document 2, an object is divided into a plurality of particles, and each is tracked and integrated. In this method, particles with low tracking confidence are lost, and a larger number of particles are generated based on particles with high tracking confidence. Next, for each particle, the position of each particle is predicted in consideration of inertia and suddenness, and the tracking certainty is obtained. Then, the tracking is continued with the average position of each particle as the position of the object with the obtained certainty of tracking as a weight.

  Further, in the method described in Patent Document 1, an object detection frame intersection is predicted, a detection result when no object intersects is stored as a template, and this template is used for template matching when the object intersects. I am trying to improve tracking accuracy.

Special table 2009-510541 gazette

"An Introduction to the Kalman Filter," Greg Welch, Gray Bishop, SIGGRAPH 2001. "A tutorial on particle filters for online nonlinear / non-gaussian bayesian tracking (2002)," M. Sanjeev Arulampalam, Simon Maskell, Neil Gordon, IEEE TRANSACTIONS ON SIGNAL PROCESSING. VOL. 50, NO. 2. "Background Subtraction: Experiments and Improvements for ViBe," M. VAN DROOGENBROECK, and O. PAQUOT, Change Detection Workshop (CDW), Providence, Rhode Island, 6 pages, June 2012. David G. Lowe, "Distinctive image features from scale-invariant keypoints," International Journal of Computer Vision, 60, 2 (2004), pp. 91-110.

  However, the tracking method described in Non-Patent Document 1 has a problem in that tracking often fails when an object to be tracked intersects or is shielded for a long time. In addition, the tracking method described in Non-Patent Document 2 has a problem that tracking cannot be continued if the object to be tracked is completely occluded.

  In the tracking method described in Patent Document 1, an intersection between detection frames of an object is detected. For this reason, if the object to be tracked crosses or is blocked by another part outside the detection frame of the same type of object, or if it crosses or is blocked by another type of moving object, the part of the same type of object in the template Or, a lot of moving body information is mixed, and tracking tends to fail at the time of crossing or blocking. Furthermore, in the tracking method described in Patent Document 1, an object is detected based on an interframe difference and an edge, and the position and range of the object are detected without using a recognition model. There is a possibility of erroneous tracking in the image area.

  In view of the above-described problems, an object of the present invention is to enable accurate tracking when an object to be tracked intersects with another object or the like.

  The object tracking device according to the present invention includes: a setting unit that sets an object search region in a frame of interest of a moving image based on a tracking result in a past frame; and the object recognition model in the search region set by the setting unit. Detecting the object using the detection means, extracting a foreground area from the detection area of the object detected by the detection means, and using the extracted foreground area and a mask corresponding to the recognition model, A detection unit for detecting an intersection, and a template indicating a feature amount of the object is updated based on a detection area of the object detected by the detection unit when the detection unit does not detect an intersection of the object. A template updating means, and when the object is not detected by the detecting means, the template updating means Therefore, collation means for extracting the image area of the object from the search area set by the setting means by collating with the template updated in the past frame, and the detection area of the object detected by the detection means, or Tracking result update means for updating the image area extracted by the matching means as a tracking result.

  According to the present invention, tracking can be performed correctly when an object to be tracked intersects with another object or the like.

It is a block diagram which shows the function structural example of the object tracking apparatus which concerns on 1st Embodiment. It is a block diagram which shows the hardware structural example of the object tracking apparatus in embodiment. It is a flowchart which shows an example of the process sequence which tracks the object in 1st Embodiment. It is a figure for demonstrating each search area | region of a pyramid layer. It is a figure for demonstrating the structure of a cascade type discriminator. It is a figure for demonstrating calculation of the ratio of the foreground area | region outside a foreground mask. It is a flowchart which shows an example of the process sequence which tracks the object in 2nd Embodiment. It is a flowchart which shows an example of the process sequence which tracks the object in 3rd Embodiment. It is a block diagram which shows the function structural example of the object tracking apparatus which concerns on 4th Embodiment. It is a flowchart which shows an example of the process sequence which tracks the object in 4th Embodiment.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In this embodiment, an example of tracking an object from each frame of a moving image will be described. In addition, this invention is not limited to the following embodiment, It shows only the specific example advantageous for implementation of this invention. What implement | achieves the function used as the characteristic of this invention is also contained in the scope of the present invention.

(First embodiment)
FIG. 1 is a block diagram illustrating a functional configuration example of the object tracking device 100 according to the present embodiment.
In FIG. 1, the object tracking device 100 includes an image acquisition unit 101, a search area setting unit 102, an object identification unit 103, a trajectory combination unit 104, and an intersection detection unit 105. Furthermore, the template update part 106, the template collation part 107, the template memory | storage part 109, and the process control part 199 are provided. The intersection detection unit 105 further includes a foreground region extraction unit 1051, a foreground mask storage unit 1052, and a ratio calculation unit 1053. The processing performed by these configurations will be described later.

FIG. 2 is a block diagram illustrating a hardware configuration example of the object tracking device 100 according to the present embodiment.
In FIG. 2, the CPU 201 executes various controls in the object tracking device 100 of the present embodiment. The ROM 202 stores a boot program executed when the object tracking device 100 is started up and various data. The RAM 203 stores a control program to be processed by the CPU 201 and provides a work area when the CPU 201 executes various controls. A keyboard 204 and a mouse 205 provide various input operation environments for the user.

  The external storage device 206 is, for example, a hard disk, flexible disk, optical disk, magnetic disk, magneto-optical disk, magnetic tape, or the like. However, the external storage device 206 is not necessarily a necessary component when all the control programs and various data are stored in the ROM 202. The display device 207 is configured by a display or the like, and displays a result of processing in the present embodiment. The network interface 208 is an interface for communicating with an external device. The video interface 209 is an interface for capturing a frame image via an imaging unit (not shown) and a coaxial cable. The bus 211 is a bus for connecting the above components.

FIG. 3 is a flowchart illustrating an example of a processing procedure for tracking an object in the present embodiment. Hereinafter, the flow of processing in this embodiment will be described in detail with reference to the flowchart of FIG.
First, in step S301, the process control unit 199 performs control so that the processes from steps S302 to S315 are repeatedly performed for all frame images acquired from the image acquisition unit 101. Next, in step S302, the process control unit 199 performs control so that the processes from steps S303 to S313 are repeatedly performed for all the trajectories obtained by the trajectory combining unit 104 described later.

  In the present embodiment, in order to simplify the description, only the processing for detecting and tracking each object will be described. Here, in order to perform the initial setting of the tracking process, a known global detector is prepared for the entire frame, for example, in addition to the tracking process according to the present embodiment. In this case, the entire first frame image acquired from the image acquisition unit 101 is detected, and the global detection result is set as the start of the trajectory. On the other hand, tracking accuracy can be improved by performing global detection on a part or all of the entire frame image and effectively integrating the global detection result with the trajectory for each object. Further, the global detector can use the same recognition model as that of the object identification unit 103, and since this process is not an essential part of the present invention, a detailed description thereof will be omitted.

  In step S303, the search area setting unit 102 predicts the tracking target object position in the target frame to be processed based on the last position and size of each trajectory obtained by the trajectory combining unit 104 in the past frame. The center position of the search area is set. Then, as shown in FIG. 4A, a size obtained by enlarging the size of the object at a predetermined ratio is set as the size of the search area.

  Next, in step S304, the object identification unit 103 associates the object search area set in step S303 with a predetermined number of pyramid layers (pyramid images), as shown in FIG. 4B. Then, the corresponding search area corresponding to each pyramid layer is scanned, and it is determined whether or not the object is present at the current scan position using each recognition model of each category. In the present embodiment, a plurality of learned recognition models are held in the external storage device 206, and objects to be recognized are classified into a plurality of categories, and a recognition model is held for each category.

  FIG. 5 is a diagram illustrating a configuration example of a cascade classifier used in the object identification unit 103. When the cascade classifier is applied to each scan position of each pyramid layer, the size of the image area recognized in each pyramid layer is the size of the cascade classifier and is constant. Since each pyramid layer has a different reduction ratio with respect to the original image, the object can be identified even when the object moves in the distance and the size of the object changes. Hereinafter, an image region normalized to the size of the cascade discriminator (that is, the recognition model) at each scan position of each pyramid layer will be referred to as a normalized local region.

  As shown in FIG. 5, the cascade type discriminator is configured by connecting a plurality of discriminators in series. Each discriminator accumulates the likelihood indicating the object likeness of each normalized local region according to the position, size, type, likelihood correspondence law, etc. of the feature amount described in each recognition model, and the identification described in the recognition model Compare with threshold. When the cumulative likelihood is smaller than the threshold value in a certain classifier, the normalized local region identified at that time is rejected as a non-object. Then, as shown in FIG. 5, the normalized local region is identified as an object when all the classifiers have been passed.

  In step S304, since the position and size of the object can be accurately identified using the recognition model, the tracking object can be calibrated to the correct position in the template matching process in step S313 described later. In addition, since identification is performed by limiting to the search area, identification can be performed at high speed.

  Next, in step S305, the object identification unit 103 integrates the identification results of the normalized local regions in each pyramid layer, which is the processing result of step S304. Usually, a single object is often identified by a plurality of neighboring layers, or a plurality of objects are often identified at neighboring positions. Therefore, in this step, all the normalized local areas identified in step S304 are separated for each object, the average value in the original image is obtained for the positions and sizes of a plurality of normalized local areas of the same object, and the position of the object And detect size.

  Next, in step S306, the processing control unit 199 determines whether or not an object is detected in the search area for the locus to be processed. For a trajectory in which an object is detected in the search area, the processing from steps S307 to S311 is performed, and for a trajectory in which no object is detected in the search area, control is performed to perform the processes from step S312 to S313.

  In step S307, the foreground region extraction unit 1051 of the intersection detection unit 105 calculates a background difference for the object region (hereinafter, detection region) detected in step S305, and extracts the foreground region. In this embodiment, since the video image | photographed with the surveillance camera is assumed, the background in the frame image acquired from the image acquisition part 101 is fixed. Therefore, the background difference can be simply calculated, and the foreground region is extracted by using the method described in Non-Patent Document 3.

  Next, in step S308, the ratio calculation unit 1053 reads the foreground mask of the recognition model of each category stored in the foreground mask storage unit 1052 based on the recognition model used in step S304. At this time, if there are a plurality of recognition model categories that have passed through the cascade discriminator in the normalized local region in step S304, the most foreground masks of recognition models that have passed through the cascade discriminator are read out.

  Here, the foreground mask of the recognition model is a normalized image obtained by binarizing the object region using the average image of the normalized images of this category used when learning the recognition model. Therefore, the number of foreground masks stored in the foreground mask storage unit 1052 is the same as the number of categories in the recognition model.

  Next, in step S309, the ratio calculation unit 1053 first normalizes the object detection area to the size of the recognition model. Then, as shown in FIG. 6, the foreground area in the normalized detection area and the background area of the foreground mask are ANDed to obtain the foreground area (number of pixels) other than the foreground mask. Then, the ratio with the normalized detection area is calculated to obtain the ratio.

  Next, in step S310, the processing control unit 199 determines whether or not the ratio of the foreground area other than the foreground mask in the normalized detection area exceeds a predetermined threshold value. As a result of this determination, if the ratio does not exceed the threshold, it is determined that the object to be tracked does not intersect with another object, and the process proceeds to step S311. On the other hand, if the ratio exceeds the threshold value, it is determined that the object to be tracked intersects with another object, and thus the process proceeds to step S314 to process the next trajectory.

  In step S <b> 311, the template update unit 106 creates a feature amount such as the color, contour, and texture of the tracking object as a template based on the image data of the detection area, and stores the template in the template storage unit 109. If a template has already been created, it is updated by replacing the existing template.

  Here, examples of the feature amount (template) of the object include an RGB histogram of the object. In this case, the RGB value of each pixel in the image area of the object is quantized to a predetermined number of bins, and voted for the index that connects the RGB quantized values to create an RGB histogram. For example, in a 24-bit image, each RGB value of each pixel is 8 bits, and when each RGB value is quantized into 3 bits (ie, 8 bins) and concatenated, a 9-bit index is obtained. Then, each pixel in the image area of the object votes for a 9-bit index, and an RGB histogram is generated.

  Another example of the feature amount of the object is a SIFT (Scale-Invariant Feature Transform) feature amount described in Non-Patent Document 4. The SIFT feature value is a feature value obtained by converting a gradient histogram into a scale and rotation-invariant vector with a key size extracted by DoG (Difference-of-Gaussian) processing as a center and a region of a predetermined size.

  On the other hand, if the result of determination in step S306 is that an object has not been detected in the search area, processing proceeds to step S312. In step S312, the template matching unit 107 reads from the template storage unit 109 the template of the object related to the trajectory to be processed, generated or updated before or in the previous frame.

  Next, in step S313, the template collation unit 107 collates the image area in the search area using the read object template, and obtains the position and size of the area most similar to the object. For example, when the template is an RGB histogram of an object, an average position is obtained by assigning a large weight to pixel positions of colors having a large change in the number of pixels between the previous frame and the current frame. And the locus | trajectory of an object is specified by utilizing the mean-shift tracking method which moves a collation position. In addition, when the area most similar to the object is significantly different from the template (when the difference is larger than a predetermined value), it is determined that tracking is not possible.

  In step S314, the process control unit 199 determines whether the processes from steps S303 to S313 have been repeated for all the trajectories. As a result of this determination, if there is a locus that has not yet been processed, control is performed so that the next locus is processed. On the other hand, if processing has been performed for all the trajectories, the process proceeds to step S315.

  In step S315, the trajectory combining unit 104 updates the trajectory state by adding a detection or tracking result to the trajectory when an object can be detected in the search region or when tracking can be performed by the collation processing in step S313. To do. If the object cannot be detected and tracking cannot be performed, the state of the trajectory is changed to unsupported. Information on the position and size of the image area of the object is stored in the external storage device 206 as image area information. When a new object is detected by the global detector, a new locus is generated. At this time, it can be determined based on the template whether or not the newly detected object is an object whose trajectory has been changed to an unsupported state. When the objects are the same, it can be determined that the object is completely shielded and the state is released. In this way, the trajectory combining unit 104 functions as a tracking result update unit.

  Next, in step S316, the process control unit 199 determines whether the processes from steps S302 to S315 have been performed for all frames. If the result of this determination is that there is a next frame, processing is performed for the next frame. If there is no next frame, the processing is terminated.

  As described above, according to the present embodiment, identification is performed using an object recognition model, and the object is tracked. When the object cannot be detected by the object identifying unit 103, the object is tracked by performing matching using a template created based on the image area in the past frame detected by the object identifying unit 103, and tracking the object. False tracking can be suppressed. This template is a feature amount based on the appearance of the object identified by the object identifying unit 103 before intersecting, and information on other objects is difficult to be mixed into the feature amount before the intersection, so that the matching accuracy can be improved. Further, since the foreground area is extracted from the detection area and the intersection is detected based on the foreground mask corresponding to the recognition model used for detecting the object, the intersection between the objects can be detected with high accuracy.

  In this embodiment, the foreground area is extracted from the object detection area in step S307. However, as a process between step S301 and step S302, the foreground area may be extracted for the entire frame image. . In this case, in step S307, information on the foreground area of the frame image in the detection area is read out.

  In the present embodiment, the intersection is determined in step S309 based on the ratio of the foreground area other than the foreground mask to the normalized detection area, but other implementations are also conceivable. For example, the intersection may be determined based on the ratio of the foreground area other than the foreground mask and the area of the foreground mask. Alternatively, the intersection may be determined based on the ratio between the number of pixels in the foreground area in the foreground mask and the number of pixels in the foreground area in the normalized detection area. Although other percentage calculation methods are conceivable, the intersection may be determined essentially using the foreground mask of the recognition model and the foreground region in the detection region.

  In this embodiment, when there are a plurality of recognition model categories that have passed through the cascade classifier in the normalized local region in step S308, the most foreground masks of the recognition models that have passed through the cascade classifier are read out. It was. On the other hand, foreground masks of all recognition models that have passed through the cascade classifier may be read out. In this case, in step S309, the number of pixels in the foreground area in the foreground mask is counted using the ratio between the number of recognition models passed and the total number of recognition models as a weight, and is subtracted from the total number of pixels in the foreground area in the normalized detection area. Thus, the number of pixels in the foreground area outside the foreground mask may be obtained.

  In the present embodiment, the normalization local region integration process is performed in step S305. However, this process may be performed immediately after step S310 or S311. In this case, in step S304, each normalized local area where the object is detected is directly used as the object detection area. In step S309, the ratio of the foreground area outside the foreground mask to the normalized local area may be calculated.

(Second Embodiment)
In the first embodiment, when no object is detected in the search area, the template is used to perform image area matching processing, and the template is not updated. On the other hand, in this embodiment, even when no object is detected in the search region, the template is updated depending on the determination result of the intersection. Note that the configuration of the object tracking device according to the present embodiment is the same as that of the first embodiment, and a description thereof will be omitted.

FIG. 7 is a flowchart illustrating an example of a processing procedure for tracking an object in the present embodiment. Note that the same processes as those in FIG. 3 are denoted by the same reference numerals, and description thereof is omitted.
In step S701, the foreground mask storage unit 1052 stores information on the recognition model number used in the object detection process. When a plurality of recognition models are used, information on the recognition model number and the number of times of use is stored.

  In step S702, the same process as step S307 is performed except that the area detected in the collation process in step S313 is used instead of the object detection area. In step S703, the ratio calculation unit 1053 reads the foreground mask from the foreground mask storage unit 1052 in accordance with the recognition model number stored in step S701. In step S704, the ratio calculation unit 1053 calculates the ratio of the foreground area outside the foreground mask as in step S309. Steps S705 and S706 are the same as steps S310 and S311, respectively.

  As described above, according to the present embodiment, it is possible to detect an intersection even when an object cannot be detected by the object identification unit 103 but can be tracked by a matching process. In addition, since the template is updated when no intersection is detected, information on the latest object can be reflected.

(Third embodiment)
In the first embodiment, it is determined whether or not an object is detected after integrating the detection results of the pyramid layers. On the other hand, in the present embodiment, when the likelihoods are all less than the threshold before integrating the detection results, the detection results are integrated based on the similarity with the template. Note that the configuration of the object tracking device according to the present embodiment is the same as that of the first embodiment, and a description thereof will be omitted.

FIG. 8 is a flowchart illustrating an example of a processing procedure for tracking an object in the present embodiment. Note that the same processes as those in FIG. 3 are denoted by the same reference numerals, and description thereof is omitted.
In step S801, the processing control unit 199 calculates the likelihood of detection results of each pyramid layer in the search area, and determines whether the likelihoods of these detection results are all smaller than a predetermined threshold. As a result of this determination, if there is a detection result having a likelihood equal to or greater than the threshold, the process proceeds to step S802, and if all are smaller than the threshold, the process proceeds to step S312.

  In step S802, the object identification unit 103 separates detection results having a likelihood equal to or greater than a threshold for each object, obtains an average value in the original image for the positions and sizes of a plurality of normalized local regions of the same object, Is detected as an object detection area.

  On the other hand, in step S312, the template matching unit 107 reads the tracking template as in the first embodiment. In step S803, the process control unit 199 performs control so that the process of step S804 is repeatedly performed for all detection results.

  In step S804, the template matching unit 107 calculates the similarity between the template read in step S312 and the detection result. For example, if the template is an RGB histogram, the similarity can be a histogram intersection. When the template is a SIFT feature quantity, the similarity can be a ratio between the number of matched SIFT feature quantities and the total number of SIFT feature quantities.

  In step S805, the process control unit 199 determines whether or not the process of step S804 has been performed for all detection results. If there is a detection result that has not been processed as a result of this determination, the process of step S804 is performed for the next detection result. On the other hand, if all the detection results have been processed, the process proceeds to the next step S806.

  In step S806, the template matching unit 107 compares the similarity of each detection result calculated in step S314 with a threshold value, and integrates the detection results whose similarity is equal to or higher than the threshold value (a predetermined value or higher) as a tracking result.

  As described above, according to the present embodiment, as in the first embodiment, it is possible to improve the matching system and suppress erroneous tracking of an object to be tracked. In addition, it is possible to accurately detect the intersection between objects.

  In the present embodiment, the integration process in step S802 is performed before the intersection detection process, but may be performed immediately after step S310 or S311. Similarly to the second embodiment, as a result of the determination in step S308, even when the likelihood of each detection result is all smaller than the threshold, cross detection may be performed and the tracking template may be updated.

(Fourth embodiment)
In the first to third embodiments, the intersection of objects is detected using the foreground area in the detection area extracted by the background difference and the foreground mask of each recognition model. On the other hand, in the present embodiment, the detection region detected by the object identification unit 103 or the image region of the object collated by the template collation unit 107 is used to intersect each other by the overlapping ratio of the image regions of the objects of each locus. to decide.

FIG. 9 is a block diagram illustrating a functional configuration example of the object tracking apparatus 900 according to the present embodiment. Note that the hardware configuration of the object tracking device according to the present embodiment is the same as that of the first embodiment, and a description thereof will be omitted.
As shown in FIG. 9, compared to the configuration shown in FIG. 1, a trajectory intersection detection unit 905 is used instead of the intersection detection unit 105. In addition, about the other structure, the code | symbol same as FIG. 1 is attached | subjected and description is abbreviate | omitted about these structures.

FIG. 10 is a flowchart illustrating an example of a processing procedure for tracking an object in the present embodiment. Note that the same processes as those in FIG. 3 are denoted by the same reference numerals, and description thereof is omitted.
First, in step S <b> 1001, the trajectory intersection detection unit 905 reads from the external storage device 206 image area information of all trajectory objects up to the previous frame updated by the trajectory combination unit 104.

  Next, in step S1002, the trajectory intersection detection unit 905 obtains an overlapping area between the image area of the object of the trajectory to be processed and the image area of the object of the other trajectory for all other trajectories. Then, for all the other trajectories, the ratio of the obtained overlap area in the image area of the object of the trajectory to be processed is calculated.

  Next, in step S1003, the trajectory crossing detection unit 905 determines whether all the ratios of the overlapping region calculated in all other trajectories in the image region of the object to be processed are less than the threshold value. To do. As a result of this determination, if all of them are less than the threshold value, it is considered that there is no trajectory intersection, and the process proceeds to step S311. On the other hand, if there is at least one ratio that is equal to or greater than the threshold, the process proceeds to step S314 without updating the template.

  As described above, according to the present embodiment, as in the first embodiment, it is possible to improve the matching system and suppress erroneous tracking of an object to be tracked. In addition, it is possible to accurately detect the intersection between objects.

  In the present embodiment, the template update process is performed only when there is an object region identified by the recognition model in the search region, as in the first embodiment. On the other hand, as in the second embodiment, the same process as in steps S1001 to S1003 may be performed using the object region obtained by the template matching process, and the template may be updated even when the trajectories do not intersect. In the present embodiment, the integration processing of each normalized local region identified in step S304 is performed in S305. However, as in the third embodiment, it may be performed after the template update processing or the template matching processing. .

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

DESCRIPTION OF SYMBOLS 102 Search area setting part 103 Object identification part 104 Trajectory coupling part 105 Crossing detection part 106 Template update part 107 Template collation part 199 Process control part

Claims (10)

Setting means for setting a search area of an object in a frame of interest of a moving image based on a tracking result in a past frame;
Detecting means for detecting the object using a recognition model of the object in the search region set by the setting means;
Detecting means for extracting a foreground area from a detection area of the object detected by the detection means, and detecting an intersection of the objects using a mask corresponding to the extracted foreground area and the recognition model;
A template updating unit for updating a template indicating a feature amount of the object based on a detection area of the object detected by the detection unit when an intersection of the objects is not detected by the detection unit;
When the object is not detected by the detecting means, the image update area is extracted from the search area set by the setting means by collating with the template updated in the past frame by the template updating means. Matching means to
Tracking result update means for updating the detection area of the object detected by the detection means or the image area extracted by the collation means as a tracking result;
An object tracking device comprising:   The object tracking device according to claim 1, wherein when the detection unit detects the intersection of the objects, the template update unit does not update the template of the object.   The object tracking device according to claim 1, wherein the detection unit detects an intersection of the objects based on a ratio of an area that does not overlap the mask in the foreground area. When the image area of the object is extracted by the matching unit, the detection unit extracts a foreground area from the image area of the object, and uses the extracted foreground area and a mask corresponding to the recognition model. Detecting the intersection of the objects,
2. The template updating unit updates the template of the object based on an image area of the object detected by the detecting unit when the detecting unit does not detect the intersection of the objects. The object tracking device according to any one of?   The detection means detects the object by integrating detection results of the plurality of pyramid images using a plurality of pyramid images obtained by reducing the images including the search region at different ratios. The object tracking device according to any one of 1 to 4. The detection means detects the object in each of the plurality of pyramid images using a plurality of pyramid images including the search area,
The collating unit calculates a similarity between the template updated in the past frame by the template updating unit and a detection result in each of the plurality of pyramid images, and the pyramid image whose similarity is a predetermined value or more. The object tracking device according to claim 1, wherein the detection results are integrated into an image area of the object.   The said detection means uses the mask of the recognition model most utilized for the detection of the said object, when the said detection means uses several recognition models, The any one of Claims 1-6 characterized by the above-mentioned. The object tracking device described in 1.   The object tracking device according to claim 1, wherein the detection unit uses a mask of the plurality of recognition models when the detection unit uses a plurality of recognition models. A setting step of setting a search area of an object in a frame of interest of a moving image based on a tracking result in a past frame;
A detection step of detecting the object using a recognition model of the object in the search region set in the setting step;
A detection step of extracting a foreground region from a detection region of the object detected in the detection step, and detecting an intersection of the objects using the extracted foreground region and a mask corresponding to the recognition model;
A template update step of updating a template indicating the feature amount of the object based on the detection area of the object detected in the detection step when an intersection of the objects is not detected in the detection step;
When the object is not detected in the detection step, the image region of the object is extracted from the search region set in the setting step by collating with the template updated in the past frame in the template update step. A matching process to
A tracking result update step of updating the detection region of the object detected in the detection step or the image region extracted in the collation step as a tracking result;
An object tracking method comprising: A setting step of setting a search area of an object in a frame of interest of a moving image based on a tracking result in a past frame;
A detection step of detecting the object using a recognition model of the object in the search region set in the setting step;
A detection step of extracting a foreground region from a detection region of the object detected in the detection step, and detecting an intersection of the objects using the extracted foreground region and a mask corresponding to the recognition model;
A template update step of updating a template indicating the feature amount of the object based on the detection area of the object detected in the detection step when an intersection of the objects is not detected in the detection step;
When the object is not detected in the detection step, the image region of the object is extracted from the search region set in the setting step by collating with the template updated in the past frame in the template update step. A matching process to
A tracking result update step of updating the detection region of the object detected in the detection step or the image region extracted in the collation step as a tracking result;
A program that causes a computer to execute.