JP2017228303A - Moving object tracking device, method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving object tracking system achieving a less calculation cost even for complicated movements of a plurality of moving objects, a method and a program.SOLUTION: A moving object tracking system 1 has: an acquisition part 10 for acquiring a plurality of frames; a detection part 11 for detecting an object from the plurality of frames; an extraction part 12 for extracting a first movement trajectory including a first frame, a second movement trajectory constituted only with other frames before the first frame and a third movement trajectory constituted only with other frames after the first frame for the detected object; and an associating part 16 for associating the second movement trajectory with the third movement trajectory when a similarity between the second movement trajectory and the third movement trajectory comes to be equal to or higher than a similarity between the first movement trajectory and the third movement trajectory.SELECTED DRAWING: Figure 1

Description

  Embodiments described herein relate generally to a moving object tracking apparatus, method, and program.

  By detecting multiple moving objects included in multiple frames in the time series of images and associating the same objects between frames, tracking of moving objects is performed, and the results of tracking are recorded. A moving object tracking system for identifying a moving object is disclosed.

  However, since the combination of branches (paths) with the appearance, disappearance, and detection failure of detected faces in a plurality of time-series images as nodes is considered, the calculation cost increases.

JP 2011-170711 A

  SUMMARY OF THE INVENTION The problem to be solved by the present invention is to provide a moving object tracking system and method with a low calculation cost even for a complicated movement of a plurality of moving objects.

  The object tracking device according to the embodiment includes an acquisition unit, a detection unit, an extraction unit, and a correspondence unit. The acquisition unit acquires a plurality of frames. The detection unit detects an object from the plurality of frames. The extraction unit includes, for the detected object, only a first movement trajectory including a first frame, a second movement trajectory including only a frame before the first frame, and only a frame after the first frame. A third movement trajectory configured is extracted. The associating unit is configured such that when the similarity between the second movement trajectory and the third movement trajectory is greater than or equal to the similarity between the first movement trajectory and the third movement trajectory, the second movement trajectory and the third movement trajectory Associate trajectories.

1 is a configuration diagram showing an example of a moving object tracking system of a first embodiment. The flowchart which shows the example of the moving object tracking system of 1st Embodiment. Explanatory drawing which shows the example of the division | segmentation of a movement locus | trajectory. Explanatory drawing which shows the example of calculation using a Euclidean distance. The block diagram which shows the example of the moving object tracking system of 2nd Embodiment. The schematic diagram showing the shielding of 2nd Embodiment.

  Hereinafter, embodiments will be described in detail with reference to the accompanying drawings.

(First embodiment)
FIG. 1 is a configuration diagram illustrating an example of a moving object tracking system 1 according to the first embodiment. As shown in FIG. 1, an acquisition unit 10, a detection unit 11, an extraction unit 12, a management unit 13, a setting unit 14, a division unit 15, a correspondence unit 16, an integration unit 17, and an output unit 18 are provided. The moving object tracking system may be realized by causing a processing device such as a CPU (Central Processing Unit) to execute a program, that is, by software, or by hardware such as an IC (Integrated Circuit). It may be realized by using software and hardware together. The moving image acquired by the acquisition unit 10 may be one stored in a storage device.

  The storage device is, for example, at least one of magnetic, optical, and electrical storage devices such as a hard disk drive (HDD), a solid state drive (SSD), a read only memory (ROM), and a memory card. Can be realized.

  FIG. 2 is a flowchart illustrating an example of a flow of processing of the moving object tracking system 1 according to the first embodiment.

  First, the moving object tracking system 1 acquires a moving image captured by the imaging device or a moving image stored in the storage device (step S101). The moving image includes a plurality of frames (images).

  Next, the detection unit 11 detects a plurality of the moving objects from the moving image acquired by the acquisition unit (step S102). A moving object refers to a person, a car, etc., for example. Hereinafter, a case where the moving object is a person will be described as an example. The following method can be applied as a specific processing method for detecting a person. For example, a method described in the literature (N. Dalal, B. Triggs, “Histograms of Oriented Gradients for Human Detection,” IEEE Computer Vision and Pattern Recognition, pp. 886-893, 2005) is applicable. In addition, it is possible to improve the accuracy of human detection by estimating the position of the same object in the frame being processed by tracking the object detected in the frame immediately before the frame being processed using the Visual Tracking technology. It is. The Visual Tracking technique is described in, for example, a method described in the literature (K. Zhang, L. Zhang, MH Yang “, Real-time compressive tracking”, European Conference Computer Vision, pp. 866-879, 2012). Applicable.

  Next, the extraction unit 12 associates each person in each frame, and extracts a movement trajectory (tracklet; hereinafter referred to as a tracklet) of the associated person (step S103). As a method of extracting tracklets, for example, literature (H. Pirsiavash, D. Ramanan, C. C. Fowles “, Globally-Optimal Greedy Algorithm for Revelation of Objet of ObEvE” 1201-1208, 2012) can be applied.

  Next, the management unit 13 manages tracklets for each person (step S104). The tracklet only needs to manage in which time zone the moving person is included in the moving image, and may be managed by the frame number of the image, playback or recording time, and the like. Further, the position and size of the person detected by the detection unit 11 may be stored together. These pieces of information may be stored in the storage unit 19. The tracklet for each person may be stored in the storage unit 19 together with the ID of the person. The person IDs need only be assigned so that they can be distinguished from each other, and the person itself may not be identified.

  Next, the setting unit 14 sets a frame whose position is to be calculated as a frame of interest (step S105).

Next, the dividing unit 15 performs, for the plurality of extracted tracklets, a first block including at least the frame of interest, a second block positioned before the first block in time series order, and time series order To divide into the third block located after the first block (step S105). For example, as shown in FIG. 3, when a time t (i-th) frame is set as a frame of interest in an image of N frames, a block including the i-th frame is more time-sequential than the first block and the first block. A block preceding in order is a second block, and a block following in time series order than the first block is a third block. More specifically, the following formula 1 is used.

  Here, start (t) is the number of the start frame of tracklet t, and end (t) is the number of the end frame of tracklet t.

  Next, the associating unit 16 acquires the tracklet included in the second block from the management unit 13, and determines the acquired tracklet and the tracklet included in the first block and the third block based on the similarity of the tracklet. Correlate (step S106).

  This is executed in two stages, that is, tracklet selection for the second block and tracklet selection for the third block.

  First, the tracklet selection of the second block selects ap such that end (ap) = i−1 from the second block. i is a frame of interest. More specifically, a track block having a length of the second block within a predetermined threshold may be used. Tracklets that end before time sequence in order of i-1 are excluded because they have been processed before this processing flow. In this case, the processing time can be greatly saved by greatly reducing the number of candidates for association.

In addition, in the third block tracklet selection, each {ap} selected in the previous step is associated with a tracklet bq that satisfies the following expression (2) from the third block.

However, D (,) is calculated from the similarity of motion between appearances and the similarity of appearance, which indicates the correspondence between two tracklets.

The motion similarity MotionMatch (t1, t2) assumes that the person takes a linear movement at t1 in a short time between the tracklets t1 and t2. t1 extends to t1 ′ by the start time of t2, and is calculated using the Euclidean distance as shown in FIG.

  The appearance similarity AppearanceMatch (t1, t2) is calculated by selecting a representative appearance of a person from each tracklet, and extracting and comparing feature values from the two appearances. Regarding the selection of a typical appearance, as shown in FIG. 4, a person between tracklets may be selected. Further, extraction and comparison of feature values are described in literature (K. Zhang, L. Zhang, MH Yang “, Real-time compressive tracking”, European Confence Computer Vision, pp. 866-879, 2012). Applied techniques can be applied.

  Here, if there is no tracklet extraction result corresponding to the person detection result in the second block, the same processing as Step S101 to Step S104 described in FIG. 2 is performed to extract a person and tracklet. Is possible.

  The integration unit 17 integrates the set of tracklets associated with the association unit 16 as a new tracklet. The management unit 14 manages the detection result of the detection unit 12 and the integrated tracklet of the integration unit 17.

  The output unit 18 outputs the associated person and tracklet results. The result of the person's tracklet may be displayed superimposed on the moving image, or only the result of the desired person's tracklet may be output. If the result of a person's tracklet is superimposed, a complicated trajectory becomes clear to the user, which is good. Also, the person ID and position may be output only for the frame of interest. As described above, the person IDs may be assigned so that they can be distinguished from each other, and the person itself may not be identified.

  As described above, according to the moving object tracking system 1 according to the first embodiment, it is possible to reduce the calculation cost even for complicated movements of a plurality of moving objects. In particular, the associating unit 16 uses the tracklets of the second block and integrates them with the tracklets of the first and third blocks, so that no overlap calculation is required in the association of the tracklets. In addition, it is possible to reduce the calculation cost because redundant calculation is unnecessary.

(Second Embodiment)
FIG. 5 is a block diagram showing a moving object tracking system 2 according to the second embodiment. The moving object tracking system 2 according to the present embodiment includes an acquisition unit 10, a setting unit 11, a detection unit 12, an extraction unit 13, a management unit 14, a division unit 15, an association unit 16, an interpolation unit 20, an integration unit 17, and an output. Part 18. The difference from the first embodiment is that an interpolation unit 20 is provided.

  The interpolating unit 20 interpolates the position information of the person in the first block from the tracklets of each person associated with the second block and the third block. Here, the position information indicates information including any one of the position in the person's frame, the size in the person's frame, and the movement trajectory (tracklet) before the target frame in time series order.

Specifically, the non-detection of the person in the time series between the tracklets associated by the associating unit 16 is interpolated. The associated tracklet is (t1, t2), and the position, size and frame number ([x1, y1], [h1, w1], f1) of the person at the end of t1 are The position, size, and frame number ([x2, y2], [h2, w2], f2) of the person are assumed. If df = f2-f1, the position and size of the person ([xs, ys], [hs, ws]) for each frame f1 + s in [f1 + 1, ..., f1 + df-1] Can be estimated by Equation 5.

  The output unit 18 outputs the position of the person in the frame of interest using the association result of the association unit 16 and the estimation result of the interpolation unit 20. Also, the size and the previous association result in chronological order from the frame of interest may be output together.

  The integration unit 17 may integrate the associated tracklet and the interpolated person area into a new tracklet using the association result of the association unit 16 and the interpolation result of the interpolation unit 20.

  Here, a specific case where interpolation is required will be described with reference to FIG.

  FIG. 6 is a schematic diagram illustrating a case where a person hides in the shadow of a building (upper stage) and shielding caused by passing of people. The situation when the drawing changes from the left drawing to the right side along the time axis is shown.

  For example, if people pass each other (the lower part of FIG. 6), the front-rear correspondence can be made based on the walking speed. The speed can be estimated by determining whether the moving object is a person or a car and learning an average speed in advance.

  The same applies to the shielding by the building (upper part of FIG. 6). For example, if there is a building that shields a person around the image to be captured, the presence or absence of a building that shields the moving object if the moving object is a car, etc., or position information Etc. in advance can be used to estimate the shielding time.

  From the estimated shielding time, an appropriate length of the first block may be set from the assumed shielding time. For example, in a general monitoring video, the length M is set to twice the frame rate (that is, the number of frames for 2 seconds). By setting the occlusion time, it is possible to appropriately respond to changes in the environment due to buildings, traffic, and the like, and the results of tracking a person can be more robust.

  As described above, according to the moving object tracking system 1 according to the second embodiment, it is possible to reduce the calculation cost even for complicated movements of a plurality of moving objects. In particular, the tracking result in the frame of interest can be output by interpolation even if the entire tracking result is unknown.

(Hardware configuration)
The moving object tracking system of the above embodiment includes a control device such as a CPU (Central Processing Unit), a storage device such as a ROM and a RAM, an external storage device such as an HDD and an SSD, a display device such as a display, an input such as a mouse and a keyboard. The apparatus includes an image capturing apparatus such as a camera and a camera, and can be realized by a hardware configuration using a normal computer.

  The program executed by the apparatus of the above embodiment is provided by being incorporated in advance in a ROM or the like.

  The program executed by the apparatus of the above embodiment can be read by a computer such as a CD-ROM, a CD-R, a memory card, a DVD, and a flexible disk (FD) in an installable or executable file. It may be provided by being stored in a different storage medium.

  The program executed by the apparatus of the above embodiment may be provided by storing it on a computer connected to a network such as the Internet and downloading it via the network. The program executed by the apparatus of the above embodiment may be provided or distributed via a network such as the Internet.

  The program executed by the apparatus of the above embodiment has a module configuration for realizing the above-described units on a computer. As actual hardware, for example, the control device reads out a program from an external storage device to the storage device and executes the program, whereby the above-described units are realized on a computer.

  Note that the present invention is not limited to the above-described embodiment as it is, and can be embodied by modifying the constituent elements without departing from the scope of the invention in the implementation stage. In addition, various inventions can be formed by appropriately combining a plurality of components disclosed in the embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, constituent elements over different embodiments may be appropriately combined.

  For example, as long as each step in the flowchart of the above embodiment is not contrary to its nature, the execution order may be changed, a plurality of steps may be performed simultaneously, or may be performed in a different order for each execution.

  As described above, according to the moving object tracking system 1 according to the embodiment, it is possible to reduce the calculation cost even for complicated movements of a plurality of moving objects. In particular, the associating unit 16 uses the tracklets of the second block and integrates them with the tracklets of the first and third blocks, so that no overlap calculation is required in the association of the tracklets.

DESCRIPTION OF SYMBOLS 1, 2 ... Moving object tracking system, 10 ... Acquisition part, 11 ... Detection part, 12 ... Extraction part, 13 ... Management part, 14 ... Setting part, 15 ... Dividing unit, 16 ... corresponding unit, 17 ... integrating unit, 18 ... output unit, 19 ... storage unit, 20 ... interpolating unit

Claims (7)

An acquisition unit for acquiring a plurality of frames;
A detection unit for detecting an object from the plurality of frames;
For the detected object, a first moving trajectory including a first frame, a second moving trajectory including only frames before the first frame, and a first moving trajectory including only frames after the first frame. An extraction unit for extracting three moving trajectories;
When the similarity between the second movement trajectory and the third movement trajectory is equal to or higher than the similarity between the first movement trajectory and the third movement trajectory, an association for associating the second movement trajectory with the third movement trajectory And
An object tracking device comprising: An interpolation unit that interpolates position information of the object in the first and subsequent frames from the second movement trajectory and the third movement trajectory associated with each other in the association unit;
The object tracking device according to claim 1, further comprising: The object tracking device according to claim 1, wherein the associating unit uses the similarity of an object and the movement of a moving track as the similarity. The object tracking device according to claim 3, wherein the associating unit uses a product of the similarity of the object and the similarity of movement of the movement locus as the similarity.   The object tracking device according to any one of claims 1 to 4, wherein a length of the second movement trajectory associated with the association unit is within a predetermined threshold. Obtaining a plurality of frames;
Detecting an object from the plurality of frames;
For the detected object, a first moving trajectory including a first frame, a second moving trajectory including only frames before the first frame, and a first moving trajectory including only frames after the first frame. Extracting three trajectories; and
Associating the second movement trajectory with the third movement trajectory when the similarity between the second movement trajectory and the third movement trajectory is greater than or equal to the similarity between the first movement trajectory and the third movement trajectory; ,
An object tracking method. Obtaining a plurality of frames;
Detecting an object from the plurality of frames;
For the detected object, a first moving trajectory including a first frame, a second moving trajectory including only frames before the first frame, and a first moving trajectory including only frames after the first frame. Extracting three trajectories; and
Associating the second movement trajectory with the third movement trajectory when the similarity between the second movement trajectory and the third movement trajectory is greater than or equal to the similarity between the first movement trajectory and the third movement trajectory; ,
A program that causes a computer to execute.