JP5754990B2 - Information processing apparatus, information processing method, and program - Google Patents

Description

  The present invention relates to an information processing apparatus, an information processing method, and a program.

Conventionally, in tracking a moving object in an image, there is a method of using an environment model in a method of improving accuracy in tracking and a recovery method when tracking fails.
For example, Patent Document 1 discloses a method of monitoring the periphery of a specific area and waiting for a moving object to reappear when the tracking target overlaps with the specific area.

JP 2006-311099 A

  However, in the technology disclosed in Patent Document 1, in an indoor environment where a plurality of objects appear to overlap each other, if only one specific area is specified, the possibility that a moving object will appear again from the specific area is limited. There was a problem that we couldn't.

  The present invention has been made in view of such problems, and an object thereof is to enable stable object tracking even in an environment where a plurality of shielding objects appear to overlap each other.

Therefore, the present invention is an information processing apparatus that detects and tracks a tracking target from an image, and determines whether the tracking target is shielded by an object when the tracking target cannot be detected. When the determination unit determines that the tracking target is shielded by an object, the extraction unit extracts the object region and the object region overlapping the object from the image, and the extraction unit extracts the region. and the object region that is associate, and the object of the area overlapping with the object, in the vicinity of the associated region, then a setting means for setting a region in which the tracking target appears, wherein the determining means The depth of the detection position of the tracking object before the detection of the tracking object fails is compared with the depth of the predicted position of the tracking object after the detection of the tracking object fails, and the detection of the tracking object is performed. If towards the depth of the location is long, it is determined that the tracking target is shielded by the object.

  According to the present invention, it is possible to enable stable object tracking even in an environment where a plurality of shielding objects appear to overlap each other.

It is a figure which shows an example of the hardware constitutions of a tracking apparatus. FIG. 3 is a diagram illustrating an example of a software configuration of the tracking device according to the first embodiment. It is a figure which shows an example of an image. It is FIG. (1) which shows an example of the process in a tracking apparatus. It is FIG. (2) which shows an example of the process of a tracking apparatus. It is FIG. (3) which shows an example of the process of a tracking apparatus. It is FIG. (4) which shows an example of the process of a tracking apparatus. It is a figure which shows an example of a list. It is FIG. (5) which shows an example of the process of a tracking apparatus. It is a flowchart which shows an example of a tracking process. It is a figure which shows an example of the software configuration of the tracking apparatus of Embodiment 2. It is FIG. (6) which shows an example of the process of a tracking apparatus. It is a figure which shows an example of the software configuration of the tracking apparatus of Embodiment 3. It is a figure for demonstrating object information.

  Hereinafter, embodiments of the present invention will be described with reference to the drawings.

<Embodiment 1>
The tracking device according to the present embodiment is used for video monitoring / monitoring, and the imaging means may be a general network camera, or a camera with a tracking function. This embodiment will be described below with reference to the drawings. The tracking device is an example of an information processing device.
In the present embodiment, a case will be described in which an image is taken in an environment with a plurality of objects, and a person to be tracked is tracked from the obtained moving image. Although the environment according to the present embodiment is indoors and the tracking target is a person, the present invention is not limited to this, and any object that moves in another environment can be used.
FIG. 1 is a diagram illustrating an example of a hardware configuration of the tracking device.
The control device 10 is a CPU or the like, and controls the entire tracking device 100. The storage device 11 is a storage device such as a RAM and / or a ROM and / or an HDD, and stores, for example, an image or a program. The communication device 12 is a device that connects the tracking device to a network or the like. In the present embodiment described below, it is assumed that the tracking device 100 is connected to a network camera or the like via the communication device 12.
When the control device 10 executes a process based on a program stored in the storage device 11, the function of the tracking device 100 and a process related to a flowchart described later are realized.

FIG. 3 described later is an example of an image targeted by this embodiment. It is assumed that the tracking target 210 is moving in the moving direction 250. In the image 200, the objects 220, 230, and 240 exist in the environment where the tracking target is present, which are in different shapes and positions, and some of them appear to overlap each other. The tracking device 100 detects the tracking object 210 by the initial detection and tracks the tracking object 210. The search area 260 represents an area where the tracking device 100 has predicted that the tracking target 210 at the next time will move to a position in the rectangular area. The tracking device 100 may use prediction based on dynamics of a moving object or prediction in a tracking framework such as a particle filter for the prediction of the tracking target 210.
FIG. 2 is a diagram illustrating an example of a software configuration of the tracking device 100 according to the first embodiment.
As shown in FIG. 2, the tracking device 100 includes an initial detection unit 101, a prediction unit 102, a target detection unit 103, a depth information acquisition unit 104, an occlusion determination unit 105, an occlusion object region extraction unit 106, A shielded object overlapping region identifying unit 107 and a region setting unit 108 are included.
The initial detection unit 101 detects the tracking target 210 from the image, and a general object detection method such as template matching can be used. The initial detection unit 101 detects the tracking target 210 and outputs the detection position 270 of the tracking target 210 to the prediction unit 102.

FIG. 3 is a diagram illustrating an example of the image 200.
As shown in FIG. 3, the prediction unit 102 receives the detection position 270 of the tracking target 210 from the initial detection unit 101, and determines the position to move based on the direction (movement direction) 250 in which the tracking target 210 moves at the next time. Prediction is performed, and a predicted position 280 and a search area 260 are set. The prediction unit 102 outputs the predicted position 280 to the target detection unit 103.
The target detection unit 103 acquires an image at the next time based on the predicted position 280 output from the prediction unit 102 and detects whether the tracking target 210 is in the predicted position 280 and the search region 260. If the tracking target 210 can be detected in the search area 260, the process returns to the prediction unit 102, and the prediction unit 102 predicts the position of the tracking target 210 one after another. If not detected, the target detection unit 103 outputs the predicted position 280 to the shielding determination unit 105.
The depth information acquisition unit 104 acquires depth information and the like of each pixel in the image. The depth information acquisition unit 104 may use distance measurement using a method of acquiring the depth of a sparse feature point by a stereo camera, or acquire depth information of all pixel pixels by an infrared distance measurement camera such as a TOF camera. May be. In this embodiment, a case where TOF is used will be described as an example. The depth information acquisition unit 104 can output depth information to the occlusion determination unit 105, the occluded object region extraction unit 106, and the occluded object overlap region identification unit 107.

As illustrated in FIG. 4, the occlusion determination unit 105 determines whether the tracking target 210 is occluded by the occluded object 220 with respect to the predicted position 280 based on the result of the target detection unit 103 and the result of the prediction unit 102. . In the present embodiment, the shielding determination unit 105 shows an example based on the result of the depth information acquisition unit 104. However, the present invention is not limited to this, and the depth disclosed in Japanese Patent Application Laid-Open No. 2006-311099 is used. The shielding determination may be performed by a method that does not. The occlusion determination unit 105 acquires the depth information 380 at the pixel at the predicted position 280 from the depth information acquisition unit 104 and compares it with the depth information 370 at the pixel at the detection position 270 of the tracking target 210 at time t.
For these two positions, when the depth information 370 of the pixel at the detection position 270 is farther than the depth information 380 of the pixel at the predicted position 280 as shown in Equation (1) described later, the shielding determination unit 105 determines that the tracking target 210 is shielded. It is determined that Then, the shielding determination unit 105 outputs the predicted position 280 and its depth information 380 to the shielding object region extraction unit 106. If it is determined that it is not due to shielding, the processing is not performed in the present embodiment, but the processing is returned to the initial detection unit 101 or the prediction unit 102, and the processing is continued.
depthposition> depthpredict (1)

  As illustrated in FIG. 5, the occluded object region extraction unit 106 extracts an area 420 of the occluded object 220 in the depth information 380 of the predicted position 280 based on the result of the occlusion determination unit 105. The occluded object region extraction unit 106 acquires the depth information 380 of the pixel at the predicted position 280 from the depth information acquisition unit 104, and has a similar depth at neighboring pixels by threshold discrimination for the depth information of the depth information 380 at the predicted position 280. Pixels are extracted to generate an object region 420. Similarly, the shielding object region extraction unit 106 extracts regions 430 and 440 of other shielding objects 230 and 240 in the image based on the depth information. In the present embodiment, an example in which the shielding object region extraction unit 106 performs object region extraction based on the depth information is shown. However, the present embodiment is not limited to this, and the occluded object region extraction unit 106 does not use depth information, and does not use Graph-Cut or You may extract by applying segmentation algorithms, such as a watershed method and a mean-shift method. The occluded object area extraction unit 106 outputs the object area 420 and the other object areas 430 and 440 that occlude the predicted position 280 to the occluded object overlapping area identification unit 107.

As illustrated in FIG. 6, the occluded object overlap area identifying unit 107 identifies another object area 430 that overlaps the object area 420 that occludes the tracking target 210, and an area that overlaps the area. The occluded object overlapping area identification unit 107 includes areas 420, 430, and 440 of the occluded objects 220, 230, and 240 that overlap in areas and boundaries as shown in Expression (2) described later in the image. It is assumed that the two occluded object regions are in an overlapping relationship when the depth is separated to some extent as shown in Equation (3) described later. The occluded object overlapping area identifying unit 107 repeats this process to create a list of overlapping occluded object areas. The overlapping relationship of the shielding objects is not limited to the list, and any overlapping relationship may be used. As an example, when the shielding object is set to 623 with respect to the environment shown in FIG. 7, a list of overlapping relations of the shielding objects by the shielding object overlapping area identification unit 107 is described as shown in FIG. The occluded object overlapping area identifying unit 107 outputs this overlapping relationship list to the area setting unit 108.
Robj1 AND Robj2 = 1 (2)
| Object_area-area |> threshold (Formula (3))

The area setting unit 108 sets an area 792 where the tracking target 210 reappears based on the output result of the occluded object overlapping area identification unit 107. In the example of FIG. 9, the area setting is set to a rectangle, but the present embodiment is not limited to this, and is not limited to a rectangle, and may be a circle or an ellipse. The region setting unit 108 outputs the set region 792 to the initial detection unit 101 or the prediction unit 102. Thus, the tracking device 100 that can detect when the tracking object 210 appears from the shielding object 220 or the overlapping shielding object 230 after the tracking loop is interrupted by being concealed by the shielding object 220. Can be realized.
The above is the components related to the tracking device 100 according to the present embodiment. As a result, the tracking can be stabilized even in an environment where there are a plurality of objects and they appear to overlap each other.

Subsequently, an information processing method in the present embodiment will be described.
FIG. 10 is a flowchart illustrating an example of the tracking process.
In step 1, the initial detection unit 101 performs initial detection to determine whether there is a tracking target in the image obtained from the imaging unit. The initial detection unit 101 creates a tracking target template from an image when the tracking target can be detected by the initial detection. The initial detection unit 101 may extract the image information as a template as it is, or may generate it using image characteristics to be tracked.
In step 2, the prediction unit 102 predicts a position where the tracking target is likely to appear in the next frame. In the prediction method, a general prediction method such as a particle filter can be used, but it is not limited thereto.
In step 3, the target detection unit 103 detects the tracking target in the next frame using the tracking target template acquired in step 1 for the position predicted in step 2.
In step 4, the target detection unit 103 determines whether the tracking target has been detected. The target detection unit 103 may calculate a correlation value between the template and the image, and may determine that the value is equal to or greater than an arbitrary threshold value, or may determine using a method other than that. If the target detection unit 103 can detect the tracking target, the target detection unit 103 returns to step 3 and performs prediction again. If not detected, go to step 7.
Steps 1 to 4 are the tracking framework. In the following steps 5 to 8, a tracking method corresponding to shielding will be described.

In Step 5, when the detection in Step 4 fails in the tracking target position predicted in Step 3, the shielding determination unit 105 determines whether or not the cause of the detection failure is shielded by an object in the environment. The occlusion determination unit 105 uses depth information acquired from a depth camera or the like, and determines that the occlusion is occluded when the depth of the detected position is far from the depth of the predicted position. When the shielding determination unit 105 determines that the tracking target is not lost due to the shielding, the process returns to Step 1 or Step 3. If the shielding determination unit 105 determines that it is due to shielding, the process proceeds to step 6. In addition to the stereo camera, a TOF camera may be used as the depth camera.
In Step 6, since it is determined that there is a shielding object at the position predicted in Step 3, the shielding object area extraction unit 106 extracts a shielding object area in the image. The occluded object region extraction unit 106 uses the information on the depth of the predicted position to extract pixels with close depths as regions. As another method, the occluded object region extraction unit 106 may extract the region by extracting the image feature of the predicted position and performing segmentation. In step 6, the occluded object area extraction unit 106 simultaneously extracts another object area in the image.
In step 7, the occluded object overlapping area identifying unit 107 identifies an area that overlaps the occluded object area in the extracted area of the occluded object, and an area that overlaps the area. Create a list of
In step 8, the region setting unit 108 sets a position where the tracking target appears again based on the list.

  As described above, according to the processing of the first embodiment, when the tracking target is hidden by the shielding object, the area of another shielding object that appears to overlap the area of the shielding object is connected, and the search area is set accordingly. Is done. Thereby, when the tracking target cannot be seen, the movement is different from the previous one, and the tracking can be performed even if it reappears from a different location from the detected occluding object.

<Embodiment 2>
Next, Embodiment 2 will be described. FIG. 11 is a diagram illustrating an example of a software configuration of the tracking device 100 according to the second embodiment. In the configuration of the tracking device 100 of the second embodiment, a time information recording unit 109 is further added to the configuration of the first embodiment.
The time information recording unit 109 records the time when the tracking target is determined to be shielded, and outputs the recorded time to the region setting unit 108.
The area setting unit 108 weights the setting area as shown in FIG. 12 based on the predicted position, the occluded object area overlapping the occluded object area, and the time information. For example, it is considered that the tracking target 1010 is shielded by the object 1021 in the time immediately after the shielding determination. Therefore, when it is assumed that the tracking target 1010 does not move extremely, a little time is required when the object to be tracked 1091 comes out behind the object 931. Therefore, the area setting unit 108 increases the weight of the surrounding area 1093 around the object 1021 and weakens the weight of the surrounding area 1092 around the object 1031 when the tracking target is blocked.
However, since the tracking target 1010 may move to the region of the object 993 as time elapses while being shielded, the tracking target 1010 may appear from either the region 1093 or the region 1094. Becomes higher. Therefore, the area setting unit 108 sets an area 1095 in which a certain weight is applied to the surrounding area between the object 1021 and the object 1031.

<Embodiment 3>
Subsequently, Embodiment 3 will be described. FIG. 13 is a diagram illustrating an example of a software configuration of the tracking device 100 according to the third embodiment. In the configuration of the tracking device 100 of the third embodiment, in addition to the configuration of the first embodiment, an object information recording unit 110 and an object recognition unit 111 are further added.
The object information recording unit 110 records object information in the environment (for example, a three-dimensional shape such as a bed 1202, a shelf 1203, a table 1204, and a chair 1205 as shown in FIG. 14). The object recognition unit 111 recognizes a shielded object in the environment based on information from the depth information acquisition unit 104 and the object information recording unit 110. The object recognizing unit 111 outputs the behavior information of the tracking target associated with the object stored in the object information recording unit 110 from the recognition result to the region setting unit 108.
The area setting unit 108 sets a weighted area based on the output of the occluded object overlapping area identification unit 107 and the output of the object recognition unit 111. For example, in the bed, the area setting unit 108 weakens the weight on the head side area and increases the weight on the side area.
According to the processing of the third embodiment, the tracking can be stabilized.

<Other embodiments>
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

According to the processing of each embodiment described above, it is possible to stably track a moving object even in an environment where a plurality of objects in an image are cluttered. Even if tracking is interrupted by occlusion, tracking can be resumed at high speed when it reappears from around the occluded object. For example, it is possible to stably track the movement of a person in an environment where a plurality of objects appear to overlap.
That is, stable object tracking can be achieved even in an environment where a plurality of shielding objects appear to overlap each other.

  The preferred embodiments of the present invention have been described in detail above, but the present invention is not limited to such specific embodiments, and various modifications can be made within the scope of the gist of the present invention described in the claims.・ Change is possible.

10 Control device 100 Tracking device

Claims (4)

An information processing apparatus for detecting and tracking a tracking target from an image,
Determining means for determining whether or not the tracking target is occluded by an object when the tracking target cannot be detected;
An extraction unit that extracts the region of the object and the region of the object that overlaps the object from the image when the determination unit determines that the tracking target is blocked by the object;
A setting unit that associates the region of the object extracted by the extraction unit with the region of the object that overlaps the object, and sets a region in which a tracking target appears next to the vicinity of the associated region;
Have
The determination means compares the depth of the detection position of the tracking object before the detection of the tracking object fails with the depth of the predicted position of the tracking object after the detection of the tracking object fails, An information processing apparatus that determines that the tracking target is shielded by an object when the depth of the detection position of the tracking target is longer . Depth information acquisition means for acquiring depth information of the detection position of the tracking object before the detection of the tracking object fails and depth information of the predicted position of the tracking object after the detection of the tracking object fails. In addition,
The information processing apparatus according to claim 1 , wherein the determination unit compares the depth information of the detection position acquired by the depth information acquisition unit with the depth information of the predicted position . An information processing method executed by an information processing apparatus that detects and tracks a tracking target from an image,
A determination step of determining whether or not the tracking target is shielded by an object when the tracking target cannot be detected;
An extraction step of extracting the region of the object and the region of the object overlapping the object from the image when it is determined in the determination step that the tracking target is blocked by the object;
A step of associating the region of the object extracted in the extraction step with the region of the object that overlaps the object, and setting a region in which a tracking target appears next in the vicinity of the associated region;
Including
In the determination step, the depth of the detection position of the tracking object before the detection of the tracking object fails is compared with the depth of the predicted position of the tracking object after the detection of the tracking object fails, An information processing method for determining that the tracking target is shielded by an object when the depth of the detection position of the tracking target is longer . A computer that detects and tracks the tracking target from the image ,
A determination step of determining whether or not the tracking target is shielded by an object when the tracking target cannot be detected;
An extraction step of extracting the region of the object and the region of the object overlapping the object from the image when it is determined in the determination step that the tracking target is blocked by the object;
A step of associating the region of the object extracted in the extraction step with the region of the object that overlaps the object, and setting a region in which a tracking target appears next in the vicinity of the associated region;
Was executed,
In the determination step, the depth of the detection position of the tracking object before the detection of the tracking object fails is compared with the depth of the predicted position of the tracking object after the detection of the tracking object fails, A program for determining that the tracking target is shielded by an object when the depth of the detection position of the tracking target is longer .

Images