JP2014164579A - Information processor, program and information processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To more accurately detect a person from a distance image even when a person is not imaged from directly above.SOLUTION: The information processor includes: an acquisition part for acquiring a first distance image generated by an imaging device and a second distance image preliminarily generated by the imaging device by imaging a background; a generation part for generating a third distance image corresponding to a foreground corresponding to the background on the basis of the first distance image and the second distance image; and a detection part for detecting one or more regions from the third distance image by using a plurality of filters applied at different positions in the third distance image, that is, the plurality of filters capable of recognizing the patterns of the head and body of the person corresponding to the applied positions.

Description

  The present invention relates to an information processing apparatus, a program, and an information processing method.

  In recent years, various techniques for detecting a desired object from an image by pattern recognition have been studied. As such a technique, for example, there is a person detection technique for detecting a person from an image.

  For example, Non-Patent Document 1 discloses a technique for detecting a person from a distance image generated by a distance image sensor installed from the ceiling toward the floor. More specifically, according to the technique, an image of only the foreground portion is generated from the generated distance image by background difference. Then, by applying a Haar-Like filter capable of recognizing the convex shape of the person's head and both shoulders to the generated image and performing clustering on the output result of the Haar-Like filter, the person ( A region corresponding to a person) is detected.

"Sho Ikemura, Shunsuke Kawai, Hironobu Fujiyoshi, Human detection by Haar-Like filtering using distance information, 16th Image Sensing Symposium (SSII2010)"

  However, in the technique disclosed in Non-Patent Document 1, it is assumed that the distance image sensor images the person from directly above the person. There is a concern that erroneous detection or non-detection of a person may occur.

  Therefore, it is desirable to provide a mechanism that enables more accurate detection of a person from a distance image even when the person is not captured from directly above.

  According to the present invention, the acquisition unit that acquires the first distance image generated by the imaging device and the second distance image generated in advance by the imaging device by imaging the background, and the first distance image And a generation unit for generating a third distance image corresponding to the foreground with respect to the background based on the second distance image, and a plurality of filters applied at different positions in the third distance image. And a detection unit for detecting one or more regions from the third distance image using the plurality of filters capable of recognizing a human head and torso pattern according to the position to be applied. An information processing apparatus is provided.

  Each of the plurality of filters may be a filter generated based on a relative relationship between the imaging device and the person when the person is imaged by the imaging device.

  The relative relationship may include a direction from the imaging device where the person's head exists and a direction from the imaging device where the person's torso exists.

  Further, each of the plurality of filters may be a filter generated based on characteristics of the imaging device.

  Further, the detection unit detects the one or more regions as regions corresponding to the target portion of the person, and the information processing device corresponds to each of at least one region of the one or more regions. And a region determination unit that determines whether each of the at least one region corresponds to the target portion based on the size of the region.

  In addition, the information processing device is a peripheral region of each of at least one of the one or more regions based on the first distance image or the third distance image, and a predetermined condition A correction unit that corrects the one or more regions by adding the peripheral region that satisfies the condition to each of the at least one region, and the region determination unit includes the corrected at least one region It may be determined whether each of the at least one region corresponds to the target portion based on a size corresponding to each of the regions.

  The target portion may be a human head.

  In addition, the information processing apparatus identifies a high possibility region that is highly likely to be determined to be a region corresponding to the target portion, and other regions out of the one or more regions. The at least one region of the one or more regions may be the other region of the one or more regions.

  The high possibility area may be an area located within a predetermined range in which an area corresponding to a person portion other than the target portion is difficult to detect.

  In addition, the high possibility region is the one region when the one or more regions include only one region, and when the one or more regions include two or more regions. A region that is separated from the other regions included in the one or more regions by a predetermined distance or more may be used.

  Further, the acquisition unit acquires a fourth distance image generated by the imaging device before the first distance image, and the generation unit includes the fourth distance image and the second distance image. And generating a fifth distance image corresponding to the foreground with respect to the background, and the detection unit detects another one or more regions from the fifth distance image using the plurality of filters. The identification unit identifies the high possibility region and the other region among the one or more other regions, and the region determination unit includes the one of the one or more other regions. It is determined whether each of the other regions corresponds to the target portion, and the identification unit includes the high possibility region and the region corresponding to the target portion among the one or more other regions. One of the one or more regions based on the determined region determined to be Serial may identify a high potential region and other regions.

  In addition, the identification unit may determine the one or more areas based on the position of the high possibility area and the determined area and the position of the one or more areas among the one or more other areas. Of these, the high possibility area and other areas may be identified.

  Further, the detection unit is a plurality of filter sets applied at different positions in the third distance image, and is capable of recognizing a human head and torso pattern according to the applied position. The one or more regions are detected using a plurality of filter sets, and each of the plurality of filter sets includes two or more filters capable of recognizing human head and torso patterns having different characteristics. May be included.

  In addition, the detection unit detects the plurality of regions by applying the two or more filters to the same position in the third distance image when using each of the plurality of filter sets. The region determination unit may determine whether each of the at least one region corresponds to the target portion based on a size corresponding to each of at least one region of the plurality of regions.

  The detection unit selects at least one filter from the two or more filters when using each of the plurality of filter sets, and applies the selected at least one filter. The above region may be detected.

  In addition, the detection unit is configured to extract the two or more filters from the two or more filters based on a person image estimated from a distance image generated by the imaging device or another distance image generated based on the distance image. At least one filter may be selected.

  Further, the information processing apparatus determines whether a region other than the one or more regions that should be detected may not be detected based on the one or more regions and the third distance image. An undetected determination unit may be further provided.

  Each of the plurality of filters may be a Haar-like filter.

  Each of the plurality of filters may be a filter capable of recognizing a person's head and shoulder pattern according to an applied position.

  Further, according to the present invention, the computer acquires the first distance image generated by the imaging device, and the second distance image generated in advance by the imaging device by imaging the background, Based on the first distance image and the second distance image, the generation unit generates a third distance image corresponding to the foreground with respect to the background, and is applied at different positions in the third distance image. Detection that detects one or more regions from the third distance image by using a plurality of filters that can recognize a human head and torso pattern according to an applied position. And a program for functioning as a part.

  In addition, according to the present invention, the first distance image generated by the imaging device and the second distance image generated in advance by the imaging device by imaging the background are acquired, and the first distance is acquired. Generating a third distance image corresponding to the foreground with respect to the background based on the image and the second distance image; and a plurality of filters applied at different positions in the third distance image. And detecting one or more regions from the third distance image using the plurality of filters capable of recognizing a human head and torso pattern according to an applied position. A processing method is provided.

  As described above, according to the present invention, it is possible to more accurately detect a person from a distance image even when the person is not imaged from directly above.

It is explanatory drawing for demonstrating the example of the conventional Haar-Like filter. It is explanatory drawing for demonstrating the example of the distance image to which the conventional Haar-Like filter is applied. It is explanatory drawing for demonstrating the example of the distance image which may generate | occur | produce a misdetection when the conventional Haar-Like filter is applied. It is explanatory drawing which shows an example of schematic structure of the information processing system which concerns on embodiment of this invention. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 1st Embodiment. It is explanatory drawing for demonstrating an example of the foreground distance image produced | generated. It is an explanatory view for explaining an example of one filter among a plurality of filters concerning an embodiment of this indication. It is explanatory drawing for demonstrating the example of the object distance image produced | generated by an imaging device. It is explanatory drawing for demonstrating the example of the foreground distance image obtained from the object distance image shown by FIG. It is explanatory drawing for demonstrating the example of the filter used for the detection of the object area | region in a foreground distance image. It is explanatory drawing for demonstrating the example of the pixel count of the distance image produced | generated by an imaging device. It is explanatory drawing for demonstrating the angle of view of an imaging device. It is explanatory drawing for demonstrating the example of the method of calculating the range of the imaging angle corresponding to a person's head. It is explanatory drawing for demonstrating the example of the method of calculating the range of the imaging angle corresponding to a person's shoulder. It is explanatory drawing for demonstrating the example of the detection of the object area | region using the some filter which concerns on embodiment of this invention. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 1st Embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the 1st example of correction | amendment of the object area | region which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the 2nd example of correction | amendment of the object area | region which concerns on 2nd Embodiment. It is explanatory drawing for demonstrating the example of the method of determining whether an object area | region is an area | region corresponding to a person's head. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 2nd Embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 3rd Embodiment. It is explanatory drawing for demonstrating an example of the predetermined range where the area | region corresponding to the parts of persons other than a target part is hard to be detected. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 3rd Embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 4th Embodiment. It is explanatory drawing for demonstrating the example of identification with the high possibility area | region and other area | region which concerns on 4th Embodiment. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 4th Embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 5th Embodiment. It is explanatory drawing for demonstrating the example of the filter contained in a filter set. It is explanatory drawing for demonstrating the example of the object area | region detected by applying the 2 or more filter contained in a filter set. It is explanatory drawing for demonstrating the example of the determination about the several object area | region detected by applying two or more filters contained in a filter set to the same position. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 5th Embodiment. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on the modification of 5th Embodiment. It is a block diagram which shows an example of a structure of the information processing apparatus which concerns on 6th Embodiment. It is explanatory drawing for demonstrating an example of the detection result of a foreground area | region. It is explanatory drawing for demonstrating the example of determination of whether the area | region has not been detected. It is a flowchart which shows an example of the schematic flow of the information processing which concerns on 6th Embodiment. It is a block diagram showing an example of hardware constitutions of an information processor concerning an embodiment of this indication.

  Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. In addition, in this specification and drawing, about the component which has the substantially same function structure, duplication description is abbreviate | omitted by attaching | subjecting the same code | symbol.

  Thereafter, << 1. Introduction >>, << 2. Schematic configuration of information processing system >>, << 3. First Embodiment >>, << 4. Second Embodiment >>, << 5. Third Embodiment >>, << 6. Fourth Embodiment >>, << 7. Fifth embodiment >>, << 8. Embodiments of the present invention will be described in the order of the sixth embodiment >>.

<< 1. Introduction >>
First, a conventional Haar-Like filter and technical problems will be described with reference to FIGS.

(Conventional Haar-Like filter)
Non-Patent Documents “Sho Ikemura, Shunsuke Kawai, Hironobu Fujiyoshi, Human Detection by Haar-Like Filtering Using Distance Information, 16th Image Sensing Symposium (SSII2010)”, distance installed from ceiling to floor A technique for detecting a person from a distance image generated by an image sensor is disclosed.

  More specifically, according to the technique, an image of only the foreground portion is generated from the generated distance image by background difference. Then, by applying a Haar-Like filter capable of recognizing the convex shape of the person's head and both shoulders to the generated image and performing clustering on the output result of the Haar-Like filter, the person ( A region corresponding to a person) is detected. Hereinafter, specific contents of this point will be described with reference to FIGS. 1 and 2.

  FIG. 1 is an explanatory diagram for explaining an example of a conventional Haar-Like filter. Referring to FIG. 1, a Haar-Like filter 10 disclosed in the above non-patent document is shown. The Haar-Like filter 10 is a filter for recognizing the shoulder, the head, and the convex shape of the shoulder, and includes one rectangle 11 and two other types of rectangles 13 (that is, a rectangle 13A and a rectangle 13B). The Haar-Like filter 10 is applied pixel by pixel in the foreground pixel, and outputs 1 when the applied pixel corresponds to the convex head and 0 otherwise. Specifically, when the Haar-Like filter 10 is applied to the target pixel, the Haar-Like filter 10 is arranged so that the target pixel is located at the center in the distance image after the background difference. Then, for example, the difference between the sum of the distances of the foreground pixels included in the rectangle 11 and half of the sum of the distances of the foreground pixels included in the rectangles 13A and 13B is calculated. Here, the Haar-Like filter 10 compares the difference with a predetermined threshold value, and outputs one of 1 and 0 based on the comparison result. In this way, if the pixels for which 1 is output by the Haar-Like filter 10 are clustered, a region corresponding to the human head is detected. The Haar-Like filter 10 is used by being inclined in four directions of 0 °, 45 °, 90 °, and 135 ° for each target pixel.

  FIG. 2 is an explanatory diagram for explaining an example of a distance image to which a conventional Haar-Like filter is applied. Referring to FIG. 2, a distance image 20A is shown. The distance image 20 includes a region 21A and a region 21B corresponding to a person. The distance image 20A is a distance image generated by the distance image sensor by imaging a person from directly above. By performing background difference processing on such a distance image 20A and then applying the Haar-Like filter 10, a region corresponding to the head of each region 21 corresponding to a person can be detected.

(Technical issues)
However, the technology disclosed in the non-patent literature has technical problems. Specifically, in the technique, since it is assumed that the distance image sensor captures the person from directly above the person, if the person is not captured from directly above, There is a concern that non-detection may occur. For example, when a person is imaged from an oblique direction, an area corresponding to a part other than the person's head may be detected in addition to the area corresponding to the person's head. That is, even if there is only one person, two or more persons (two areas corresponding to the persons) can be detected. Hereinafter, a specific example of such a distance image will be described with reference to FIG.

  FIG. 3 is an explanatory diagram for explaining an example of a distance image in which erroneous detection may occur when a conventional Haar-Like filter is applied. Referring to FIG. 3, a distance image 20B is shown. The distance image 20 includes a region 21C and a region 21D corresponding to a person. The distance image 20B is a distance image generated by the distance image sensor by imaging a person from an oblique direction. In this example, a person is positioned near the end of the imaging range of the distance image sensor, and as a result, the person is imaged from an oblique direction. When background difference processing is performed on such a distance image 20B and then the Haar-Like filter 10 is applied, in addition to the region corresponding to the head of each region 21 corresponding to the person, the portion other than the head of the person An area corresponding to (for example, a part of a torso such as a shoulder or a back) can also be detected. That is, even if there is only one person, two or more persons (two areas corresponding to the persons) can be detected.

  As described above, there is a concern that erroneous detection may occur due to the application of the Haar-Like filter 10 when a person is positioned near the end of the imaging range of the distance image sensor. In particular, when the distance image sensor is installed at a lower position, the possibility of such erroneous detection increases. This is because a person is more likely to be captured from an oblique direction by the distance image sensor. Of course, when the distance image sensor is installed not in the downward direction but in the oblique direction, the possibility of the above-described erroneous detection increases.

  Therefore, in the embodiment of the present invention, it is possible to more accurately detect a person from a distance image even when the person is not captured from directly above. Hereinafter, specific contents of the embodiment of the present invention will be described.

<< 2. Schematic configuration of information processing system >>
With reference to FIG. 4, a schematic configuration of the information processing system 1 according to the embodiment of the present invention will be described. FIG. 4 is an explanatory diagram showing an example of a schematic configuration of the information processing system 1 according to the embodiment of the present invention. Referring to FIG. 4, the information processing system 1 includes an imaging device 30 and an information processing device 100.

  The imaging device 30 generates a distance image. The imaging device 30 is a distance image sensor, for example. More specifically, for example, the imaging device 30 is a TOF (Time-of-Flight) range image sensor. Each pixel value of the distance image is the distance from the imaging device 30 to the subject or the three-dimensional coordinates of the subject when the imaging device 30 is the origin.

  For example, the imaging device 30 is installed on the ceiling from the ceiling toward the floor (that is, directly below).

  For example, the imaging device 30 captures an image of the person 40 when the person 40 is located within the imaging range. As a result, a distance image including an area corresponding to the person 40 is generated by the imaging device 30.

  The information processing apparatus 100 detects a person from the distance image. The information processing apparatus 100 is a server, for example.

  For example, the information processing apparatus 100 communicates with the imaging apparatus 30 via the network 50 and acquires a distance image generated by the imaging apparatus 30. Then, the information processing apparatus 100 detects, for example, the person 40 from the acquired distance image.

  Hereinafter, on the premise of such an information processing system 1, first to sixth embodiments of the present invention will be described.

<< 3. First Embodiment >>
First, a first embodiment of the present invention will be described with reference to FIGS. According to the first embodiment of the present invention, a plurality of filters applied at different positions in the distance image after the background difference are used. This makes it possible to detect the person more accurately from the distance image even when the person is not imaged from directly above.

<3-1. Configuration of information processing apparatus>
An example of the configuration of the information processing apparatus according to the first embodiment will be described with reference to FIGS. FIG. 5 is a block diagram illustrating an example of the configuration of the information processing apparatus 100-1 according to the first embodiment. Referring to FIG. 5, the information processing apparatus 100-1 includes a communication unit 110, a storage unit 120, and a control unit 130.

(Communication unit 110)
The communication unit 110 communicates with other devices. For example, the communication unit 110 communicates with the imaging device 30 via the network 50.

  For example, the communication unit 110 receives a distance image generated by the imaging device 30. Then, the communication unit 110 provides the distance image to the control unit 130 (distance image acquisition unit 131).

(Storage unit 120)
The storage unit 120 stores data to be temporarily or permanently stored in the information processing apparatus 100-1.

  For example, the storage unit 120 stores a second distance image (hereinafter referred to as “background distance image”) generated in advance by the imaging device 30 by imaging the background.

  For example, the storage unit 120 stores information on characteristics of the imaging device 30. For example, the information on the characteristic includes information such as the number of pixels of the distance image 20 generated by the imaging device 30 and the angle of view of the imaging device 30.

  For example, the storage unit 120 stores a filter for detecting a region. The filter will be described later.

  For example, the storage unit 120 includes information regarding the installation state of the imaging device 30. The information regarding the installation state includes, for example, information on the height at which the imaging device 30 is installed. As an example, the height is a height from the floor.

  For example, the storage unit 120 includes information related to a person assumed to be imaged by the imaging device 30. For example, the information includes information such as the height, head width, and shoulder width of a person assumed to be imaged.

(Control unit 130)
The control unit 130 provides various functions of the information processing apparatus 100-1. The control unit 130 includes a distance image acquisition unit 131, a foreground distance image generation unit 133, and an area detection unit 135.

(Distance image acquisition unit 131)
The distance image acquisition unit 131 acquires a first distance image (hereinafter referred to as “target processed image”) generated by the imaging device 30. For example, when the communication unit 110 receives a distance image generated by the imaging device 30, the distance image acquisition unit 131 acquires the distance image.

  The distance image acquisition unit 131 acquires a background distance image. For example, the distance image acquisition unit 131 acquires the background distance image stored in the storage unit 120 from the storage unit 120.

  The distance image acquisition unit 131 provides the acquired target distance image and background distance image to the foreground distance image generation unit 133. For example, the distance image acquisition unit 131 stores the acquired target distance image in the storage unit 120.

(Foreground distance image generation unit 133)
The foreground distance image generation unit 133 generates a third distance image corresponding to the foreground with respect to the background (hereinafter referred to as “foreground distance image”) based on the target distance image and the background distance image.

  For example, when the pixel value at the coordinates (x, y) of the target distance image I is I (x, y) and the pixel value at the coordinates (x, y) of the background distance image B is B (x, y), The foreground distance image is expressed by the following equation.

  That is, the foreground distance image generation unit 133 calculates the difference between the pixel value I (x, y) of the target distance image I and the pixel value of the background distance image B as B (x, y), and the magnitude of the difference Is greater than the threshold th. Then, the foreground distance image generation unit 133 maintains the pixel value of the pixel in which the magnitude of the difference exceeds the threshold th as the pixel value I (x, y), and the pixel of which the magnitude of the difference is the threshold th Set the pixel value to 0. Accordingly, the foreground distance image generation unit 133 generates a foreground distance image corresponding to the foreground in the target distance image I. Hereinafter, a specific example of the foreground distance image will be described with reference to FIG.

  FIG. 6 is an explanatory diagram for explaining an example of the generated foreground distance image. Referring to FIG. 6, a foreground distance image 60 is shown. In the foreground distance image 60, the pixel values of the pixels in the foreground area 61A and the foreground area 61B corresponding to the foreground are maintained as the pixel values of the target distance image, and the pixel values of the other pixels are 0.

  Note that the foreground distance image may also be referred to as a background difference image.

(Area detection unit 135)
The area detection unit 135 includes a plurality of filters applied at different positions in the foreground distance image, and the plurality of filters capable of recognizing a human head and torso pattern according to the applied position. And one or more regions (hereinafter referred to as “object regions”) are detected from the foreground distance image.

-Filter shape For example, the region detection unit 135 detects one or more object regions as regions corresponding to the target portion of a person. For example, the target portion is a human head. That is, the area detection unit 135 detects one or more object areas as an area corresponding to a person's head.

  For example, each of the plurality of filters is a Haar-Like filter. However, each of the plurality of filters has a shape different from that of the Haar-Like filter 10 shown in FIG. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 7 is an explanatory diagram for describing an example of one of the plurality of filters according to the embodiment of the present disclosure. Referring to FIG. 7, a filter 70 according to an embodiment of the present invention is shown. The filter 70 is a Haar-Like filter. The filter 70 is a filter for recognizing a pattern of a person's head and torso (for example, shoulder, back, chest, etc.), and includes a smaller rectangle 71 and a larger rectangle 73. The filter 70 is applied pixel by pixel in the foreground area 61 of the foreground distance image 60, and outputs 1 when the applied pixel corresponds to the head in the pattern, and 0 otherwise. Specifically, when the filter 70 is applied to the target pixel, the filter 70 is arranged so that the target pixel is located at the center of the rectangle 71 in the foreground distance image 60. Then, for example, by subtracting the average height corresponding to the foreground pixels included in the rectangle 73 from the average height corresponding to the foreground pixels included in the rectangle 71, the difference in the average height is calculated. Is done. Here, the filter 70 outputs 1 when the difference exceeds a predetermined threshold, and outputs 0 otherwise. In this way, if the pixels for which 1 is output by the filter 70 are clustered, a region corresponding to the person's head is detected.

  Note that the height corresponding to each pixel is, for example, the height of the subject corresponding to each pixel from the floor. Then, the height can be calculated based on the pixel value (that is, distance) of each pixel, the installation state of the imaging device 30, and information on the characteristics of the imaging device 30. Hereinafter, a specific method for calculating the height of the subject corresponding to a certain pixel of interest will be described.

First, from the angle of view of the imaging device 30 and the number of pixels of the distance image generated by the imaging device 30, an imaging angle θ _X corresponding to the pixel of interest (for example, an angle with respect to the downward direction of the imaging device 30) is specified. it can. Further, the distance d _X from the imaging device 30 to the subject corresponding to the target pixel (that is, the subject in the direction of the imaging angle) is obtained from the pixel value of the target pixel. Therefore, by multiplying the cosine (cos θ _X ) corresponding to the imaging angle θ _X by the distance d _X , the distance in the height direction between the subject and the imaging device 30 is calculated. For example, since the height D at which the imaging device 30 is installed is stored in the storage unit 120, the height direction between the subject and the imaging device 30 is determined from the height D at which the imaging device 30 is installed. The height D _{X of the} subject (that is, the height corresponding to the target pixel) is calculated by subtracting the distance. That is, the height D _X corresponding to the target pixel is expressed as follows.

-Multiple filters In particular in the first embodiment, multiple filters are used that are applied at different positions in the foreground distance image.

  As described above, the imaging device 30 does not always capture the person from directly above the person. For example, when a person is located near the end of the imaging range of the imaging device 30, the person is imaged from an oblique direction. Hereinafter, a specific example of this point will be described with reference to FIGS.

  FIG. 8 is an explanatory diagram for describing an example of a target distance image generated by the imaging device 30. Referring to FIG. 8, a target distance image 20 is shown. The target distance image 20 is a distance image generated when a person is located at nine locations within the imaging range of the imaging device 30, and includes nine foreground areas 21A to 21I corresponding to these persons. For example, the foreground area 21E is an area corresponding to a person located directly below the imaging device 30. On the other hand, each of the foreground areas 21 other than the foreground area 21E is an area corresponding to a person located near the end of the imaging range of the imaging device 30. As described above, the foreground area 21E includes areas corresponding to the head and shoulders of the person, but the other foreground areas 21E include portions other than the person's head and shoulders (for example, the back, chest, etc.). ) Is also included.

  FIG. 9 is an explanatory diagram for explaining an example of the foreground distance image obtained from the target distance image shown in FIG. Referring to FIG. 9, a foreground distance image 60 is shown. The foreground distance image 60 includes nine foreground areas 61A to 61I corresponding to the nine foreground areas 21A to 21I. As described above, in the foreground distance image 60, the pixel value of each foreground region 61 corresponding to the foreground is maintained as the pixel value of the target distance image 20 (that is, the pixel value of each pixel of the foreground region 21). The pixel values of other pixels are 0.

  As described above, for example, the target distance image 20 as shown in FIG. 8 is acquired, and the foreground distance image 60 as shown in FIG. 9 is generated. In such an example, when a filter (for example, the Haar-Like filter 10 shown in FIG. 1) on the premise that a person is imaged from directly above is used, the other foreground areas 61 ( That is, in the foreground areas 61A to 61D and 61F to 61I), in addition to the area corresponding to the person's head, areas corresponding to other parts can be erroneously detected. For example, a region corresponding to a person's shoulder, back, or chest may be erroneously detected. That is, even if there is only one person, two or more persons (two areas corresponding to the persons) can be detected. Therefore, in the embodiment of the present invention, a plurality of filters applied at different positions in the foreground distance image 60 are used. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 10 is an explanatory diagram for describing an example of a filter used for detecting an object region in a foreground distance image. Referring to FIG. 10, nine filters 70A-70I are shown. Thus, in the embodiment of the present invention, different filters 70 are applied depending on the position in the foreground distance image 60. As an example, a separate filter 70 is applied to each pixel to which the filter 70 is applied. The filter 70 can recognize the pattern of the person's head and torso according to the applied position. For example, the filter 70 has a shape corresponding to the position to be applied in order to recognize the pattern of the head and torso of the person according to the position to be applied.

-Filter Generation Method Next, the filter 70 generation method will be described.

-Information to be used (characteristics of imaging device)
For example, each of the plurality of filters is generated further based on the characteristics of the imaging device 30.

For example, the characteristic of the imaging device 30 includes the number of pixels (N _h , N _v ) of the distance image generated by the imaging device 30. The number of pixels _Nh is the number of pixels in the horizontal direction of the distance image, and the number of pixels _Nv is the number of pixels in the vertical direction of the distance image. Hereinafter, a specific example of this point will be described with reference to FIG.

FIG. 11 is an explanatory diagram for explaining an example of the number of pixels of the distance image generated by the imaging device. Referring to FIG. 11, the target distance image 20 generated by the imaging device 30 is shown as in FIG. 3. As shown in FIG. 11, the target distance image 20 is accompanied by the number of pixels N _{h in} the horizontal direction and the number of pixels N _v in the vertical direction. Since the target distance image 20 and the foreground distance image 60 have the same number of pixels, the number of pixels N _h and the number of pixels N _v can also be said to be the number of pixels of the foreground distance image 60.

For example, the characteristic of the imaging device 30 includes the angle of view (Θ _h , Θ _v ) of the imaging device 30. The field angle Θ _h is a field angle corresponding to the horizontal direction of the target distance image 20, and the field angle Θ _v is a field angle corresponding to the vertical direction of the target distance image 20. Hereinafter, a specific example of this point will be described with reference to FIG.

FIG. 12 is an explanatory diagram for explaining the angle of view of the imaging apparatus. Referring to FIG. 12, an imaging device 30 and its angle of view Θ are shown. The angle of view Θ is the angle of view Θ _h corresponding to the horizontal direction of the target distance image 20 or the angle of view Θ _v corresponding to the vertical direction of the target distance image 20. The imaging device 30 captures an imaging range corresponding to such an angle of view Θ.

-Information to be used (relative relationship between imaging device and person)
For example, each of the plurality of filters 70 is generated based on the relative relationship between the imaging device 30 and the person when the person is imaged by the imaging device 30.

  Specifically, for example, each of the plurality of filters 70 is applied to a separate pixel. That is, a pixel-unit filter 70 is generated. In this case, the relative relationship when generating the filter 70 applied to a certain pixel is that the person is assumed to exist when the center of the certain pixel corresponds to the center of the person's head. And the imaging device 30.

  Further, for example, the relative relationship includes a direction from the imaging device 30 in which the person's head is present and a direction from the imaging device 30 in which the person's torso is present. For example, each of the plurality of filters is a filter capable of recognizing a person's head and shoulder pattern according to an applied position. In this case, the relative relationship includes a direction from the imaging device 30 in which the person's head exists and a direction from the imaging device 30 in which the person's shoulder exists.

  As an example, the direction from the imaging device 30 in which a person's head exists is a range of imaging angles corresponding to the person's head. Further, the direction from the imaging device 30 where the person's shoulder exists is the range of the imaging angle corresponding to the person's shoulder.

  Specifically, for example, the above relative relationship when generating the filter 70 applied to a certain pixel is based on the assumption that a person exists such that the center of the certain pixel corresponds to the center of the head of the person. A range of imaging angles corresponding to the head of the person and a range of imaging angles corresponding to the shoulder of the person. Hereinafter, a specific example of this point will be described with reference to FIGS. 13A and 13B.

FIG. 13A is an explanatory diagram for describing an example of a technique for calculating a range of imaging angles corresponding to a person's head. Referring to FIG. 13A, an imaging device 30 and a person 40 are shown. Head width of the person 40 is a _{W H,} width of the shoulders of the person 40 is a _{W S.} The distance in the height direction of the head of the image pickup device 30 and the person 40 are D _H, the distance in the height direction between the shoulder of the imaging device 30 and the person 40 is D _S. Further, in consideration of the assumption that the pixel to which the filter 70 is applied corresponds to the center of the head of the person 40, it is based on the number of pixels of the target distance image 20 generated by the imaging device 30 and the angle of view of the imaging device 30. Thus, the imaging angle θ _H at the center of the head corresponding to each pixel of the target distance image can be calculated. Then, for example, the head of the person 40 using the imaging angle θ _{H at} the center of the head, the head width W _{H of} the person 40, and the distance _DH in the height direction between the imaging device 30 and the head of the person 40. The imaging angle range θ _{H ±} corresponding to the part can be calculated as follows.

FIG. 13B is an explanatory diagram for describing an example of a method for calculating a range of imaging angles corresponding to a person's shoulder. Referring to FIG. 13B, similarly to FIG. 13A, the shoulder width W _{S of} the person 40 and the distance D _S in the height direction between the imaging device 30 and the shoulder of the person 40 are shown. The head width W _{H of} the person 40, the distance D _H in the height direction between the imaging device 30 and the head of the person 40, and the imaging angle θ _H of the center of the head are not shown. It is the same as 13A. From the imaging angle θ _{H at} the center of the head, the distance D _H, and the distance D _S , the imaging angle θ _S at the center of the shoulder can be calculated. Then, for example, using the imaging angle θ _{S at} the center of the shoulder, the shoulder width W _{S of} the person 40, and the distance D _S in the height direction between the imaging device 30 and the shoulder of the person 40, it corresponds to the shoulder of the person 40. The imaging angle range θ _{S ±} can be calculated as follows.

  For example, as described above, the relative relationship between the imaging device 30 and the person when the person is imaged by the imaging device 30 is calculated.

In addition, the calculation method mentioned above is the range of the imaging angle about one direction. Therefore, as the above-described relative relationship, a range of imaging angles corresponding to the vertical direction of the target distance image 20 and a range of imaging angles corresponding to the horizontal direction of the target distance image 20 are calculated. That is, the horizontal and vertical imaging angle ranges corresponding to the person's head (θ _{Hh ±} , θ _{Hv ±} ) and the horizontal and vertical imaging angle ranges corresponding to the person's shoulder (θ _{Sh ±} , θ _{Sv ±} ) is calculated.

-Generation of Filter As described with reference to FIG. 7, for example, the filter 70 includes a rectangle 71 and a rectangle 73. The horizontal and vertical pixel ranges (P _{Hh ±} , P _{Hv ±} ) of the rectangle 71 corresponding to the head are the horizontal and vertical imaging angle ranges (θ _{Hh ±} , θ _{Hv ±} ), the number of pixels of the target distance image 20 (N _h , N _v ), and the angle of view (Θ _h , Θ _v ) of the imaging device 30 are calculated as follows. The following pixel range is expressed as pixel coordinates when the center of the target distance image 20 is the origin. In addition, in consideration of the characteristics of the distance image, it is assumed that the coordinates of the pixel are proportional to the imaging angle.

Similarly, the horizontal and vertical pixel ranges (P _{Sh ±} , P _{Sv ±} ) of the rectangle 73 corresponding to the shoulder are calculated as follows.

  As described above, for example, the filter 70 in units of pixels is generated. For example, the filter 70 in units of pixels is generated in advance and stored in the storage unit 120, for example. Then, it is acquired when the filter 70 is applied to each pixel.

-Example of detection of object region using a plurality of filters An example of detection of an object region using the plurality of filters will be described with reference to Fig. 14. FIG. 14 is an explanatory diagram for explaining an example of object region detection using a plurality of filters according to the embodiment of the present invention. Referring to FIG. 14, object areas 81A and 81B detected when the object area is detected from the foreground distance image 60 shown in FIG. 6 and a binary image 80 including only the object areas 81A and 81B are shown. Has been. In the binary image 80, the pixel value at the pixel where 1 is output by the filter 70 is 1, and the pixel values at the other pixels are 0. That is, each of the object areas 81A and 81B is a set of pixels having a pixel value of 1. In this way, the regions corresponding to the person's head are detected as the individual object regions 81A and 81B.

  Note that the area detection unit 135 performs labeling on the pixels having the pixel value 1 among the pixels included in the binary image 80, thereby causing the individual object areas 81 (for example, the object area 81A and the object area 81B) to be labeled. Is detected. For example, the region detection unit 135 adds the same label (for example, a number) to pixels that are continuous in the vicinity of eight diagonally up, down, left and right. As a result, a set of pixels having the same label becomes individual object regions 81A and 81B.

<3-2. Process flow>
Next, an example of information processing according to the first embodiment will be described with reference to FIG. FIG. 15 is a flowchart illustrating an example of a schematic flow of information processing according to the first embodiment.

  In step S <b> 310, the distance image acquisition unit 131 acquires the background distance image stored in the storage unit 120 from the storage unit 120. In step S320, the distance image acquisition unit 131 acquires the target distance image 20 generated by the imaging device 30.

  Next, in step S330, the foreground distance image generation unit 133 generates a foreground distance image based on the target distance image and the background distance image.

  In step S <b> 340, the region detection unit 135 detects one or more object regions from the foreground distance image using the plurality of filters 70. Then, the process ends.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, the process may return to step S320 after step S340.

  The first embodiment of the present invention has been described above. According to the first embodiment, a plurality of filters applied at different positions in the foreground distance image, the plurality of filters capable of recognizing a human head and torso pattern according to the applied position. A filter is used. This makes it possible to detect the person more accurately from the distance image even when the person is not imaged from directly above.

  More specifically, for example, even when the person 40 is located near the end of the imaging range of the imaging device 30, it corresponds to a part other than the person's head (eg, shoulder, back, chest, etc.). The possibility that the area to be detected is erroneously detected is reduced. In other words, the possibility of detecting two or more persons (two areas corresponding to the persons) even though there is only one person is reduced, so that erroneous detection of persons can be reduced.

  Furthermore, the use of the plurality of filters 70 increases the degree of freedom regarding the installation of the imaging device 30. As an example, even when the imaging device can be installed only at a low position (for example, when the ceiling is low), the imaging device 30 is installed at the low position, and a person is detected from the distance image generated by the imaging device 30. It becomes possible to detect more accurately. As another example, it is also possible to install the imaging device 30 in an oblique direction without installing the imaging device 30 directly below. Also in this case, the filter 70 is generated based on the installation state of the imaging device 30.

  According to the first embodiment, for example, the filter 70 is generated based on the relative relationship between the imaging device 30 and the person 40 when the imaging device 30 captures the person 40. Thereby, if the information on the relative relationship is acquired, the filter 70 can be automatically generated. Therefore, it is possible to reduce the trouble of changing the installation state of the imaging device 30 or newly installing it.

  Further, according to the first embodiment, for example, the filter 70 is generated further based on the characteristics of the imaging device 30. Thereby, even if the imaging device 30 having various specifications is used, the filter 70 can be automatically generated. Therefore, it is possible to reduce the trouble of changing the installation state of the imaging device 30 or newly installing it.

<< 4. Second Embodiment >>
Subsequently, a second embodiment of the present invention will be described with reference to FIGS. According to the second embodiment of the present invention, it is determined whether the area detected using the plurality of filters 70 corresponds to the target portion (that is, the head) of the person based on the size of the area. The This makes it possible to more accurately detect a person from a distance image.

<4-1. Configuration of information processing apparatus>
An example of the configuration of the information processing apparatus according to the second embodiment will be described with reference to FIGS. FIG. 16 is a block diagram illustrating an example of the configuration of the information processing apparatus 100-2 according to the second embodiment. Referring to FIG. 16, the information processing apparatus 100-2 includes a communication unit 110, a storage unit 120, and a control unit 140.

  Here, the communication unit 110, the storage unit 120, and the distance image acquisition unit 131, the foreground distance image generation unit 133, and the region detection unit 135 included in the control unit 140 are the first embodiment and the second embodiment. There is no difference between Therefore, only the region correction unit 141 and the region determination unit 143 included in the control unit 140 will be described.

(Region Correction Unit 141)
The region correction unit 141 is a peripheral region of each of at least one object region 81 among the one or more object regions 81 detected based on the target distance image 20 or the foreground distance image 60, and has a predetermined condition. The one or more object regions 81 are corrected by adding the peripheral region satisfying the condition to each of the at least one object region 81.

  For example, the area correction unit 141 corrects all of the detected one or more object areas 81. That is, the region correction unit 141 adds the peripheral region that satisfies each of the one or more object regions 81 that is a peripheral region of each of the one or more regions 81 to be detected that satisfies a predetermined condition. Thus, the one or more object regions 81 are corrected.

  Further, for example, the region correction unit 141 selects a pixel corresponding to a certain height or higher among pixels having a pixel value of 0 located around the object region 81 (hereinafter referred to as “peripheral pixel”). The object area 81 is added. That is, the area correction unit 141 sets the pixel value of the pixel corresponding to a certain height or higher to 1.

More specifically, for example, first, the region correction unit 141 sets a threshold value for each object region 81. As an example, as the threshold value, the difference from the average value of the height corresponding to the pixels of each object region 81, the height of the head is assumed (i.e., the distance D _S and the distance D _H shown in FIG. 12A ) Is subtracted. Then, the region correction unit 141 compares the height corresponding to each peripheral pixel with the set threshold value, and adds the peripheral pixel whose height exceeds the threshold value to the object region 81. As an example, the area correction section 141, the pixels located within a distance of 2 times the head width W _H from center of the object region 81 is selected as peripheral pixels.

  For example, the object region 81 is corrected as described above. Hereinafter, a specific example of the correction of the object region 81 will be described with reference to FIGS. 17 and 18.

  FIG. 17 is an explanatory diagram for describing a first example of correction of the object region 81 according to the second embodiment. Referring to FIG. 17, a binary image 80 and object regions 81A and 81B before correction, and a binary image 80 and object regions 81A and 81B after correction are shown. Each of the object areas 81A and 81B before correction does not include a part of the area corresponding to the head. Among the regions corresponding to the head, such as the corrected object regions 81A and 81B, by adding peripheral pixels satisfying a predetermined condition that are peripheral pixels of the object region 81A and the object region 81B. The area that was not included before the correction is added.

  FIG. 18 is an explanatory diagram for describing a second example of the correction of the object region 81 according to the second embodiment. Referring to FIG. 18, unlike the example of FIG. 17, an object region 81C is further detected. The object region 81C is a region corresponding to a part of a person's shoulder, for example. As described above, even when the filter 70 is used, if the object region 81C is detected, the object region 81C is also corrected in the same manner as the object region 81A and the object region 81B.

  Note that the method of correcting the object region 81 is not limited to the above-described example. For example, the binary image 80 may be expanded by referring to the target distance image 20 or the foreground distance image 60. Specifically, for example, for a target pixel having a pixel value of 0 in the binary image 80, it may be determined whether the object region 81 is in the vicinity of the target pixel. If the object region 81 is in the vicinity of the target pixel, the height corresponding to the target pixel may be calculated with reference to the target distance image 20 or the foreground distance image 60. As a result, when the calculated height exceeds the threshold, the target pixel may be added to the nearby object region 81.

(Area determination unit 143)
Based on the size corresponding to each of at least one of the one or more object areas 81 to be detected, the area determination unit 143 determines that each of the at least one object area 81 is a target portion of a person. It is determined whether the area corresponds to.

  For example, the region determination unit 143 determines whether each of the at least one object region 81 corresponds to a target portion of a person based on the corrected size corresponding to each of the at least one object region 81. To do.

  More specifically, for example, the target portion is a person's head, and each of the detected one or more regions 81 is corrected. That is, the region determination unit 143 determines whether each of the one or more object regions 81 is a region corresponding to the head based on the corrected size corresponding to each of the one or more object regions 81. .

-Calculation of size of object region For example, the region determination unit 143 calculates the size of the object region 81 using the moment feature of the binary image. By using the moment feature, an ellipse (that is, an inertia equivalent ellipse) in which the center of gravity and the second moment around the center of gravity are equal to those of the object region 81 is calculated. That is, the major axis and minor axis lengths of the ellipse are calculated. Then, the region determination unit 143 regards the size of the ellipse as the size of the object region 81.

Specifically, for example, the region determination unit 143 performs the following process for each individual object region 81. First, the region determination unit 143 pays attention to one object region 81, sets the pixel value of the object region 81 of interest in the binary image J (x, y) to 1, and sets the other pixel values to 0. To do. Then, the region determination unit 143 calculates the zeroth moment M ₀₀ , the first moment M _10, and the second moment M ₂₀ of the object region 81 of interest as follows.

Then, the secondary moments U _xx , U _xy and U _yy around the center of gravity are calculated as follows.

Then, the major axis L _m and the minor axis L _n of the inertia equivalent ellipse are calculated as follows.

As described above, the region determination unit 143 calculates the lengths of the major axis L _m and the minor axis L _n of the ellipse as the size of the object region 81.

-Determining whether or not the object region corresponds to the head The region determining unit 143 determines that the object region 81 is determined if the size of the calculated object region 81 is within a predetermined size corresponding to the head. It is determined that the region corresponds to the head.

For example, the range of the predetermined size is a range in which a difference from the size of the rectangle 71 of the filter 70 is equal to or less than a predetermined value. That is, _assuming that the horizontal and vertical widths of the rectangle 71 are (W _Fh , W _Fv ) and the threshold is T, the object region 81 is peeled off when the major axis L _m and the minor axis L _n satisfy the following conditions. It is determined that the area corresponds to a person.

FIG. 19 is an explanatory diagram for explaining an example of a method for determining whether the object region 81 is a region corresponding to a person's head. Referring to FIG. 19, a rectangle 71 having widths W _Fh and W _Fv and a major axis L _m and a minor axis L _{n of the} calculated ellipse are shown. Thus, the major axis L _m and the minor axis L _n of the ellipse are compared with the widths W _Fh and W _Fv . If the major axis L _m and the minor axis L _n are not significantly different from the widths W _Fh and W _Fv , it is determined that the object region 81 is a region corresponding to the person's head.

  As described above, it is determined whether the object region 81 corresponds to the person's head. For example, even if an object region 81 such as the object region 81C shown in FIG. 18 is detected by such determination, such an object region 81C can be finally excluded.

In the determination, instead of the widths W _Fh and W _Fv of the rectangle 71, the height is estimated based on the height corresponding to the object region 81 (for example, the average value of the height corresponding to the pixels), A head width corresponding to the height may be used.

<4-2. Process flow>
Next, an example of information processing according to the second embodiment will be described with reference to FIG. FIG. 20 is a flowchart illustrating an example of a schematic flow of information processing according to the second embodiment.

  Here, steps S410 to S440 are the same as steps S310 to S340 according to the first embodiment described with reference to FIG. Therefore, only steps S450 and S460 will be described here.

  In step S450, the region correction unit 141 is a peripheral region of each of the one or more object regions 81 detected based on the target distance image 20 or the foreground distance image 60 and satisfies the predetermined condition. Is added to each of the one or more object regions 81 to correct the one or more object regions 81.

  In step S460, the region determination unit 143 determines the region in which each of the one or more object regions 81 corresponds to the head based on the corrected size corresponding to each of the one or more object regions 81. Determine whether. Then, the process ends.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, after step S460, the process may return to step S420.

  Heretofore, the second embodiment of the present invention has been described. According to the second embodiment, based on the size corresponding to the object region 81, it is determined whether the object region 81 corresponds to the head. This makes it possible to more accurately detect a person from a distance image.

  More specifically, for example, when a region corresponding to a portion other than the target portion of a person (for example, shoulder, back, chest, etc.) is still erroneously detected using the filter 70, the region Can eventually be excluded. In other words, the possibility of detecting two or more persons (two areas corresponding to the persons) even though there is only one person is reduced, so that erroneous detection of persons can be reduced.

  In addition, when an area corresponding to an object other than a person (for example, an animal, a transported object, etc.) is erroneously detected, the area can be finally excluded. That is, the possibility of detecting an object other than a person can be reduced.

  Further, according to the second embodiment, the object region 81 is corrected by adding the peripheral region. Thereby, even if the region corresponding to a part of the head is not included in the object region 81, the region can be included in the object region 81 afterwards. As a result, the object region 81 does not include a region corresponding to a part of the head even though the object region 81 corresponds to the target portion (for example, the head) of the person. Can be determined not to be an area corresponding to the head. Conversely, it is determined that the object region 81 is a region corresponding to the head, although the object region 81 corresponds to a portion other than the target portion of the person (for example, an animal, a transported item, etc.). The possibility of being lost can be reduced.

<< 5. Third Embodiment >>
Subsequently, a third embodiment of the present invention will be described with reference to FIGS. According to the third embodiment of the present invention, there is a high possibility that it is determined that the region corresponds to the target portion (for example, the head) of one or more object regions 81. Regions and other regions are identified. Then, it is determined whether the other region is a region corresponding to the target portion of the person. Thereby, it is possible to reduce processing until a person is finally detected.

<5-1. Configuration of information processing apparatus>
An example of the configuration of the information processing apparatus according to the third embodiment will be described with reference to FIGS. 21 and 22. FIG. 21 is a block diagram illustrating an example of the configuration of the information processing apparatus 100-3 according to the third embodiment. Referring to FIG. 21, the information processing apparatus 100-3 includes a communication unit 110, a storage unit 120, and a control unit 150.

  Here, the communication unit 110, the storage unit 120, and the distance image acquisition unit 131, the foreground distance image generation unit 133, and the region detection unit 135 included in the control unit 150 are the second embodiment and the third embodiment. There is no difference between Therefore, only the region identification unit 151, the region correction unit 153, and the region determination unit 155 included in the control unit 150 will be described.

(Area identification unit 151)
The region identifying unit 151 identifies a high possibility region that is highly likely to be determined to be a region corresponding to the target portion of the person, and other regions in the one or more object regions 81.

-1st example of high possibility area | region As a 1st example, the said high possibility area | region is the object area | region 81 located in the predetermined range in which the area | region corresponding to a person's part other than the said target part is hard to be detected. is there. For example, as shown in FIG. 4, the predetermined range is a range close to the center of the binary image 80 when the imaging device 30 is installed directly below. A specific example of this point will be described with reference to FIG.

  FIG. 22 is an explanatory diagram for explaining an example of a predetermined range in which a region corresponding to a person portion other than the target portion is difficult to be detected. Referring to FIG. 22, a binary image 80 and object areas 81D and 81E are shown. Further, a predetermined range 85 in which a region corresponding to a portion other than the target portion of the person (for example, shoulder, back, chest, etc.) is difficult to be detected is shown. Thus, the predetermined range 85 is a range close to the center of the binary image 80, for example. In this case, the area identifying unit 151 identifies the object area 81D as a high possibility area and identifies the object area 81E as another area.

  As described above, when the imaging device 30 is installed directly below, a region corresponding to a portion other than the person's head (for example, shoulder, back, chest, etc.) may be detected. Is expensive. Therefore, it is desirable to determine whether the object region 81 is a region corresponding to the head in a range near the end in the binary image 80. On the other hand, in the range close to the center in the binary image 80, there is a low possibility that a region corresponding to a portion other than the person's head will be detected. Therefore, as illustrated in FIG. 4, when the imaging device 30 is installed directly below, the region identification unit 151 displays an object region 81 located within a range close to the center of the binary image 80 as a high It can be identified as a potential area.

-2nd example of high possibility area | region As a 2nd example, when the one or more detected object area | regions 81 contain only one object area | region 81, the said high possibility area | region is the said 1 object This is a region 81. In addition, when the one or more object regions 81 include two or more object regions 81, the high possibility region is the same as any other object region 81 included in the one or more object regions 81. The object region 81 is separated by a predetermined distance or more.

  For example, when only one object region 81 is detected, the one object region 81 is identified as a high possibility region. When two or more object areas 81 are detected, an object area 81 that is not located in the vicinity of another object area 81 in the binary image 80 is identified as a high possibility area. The object area 81 where the other object area 81 is located in the vicinity is identified as the other area. Referring to the example shown in FIG. 18, for example, the object area 81B is identified as a high possibility area. On the other hand, the object region 81A and the object region 81C are identified as other regions.

  When an object region 81 corresponding to a target portion (for example, a head) of a person and an object region 81 corresponding to another portion (for example, shoulder, back, chest, etc.) corresponding to the person are detected. These object regions 81 are located in the vicinity of each other. Therefore, if there are object regions 81 located in the vicinity of each other, it is desirable to determine whether these object regions 81 are regions corresponding to the head. On the other hand, it can be estimated that the object region 81 in which the other object region 81 is not located in the vicinity is not a region corresponding to the other part. Further, even if the region corresponding to the target portion is not detected as the object region 81 and the region corresponding to the other portion is detected as the object region 81, two people are used even though there is only one person. The above person (two areas corresponding to the person) is not detected. Therefore, the area identification unit 151 can identify the object area 81 in which the other object area 81 is not located in the vicinity as the high possibility area.

(Region Correction Unit 153)
The area correction unit 153 is a peripheral area of each of at least one object area 81 among the one or more object areas 81 detected based on the target distance image 20 or the foreground distance image 60, and has a predetermined condition. The one or more object regions 81 are corrected by adding the peripheral region satisfying the condition to each of the at least one object region 81.

  In particular, in the third embodiment, the at least one region is an object region 81 other than the high possibility region among the one or more regions. That is, the area correction unit 153 corrects the other object area 81 other than the high possibility area. A specific correction method is as described in the second embodiment.

(Area determination unit 155)
The area determination unit 155 determines that each of the at least one object area 81 is a target portion of a person based on a size corresponding to each of the at least one object area 81 among the one or more object areas 81 to be detected. It is determined whether the area corresponds to.

  In particular, in the third embodiment, the at least one region is an object region 81 other than the high possibility region among the one or more regions. That is, the region determination unit 155 determines whether the other object region 81 other than the high possibility region is a region corresponding to a target portion (for example, a head) of a person. The specific determination method is as described in the second embodiment.

<5-2. Process flow>
Next, an example of information processing according to the third embodiment will be described with reference to FIG. FIG. 23 is a flowchart illustrating an example of a schematic flow of information processing according to the third embodiment.

  Here, steps S510 to S540 are the same as steps S410 to S440 (that is, steps S310 to S340 described with reference to FIG. 15) according to the second embodiment described with reference to FIG. Therefore, only steps S550 to S570 will be described here.

  In step S550, the area identifying unit 151 includes a high possibility area that is highly likely to be determined to be an area corresponding to a person's head among the corrected one or more object areas 81, and other areas. Identify the area.

  In step S560, the region correction unit 153 determines the peripheral region that satisfies the predetermined condition, which is the peripheral region of each of the other regions identified based on the target distance image 20 or the foreground distance image 60. Correcting other areas by adding to each of the other areas

  In step S570, the region determination unit 155 determines whether each of the other regions corresponds to the head based on the corrected size corresponding to each of the other regions. Then, the process ends.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, after step S570, the process may return to step S520.

  Heretofore, the third embodiment of the present invention has been described. According to the third embodiment, one of the one or more object areas 81 to be detected is identified from the other areas with the high possibility area. Then, it is determined whether the other area corresponds to the target portion of the person. Thereby, it is possible to reduce processing until a person is finally detected.

  More specifically, for example, it is possible to reduce the number of determination processing (and correction processing) targets. As a result, determination processing (and correction processing) can be reduced. In addition, the possibility of false detection is expected to hardly increase. That is, it is possible to reduce processing while maintaining accurate detection.

  Further, for example, the high possibility area is an object area 81 located within a predetermined range in which an area corresponding to a person part other than the target part is difficult to detect. Thereby, the target of the determination process (and the correction process) can be narrowed down to only the object region 81 located in a part of the range.

  Further, for example, the high possibility area is an object area 81 in which another object area 81 is not located in the vicinity. Thereby, the target of the determination process (and the correction process) may be limited to only an area that is highly likely to be an area corresponding to a part other than the target part (head) (for example, shoulder, back, chest, etc.). it can.

<< 6. Fourth Embodiment >>
Subsequently, a fourth embodiment of the present invention will be described with reference to FIGS. According to the fourth embodiment of the present invention, as in the third embodiment, it is determined that the region corresponds to the target portion (for example, the head) of one or more object regions 81. A high likelihood region that is likely to be identified and other regions are identified. Then, it is determined whether the other region is a region corresponding to the target portion of the person. In particular, in the fourth embodiment, when the target distance image 20 is continuously generated and the object region 81 is continuously detected, the current object region 81 is detected based on the previous detection result of the object region 81. Our high probability areas and other areas are identified. Thereby, when the object distance image 20 is continuously generated and the object region 81 is continuously detected, it is possible to reduce processing until the person is detected.

<6-1. Configuration of information processing apparatus>
With reference to FIGS. 24 and 25, an example of the configuration of the information processing apparatus according to the fourth embodiment will be described. FIG. 24 is a block diagram illustrating an example of a configuration of an information processing device 100-4 according to the fourth embodiment. Referring to FIG. 24, the information processing apparatus 100-4 includes a communication unit 110, a storage unit 120, and a control unit 160.

  Here, the communication unit 110, the storage unit 120, and the distance image acquisition unit 131, the foreground distance image generation unit 133, the region detection unit 135, the region correction unit 153, and the region determination unit 155 included in the control unit 160 There is no difference between the third embodiment and the fourth embodiment. Therefore, only the area identification unit 161 included in the control unit 160 will be described.

(Area identification unit 161)
The area identifying unit 161 identifies a high possibility area that is highly likely to be determined to be an area corresponding to a target portion of a person, and other areas of the one or more object areas 81.

  In particular, in the fourth embodiment, the target distance image 20 (that is, the previous target distance image 20) generated by the imaging device 30 before the target distance image 20 (that is, the current target distance image 20) is presupposed. To be acquired. Based on the previous target distance image 20 and the background distance image, a foreground distance image 60 (that is, the previous foreground distance image 60) is generated. Furthermore, using the plurality of filters 70, another one or more object areas (that is, the previous one or more object areas 81) are detected from the previous foreground distance image 60. Then, the high possibility area | region and other area | regions of the said 1 or more area | region of the last time are identified. Then, it is determined whether each of the other regions of the one or more previous regions corresponds to the target portion.

  In particular, the area identifying unit 161 is based on the high possibility area and the determined area determined to be the area corresponding to the target portion among the one or more previous object areas 81. Among the one or more object regions 81 this time, the high possibility region and other regions are identified.

  In other words, the region identifying unit 161 traces a person using the previous object region 81 that is regarded as a region corresponding to the target portion (for example, the head), thereby Identify high likelihood areas and other areas.

  Specifically, the area identification unit 161 is based on the positions of the high possibility area and the determined area of the one or more previous object areas 81 and the positions of the one or more current object areas 81. The high possibility area and the other areas of the one or more object areas 81 of this time are identified.

For example, the target distance image 20 is generated and acquired at predetermined time intervals. Then, each time the target distance image 20 is acquired at a predetermined time T _N and one or more object regions 81 are detected, the region identifying unit 161 determines the position of each object region 81 at the time T _N. get. The position at time _TN is the position of the object region 81 in the binary image 80 generated from the distance image 20 acquired at time _TN .

The area identifying unit 161, the object region 81 at time T _N from the front by the predetermined time interval time T _N-1 (i.e., the last object region 81) from the position and the moving speed of the previous object region The position after each of the predetermined time intervals 81 (ie, time T _N ) is estimated. The moving speed will be described later. Then, the area identification unit 161 compares the estimated position of the previous object area 81 with the actual position of the object area 81 at the time _TN (that is, the current object area 81), and the previous position where the position is close The object area 81 and the current object area 81 are associated with each other. As an example, the previous object region 81 and the current object region 81 are associated with each other in ascending order of distance.

Thereafter, the area identifying unit 161 acquires and holds the moving speed at the time _TN of the current object area 81 (ie, the high possibility area) associated with the previous object area 81. The moving speed at time _TN is a difference between the current position of the object region 81 and the previous position of the associated object region 81. Similarly, since the movement speed at the time T _N-1 of the previous object region 81 is acquired, the movement speed at the time T _N-1 can be used as described above. The moving speed of the current object area 81 that is not associated with the previous object area 81 is zero.

Then, the area identifying unit 161 identifies the current object area 81 that is not associated with the previous object area 81 as another area other than the high possibility area. In addition, the region identification unit 161 is the current object region 81 associated with the previous object region 81, and the direction of the moving speed at the time T _N is the outward direction of the binary image 80 (that is, binary). The current object region 81 (in the direction away from the center of the image 80) is also identified as another region. The area identifying unit 161 identifies the remaining object area 81 other than the object area 81 described above as a high possibility area.

  As described above, the high possibility area and the other areas are identified. Generally, as a posture of a person, the head tends to be positioned slightly forward of the torso. Therefore, when the person moves away from the imaging device 30 (that is, when the target distance image 20 has an area corresponding to the person's back), an area corresponding to the person's shoulder or back is erroneously detected. Easy to be. Conversely, when the person moves in a direction approaching the imaging device 30 (that is, when there is no area corresponding to the person's back in the target distance image 20), the area corresponding to the person's shoulder or back is erroneously set. It is difficult to detect. Therefore, the area identification unit 161 can identify the high possibility area and the other areas as described above.

  FIG. 25 is an explanatory diagram for explaining an example of identification between a high possibility area and other areas according to the fourth embodiment. Referring to FIG. 25, a binary image 80A and object areas 81F and 81G obtained from the previous target distance image 20, and a binary image 80B and object areas 81H and 81I obtained from the current target distance image, It is shown. For example, the previous object region 81F and the current object region 81H are associated with each other, and the previous object region 81G and the current object region 81I are associated with each other. Further, since the direction of the moving speed of the current object region 81H is the direction toward the inside of the binary image 80B, the current object region 81H is identified as a high possibility region. On the other hand, since the direction of the moving speed of the current object region 81I is the outward direction of the binary image 80B, the current object region 81I is identified as another region.

<6-2. Process flow>
Next, an example of information processing according to the fourth embodiment will be described with reference to FIG. FIG. 26 is a flowchart illustrating an example of a schematic flow of information processing according to the fourth embodiment.

  Here, steps S610 to S640, S670, and S680 are the same as steps S510 to S540, S560, and S570 according to the third embodiment described with reference to FIG. Therefore, only steps S650 and S660 will be described here.

  In step S650, the region identification unit 161 acquires the positions of the current one or more object regions 81.

  In step S660, the area identifying unit 161 determines the position of the high possibility area and the determined area from the previous one or more object areas 81, and the current one or more object areas 81, Among the one or more object areas 81 this time, the high possibility area and other areas are identified.

  Note that after step S680, the process returns to step S620.

  The fourth embodiment of the present invention has been described above. According to the fourth embodiment, one of the one or more object areas 81 to be detected is identified from the other areas with the high possibility area. Then, it is determined whether the other area corresponds to the target portion of the person. Thereby, it is possible to reduce processing until a person is finally detected.

  In particular, in the fourth embodiment, based on the previous detection result of the object area 81, the high possibility area and other areas in the current object area 81 are identified. Thereby, when the object distance image 20 is continuously generated and the object region 81 is continuously detected, it is possible to reduce processing until the person is detected.

  In addition, if the target distance image 20 is generated in a short period and one or more object regions 81 are continuously generated, the object region 81 corresponding to the head of the same person is continuously detected. It is possible to detect a person more accurately by using not only information in the target distance image 20 but also information in the time direction.

<< 7. Fifth embodiment >>
Subsequently, a fifth embodiment of the present invention will be described with reference to FIGS. According to the fifth embodiment of the present invention, a plurality of filter sets applied at different positions in the foreground distance image 60 are used for detection of the object region 81. Each filter set includes two or more filters 70 capable of recognizing human head and torso patterns having different characteristics. This makes it possible to more accurately detect a person in the presence of various persons with different characteristics (for example, physique).

<7-1. Configuration of information processing apparatus>
An example of the configuration of the information processing apparatus according to the fifth embodiment will be described with reference to FIGS. FIG. 27 is a block diagram illustrating an example of a configuration of an information processing device 100-5 according to the fifth embodiment. Referring to FIG. 27, the information processing apparatus 100-5 includes a communication unit 110, a storage unit 120, and a control unit 170.

  Here, the communication unit 110, the storage unit 120, and the distance image acquisition unit 131, the foreground distance image generation unit 133, and the region correction unit 141 included in the control unit 170 are the second embodiment and the fifth embodiment. There is no difference between Therefore, only the region detection unit 171 and the region determination unit 173 included in the control unit 150 will be described.

(Area detection unit 171)
-Plural Filter Sets In the fifth embodiment, the region detection unit 171 is a plurality of filter sets applied at different positions in the foreground distance image, and a human head corresponding to the applied position One or more object regions are detected from the foreground distance image using the plurality of filter sets capable of recognizing the body pattern.

  Each of the plurality of filter sets includes two or more filters 70 capable of recognizing human head and torso patterns having different characteristics. For example, each filter set includes two or more filters 70 capable of recognizing head and torso patterns of persons having different physiques. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 28 is an explanatory diagram for describing an example of filters included in the filter set. Referring to FIG. 28, filters 70A, 70B and 70C included in one filter set are shown. In this example, one filter set includes a filter 70A for a person having a height of 180 cm, a filter 70B for a person having a height of 160 cm, and a filter 70C for a person having a height of 140 cm. For example, as described above, each filter set includes two or more filters capable of recognizing head and trunk patterns of persons having different physiques (for example, height). Then, for example, a filter set is generated for each pixel.

—Shape of Filter The shape of each filter 70 included in the filter set is as described in the second embodiment.

-Generation Method of Multiple Filter Sets Each filter 70 included in the filter set can also be generated by the method described in the second embodiment.

  In particular, the two or more filters 70 included in each filter set according to the fifth embodiment can recognize human head and trunk patterns having different characteristics as described above. Therefore, each filter 70 included in the filter set is generated in consideration of the characteristics of the target person.

For example, as described with reference to FIGS. 13A and 13B, each filter 70 is generated based on the relative relationship between the imaging device 30 and the person 40 when the imaging device 30 images the person 40. In particular, in the fifth embodiment, two or more filters 70 included in the filter set are generated based on different relative relationships. This is because the two or more filters included in the filter set each identify a pattern of a person's head and torso (eg, shoulders) having different person characteristics. For example, as shown in FIGS. 13A and 13B, the distance D _H in the height direction between the imaging device 30 and the head of the person 40, and the distance D in the height direction between the imaging device 30 and the shoulder of the person 40. _S , the head width W _{H of} the person 40, and the shoulder width W _{S of the} person 40 are changed depending on which filter 70 included in the filter set is generated. For example, when the filter 70A shown in FIG. 28 is generated, the information related to the person 40 is set under the assumption that the height of the person 40 is 180 cm. Further, when the filter 70C shown in FIG. 28 is generated, the information related to the person 40 is set on the assumption that the height of the person 40 is 140 cm.

-Example of detection of object region using a plurality of filter sets For example, the region detection unit 171 uses the two or more filters 70 in the foreground distance image 60 when using each of the plurality of filter sets. A plurality of areas are detected by applying to the position of.

  Specifically, for example, as illustrated in FIG. 28, when one filter set includes three filters 70, the region detection unit 171 sets the three filters 70 at the same position in the foreground distance image 60. Applies to That is, the region detection unit 171 places the filter 70A for a person having a height of 180 cm, the filter 70B for a person having a height of 160 cm, and the filter 70C for a person having a height of 140 cm at the same position (for example, , The same pixel). Hereinafter, an example of the object region 81 detected by applying such a filter will be described with reference to FIG.

  FIG. 29 is an explanatory diagram for explaining an example of the object region 81 detected by applying two or more filters included in the filter set. Referring to FIG. 29, the filters 70A, 70B and 70C shown in FIG. 28 are shown. For example, such a filter set including the filters 70 </ b> A, 70 </ b> B, and 70 </ b> C is generated in units of pixels and applied to the pixels included in each foreground region 61 of the foreground distance image 60. As a result, for example, the object region 81A is detected from the foreground region 61A corresponding to a person having a height of 180 cm using the filter 70A for a person having a height of 180 cm. Similarly, the object region 81B is detected using the filter 70B for a person having a height of 160 cm, and the object region 81C is detected using the filter 70C for a person having a height of 140 cm. In addition, the object region 81D, the object region 81E, and the object region 81F are detected from the foreground region 61B corresponding to a person having a height of 140 cm. For example, in this way, among the filters 70 included in each filter set, a filter corresponding to a specific height (that is, a filter 70 that can identify the head and trunk patterns of a person having a specific height) Applied in pixels. Accordingly, the object region 81 can be detected for each set of filters 70 corresponding to the specific height (that is, the filter 70 that can identify the pattern of the head and torso of a person having a specific height).

(Area determination unit 173)
Based on the size corresponding to each of at least one of the one or more object areas 81 to be detected, the area determination unit 173 determines that each of the at least one object area 81 is a target portion of a person. It is determined whether the area corresponds to.

  For example, the region determination unit 173 determines whether each of the at least one object region 81 corresponds to a target portion of a person based on the corrected size corresponding to each of the at least one object region 81. To do.

  For example, in the fifth embodiment, a plurality of object regions 81 are detected by applying two or more filters 70 to the same position in the foreground distance image 60 when using each of a plurality of filter sets. . Further, for example, each of the plurality of object regions 81 is corrected. Then, based on the size corresponding to each of at least one object region 81 among the plurality of object regions 81, the region determination unit 173 is a region in which each of the at least one object region 81 corresponds to the target portion. Determine whether.

For example, the region determination unit 173 evaluates the plurality of object regions 81 based on the size corresponding to each of the plurality of object regions 81. For example, as described with reference to FIG. 19, compared with the width W _F of the ellipse major axis L _m and a minor axis L _n and a rectangular 71 corresponding to each object region 81 is made, as an example, both Similarity is calculated as an evaluation result. Then, the region determination unit 173 selects one object region 81 with the best evaluation result (for example, the highest similarity) from the plurality of object regions 81 corresponding to the same foreground region 61. For example, such a selection is performed for each foreground region 61. Then, the region determination unit 173 determines whether the selected at least one object region 81 is a region corresponding to a person's head. Hereinafter, a specific example of such a determination method will be described with reference to FIG.

FIG. 30 is an explanatory diagram for explaining an example of determination for a plurality of object regions 81 detected by applying two or more filters included in the filter set to the same position. Referring to FIG. 30, ellipses 83 </ b> A to 83 </ b> F respectively corresponding to the plurality of detected object regions 81 </ b> A to 81 </ b> F shown in FIG. 29 are shown. Further, the rectangle of the filter 70 (that is, the filter 70A, 70B, or 70C) used for detecting each of the object regions 81A to 81F corresponding to the ellipses 83A to 83F is displayed so as to be superimposed on each of the ellipses 83A to 83F. Yes. The area determination unit 173 calculates the major axis L _m and the minor axis L _n of the corresponding ellipse 83 for each of the detected object areas 81A to 81F, and the smaller one of the major axis L _m and the minor axis L _n. rectangular (i.e., square 71) is calculated as follows similarity R between the width _{W Fh} and _{W Fv} of.

  Then, one object region 81 is selected for each foreground region 61 based on the similarity. As an example, the object region 81 having the highest similarity R is selected. That is, in the example of FIGS. 29 and 30, the object region 81A corresponding to the ellipse 83A is selected for the foreground region 61A. For the foreground area 61B, the object area 81F corresponding to the ellipse 83F is selected. Then, it is determined whether the selected at least one object region corresponds to a person's head. This determination is as described with reference to FIG. 19 in the second embodiment.

  For example, as described above, the evaluation, selection, and determination of the object region 81 are performed.

<7-2. Process flow>
Next, an example of information processing according to the fifth embodiment will be described with reference to FIG. FIG. 31 is a flowchart illustrating an example of a schematic flow of information processing according to the fifth embodiment.

  Here, steps S710 to S730 are the same as steps S410 to S430 according to the second embodiment described with reference to FIG. 20 (S310 to S330 according to the first embodiment described with reference to FIG. 15). It is. Therefore, only steps S740 to S770 will be described here.

  In step S740, the region detection unit 171 is a plurality of filter sets applied at different positions in the foreground distance image 60, and can recognize the pattern of the human head and trunk corresponding to the applied positions. A plurality of object regions 81 are detected from the foreground distance image 60 using the plurality of filter sets. Here, when each of the plurality of filter sets is used, two or more filters 70 included in the filter set are applied to the same position in the foreground distance image 60.

  In step S750, the region correction unit 141 determines the peripheral region that is a peripheral region of each of the plurality of object regions 81 detected based on the target distance image 20 or the foreground distance image 60 and satisfies a predetermined condition. By adding to each of the plurality of object regions 81, the plurality of object regions 81 are corrected.

  In step S760, the region determination unit 173 evaluates the plurality of object regions 81 based on the size corresponding to each of the plurality of object regions 81, and selects at least one object region 81 with high evaluation. . Each of the at least one object region 81 with high evaluation is, for example, the object region 81 with the best evaluation result among the object regions 81 corresponding to the same foreground region 61.

  In step S770, the region determination unit 173 determines whether each of the selected at least one object region 81 corresponds to a person's head. Then, the process ends.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, after step S770, the process may return to step S720.

<7-3. Modification>
Next, a modification of the fifth embodiment will be described with reference to FIG. According to the modification of the fifth embodiment, two or more filters included in each filter set are selectively applied. This makes it possible to reduce processing required to detect the object region 81 while more accurately detecting a person in the presence of various persons having different physiques.

(Area detection unit 171)
In the example of the fifth embodiment described above, the region detection unit 171 applies the two or more filters 70 to the same position in the foreground distance image 60 when using each of the plurality of filter sets. Detect multiple areas.

  On the other hand, in the modification of the fifth embodiment, the region detection unit 171 selects at least one filter 70 from the two or more filters 70 when using each of the plurality of filter sets, and the selected By applying at least one filter 70, one or more object regions 81 are detected.

  In addition, for example, the region detection unit 171 may perform the two or more of the above based on a person image estimated from a distance image generated by the imaging device 30 or another distance image generated based on the distance image. The at least one filter 70 is selected from the filters 70.

  More specifically, for example, an average height corresponding to each foreground area 61 of the foreground distance image 60 generated from the target distance image 20 is calculated. That is, the height corresponding to each pixel included in the foreground area 61 is calculated, and the average value of the calculated heights is calculated. Further, the height of the person corresponding to the foreground area 61 is estimated from the average value. Then, one or more filters 70 corresponding to the estimated height are selected from the two or more filters 70 included in the filter set. As an example, one filter 70 is selected.

  Note that two or more filters included in the filter set may be stored in advance and acquired when selected, or may exist in advance as candidates but may actually be generated when selected. .

(Area determination unit 173)
Based on the size corresponding to each of at least one of the one or more object areas 81 to be detected, the area determination unit 173 determines that each of the at least one object area 81 is a target portion of a person. It is determined whether the area corresponds to.

  For example, the region determination unit 173 determines whether each of the at least one object region 81 corresponds to a target portion of a person based on the corrected size corresponding to each of the at least one object region 81. To do.

  As described above, for example, one of the two or more filters 70 included in the filter set is selected. In this case, a filter set is used, but only one filter 70 is actually applied. Therefore, in this case, similarly to the region determination unit 143 according to the second embodiment, it is determined whether each object region 81 is a region corresponding to a target portion of a person.

(Process flow)
Next, an example of information processing according to the fifth embodiment will be described with reference to FIG. FIG. 32 is a flowchart illustrating an example of a schematic flow of information processing according to a modification of the fifth embodiment.

  Here, steps S810 to S830, S850, and S860 are the same as steps S410 to S430, S450, and S460 according to the second embodiment described with reference to FIG. Therefore, only step S840 will be described here.

  In step S840, the region detection unit 171 selects one filter 70 from two or more filters 70 when using each of the plurality of filter sets, and applies the selected one filter 70. One or more object regions 81 are detected.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, after step S860, the process may return to step S820.

(Other)
As described above, the modification of the fifth embodiment has been described, but the modification is not limited to this example.

  For example, although the example in which the filter 70 is selected from the filter set based on the estimated human characteristics has been described, the method for selecting the filter 70 from the filter set is not limited thereto. For example, the filter 70 applied to the foreground area 61 in the previous foreground distance image 60 generated from the previously acquired target distance image 20 is retained and used for the foreground area 61 in the current foreground distance image 60. Also good. For example, for the current foreground area 61 located in the vicinity of the previous foreground area 61, the filter 70 applied in the previous foreground area 61 may be selected.

  Moreover, although the example in which one filter 70 of two or more filters 70 included in the filter set is selected has been described, the number of filters 70 to be selected is not limited thereto. For example, two or more filters 70 may be selected from the filter set and applied. And also in the modification of 5th Embodiment, determination of the object area | region 81 similar to 5th Embodiment may be performed instead of the determination of the object area | region 81 similar to 2nd Embodiment.

  The fifth embodiment of the present invention has been described above. According to the fifth embodiment, a plurality of filter sets applied at different positions in the foreground distance image are used to detect the object region 81. Each filter set includes two or more filters 70 capable of recognizing human head and torso patterns having different characteristics. This makes it possible to more accurately detect a person in the presence of various persons with different characteristics (for example, physique).

  Specifically, it is assumed that the person imaged by the imaging device 30 does not have the same characteristic (for example, physique) uniformly but has different characteristics. Therefore, it is possible to reduce the number of undetected persons by using the filter 70 according to various features, not limited to a single feature.

  Further, as described in the fifth embodiment, for example, when each of a plurality of filter sets is used, by applying two or more filters to the same position in the foreground distance image 60, a plurality of regions can be obtained. Detected. Thereby, even when the characteristics of the person imaged by the imaging device 30 cannot be specified, it is possible to reduce the undetected person.

  Further, as described in the modification of the fifth embodiment, for example, when using each of a plurality of filter sets, at least one filter is selected from the two or more filters, and the selected filter is applied. The As a result, it is possible to reduce the number of undetected persons while reducing the processing required to detect the object region 81.

  Further, for example, the at least one filter is selected based on a person image estimated from a distance image generated by the imaging device 30 or another distance image generated based on the distance image. As a result, the selected filter 70 becomes more suitable for the characteristics of the person, so that it is possible to further reduce the number of undetected persons while reducing the processing required to detect the object region 81.

  In addition, for example, there are some filter sets other than those shown so far, such as tilted postures such as bent waist, and carry items integrated with the body such as golf bags and babies. A filter that takes into account the characteristics of a person having a simple shape may be included.

<< 8. Sixth Embodiment >>
Subsequently, a sixth embodiment of the present invention will be described with reference to FIGS. According to the sixth embodiment of the present invention, it is determined based on the foreground distance image 60 whether there is an undetected object region 81. This can reduce the possibility that an undetected object is missed.

<8-1. Configuration of information processing apparatus>
An example of the configuration of the information processing apparatus according to the sixth embodiment will be described with reference to FIGS. FIG. 33 is a block diagram illustrating an example of a configuration of an information processing device 100-6 according to the sixth embodiment. Referring to FIG. 33, the information processing apparatus 100-6 includes a communication unit 110, a storage unit 120, and a control unit 180.

  Here, the communication unit 110, the storage unit 120, and the distance image acquisition unit 131, the foreground distance image generation unit 133, and the region detection unit 135 included in the control unit 180 are described in the first embodiment and the sixth embodiment. There is no difference between Therefore, only the undetected area determination unit 181 included in the control unit 180 will be described.

(Undetected area determination unit 181)
The undetected area determination unit 181 may not detect an area to be detected other than the one or more object areas 81 based on the one or more object areas 81 and the foreground distance image 60 that are detected. Determine if there is.

-Detection of the foreground area from the foreground distance image For example, first, the undetected area determination unit 181 detects the foreground area 61 from the foreground distance image 60.

  For example, the undetected area determination unit 181 detects a pixel whose pixel value (that is, distance) is not 0 in the foreground distance image 60 as the foreground area 61. More specifically, for example, the undetected area determination unit 181 maintains the pixel value of the foreground distance image 60 with the pixel value of 0 as it is, and sets the pixel value of the pixel with a pixel value of 0 to 1 as it is. By changing, a foreground binary image is generated. Then, the undetected area determination unit 181 detects individual foreground areas by labeling pixels having a pixel value 1 among the pixels included in the foreground binary image. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 34 is an explanatory diagram for explaining an example of the detection result of the foreground area 61. The example shown in FIG. 34 is an example when the foreground regions 61A and 61B are detected from the foreground distance image 60 shown in FIG. Referring to FIG. 34, a foreground binary image 90 is shown. The foreground area 61A and the foreground area 61B shown in FIG. 6 are detected as a foreground area 91A and a foreground area 91B that are a set of pixels having a pixel value of 1.

-Determination of presence / absence of undetected area For example, the undetected area determination unit 181 then performs the determination based on the detected one or more object areas 81 and the detected one or more foreground areas 91. It is determined whether there is a possibility that a region other than the object region 81 to be detected is not detected.

  For example, the undetected area determination unit 181 calculates a rectangle circumscribing each of the detected one or more object areas 81 (hereinafter referred to as “object area rectangle”). The undetected area determination unit 181 also calculates a rectangle circumscribing each of the detected one or more foreground areas 91 (hereinafter referred to as “foreground area rectangle”). Then, the undetected area determination unit 181 compares the position and size of each of the one or more object area rectangles with the position and size of each of the one or more foreground area rectangles. The foreground area 61 that is not involved in the detection of any object area 81 is specified. Then, when the undetected area determination unit 181 identifies the foreground area 61 that is not involved in the detection of any object area 81, the undetected area determination unit 181 determines that there is a possibility that the area has not been detected with respect to the identified foreground area 61. To do. Hereinafter, a specific example of this point will be described with reference to FIG.

  FIG. 35 is an explanatory diagram for explaining an example of determining whether or not there is a possibility that an area has not been detected. Referring to FIG. 35, a foreground binary image 90 and a binary image 80 are shown. In the foreground binary image 90, foreground areas 91A and 91B and foreground area rectangles 93A and 93B of the foreground areas 91A and 91B are shown. In the binary image 80, one object area 81A and an object area rectangle 87A of the object area 81A are shown. In this example, assuming that the foreground binary image 90 and the binary image 80 are overlapped, an object region rectangle 87A exists in the foreground region rectangle 93A, but any object region is present in the foreground region rectangle 93B. There is also no rectangle 87. Therefore, it is determined that there is a possibility that no area has been detected for the foreground area 90B.

  As described above, it is determined whether there is a possibility that a region other than the one or more object regions 81 is not detected.

-Notification of undetected possibility For example, when the undetected area determination unit 181 determines that there is a possibility that an area to be detected other than the one or more object areas 81 may not be detected. Notify the result.

  For example, the undetected area determination unit 181 notifies the determination result to the user or administrator of the information processing apparatus 100-6 via the communication unit 110 (or a display unit not shown).

  Also, for example, the notified determination result includes information indicating that there is a possibility that the area has not been detected, information about the foreground area 61 that may be related to the non-detection, and the target distance image 20.

<8-2. Process flow>
Next, an example of information processing according to the sixth embodiment will be described with reference to FIG. FIG. 36 is a flowchart illustrating an example of a schematic flow of information processing according to the sixth embodiment.

  Here, steps S910 to S940 are the same as steps S310 to S340 according to the first embodiment described with reference to FIG. Therefore, only steps S950 to S970 will be described here.

  In step S950, the undetected area determination unit 181 detects the foreground area 61 from the foreground distance image 60.

  In step S960, the undetected area determination unit 181 detects other than the one or more object areas 81 based on the detected one or more object areas 81 and the detected one or more foreground areas 91. It is determined whether there is a possibility that a region to be detected is not detected. If there is a possibility that the area has not been detected, the process proceeds to step S970. Otherwise, the process ends.

  In step S970, the undetected area determination unit 181 notifies the determination result to the user or administrator of the information processing apparatus 100-6. Then, the process ends.

  Note that the target distance image 20 may be continuously generated and acquired. In this case, the process may return to step S920 instead of ending after step S960 or S970.

  The sixth embodiment of the present invention has been described above. According to the sixth embodiment, based on the foreground distance image 60, it is determined whether or not the object region 81 has not been detected. This can reduce the possibility that an undetected object is missed.

<< 9. Hardware configuration >>
Subsequently, an example of a hardware configuration of the information processing apparatus 100 according to the embodiment of the present disclosure will be described with reference to FIG. FIG. 37 is a block diagram illustrating an example of a hardware configuration of the information processing apparatus 100 according to an embodiment of the present disclosure. Referring to FIG. 37, the information processing apparatus 100 includes a CPU (Central Processing Unit) 1001, a ROM (Read Only Memory) 1003, a RAM (Random Access Memory) 1005, a bus 1007, a storage device 1009, and a communication device 1011.

  The CPU 1001 executes various processes in the information processing apparatus 100. The ROM 1001 stores a program and data for causing the CPU 1001 to execute processing in the information processing apparatus 100. The RAM 1005 temporarily stores programs and data when the CPU 1001 executes processing.

  A bus 1007 connects the CPU 1001, the ROM 1003, and the RAM 1005 to each other. A storage device 1009 and a communication device 1011 are further connected to the bus. The bus 1007 includes, for example, a plurality of types of buses. As an example, the bus 1007 includes a bus that connects the CPU 1001, the ROM 1003, and the RAM 1005, and one or more other buses that are slower than the bus.

  The storage device 1009 stores data to be temporarily or permanently stored in the information processing device 100. The storage device 1009 may be a magnetic storage device such as a hard disk, or may be an EEPROM (Electrically Erasable and Programmable Read Only Memory), a flash memory, a MRAM (Magnetoresistive Random Access Memory), or the like. Further, non-volatile memories such as FeRAM (Ferroelectric Random Access Memory) and PRAM (Phase change Random Access Memory) may be used.

  The communication device 1011 is a communication unit included in the information processing device 100 and communicates with an external device via a network (or directly). The communication device 1011 may be a communication device for wired communication. In this case, for example, the communication device 1011 may include a LAN terminal, a transmission circuit, and other communication processing circuits. The communication device 1011 may be a communication device for wireless communication. In this case, for example, the communication device 1011 may include a communication antenna, an RF circuit, and other communication processing circuits.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, although the example in which the shape of the filter used when detecting the object region is rectangular has been described, the present invention is not limited to such an example. The shape of the filter may be other than a rectangle. As an example, the shape of the filter may be an ellipse.

  Moreover, although the example in which one filter is applied to each position of the foreground distance image has been described except for the fifth embodiment, the present invention is not limited to such an example. Two or more filters may be applied to each position. As an example, two or more directions may be set as the direction in which the person is facing, and a filter corresponding to each direction may be generated. A filter corresponding to each direction may be applied to each position. Further, after one or more object regions are detected for each direction, the object regions in all directions may be integrated, and the one or more object regions after integration may be regarded as a final object region.

  Moreover, although the example in which the filter is generated in units of pixels has been described, the present invention is not limited to such an example. The filter may be generated in a coarser unit than the pixel unit. As an example, several sections may be set in the foreground distance image, and a filter may be generated for each section.

  In addition, the description has been mainly made on the assumption that the filter is generated in advance, but the present invention is not limited to such an example. The filter may be generated at any time. As an example, the filter may be generated at any time when applied to a foreground distance image.

  Moreover, although the example in which the imaging device is installed directly below has been described, the present invention is not limited to such an example. The imaging device may be installed in a direction other than directly below. For example, the imaging device may be installed in a diagonally downward direction. Also in this case, a filter is generated based on information regarding the installation state of the imaging device.

  In addition, although the example in which the information processing system includes only one imaging device has been described, it is needless to say that the present invention is not limited to such an example. The information processing system may include two or more imaging devices, and the information processing device may detect a person from a distance image generated by each imaging device.

  Further, although the example in which the imaging device and the information processing device are separate devices has been described, the present invention is not limited to such an example. For example, the imaging device and the information processing device are the same device, and both generation of a distance image and detection of a person from the distance image may be performed.

  Further, the processing steps in the information processing of the present specification do not necessarily have to be executed in time series in the order described in the flowchart. For example, the processing steps in the information processing may be executed in an order different from the order described in the flowchart, or may be executed in parallel.

  It is also possible to create a computer program for causing hardware such as a CPU, a ROM, and a RAM built in the information processing apparatus to exhibit functions equivalent to those of each configuration of the information processing apparatus. A storage medium storing the computer program is also provided.

10 (Conventional) Haar-Like filter 20 Distance image 30 Imaging device 40 Person 60 Foreground distance image 61 Foreground area 70 Filter 71, 73 Rectangular 80 Binary image 81 Object area 83 Ellipse 90 Foreground binary image 91 Foreground area 100 Information processing Device 131 Distance image acquisition unit 133 Foreground distance image generation unit 135, 171 Area detection unit 141, 153 Area correction unit 143, 155, 173 Area determination unit 151, 161 Area identification unit 181 Undetected area determination unit

Claims (21)

An acquisition unit that acquires a first distance image generated by the imaging device and a second distance image generated in advance by the imaging device by imaging a background;
A generating unit that generates a third distance image corresponding to a foreground with respect to the background based on the first distance image and the second distance image;
A plurality of filters applied at different positions in the third distance image, the plurality of filters capable of recognizing human head and torso patterns according to the applied positions, A detection unit for detecting one or more regions from the third distance image;
An information processing apparatus comprising:   2. The information processing apparatus according to claim 1, wherein each of the plurality of filters is a filter generated based on a relative relationship between the imaging apparatus and the person when the person is imaged by the imaging apparatus.   The information processing apparatus according to claim 2, wherein the relative relationship includes a direction from the imaging apparatus in which the head of the person is present and a direction from the imaging apparatus in which the torso of the person is present.   The information processing apparatus according to claim 2, wherein each of the plurality of filters is a filter generated further based on characteristics of the imaging apparatus. The detection unit detects the one or more regions as regions corresponding to a target portion of a person,
The information processing apparatus includes:
A region determination unit that determines whether each of the at least one region corresponds to the target portion based on a size corresponding to each of at least one region of the one or more regions;
The information processing apparatus according to any one of claims 1 to 4, further comprising: Based on the first distance image or the third distance image, the peripheral area of each of at least one of the one or more areas, the peripheral area satisfying a predetermined condition is A correction unit that corrects the one or more areas by adding to each of the at least one area;
Further comprising
6. The region determination unit according to claim 5, wherein the region determination unit determines whether each of the at least one region corresponds to the target portion based on the corrected size corresponding to each of the at least one region. Information processing device.   The information processing apparatus according to claim 5, wherein the target portion is a human head. An identification unit that identifies a high possibility region that is highly likely to be determined to be a region corresponding to the target portion, and another region of the one or more regions,
Further comprising
The at least one region of the one or more regions is the other region of the one or more regions;
The information processing apparatus according to any one of claims 5 to 7.   The information processing apparatus according to claim 8, wherein the high possibility region is a region located within a predetermined range in which a region corresponding to a person portion other than the target portion is difficult to be detected.   The high possibility region is the one region when the one or more regions include only one region, and when the one or more regions include two or more regions, The information processing apparatus according to claim 8, wherein the information processing apparatus is an area that is separated from any other area included in the one or more areas by a predetermined distance or more. The acquisition unit acquires a fourth distance image generated by the imaging device before the first distance image;
The generation unit generates a fifth distance image corresponding to the foreground with respect to the background based on the fourth distance image and the second distance image,
The detection unit detects another one or more regions from the fifth distance image using the plurality of filters,
The identification unit identifies the high possibility region and the other region among the one or more other regions,
The region determination unit determines whether each of the other regions of the one or more other regions corresponds to the target portion;
The identification unit is configured to determine the one or more one or more regions based on the high possibility region and the determined region determined to be a region corresponding to the target portion. Identifying the high likelihood area and other areas of the area;
The information processing apparatus according to claim 8.   The identification unit is configured based on a position of the high possibility region and the determined region in the one or more other regions, and a position of the one or more regions. The information processing apparatus according to claim 11, wherein the high possibility area and other areas are identified. The detection unit is a plurality of filter sets applied at different positions in the third distance image, and the plurality of the plurality of filter sets capable of recognizing human head and torso patterns according to the applied positions. Using a filter set to detect the one or more regions;
Each of the plurality of filter sets includes two or more filters capable of recognizing a human head and torso pattern having different characteristics.
The information processing apparatus according to any one of claims 5 to 7. The detection unit detects a plurality of regions by applying the two or more filters to the same position in the third distance image when using each of the plurality of filter sets;
The region determination unit determines whether each of the at least one region corresponds to the target portion based on a size corresponding to each of at least one region of the plurality of regions.
The information processing apparatus according to claim 13.   The detecting unit selects at least one filter from the two or more filters when using each of the plurality of filter sets, and applies the selected at least one filter to the one or more filter sets. The information processing apparatus according to claim 13, wherein the information processing apparatus detects an area.   The detection unit is configured to detect at least one of the two or more filters based on a person image estimated from a distance image generated by the imaging device or another distance image generated based on the distance image. The information processing apparatus according to claim 15, wherein one filter is selected.   An undetected determination unit that determines whether there is a possibility that a region other than the one or more regions to be detected is not detected based on the one or more regions and the third distance image; The information processing apparatus according to claim 1, further comprising:   The information processing apparatus according to claim 1, wherein each of the plurality of filters is a Haar-like filter.   19. The information processing apparatus according to claim 1, wherein each of the plurality of filters is a filter capable of recognizing a human head and shoulder pattern according to an applied position. Computer
An acquisition unit that acquires a first distance image generated by the imaging device and a second distance image generated in advance by the imaging device by imaging a background;
A generating unit that generates a third distance image corresponding to a foreground with respect to the background based on the first distance image and the second distance image;
A plurality of filters applied at different positions in the third distance image, the plurality of filters capable of recognizing human head and torso patterns according to the applied positions, A detection unit for detecting one or more regions from the third distance image;
Program to function as. Acquiring a first distance image generated by the imaging device, and a second distance image generated in advance by the imaging device by imaging a background;
Generating a third distance image corresponding to the foreground with respect to the background based on the first distance image and the second distance image;
A plurality of filters applied at different positions in the third distance image, the plurality of filters capable of recognizing human head and torso patterns according to the applied positions, Detecting one or more regions from the third distance image;
An information processing method including:

Images