JP4082190B2 - Human presence position detection device, detection method thereof, and autonomous mobile device using the detection device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a method of automatically detecting a person from an image by a computer in a human machine interface, personal robot, image monitoring, autonomous mobile device, or the like.
[0002]
[Prior art]
Conventionally, as a method of automatically finding a person by a computer from an image, a method in which a moving part is regarded as a person and detected, a face or head that is a representative part of a person is detected by skin color or facial parts. There is a way to detect. As an example of the former, there is one that detects a changed portion as a person by taking a difference between a background image created in advance and an input image (see, for example, Patent Document 1). As an example of the latter, there is one that models a face as an ellipse and detects the face by ellipse detection (see, for example, Patent Document 2). In addition, there is one that models the head as a circle and performs head detection by circular shape detection (see, for example, Patent Document 3).
[0003]
[Patent Document 1]
JP-A-11-311682
[Patent Document 2]
Japanese Patent Laid-Open No. 11-185026
[Patent Document 3]
JP 2001-222719 A
[0004]
[Problems to be solved by the invention]
However, in the detection method as shown in the above-mentioned document 1, since a region where the brightness on the image has changed greatly is a person, when the brightness changes due to light being inserted or an object such as a chair moves. In some cases, there is a problem of detecting as a person. Further, in the detection method as shown in Document 2, it is difficult to stably detect the skin color in various scenes, and the skin color region is applied to the ellipse so that it is correct when it is in front. In this case, it is difficult to detect, and there is a problem that the head cannot be detected when facing backward. Further, in the detection method as shown in Document 3, since the shape of the template used for voting is concentric, there are many votes that do not contribute to detection, processing time is wasted, and the peak of the false vote value Has a problem that false detection occurs.
[0005]
The present invention solves the above-described problem, and can detect a person from an image at high speed with a simple configuration and can obtain an approximate position from the imaging device to the person in the real world with respect to the image world. An object of the present invention is to provide a human presence position detection device, a detection method thereof, and an autonomous mobile device using the detection device.
[0006]
[Means for Solving the Problems and Effects of the Invention]
  In order to achieve the above object, the invention of claim 1 is a device for detecting a person's presence and position by extracting a person's head image from an image captured by an imaging device. Approximate the shape with a figure model surrounded by an outwardly convex curve, and based on the focal length of the lens of the imaging device, the detection distance from the imaging device to the person to be detected, and the standard size of the human head A standard size and a size range of the shape model are determined in advance by estimating a range of the contour size of the human head image to be determined, and density is determined at each point on the boundary curve of the shape model of the standard size. A gradient direction is defined, a representative point is defined inside the shape model, a voting unit corresponding to the standard size of the shape model is provided at the representative point, and a representative point is selected from points of interest on the boundary curve of the shape model. Defined as connecting line A voting unit that includes a voting unit corresponding to the size range before or after the representative point on the line of the representative point direction line, and prepares the voting template created by the boundary curve of the shape model of the standard size so that it can be referred to Corresponding to a template preparation unit, a captured image data storage unit that digitizes an image input from the imaging device and stores the digitized captured image in a memory, and a person's outline or facial contour with respect to the captured image A differential process for extracting an edge portion composed of pixels is performed, pixels having a predetermined differential intensity or higher are extracted as edge pixels, a density gradient direction is calculated from the captured image, and a density is determined for each extracted edge pixel. Edge image data generating means for generating an edge image to which gradient direction data is added and storing it in a memory, and each pixel of the captured image and the edge image A voting space generating means for generating a voting space having voting cells at positions corresponding to the positions and storing them in a memory; and a density gradient corresponding to the density gradient direction added to the edge pixel on each edge pixel of the edge image The points on the boundary curve of the voting template having a direction are overlapped, and at this time, the voting cells in the voting space corresponding to the pixel positions on the edge image where the voting part of the voting template is located are voted and stored in the memory For each edge pixel, and a voting processing means for counting the voting values of each cell in the voting space, and detecting and detecting a cell having a voting value equal to or higher than a predetermined threshold in the voting space where the voting is performed. And a human presence position detecting means for detecting a position on the captured image corresponding to the selected cell as a human head position.The voting template preparation means conjugates each point on the boundary curve having a density gradient direction opposite to the density gradient direction defined for each point on the boundary curve of the standard size shape model. Defined as a conjugate point, one of the points conjugate to each other maintains the same relative positional relationship as the voting unit inside the shape model, and the other of the points conjugated to each other further has a voting unit outside the shape model. PreparationIt is a person's presence position detection device.
[0007]
  In the above configuration, a voting template in which the position of a person is represented by the position of the person's head on the captured image, and the shape of the person's head is modeled with a simple figure, and the range of the detection distance is wide. The voting template, which is provided with a plurality of voting sections in the form of line segments, is used to perform voting processing for each edge pixel on the edge image obtained from the captured image, so that variations in head shape and size are absorbed. The head shape can be detected reliably. In addition, since each edge pixel of the edge image is provided with information on the density gradient direction, and voting is performed only to the voting unit determined by the density gradient direction of each edge pixel, so-called circle detection hough that detects a circular curve from the image. Perform voting in all directions, such as (Hough) conversionNaTherefore, the number of voting processes for human detection can be reduced, and the human head shape can be detected at high speed.In the above configuration, the voting template conjugate to each other is provided in the voting template so that the voting process can be performed simultaneously on the image in which the image density in the human head is inverted, so that the image density is inverted in one voting process. Voting can be performed for two types of human head images, and in addition to the above-described effects, more stable and reliable detection can be performed at high speed.
[0012]
  Claim2The invention of claim1In the human presence position detecting device described in the above, the voting processing means obtains the curvature of the contour composed of the edge pixel of interest and a plurality of edge pixels in the vicinity of each edge pixel of the edge image, and the center of curvature exists. A direction is determined, and when voting for the edge pixel of interest, the voting cell in the voting space corresponding to the voting part position in the direction in which the center of curvature exists is voted.
[0013]
In the above configuration, only the voting process on the side having the center of curvature is sufficient, so that the head of the person can be extracted more efficiently and the position of the person can be detected at high speed. Note that the calculation for obtaining the curvature of the necessary contour is only performed once at the beginning of the voting process.
[0014]
  Claim3The invention of claim1The voting template preparation means includes a voting unit in the vicinity of the representative point of the voting template and on both sides of the representative point direction line.
[0015]
In the above configuration, even if the concentration gradient direction near the edge fluctuates, voting units are also provided near both sides of the representative point. Therefore, voting to the voting unit having such a width absorbs fluctuations in voting positions. Thus, voting can be performed near the center of the shape model, and detection with high reliability can be performed.
[0016]
  Claim4The invention of claim1The voting template preparation means stores the relative position of the voting unit with respect to each point on the boundary curve of the shape model in a tabular format using the concentration gradient direction at each point as an index. To remember.
[0017]
In the above configuration, since the position of the voting part of the voting template is stored in the memory in a table format in advance, it is only necessary to obtain the voting position by referring to this table at the time of voting processing. Can be realized.
[0018]
  Claim5The invention of claim1The voting template preparation means stores the voting template data in advance for each distance from the imaging device to the person, and the voting processing means measures the distance to the detection target. The voting template is set by referring to the data of the voting template based on the distance measurement result of the distance sensor, and voted.
[0019]
In the above configuration, the distance from the imaging device to the person is obtained by the distance sensor, and the detection is performed by limiting the size of the head to be detected on the image, so that highly reliable detection can be performed at high speed. it can.
[0020]
  Claim6The invention of claim1In the human presence position detecting device according to the above, the human presence position detecting means, when the detected cell having a vote value greater than or equal to a predetermined threshold is divided into a plurality of chunks, the distance between the chunks If the distances are within a predetermined range, they are grouped and integrated, and the position on the captured image corresponding to the representative point of the integrated mass is detected as the human head position.
[0021]
In the above configuration, by integrating cells having voting values equal to or greater than a predetermined threshold, it is possible to detect a human head position accurately by detecting a shape model even when voting is dispersed. it can.
[0022]
  Claim7The invention of claim1In the human presence position detection device according to the above, the voting template preparation unit arbitrarily sets a plurality of detection distance ranges from the imaging device to the person to be detected, and the standard size and size of the shape model for each detection distance range Ranges and voting templates based thereon are prepared, the voting space generation means generates a voting space for each detection distance range, and the voting processing means detects a detection distance range for each voting template corresponding to each detection distance range. Performs the above voting process on the corresponding voting space, and when the person presence position detecting means detects a cell having a voting value equal to or larger than a predetermined threshold in the voting space corresponding to each detection distance range, Are made to correspond to those detection distance ranges.
[0023]
In the above configuration, among many human candidates in the perspective direction from the imaging apparatus, only the human candidates within a specific distance range can be selected and the detection process can be performed.
[0024]
  Claim8The invention of claim1In the human presence position detecting device described in the above, when the size range of the shape model is a predetermined value or more, the voting template preparation unit divides the human detection distance range so that the size range is a predetermined value or less. Each of the divided ranges has a standard size and a size range of the shape model and a voting template based thereon, and the voting space generation unit generates a voting space for each of the divided detection distance ranges, and the voting processing unit Performs the voting process on the voting space corresponding to the detection distance range for each vote template corresponding to each of the divided detection distance ranges, and the person presence position detecting means When a cell having a vote value equal to or greater than a predetermined threshold value is detected in the corresponding vote space, the person presence position is made to correspond to the detection distance range.
[0025]
In the above configuration, when the range where the person exists is wide in the perspective direction, the size range of the shape model is set to a predetermined value or less by dividing the range where the person exists, so only the head of a specific size Can be detected, and the detection accuracy can be increased by suppressing the occurrence of erroneous detection.
[0026]
  Claim9The invention of claim1In the human presence position detecting device described in the above, the voting template preparation unit approximates the human head shape with a plurality of shapes with different postures using the same shape model, and prepares a voting template for each shape of each posture. The voting space generating unit generates a voting space for each shape of the posture, and the voting processing unit performs voting processing on each cell on the voting space for each shape having a different posture.
[0027]
In the above configuration, since a plurality of shape models having different postures are prepared, it is possible to cope with the inclined heads with distinction, and highly accurate human detection can be performed.
[0028]
  Claim10The invention of claim1The voting template preparation means approximates the human head shape with a plurality of different shape model shapes, prepares a voting template for each shape model shape, and the voting space The generation unit generates a voting space for each shape of the shape model, and the voting processing unit performs voting processing on each cell on the voting space for each shape model.
[0029]
In the above configuration, since a plurality of shape models, voting templates and voting spaces corresponding to them are provided, people can be detected using different shape models, and various head shapes can be detected. .
[0030]
  Claim11The invention of claim1In the human presence position detecting device according to the above, the human presence position detecting unit is configured to project an image having a size of a projection image obtained by projecting a shape modeling a head on the head position on the detected captured image. It is what is cut out.
[0031]
In the above configuration, since the head image is cut out for each head size on the captured image, a cut-out head image having an appropriate size with a small area of an unnecessary background portion can be obtained.
[0032]
  Claim12The invention of claim11In the human presence position detecting device described above, the human presence position detecting means compares the clipped image with head data that is classified and stored in advance for each head orientation, thereby determining the head orientation. Identify and detect.
[0033]
In the above configuration, since the detected head image can be recognized for each head orientation, the head orientation can be known together with the head orientation.
[0034]
  Claim13The invention of claimTo 12In the human presence position detection apparatus described above, the human presence position detection unit divides the cut-out image into a plurality of predetermined small images when a plurality of candidates are output as the head orientation. The orientation of the head is determined based on the difference in density distribution between the small images.
[0035]
In the above configuration, since the density distribution of the extracted head image can be examined in more detail, the orientation of the head can be determined with high accuracy.
[0036]
  Claim14The invention of claim7In the human presence position detecting device described above, the human presence position detecting means is configured to determine a center position of a human detection distance range corresponding to the voting space used for detecting the head position of the person as a distance from the imaging device to the person. It is estimated.
[0037]
In the above configuration, not only the detection of the position of the person in the captured image but also the distance from the imaging apparatus to the person in the real world can be estimated.
[0038]
  Claim15The invention of claim7In the human presence position detection device described above, the human presence position detection means obtains a human presence position direction with respect to the imaging device based on the detected position of the head on the captured image and the focal length of the imaging device, and the human head The center position of the human detection distance range corresponding to the voting space used for position detection is estimated as the distance from the imaging apparatus to the person, and the position of the person relative to the imaging apparatus is obtained from the direction of the person and the distance to the person. Is.
[0039]
In the above configuration, not only the detection of the position of the person in the captured image but also the distance from the imaging apparatus to the person and the direction in the real world can be known.
[0040]
  Claim16The invention of claim7Or claims14In the human presence position detecting device described in the above, the human presence position detecting unit divides the space in the line-of-sight direction of the imaging device into predetermined zones from the imaging device, and the previous presence state of each person in each zone Compare the detection result with the current detection result, and if the zone to which the currently detected person belongs is closer to the zone to which the previously detected person belongs, determine that the person is approaching, If the zone to which the previously detected person belongs is farther than the zone to which the previously detected person belongs, it is determined that the person is moving away, and if the zone to which the currently detected person belongs and the zone to which the previously detected person belongs are the same, the person is stopped It is determined.
[0041]
In the above-described configuration, the movement of the person whether the person is approaching or moving away from the imaging device by dividing the zone at a predetermined distance from the imaging device and comparing the presence or absence of the person in each zone Can know.
[0042]
  Claim17The invention of claimTo 15In the human presence position detection device described above, the human presence position detection means creates a human presence map in which the space in the line-of-sight direction of the imaging device is divided into a plurality of areas based on the imaging device position, A person's location is detected by classifying an area where a person is detected, an unconfirmed area, and an area where no person is present.
[0043]
In the above configuration, a human presence map is created, and it is possible to know the existence of a person from the imaging device by distinguishing the area where the person was detected in the real world, the unconfirmed area, and the area where there is no person. It can be applied to the control of autonomous mobile devices that perform advanced mobility control.
[0044]
  Claim18The invention of claimTo 15In the human presence position detection device described above, the human presence position detection unit detects a person by creating a human presence map in which a space in the line-of-sight direction of the imaging device is divided into a plurality of areas based on the imaging device position. Register the position at the corresponding position on the map, store the cut-out image of the detected person's head in association with the detected position, compare the previous detection point map with the current map, The current person detected within a movable range from the detection position of the person of interest at the time of detection and its detection position are used as candidates for association. If there are multiple attachment candidates, calculate the image correlation between the extracted image of the head stored in association with the detection position of the person at the time of the previous detection and the extracted image of the person's head that is the matching candidate And correlation It is to track the human and detecting the position of the highest correlation candidate as a correct correspondence.
[0045]
In the above configuration, since the detected person is identified and tracked, the movement of the person can be accurately known.
[0046]
  Claim19In the method of detecting the presence and position of a person by extracting a person's head image from a captured image captured by an imaging device, the shape of the person's head image is surrounded by an outwardly convex curve The contour of the human head image, which is approximated by the shape model of the figure, and determined based on the focal length of the lens of the imaging device, the detection distance from the imaging device to the person to be detected, and the standard size of the human head A standard size and a size range of the shape model are determined in advance, a concentration gradient direction is defined at each point on the boundary curve of the shape model of the standard size, and the shape model A representative point direction line defined as a line connecting a representative point to a point of interest on a boundary curve of the shape model, with a representative point defined inside, a voting unit corresponding to the standard size of the shape model at the representative point On the line A voting template preparation process that includes a voting unit corresponding to the size range before or after the representative point and prepares a voting template created by a boundary curve of the shape model of the standard size so that it can be referred to, and input from the imaging device , A captured image data storage process for storing the digitized captured image in a memory, and a differential process for extracting an edge portion composed of pixels corresponding to a contour of a person, a contour of a face, and the like on the captured image, Edge image data generation that extracts pixels having a predetermined differential intensity or more as edge pixels, calculates a density gradient direction from the captured image, and generates an edge image with density gradient direction data added to each extracted edge pixel And a voting space having voting cells at positions corresponding to the pixel positions of the captured image and the edge image. A voting space generation process to be performed, and a point on the boundary curve of the voting template having a density gradient direction corresponding to the density gradient direction added to the edge pixel is superimposed on each edge pixel of the edge image. The voting cells in the voting space corresponding to the pixel positions on the edge image where the voting part of the template is located are voted and stored in the memory for each edge pixel, and the voting values of each cell in the voting space are totaled Voting process to be performed and a cell having a voting value equal to or higher than a predetermined threshold in the voting space where the voting is performed, and a position on the captured image corresponding to the detected cell is detected as a human head position. Human location detection processIn the voting template preparation process, for each point on the boundary curve of the shape model of the standard size, the points on the boundary curve having a concentration gradient direction opposite to the concentration gradient direction defined for the point are conjugated with each other. Defined as a conjugate point, one of the points conjugate to each other maintains the same relative positional relationship as the voting unit inside the shape model, and the other of the points conjugated to each other further has a voting unit outside the shape model. Prepare the voting template created by the boundary curve of the standard size shape model so that it can be referred toThis is a human location detection method.
[0047]
  In the above detection method, the person's position is represented by the position of the person's head, and a simple figure shape model surrounded by an outwardly convex curve such as a circle or an ellipse is used as the person's head model, so-called Hough transform Since the person's head is detected by the above, the presence position of the person can be detected by a simple voting process. In addition, the Hough transform voting process is performed only on both sides of the edge pixel based on the density gradient direction of the target edge pixel without performing it in all directions of the target edge pixel. Processing can be realized. In addition, since a voting template that uses only the size of the head of a person within a specific detection distance range from the imaging device as a detection target is used, the detection distance can be estimated and the human head highly reliable on the captured image Detection can be performed.Further, in the above detection method, the voting template conjugate to each other is provided in the voting template so that voting processing can be simultaneously performed on an image in which the image density in the human head is inverted, so that the image density is inverted by a single voting process. In addition to the above-described effects, more stable and reliable detection can be performed at high speed.
[0048]
  Claim20The invention of claim15To claims17It is an autonomous mobile device which controls movement using the position information of the person detected using the person presence position detecting apparatus described in any of the above.
[0049]
In the above configuration, the imaging device is mounted on the autonomous mobile device, the person is detected using the human presence position detection device, the position of the person around the autonomous mobile device is recognized, and the autonomous mobile device is moved. Therefore, it is possible to realize an operation of avoiding a person by adapting to the presence of the person and a retreating operation of giving up a passage to the person.
[0050]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, a human presence position detection device, a detection method thereof, and an autonomous mobile device using the detection device according to an embodiment of the present invention will be described with reference to the drawings. FIG. 1 shows a block configuration of a human presence position detection apparatus. The human presence position detection device performs image processing on an image captured by the imaging device 2 in the human presence position detection device main body 1 and detects a human presence and its position by extracting a human head image. The result is output to the outside as an alarm sound using, for example, the alarm generation means 9. A method applying generalized Hough transform is used as a method for extracting a human head image.
[0051]
An outline of the human presence position detection process will be described. The image input from the imaging device 2 is stored in the memory as a captured image that is A / D converted from an analog image to a digital image in the captured image data storage unit 3. Next, in the edge image data generation means 4, the captured image is differentiated with respect to the image density, a point having a differential intensity value equal to or greater than a predetermined threshold is extracted as an edge pixel, and an edge image composed of the extracted edge pixels is obtained. Generated. The generated edge image is stored in the memory with the data of the density gradient direction at each edge pixel point added.
[0052]
Subsequently, the voting processing means 5 uses a voting template for each detection distance prepared by the voting template preparation means 6 and applies a generalized Hough transform to the voting space prepared by the voting space generation means 7. The vote is performed. Next, in the human presence position detecting means 8, a cell having a vote value equal to or greater than a predetermined threshold is detected for each voting space corresponding to each detection distance range, and a captured image corresponding to the detected cell position is detected. The presence of the human head is detected at the upper position. The person presence position detecting means 8 determines the center distance value of the detection distance range (zone) corresponding to the voting space in which the person's head is detected as the distance from the imaging device to the person, and the zone number is determined. Output to the alarm generation means 9. The alarm sound generating means 9 selects an alarm sound corresponding to each zone number and generates an alarm sound.
[0053]
Such a human presence position detecting device can be used as an area sensor, for example. FIG. 2 shows the concept of the area sensor. The imaging area AR in front of the visual field of the imaging device 2 of the human presence position detection device is divided into a plurality of detection zones ZN1, ZN2, and ZN3 by distances L1 to L4 from the reference point O of the imaging device 2. The operation as an area sensor when a person is detected is set for each zone. For example, as an operation based on the distance and urgency for each zone, a strong warning sound is generated in the first zone ZN1, a medium warning sound is generated in the second zone ZN2, and a weak warning sound is generated in the third zone ZN3. it can. Hereinafter, each constituent unit of the human presence position detection device will be described.
[0054]
(Captured image data storage means and edge image data generation means)
Processing in the captured image data storage unit 3 and the edge image data generation unit 4 will be described with reference to FIGS. FIG. 3 shows a processing flow of the captured image data storage means 3. An analog signal of an image captured by the imaging device 2 is A / D converted, and a captured image P0 is created (S101) and stored in a memory (S102). FIG. 4A shows an example of the captured image P0.
[0055]
Further, the edge image data generation means 4 performs a differentiation process on the captured image P0, and an edge image P1 shown in FIG. 4B is obtained. FIG. 5 shows a flow of edge image generation. The differentiation step (S201) will be described. The captured image P0 is a grayscale image. The differentiation process for extracting the edge portion composed of pixels corresponding to the outline of the person, the contour of the face, etc. from this grayscale image is based on the idea that “the edge corresponds to the portion where the density change is large”. . As shown in FIG. 6, the differentiation process is performed by scanning the captured image P0 in the horizontal direction x and the vertical direction y, respectively, by a 3 × 3 pixel local parallel window W33. As shown in FIG. 7A, the local parallel window W33 includes a pixel I (i, j) = a5 of interest, and eight pixels (near 8) a1 to a4 and a6 to a9 around the pixel a5. Composed. An x-direction differential value Gx (i, j) is calculated by applying the x-direction differential operator OPx shown in FIG. 7B to each pixel a1 to a9 in the local parallel window W, and FIG. The y-direction differential value Gy (i, j) is calculated by operating the y-direction differential operator OPy shown in FIG. The calculation formula is as follows. However, in the following, the arguments i and j are partially omitted.
Gx = a3 + 2 * a6 + a9-a1-2 * a4-a7
Gy = a1 + 2 × a2 + a3-a7-2 × a8-a9
Based on these, the differential intensity value J (i, j) and the concentration gradient direction Ψ (i, j) are obtained by the following equation.
J = sqr (Gx × Gx + Gy × Gy)
Ψ = arctan (Gy / Gx)
Here, sqr is a square root operation, and arctan is an arc tangent. The concentration gradient direction ψ when Gx or Gy is 0 is ψ = 90 ° when Gx = 0, Gy> 0, ψ = 270 ° when Gx = 0, Gy <0, Gy = 0, Gx> When 0, ψ = 0 °, Gy = 0, and when Gx <0, ψ = 180 °. The above processing is performed in the differentiation step (S201).
[0056]
Subsequently, the differential intensity value J (i, j) in each pixel is compared with a predetermined threshold value, and a pixel I (i, j) having a differential intensity value greater than or equal to the predetermined threshold value is extracted as an edge pixel. (Yes in S202). The density gradient direction Ψ (i, j) data is stored at the edge pixel position on the memory (S203). After differential processing is performed on all the pixels of the captured image P0 (Yes in S204), the edge image P1 is stored in the memory (S205).
[0057]
(Voting template preparation means)
Processing in the voting template preparation means 6 will be described with reference to FIGS. FIG. 8 shows a shape model of the human head. In order to detect the human head image in the captured image, and to detect the presence and position of the person in the captured image, the shape model of the figure surrounded by an outwardly convex curve Approximate. For example, the triangular shape T1 in FIG. 8A, the square shape T2 in FIG. 8B, the elliptic shape T3 in FIG. 8C, and the circular shape T4 in FIG. 8D are used as the shape models. Inside each convex shape model, for example, a representative point G is defined at the center of gravity of the figure.
[0058]
The sizes of these shape models will be described. FIG. 9 shows the dimensional relationship between the real person and the person in the image. Based on the focal length f of the lens of the imaging device, the detection distance L from the imaging device to the person to be detected, and the standard size of the human head in the real world (head width D and head height H), The size (head width d and head height h) of the head image of the person in the captured image is determined. The standard size and size range of the shape model are determined in advance in consideration of the size range and the width of the detection distance L to the standard size of the human head in the real world.
[0059]
The voting template created from the shape model will be described. 10A and 10B show voting templates based on the triangular shape model T1. As shown in FIG. 10A, each point on the boundary curve constituting the standard size shape model T10 is, for example, a direction (normal direction) orthogonal to the tangential direction at each point, and the inside of the shape model. A density gradient direction vector N is defined in the direction (inner side surrounded by the convex curve). The density gradient direction vector N is specified by, for example, the density gradient direction Ψ with reference to the horizontal direction x on the image. In addition, a voting unit Vp0 corresponding to the standard size of the shape model is provided at the representative point G defined inside the shape model. Also, as shown in FIG. 10B, the representative point direction line r defined as a line connecting the representative point to the point of interest on the boundary curve of the shape model, and before or after the representative point G Voting units Vp1 and Vp2 corresponding to the size range are provided. These voting units Vp1 and Vp2 are voting units that can simultaneously vote for a smaller shape model T11 and a larger shape model T12 than a standard size shape model T10, and absorb fluctuations from the standard size range. is there. In this manner, a voting template having a shape model having a concentration gradient direction Ψ and voting units Vp0, Vp1, and Vp2 is created for each standard shape model and prepared so that it can be referred to the memory. If the size range of the shape model is large, a larger number of such voting units can be provided on the representative point direction line r as necessary.
[0060]
A voting template corresponding to an image that is light and dark is described. 11 and 12 show voting templates when the image density of the human head is reversed. As will be described later, the voting template is used by referring to the density gradient direction at the pixel points of the edge image and the density gradient direction of the voting template. As shown in FIG. 11A, the density gradient direction vector N is directed from an image part having a low density level, that is, a dark image part A2, to an image part having a high density level, that is, a bright image part A1, at each edge pixel px of the edge image. Defined as vector N. By the way, depending on the lighting condition when the human head image is captured, there are cases where the density gradient direction with respect to the human head is inward of the head and outward of the head. For example, when the head is bright with respect to the background (imaging mode p), the density gradient direction vectors N1 and N2 are inward as shown in FIG. 11B, and when the head is dark with respect to the background (imaging mode). q), the density gradient direction vector N is outward as shown in FIG.
[0061]
The efficiency of human head detection can be increased by providing a voting unit so that it can cope with such two types of head images. The method will be described. For each point on the boundary curve of the standard size shape model, for example, the density gradient direction vector N2 opposite to the concentration gradient direction vector N1 defined at the point P on the shape model Tp in FIG. Another point Q on the boundary curve is defined as a conjugate point conjugate to each other. Then, one point Q of the conjugate points maintains the same relative positional relationship as the voting unit Vp2 included in the shape model, and the other point P of the mutually conjugate points is further outside the shape model Tp. Be prepared. FIG. 12 shows a voting template having a voting unit corresponding to an image that has been inverted in this way. The density gradient directions ψ and Φ correspond to the density gradient directions of the voting template for the imaging modes p and q, respectively. Vp1 and Vp2 in FIG. 11B correspond to Vp and Vq in FIG.
[0062]
A description will be given of how to deal with fluctuations in the concentration gradient direction. FIG. 13 shows a voting template having a two-dimensional voting unit. Since the voting template is used by referring to the density gradient direction at the pixel point of the edge image and the density gradient direction of the voting template, the original grading direction of the edge pixel is assumed from the head contour in the head image. When a deviation from the concentration gradient direction occurs, a highly reliable detection can be performed by providing a voting unit that absorbs the deviation. As such a voting unit, as shown in FIG. 13A, a voting unit is further provided near the representative point of the voting template and on both sides of the representative point direction line r. In the example shown in the figure, for each point on the boundary curve of the shape model, nine voting parts Vp0 to Vp8 are provided inside the shape model, and nine voting parts Vq0 to Vq8 are provided outside the shape model. FIG. 13B shows how the added voting unit absorbs fluctuations in the concentration gradient direction.
[0063]
The voting template table will be described. FIG. 14 shows the coordinates of the voting unit, and FIGS. 15 and 16 show voting template tables. A voting template table is prepared to facilitate comparison reference between the density gradient direction of the voting template and the density gradient direction at the pixel points of the edge image. As shown in FIGS. 14A and 14B, the voting template table indicates the relative positions Δxp, Δyp, Δxq, Δyq (argument omitted) of the voting parts Vp, Vq with respect to each point on the boundary curve of the shape model. The density gradient directions Ψ and Φ at the points are stored as an index in a memory in a tabular format.
[0064]
Next, types of shape models and voting templates based on these shape models will be described. FIG. 17 shows examples of types of shape models used in the present invention. As the types of shape models, in addition to the shape types T1 to T4 described above, standard sizes for the zones ZN1, ZN2, and ZN3 of each detection distance range, and shape models with left and right tilts with tilted heads are used. . The voting template preparation unit arbitrarily sets a plurality of detection distance ranges from the imaging device to the person to be detected, and prepares a standard size and size range of the shape model for each detection distance range and a voting template based on these. The voting space generating means generates a voting space for each detection distance range, and the voting processing means performs the voting process for the voting space corresponding to the detection distance range for each voting template corresponding to each detection distance range. Further, when the person presence position detecting means detects cells having a vote value equal to or larger than a predetermined threshold in the voting space corresponding to each detection distance range, the person presence position is made to correspond to the detection distance range. Only a person within a specific distance range from the imaging device can be selected and detected from many people.
[0065]
In addition, when the size range of the shape model is equal to or larger than the predetermined value, the voting template preparation unit divides the human detection distance range so that the size range is equal to or smaller than the predetermined value (in the above description, the total divided range is 3 The standard size and size range of the shape model and a voting template based on these are prepared for each of the divided ranges, and the voting space generation means generates a voting space for each of the divided detection distance ranges. The voting processing means performs the voting process on the voting space corresponding to the detection distance range for each voting template corresponding to each of the divided detection distance ranges. If the range to detect people is wide and the size range becomes large, the voting part of the voting template becomes longer corresponding to the size range of the shape model, and an error in creating a cell with a high false voting value due to incorrect voting The possibility of occurrence of detection increases. By dividing the range in which a person exists and setting the size range of the shape model to a predetermined value or less, only a head having a specific size can be detected, and the detection accuracy can be increased.
[0066]
The voting template preparation means approximates a human head shape with a plurality of shapes having the same shape model and different postures. A voting template is prepared for each shape, the voting space generating unit generates a voting space for each shape of each posture, and the voting processing unit performs voting processing on each cell on the voting space for each shape having a different posture. . Since detection is performed for each shape model having a different posture, a tilted head can also be detected.
[0067]
Further, the voting template preparation means approximates a human head shape with a plurality of different shape model shapes, prepares a voting template for each shape model shape, and the voting space generation means for each shape model shape. The voting space is generated for each cell on the voting space for each of the shape models. Since detection is performed using different shape models, various head shapes can be detected.
[0068]
(Voting space generating means and voting processing means)
The relationship between the voting space and its generation and the captured image / edge image will be described. FIG. 18 shows the relationship between the coordinates of edge pixels and voting cells. As shown in FIG. 18, the voting space generating means generates a voting space P2 having a voting cell VC at a position corresponding to each pixel position of the captured image P0 and the edge image P1, and stores it in the memory. A plurality of voting spaces P2 are generated according to the types of voting templates.
[0069]
The voting and counting process will be described. The voting processing means superimposes a point on the boundary curve of the voting template having a density gradient direction vector N2 corresponding to the density gradient direction vector N1 added to the edge pixel on each edge pixel on the edge image P1. When voting cells Vp and Vq in the voting space P2 corresponding to the pixel position on the edge image P1 where the voting part of the voting template is located are voted and stored in the memory, these are performed for each edge pixel and voted. The voting value of each cell in the space is aggregated. FIG. 19 shows the distribution of voting values in a voting space that is voted and aggregated. When a plurality of types of voting templates are used, the voting processing means performs these voting and counting processes on the voting space provided for each voting template.
[0070]
The use of the curvature formed by the edge pixel row at the time of voting will be described. FIG. 20 shows the relationship between the curvature center direction and the voting part. For each edge pixel of the edge image, the voting processing means determines the direction ρ in which the center of curvature exists by obtaining the curvature of the contour composed of the edge pixel px of interest and a plurality of edge pixels in the vicinity thereof, and the edge pixel px of interest When voting for voting, the voting cell in the voting space corresponding to the position of the voting part V in the direction ρ where the center of curvature exists is voted. Since the center of curvature of the contour composed of edge pixels generated from the contour of the human image faces the head image center direction regardless of the density mode of the human head image, the voting process can be performed only on one effective side.
[0071]
(Person presence detection means and alarm sound generation means)
The determination of the person presence zone will be described. FIG. 21 shows a voting space for each zone where voting is performed, and FIG. 22 shows a relationship between human detection in the voting space and a zone in the real space. The human presence position detecting means detects a cell having a voting value equal to or larger than a predetermined threshold in the voting spaces P21, P22, and P23 where voting is performed, and determines the position on the captured image corresponding to the detected cell. Detect as head position. In the case of FIG. 21, it is assumed that the cell C5 in the voting space P22 indicates a human head, and the coordinates x2 and y2 are also the position coordinates on the captured image. The human head detected in the captured image, and thus the presence position of the detected person in the real world, depends on the division of the detection distance range shown in FIG. 22 when the human head image is detected in the voting space P22. The second zone is ZN2. In this case, the average value of the zone boundary distances L2 and L3 (the center position of the detection distance range) LZ2 = (L2 + L3) / 2 is used as the distance from the imaging device to the person.
[0072]
A case will be described in which voted cells having vote values greater than or equal to a predetermined value are distributed within a specific range. FIG. 23 shows a case where a hill of vote values has a plurality of ridges. When the cell C6 having a vote value equal to or more than a predetermined threshold value detected as shown in the figure is divided into lumps having a plurality of folds, the human presence position detecting means examines the distance between the lumps, When the distance is within a predetermined range, the whole is grouped and integrated into one, and the position (x3, y3) on the captured image corresponding to the representative point of the integrated lump is detected as the human head position. To do. By integrating cells having voting values equal to or greater than a predetermined threshold value, it is possible to accurately detect a contour model and accurately detect a human head position even when voting is dispersed.
[0073]
The human presence position detecting means detects a cell having a voting value equal to or greater than a predetermined threshold value for each voting space of the voting space provided for each divided detection range, and an image corresponding to the detected cell. The upper position is detected as a human head candidate position. Thereafter, the human presence position detecting means examines the distance between the head candidate positions detected in the voting space for each of the divided detection ranges, and if the distance is within the predetermined range, it is grouped and integrated. The position on the image corresponding to the representative point of the integrated head candidate is set as the human head position. By integrating the results obtained for each detection range, highly reliable detection can be performed.
[0074]
The human presence position detection means processes the detection processing of the human presence position in the order of the detection range farther from the detection range closer to the imaging device, and when the human head position is detected, the detected captured image is detected. A predetermined range around the upper position may be stored as a detection end region, and when the detection range far from the imaging apparatus is processed, the detection end region may not be detected. When the head is detected by performing the processing of the detection distance range on the side closer to the imaging device, the head is projected greatly on the captured image, in that part of the captured image, Since it is behind the detected image, there is no possibility of human detection. By stopping detection, the processing amount can be reduced and high-speed processing can be realized.
[0075]
The detection distance range and alarm sound generation will be described. FIG. 24 shows a processing flow of the alarm sound generating means. The warning sound generating means operates based on the distance and urgency for each zone, and is a weak warning sound (S302) in the third zone ZN3, a medium warning sound (S304) in the second zone ZN2, and in the first zone ZN1. A strong warning sound is generated (S306).
[0076]
(Person presence detection device that uses a distance sensor)
A combination of the distance sensor and the human presence position detection device will be described. FIG. 25 shows a block configuration of the detection apparatus. A more efficient operation can be realized by using a distance sensor in combination with a human presence position detection device. The distance to the object is measured by the distance sensor 10 arranged in the vicinity of the imaging device 2, for example, vertically above the imaging device 2. Based on the measurement result, the zone selection means 11 selects a detection distance range from the zones ZN1, ZN2, and ZN3, and determines the size of the head image. Using a voting template corresponding to the determined head image size, a voting space is generated only for the selected zone, and subsequent processes such as voting are advanced. The number of person presence position detection processes by voting usually requires (number of zones) x (number of head shapes) x (number of head postures) times, but by using a distance sensor, (number of head shapes) X (the number of head postures) times, and the number of processes can be reduced to 1 / (number of zones). In order to perform such processing, the voting template preparation means stores the voting template data in advance for each distance from the imaging device to the person, and the voting processing means measures the distance of the distance sensor that measures the distance to the detection target. Based on the measurement result, refer to the data of the voting template to set a voting template and vote. Since the distance from the imaging device to the person is obtained by a distance sensor and detection is performed by limiting the size of the head detected on the image, highly reliable detection can be performed.
[0077]
(Movement control of autonomous mobile devices)
An application of the human presence position detection device to the autonomous mobile device will be described. FIG. 26 is an external view of the autonomous mobile device, and FIG. 27 shows a block configuration thereof. The autonomous mobile device 12 detects obstacles and people M1, M2 and M3, and controls the movement of the obstacles and people while avoiding / stopping and generating alarm sounds. It is a device that performs the pulling. The autonomous mobile device 12 includes an imaging device 2a and a human presence position detection device main body 1a in the front, and an imaging device 2b and a human presence position detection device main body 1b in the rear. Outputs of human detection results from the front and rear human presence position detection device main bodies 1a and 1b are transmitted to the travel route planning unit 101 and the wheel drive control unit 102, and the drive motor 104 is driven by the motor driver 103. Autonomous movement is performed. Further, the rotation speed of the drive motor 104 is measured by the encoder 105, and the travel distance and travel speed are obtained by the travel distance measurement unit 106 and fed back to the wheel drive control unit 102.
[0078]
(Detection of human presence direction)
Determining the presence direction of the detected person will be described. FIG. 28 shows the determination method. The human presence position detection means obtains an angle φ = arctan (b / f) in the human presence position direction with respect to the imaging device 2 from the detected head position b on the captured image and the focal length f of the imaging device. The central position of the person's detection distance range corresponding to the voting space (the shape model size is determined by the distance, the voting template and the corresponding voting space are determined) used for detecting the human head position, for example, For the detected third zone ZN3, LZ3 = (L3 + L4) / 2 is estimated as the distance from the imaging device to the person. Thus, the position of the person relative to the imaging device in the real world can be obtained from the direction (angle φ) of the person M and the distance to the person (distance LZ3).
[0079]
(People presence map, human approach / distraction, and tracking)
Human movement detection between zones (person detection distance range) will be described. FIG. 29 shows an example of the passage of time of the person presence position. The human presence position detection means divides the space in the line-of-sight direction of the imaging device into a third zone ZN3 at a long distance, a second zone ZN2 at a middle distance, and a first zone ZN1 at a short distance at a predetermined distance from the imaging device. Then, the detection result at the previous detection time t = T (−1), the detection result at the next detection time t = T (0), and the subsequent detection time t = T ( As in +1), the movement of the person in the real world is determined by comparing the position of the person in time series. In the example shown in FIG. 29, it is determined that the person is approaching because the zone ZN1 to which the currently detected person belongs is closer to the zone ZN2 to which the previously detected person belongs. Further, since the zone ZN2 to which the person detected at the subsequent time t = T (+1) belongs is a farther zone than the zone ZN1 to which the currently detected person belongs, it is determined that the person is moving away. Further, when the zone to which the currently detected person belongs and the zone to which the previously detected person belongs are the same, it is determined that the person is stopped. The identification of the detected person's identity, so-called tracking will be described later.
[0080]
Explain the human presence map in the real world. FIG. 30 shows the meaning of the person presence map. The human presence position detection means creates a human presence map 30 in which the space in the line-of-sight direction of the imaging device 2 is divided into a plurality of, for example, 15 × 9 areas based on the position of the imaging device 2, The presence position of the person is detected by classifying the area 31 where the person is detected, the area 32 where no person is present, and the unconfirmed area 33. Such a person presence map 30 can be applied to advanced movement control that does not collide with a person in the real world, for example, when controlling an autonomous mobile device.
[0081]
The tracking of a person using a person presence map will be described. FIG. 31 shows a person tracking method. The person presence position detecting means creates the person presence map 30 as described above, registers the position where the person is detected in the corresponding positions E1, E2, E3 on the person presence map 30, and detects the head of the detected person. The extracted image is stored in association with the detection position (the extracted image will be described later). Then, the map of the previous detection time t = T (−1) is compared with the map of the next detection time t = T (0), and the detection positions E1, E2, E3 of the person of interest at the previous detection time are respectively compared. Are detected within the movable ranges AR1, AR2, AR3 and their detected positions F1, F2, F3 as candidates for association. When there is one association candidate such as the position E1 and the position F1, the association candidates are tracked. When there are a plurality of association candidates such as positions F2 and F3 with respect to the position E2 and positions F2 and F3 with respect to the position E3, a cut-out image of the head stored in association with the detection position of the person at the previous detection time point, The image correlation with the cut-out image of the person's head as the candidate for association is calculated, and the person with the highest degree of correlation and the detected position are tracked as correct association.
[0082]
(Verification of the head image of the detected position)
A description will be given of the extraction of the detected human head image from the captured image. FIG. 32 shows the extraction of the head image. The human presence position detecting means cuts out an image having a size (head width d, head height h) of a projection image obtained by projecting a shape modeling the head on the detected head position on the captured image P0. . The clipped image 40 is normalized by brightness and size, compared with head / non-head data stored in advance, and the detection of the head is verified. The sizes d and h of the projected image are obtained as follows. The human presence position detecting means detects a cell having a voting value equal to or greater than a predetermined threshold value for each voting space corresponding to each detection distance range, and determines the position on the captured image P0 corresponding to the detected cell. Detect as head position candidate. The center position of the detection distance range corresponding to the voting space in which the human head candidate is detected is output as the distance from the imaging device to the human candidate, and the zone number is output as the number of the zone where the human candidate exists. For example, as shown in FIG. 2 above, when there are human candidates in the third zone ZN3, the detection range of the human candidates is L3 on the side closer to the imaging device and L4 on the far side, and accordingly, from the imaging device to the human candidate. Is set to LZ3 = (L3 + L4) / 2. The head image size is d = f · D / LZ3 and h = f · H / LZ3. Here, D and H are standard head sizes of the shape model used for detection.
[0083]
(Pattern recognition)
The pattern matching between the head standard image and the cut-out image will be described. FIG. 33 shows a head pattern space, and FIG. 34 shows a pattern recognition flow of a head image. By performing pattern recognition of the head image, it is possible to determine the identity of the detected person and to track the movement of that person. In addition, error detection can be prevented. Pattern recognition of the head image is performed by comparing the head image with the head pattern partial space HS0 in the n-dimensional pattern space that has been learned and stored in advance in the memory shown in FIG. First, a voting space corresponding to the detection distance range of the zone number m in the imaging area AR is selected as a processing target (S401). If the voting cell in the voting space does not have a voting value greater than or equal to a predetermined threshold, the process proceeds to evaluation of the next cell (No in S402), and if there is a voting value greater than or equal to the predetermined threshold, a human candidate is detected in that cell As a result, the distance LZm = (L (m + 1) + L (m)) / 2 in the real world to the human candidate is calculated (S403). Next, the head image size, width d = f · D / LZm, and height h = f · H / LZm are calculated (S404). A head image is cut out from the captured image with the obtained head image sizes d and h (S405).
[0084]
Subsequently, normalization is performed to transfer the cut out head image to the n-dimensional pattern space. In the normalization, the head standard image size dn, hn whose image size is determined in advance, and the density average value Av and the density average value Av and density of the pixels in the head standard image size whose density dispersion value is determined in advance for the image density. The distribution value Var is set (S406). An n (n = dn × hn) -dimensional vector is created using the values of the pixels of the normalized head image as vector elements, and this is set as a normalized head image vector Vhn (S407). Next, the position occupied by the normalized head image vector Vhn in the n-dimensional pattern space is examined (S408), and an n-dimensional image is obtained using a number of known normalized head images and non-head images in advance. It is checked whether or not the normalized head image vector Vhn is included in the head pattern partial space among the partial spaces provided in the vector space (S409). If the normalized head image vector Vhn is included in the head pattern subspace (Yes in S409), the clipped image is determined to be a human head image (S410), and the head image position LZm is output ( S411). If the normalized head image vector Vhn is not included in the head pattern subspace (No in S409), it is determined that the cut-out image is not a human head image (S412). After the above processing is performed for all cells (S413), the processing of the voting space corresponding to the next zone number (m + 1) is performed (S414). Note that steps S403 and S404 may be processed only once immediately after step S401.
[0085]
(Average value, variance value)
The determination using the density average value and the variance value of the head image will be described. FIG. 35 shows a flow of the same determination. The average value Avx and variance value Varx of the clipped head image are compared with the average value Av and variance value Var of a known head image calculated in advance (S501). The average value Avx and variance value Varx of the clipped head image are within a predetermined allowable range α, β, that is,
Av−α <Avx <Av + α
Var−β <Varx <Var + β
In this case, it is determined that the clipped image is a head image (Yes in S502, S503). In this way, the average value / dispersion value of the brightness of the clipped image is obtained, and by comparing the average value / dispersion value with a preset average value / dispersion value threshold, By verifying the detection, a highly reliable head position can be detected.
[0086]
(Skin color)
The determination using the skin color of the head image will be described. FIG. 36 shows a flow of the same determination. As another verification method, the ratio Sx of the skin color pixels in the cut-out image is compared with the ratio Sk of the skin color pixels in the face image facing forward, which is obtained in advance (S61). The skin color pixel ratio Sx is within a predetermined shared range γ, that is,
Sk-γ <Sx <Sk + γ
At this time, it is determined that the clipped image is a head image facing the front (Yes in S602, S603). As described above, the head position can be detected with high reliability by verifying whether or not the detected head image is the head using the ratio of the skin color included in the clipped image.
[0087]
(Head direction detection)
The detection of a head that is not a front-facing head using the head pattern space will be described. FIG. 37 shows the head pattern space. As described above, an n (n = dn × hn) -dimensional vector is created using the values of each pixel of the normalized head image as vector elements, and this is defined as a normalized head image vector Vhn. Using a number of known normalized frontal head images, leftward head images, rightward head images, and non-head images in advance, the frontal head pattern subspace HS0, leftward heads in an n-dimensional vector space The sub-pattern subspace HSL and the right-facing head pattern subspace HSR are learned, and when a human head is detected, whether or not the normalized head image vector Vhn falls within any head pattern space. Then, it is verified whether or not the cut out head image is surely a human head, and the orientation of the head is identified. When it is verified that the head is a human head, the head position on the captured image is output as the position of the person on the captured image, and the head orientation is also output.
[0088]
(Determining head orientation)
The determination of the head orientation using the head pattern space will be described. FIG. 38 shows an example of a head pattern space, and FIG. 39 shows an example of a head image. When a plurality of candidates are output as the head direction, the human presence position detection unit divides the cut image into a plurality of predetermined small images, and the difference in density distribution between the divided small images Determine head orientation. As shown in FIG. 38, the head pattern partial spaces HS0, HSL, HSR for each head orientation described above overlap, and further, the normalized head image vector Vhn exists in the overlapping space. May be simultaneously output as head orientation candidates. In order to identify the correct head orientation even in such a case, as shown in FIG. 39A, a dividing line 41 is drawn at the center of the cut-out head image 40, and regions 42 on both sides of the dividing line. For 43, the ratio of the sum of the densities in each region is compared. The sum of the density on the left side of the dividing line is SL, the sum of the density on the right side is SR, and the determination thresholds are th1 and th2.
If th1 ≦ SL / SR, turn right
When th2 <SL / SR <th1, facing front
If SL / SR ≦ th2, turn left
The head orientation is determined as.
[0089]
(Method of detecting the presence and position of a person)
A method for detecting the presence and position of a person by extracting a head image of the person from a captured image captured by the imaging apparatus will be described. FIG. 40 shows the flow of the detection method. First, a voting template is prepared in a voting template preparation process (S701). In this voting template preparation process, (1) a shape model of a figure surrounded by an outwardly convex curve is prepared, and (2) the focal length of the lens of the image pickup device and the image pickup device are prepared. The size of the contour of the human head image on the captured image is estimated based on the detection distance from the target to the person to be detected and the standard size of the human head, and (3) the standard size of the shape model and Size range is determined. (4) A density gradient direction is defined at each point on the boundary curve of the standard size shape model, (5) a representative point is defined inside the shape model, and (6) the shape model is defined at the representative point. A voting part corresponding to the standard size of the system is provided. And (7) a voting unit corresponding to the size range on the line of the representative point direction defined as a line connecting the representative point to the representative point on the boundary curve of the shape model and before or after the representative point. Provided. (8) It is also possible to provide a voting unit for dealing with the head image whose density has been inverted on the outside of the boundary curve of the shape model as a pair of voting units inside and outside the shape model. In this way, a standard size shape model is prepared for each human detection distance range and human head shape model, and the density gradient direction and a plurality of voting units are added to the standard shape model. A prepared voting template is created and stored in a memory so that it can be referred to.
[0090]
In the voting space generation process, a voting space having voting cells at positions corresponding to the pixel positions of the captured image and the edge image is generated and stored in the memory (S702).
[0091]
Next, in the captured image data storage process, the digitized captured image input from the imaging device is stored in the memory (S703). Next, in the edge image data generation process, an edge portion composed of pixels corresponding to the outline of the person, the contour of the face, etc. is extracted from the captured image by differentiation processing, and pixels having a predetermined differential intensity or higher are used as edge pixels. In addition to the extraction, the density gradient direction calculated from the captured image is added to each extracted edge pixel to generate an edge image (S704).
[0092]
Next, in the voting process, (1) one of the prepared voting templates is selected, and (2) the density corresponding to the density gradient direction added to the edge pixel on each edge pixel of the edge image. The points on the boundary curve of the voting template having the gradient direction are overlapped. (3) At this time, the voting cells in the voting space where the voting part of the voting template is located are voted and stored in the memory. (4) Voting processing is performed for all edge pixels. Next, (5) another voting template is selected, and the same voting process as described above is performed on the voting space corresponding to the voting template. (6) Such a voting process is performed for all voting templates. In each vote, the vote values of each cell in the voting space are totaled (S705).
[0093]
Next, in the person presence position detection process, (1) a cell having a voting value equal to or higher than a predetermined threshold value is detected in each voting space where voting is performed, and (2) a predetermined amount of voting value is detected. The position on the captured image corresponding to the cell is detected as the human head position on the captured image (S706).
[0094]
When detection is performed again after the above-described series of human presence position detection processes is completed (No in S707), the above processes are repeated from step S703.
[0095]
Unlike the conventional Hough transform voting process in which voting is performed in all directions of the edge pixel of interest, this method performs voting only in one side area or only in both side areas of each edge pixel. A small amount and high-speed processing can be realized. Since only the size of a person's head within a specific range from the imaging device is set as a detection target, highly reliable human head detection can be performed.
[0096]
The present invention is not limited to the above-described configuration, and various modifications can be made. For example, in the above description, the voting cells in the voting space correspond to the pixels of the captured image and the edge image on a one-to-one basis, but a plurality of pixels may correspond to one voting cell.
[Brief description of the drawings]
FIG. 1 is a block diagram of a human presence position detection apparatus according to an embodiment of the present invention.
FIG. 2 is a conceptual diagram of an area sensor to which the same device is applied.
FIG. 3 is a processing flow diagram of captured image data storage means of the apparatus.
4A is a captured image diagram, and FIG. 4B is an edge image diagram.
FIG. 5 is a flowchart in edge image data generation means according to an embodiment of the present invention.
FIG. 6 is a coordinate diagram of a captured image.
7A is a diagram of a 3 × 3 pixel local window, FIG. 7B is a diagram of an x-direction differential operator, and FIG. 7C is a y-direction differential operator diagram.
FIGS. 8A to 8D are diagrams showing a shape model of a human head according to an embodiment of the present invention.
FIG. 9 is a diagram showing a dimensional relationship between a person in the real world and a person in the image.
10A and 10B are voting template diagrams according to an embodiment of the present invention.
FIGS. 11A and 11B are diagrams for explaining a concentration gradient direction, and FIGS. 11B and 11C are diagrams of voting templates according to an embodiment of the present invention. FIGS.
FIG. 12 is a diagram of a voting template according to one embodiment of the present invention.
FIG. 13 is a diagram of a voting template according to one embodiment of the present invention.
FIGS. 14A and 14B are views showing coordinates of a voting part of a voting template according to an embodiment of the present invention.
FIG. 15 is a diagram of a voting template table according to an embodiment of the present invention.
FIG. 16 is a diagram of a voting template table according to an embodiment of the present invention.
FIG. 17 is a diagram showing types of shape models according to an embodiment of the present invention.
18A is an edge image diagram according to an embodiment of the present invention, a relationship diagram of an imaging space and a voting space, FIG. 18B is a diagram of an edge image, and FIG. 18C is a diagram of a voting space.
FIG. 19 is a distribution diagram of voting values in a voting space according to an embodiment of the present invention.
FIGS. 20A and 20B are views showing a relationship between a radius of curvature direction and a voting unit according to an embodiment of the present invention.
FIG. 21 is a diagram of a voting space for each zone after voting according to an embodiment of the present invention.
FIG. 22 is a diagram of a zone in real space according to one embodiment of the present invention.
FIG. 23 is a diagram showing a case where a hill of vote values has a plurality of ridges.
FIG. 24 is a process flow diagram of an alarm sound generating means according to an embodiment of the present invention.
FIG. 25 is a block diagram of a detection apparatus according to an embodiment of the present invention.
FIG. 26 is an external view of an autonomous mobile device according to an embodiment of the present invention.
FIG. 27 is a block diagram of the above autonomous mobile device.
FIG. 28 is a diagram for explaining determination of the presence direction of an imaging device and a person according to an embodiment of the present invention.
FIG. 29 is a diagram showing a time lapse of a person location according to an embodiment of the present invention.
FIG. 30 is a human presence map according to one embodiment of the present invention.
FIGS. 31A and 31B are person presence map diagrams according to one embodiment of the present invention.
32A is a captured image diagram, and FIG. 32B is a cut head image diagram.
FIG. 33 is a diagram of an n-dimensional pattern space.
FIG. 34 is a pattern recognition flow diagram of a head image according to an embodiment of the present invention.
FIG. 35 is a flowchart of head image determination according to an embodiment of the present invention.
FIG. 36 is a flowchart of head orientation determination according to an embodiment of the present invention.
FIG. 37 is a diagram of an n-dimensional pattern space.
FIG. 38 is a diagram of an n-dimensional pattern space.
FIGS. 39A and 39B are diagrams of head images. FIGS.
FIG. 40 is a flowchart of a human location detection method according to an embodiment of the present invention.
[Explanation of symbols]
One person location detection device
2 Imaging device
3 Captured image data storage means
4 Edge image data generation means
5 Voting processing means
6 Voting template preparation means
7 Voting space generation means
8 person presence position detection means
9 Alarm generation means
12 Autonomous mobile devices
30 person presence map
31 areas where people were detected
32 area without people
33 Unidentified area
G representative point
P, Q conjugate point
P0 captured image
P1 edge image
P2 voting space
r Representative point direction line
T voting template
T1-T4 shape model
V, Vp, Vp0 to Vp8, Vq0 to Vq8 Voting section
VC voting cell
Ψ, Φ concentration gradient direction

Claims (20)

In an apparatus for detecting the presence and position of a person by extracting a person's head image from an image captured by the imaging apparatus,
The shape of the human head image is approximated by a shape model of a figure surrounded by an outwardly convex curve, the focal length of the lens of the imaging device, the detection distance from the imaging device to the person to be detected, and the human head Presuming the standard size and size range of the shape model by estimating the size range of the contour of the human head image determined based on the standard size of
A density gradient direction is defined at each point on the boundary curve of the standard size shape model, a representative point is defined inside the shape model, and a voting unit corresponding to the standard size of the shape model is provided at the representative point. And a voting unit corresponding to the size range before or after the representative point on a representative point direction line defined as a line connecting the representative point to a representative point on the boundary curve of the shape model, A voting template preparation means for preparing a voting template created by a boundary curve of a standard size shape model so that it can be referred to;
Captured image data storage means for digitizing an image input from the imaging device and storing the digitized captured image in a memory;
The captured image is subjected to differential processing for extracting an edge portion composed of pixels corresponding to the outline of a person, the contour of a face, etc., and pixels having a predetermined differential intensity or higher are extracted as edge pixels, and the captured image An edge image data generating means for calculating a density gradient direction from the image data, generating an edge image with density gradient direction data added for each extracted edge pixel, and storing the edge image in a memory;
Voting space generating means for generating a voting space having a voting cell at a position corresponding to each pixel position of the captured image and the edge image, and storing the voting space in a memory;
A point on the boundary curve of the voting template having a density gradient direction corresponding to the density gradient direction added to the edge pixel is superimposed on each edge pixel of the edge image, and at this time, the voting part of the voting template is located. Voting processing means for voting to a voting cell in a voting space corresponding to a pixel position on an edge image and storing it in a memory for each edge pixel, and counting the voting values of each cell in the voting space;
Human presence position detecting means for detecting a cell having a voting value equal to or greater than a predetermined threshold in a voting space where voting is performed, and detecting a position on a captured image corresponding to the detected cell as a human head position , equipped with a,
The voting template preparation means conjugates each point on the boundary curve having a density gradient direction opposite to the density gradient direction defined for each point on the boundary curve of the standard size shape model. A conjugate point is defined, and one of the points conjugate to each other maintains the same relative positional relationship as the voting unit inside the shape model, and the other of the points conjugate to each other further includes a voting unit outside the shape model. A human presence position detecting device characterized by that. For each edge pixel of the edge image, the voting processing means determines a direction in which the center of curvature exists by obtaining a curvature of a contour composed of the edge pixel of interest and a plurality of edge pixels in the vicinity of the edge pixel of interest. 2. The human presence position detecting apparatus according to claim 1 , wherein, in the voting, a voting cell is voted on a voting cell in a voting space corresponding to a voting position in a direction in which the center of curvature exists. 2. The human presence position detection apparatus according to claim 1 , wherein the voting template preparation unit further includes voting units in the vicinity of a representative point of the voting template and on both sides of the representative point direction line. The voting template preparation means according to claim a relative position of the voting unit for each point on the boundary curve of the shape model, the density gradient direction in the respective points as an index, and to store in memory in a table format 1 The human presence position detecting device described in 1. The voting template preparation means stores voting template data for each distance from the imaging device to the person in advance,
The voting process unit, according to claim 1, referring to the data of the vote templates based on the distance measurement result of the distance sensor that measures the distance to the detection target to set the voting template, wherein the vote Position detection device. When the detected cell having a vote value equal to or greater than a predetermined threshold is divided into a plurality of chunks, the human presence position detecting means checks the distance between the chunks and the distance is within a predetermined range. which is integrated by grouping if, integrated existing position detection of a person as claimed in claim 1, characterized in that to detect the position on the corresponding captured image to the representative point of the mass as a head position of the person apparatus. The voting template preparation means optionally sets a plurality of detection distance ranges from the imaging device to the person to be detected to prepare a standard size and size range of the shape model for each detection distance range, and a voting template based thereon,
The voting space generating means generates a voting space for each detection distance range,
The voting processing means performs the voting process on the voting space corresponding to the detection distance range for each voting template corresponding to each detection distance range,
The human presence position detecting means, when detecting a cell having a vote value equal to or greater than a predetermined threshold in a voting space corresponding to each detection distance range, causes the human presence position to correspond to the detection distance range. The human presence position detecting device according to claim 1 . The voting template preparation means divides the human detection distance range so that the size range is equal to or smaller than a predetermined value when the size range of the shape model is equal to or larger than a predetermined value, and the shape model standard is divided into the divided ranges. Has a size and size range and voting templates based on them,
The voting space generating means generates a voting space for each of the divided detection distance ranges,
The voting processing means performs the voting process on the voting space corresponding to the detection distance range for each voting template corresponding to each divided detection distance range,
When the person presence position detecting means detects a cell having a vote value equal to or larger than a predetermined threshold in the corresponding voting space for each of the divided detection distance ranges, the person presence position is made to correspond to those detection distance ranges. The human presence position detecting device according to claim 1 . The voting template preparation means approximates a human head shape with a plurality of shapes having different postures in the same shape model, and prepares a voting template for each shape of each posture,
The voting space generating means generates a voting space for each shape of the posture,
2. The human presence position detection apparatus according to claim 1 , wherein the voting processing means performs voting processing on each cell in the voting space for each shape having a different posture. The voting template preparation means approximates a human head shape with a plurality of different shape model shapes, prepares a voting template for each shape model shape,
The voting space generating means generates a voting space for each shape of the shape model,
2. The human presence position detecting apparatus according to claim 1 , wherein the voting processing means performs voting processing on each cell on the voting space for each shape model. The person present position detecting means to claim 1, characterized in that cutting out size image of the projection image obtained by projecting the detected shape that models the head to the head position on the captured image The human presence position detection device described. The human presence position detecting means identifies and detects the orientation of the head by comparing the cut image with head data previously classified and stored for each orientation of the head. The human presence position detection apparatus according to claim 11 . The human presence position detection unit divides the cut image into a plurality of predetermined small images when a plurality of candidates are output as the head direction, and a difference in density distribution between the divided small images 13. The human presence position detecting device according to claim 12, wherein the orientation of the head is determined by the method. The human presence position detecting means estimates a central position of a human detection distance range corresponding to a voting space used for detecting a human head position as a distance from the imaging device to the human. present position detecting device of a person according to 7. The human presence position detecting means obtains the human presence position direction with respect to the imaging device from the detected head position on the captured image and the focal length of the imaging device, and corresponds to the voting space used for detecting the human head position. the center position of the detection distance range of human estimates the distance from the imaging apparatus to humans, by the distance to the direction and human human, according to claim 7, wherein the determination of the position of the person relative to the imaging device Position detection device. The human presence position detection means divides the space in the line-of-sight direction of the imaging device into predetermined zones from the imaging device, and the result of the previous detection and the current detection result of the presence / absence state of a person in each zone In comparison, if the zone to which the currently detected person belongs is closer to the zone to which the previously detected person belongs, it is determined that the person is approaching, and the zone to which the currently detected person belongs is more than the zone to which the previously detected person belongs claims determined to stay away person when the far zone, the zone a person who detected current has zones and previously detected belonging belongs and judging the same time is stopped human 7 Or the human presence position detection apparatus of Claim 14 . The human presence position detecting means creates a human presence map in which the space in the line-of-sight direction of the imaging device is divided into a plurality of areas based on the imaging device position, and each area is divided into an area where a person is detected and an unidentified area. The human presence position detection apparatus according to claim 15, wherein the human presence position is detected by classifying the areas into areas where no people are present. The human presence position detection means creates a human presence map in which the space in the line-of-sight direction of the imaging device is divided into a plurality of areas based on the imaging device position, and sets the position where the person is detected as a corresponding position on the map. Register and store the cut-out image of the detected person's head in association with the detection position, compare the map at the previous detection time with the current map, and detect the detection position of the person of interest at the previous detection time And the current person detected within the movable range and its detection position as a candidate for association, and if there is one association candidate, associate and track,
When there are a plurality of association candidates, the image correlation between the cut-out image of the head stored in association with the detection position of the person at the previous detection time and the cut-out image of the head of the person that is the association candidate is calculated. 16. The person presence position detecting apparatus according to claim 15, wherein the person's presence position detecting apparatus according to claim 15, wherein the person's presence position detecting apparatus according to claim 15 is calculated and tracks the person having the highest correlation degree and the detected position thereof as correct associations. In a method for detecting the presence and position of a person by extracting a head image of the person from a captured image captured by an imaging device,
The shape of the human head image is approximated by a shape model of a figure surrounded by an outwardly convex curve, the focal length of the lens of the imaging device, the detection distance from the imaging device to the person to be detected, and the human head Presuming the standard size and size range of the shape model by estimating the size range of the contour of the human head image determined based on the standard size of
A density gradient direction is defined at each point on the boundary curve of the standard size shape model, a representative point is defined inside the shape model, and a voting unit corresponding to the standard size of the shape model is provided at the representative point. And a voting unit corresponding to the size range before or after the representative point on a representative point direction line defined as a line connecting the representative point to a representative point on the boundary curve of the shape model, A voting template preparation process in which a voting template created by a boundary curve of a standard size shape model can be referred to, and
A captured image data storage process of storing a digitized captured image input from the imaging device in a memory;
The captured image is subjected to differential processing for extracting an edge portion composed of pixels corresponding to the outline of a person, the contour of a face, etc., and pixels having a predetermined differential intensity or higher are extracted as edge pixels, and the captured image An edge image data generation process for calculating a density gradient direction from the image and generating an edge image with density gradient direction data added for each extracted edge pixel;
A voting space generation process for generating a voting space having a voting cell at a position corresponding to each pixel position of the captured image and the edge image and storing the voting space in a memory;
A point on the boundary curve of the voting template having a density gradient direction corresponding to the density gradient direction added to the edge pixel is superimposed on each edge pixel of the edge image, and at this time, the voting part of the voting template is located. Voting process for voting to a voting cell in a voting space corresponding to a pixel position on an edge image and storing it in a memory for each edge pixel, and totaling the voting value of each cell in the voting space;
A human presence position detection process for detecting a cell having a voting value equal to or higher than a predetermined threshold in a voting space where voting is performed, and detecting a position on a captured image corresponding to the detected cell as a human head position; , equipped with a,
In the voting template preparation process, for each point on the boundary curve of the shape model of the standard size, a point on the boundary curve having a concentration gradient direction opposite to the concentration gradient direction defined for the point is conjugated with each other. A conjugate point is defined, and one of the points conjugate to each other maintains the same relative positional relationship as the voting unit inside the shape model, and the other of the points conjugate to each other further includes a voting unit outside the shape model. A method for detecting the presence position of a person , comprising preparing a voting template created by a boundary curve of a standard model of a shape model . An autonomous mobile device that controls movement by using position information of a person detected by using the human presence position detection apparatus according to any one of claims 15 to 17 .

Images